Originally formulated in the context of topology optimization, the concept of topological derivative has also proved effective as a qualitative inversion tool for a wave-based identification of finite-sized objects. This approach remains, however, largely based on a heuristic interpretation of the topological derivative, whereas most other qualitative approaches to inverse scattering are backed by a mathematical justification. As an effort toward bridging this gap, this study focuses on a topological derivative approach applied to the L2-norm of the misfit between far-field measurements. Either an inhomogeneous medium or a finite number of point-like scatterers are considered, using either the Born approximation or a full-scattering model. Topological derivative-based imaging functionals are analyzed using a suitable factorization of the far-field operator, for each of the considered cases, in order to characterize their behavior and assess their ability to reconstruct the unknown scatterer(s). Results include the justification of the usual sign heuristic underpinning the method for (i) the Born approximation and (ii) full-scattering models limited to moderately strong scatterers. Semi-analytical and numerical examples are presented. Within the chosen framework, the topological derivative approach is finally discussed and compared to other well-known qualitative methods.