To establish an alternative analytical framework for the elastic-wave imaging of underground cavities, the focus of this study is an extension of the concept of topological derivative, rooted in elastostatics and shape optimization, to three-dimensional elastodynamics involving semi-infinite and infinite solids. The main result of the proposed boundary integral approach is a formula for topological derivative, explicit in terms of the elastodynamic fundamental solution, obtained by an asymptotic expansion of the misfit-type cost functional with respect to the creation of an infinitesimal hole in an otherwise intact (semi-infinite or infinite) elastic medium. Valid for an arbitrary shape of the infinitesimal cavity, the formula involves the solution of six canonical exterior elastostatic problems, and becomes fully explicit when the vanishing cavity is spherical. A set of numerical results is included to illustrate the potential of topological derivative as a computationally efficient tool for exposing an approximate cavity topology, location, and shape via a grid-type exploration of the host solid. For a comprehensive solution to three-dimensional inverse scattering problems involving elastic waves, the proposed approach can be used most effectively as a pre-conditioning tool for more refined, albeit computationally intensive minimization-based imaging algorithms. To the authors' knowledge, an application of topological derivative to inverse scattering problems has not been attempted before; the methodology proposed in this paper could also be extended to acoustic problems.