A comprehensive treatment of the topological derivative for anisotropic elasticity is presented, with both the background material and the trial small inhomogeneity assumed to have arbitrary anisotropic elastic properties. A formula for the topological derivative of any cost functional defined in terms of regular volume or surface densities depending on the displacement is established, by combining small-inhomogeneity asymptotics and the adjoint solution approach. The latter feature makes the proposed result simple to implement and computationally efficient. Both three-dimensional and plane-strain settings are treated; they differ mostly on details in the elastic moment tensor (EMT). Moreover, the main properties of the EMT, a critical component of the topological derivative, are studied for the fully anisotropic case. Then, the topological derivative of strain energy-based quadratic cost functionals is derived, which requires a distinct treatment. Finally, numerical experiments on the numerical evaluation of the EMT and the topological derivative of the compliance cost functional are reported.