Solid bodies in free fall in a viscous fluid generally fall along a non-straight path, and a variety of periodic (fluttering, tumbling) and non-periodic regimes can be observed. We analyze the structure of the couplings between the fluid and the body, restricting to a linear stability framework. Introducing a simple toy model consisting of a infinitely long plate sliding along a vertical wall, we show that in the limit of large solid-to-fluid masses a decoupling takes place, allowing us to distinguish two kinds of modes: ``wake'' modes in which the body motion has virtually no influence, and ``body'' modes for which the intrinsic wake dynamics can be neglected. Turning to more realistic objects, we show that the ``body'' modes can be described through a rationally derived aerodynamic model (based on quasi-static assumptions), yielding either a static instability, or a dynamic, low-frequency, instability. Considering 2D rectangular rods and 3D disks, we explore the competition between the three kinds of instabilities. For objects elongated in the spanwise direction, it is found that wake instability dominates in case of 2D rectangles and low-frequency instability dominates in case of disks. For objects elongated in the streamwise direction, static instability always dominate.