A comprehensive understanding of the mechanism by which type I collagen (Col) interacts with hydroxyapatite nanoparticles (Hap NPs) in aqueous solutions is a pivotal step for guiding the design of biologically relevant nanocomposites with controlled hierarchical structure. In this paper we use a variety of Hap NPs differing by their shape (rod vs platelet) and their size (~30 vs ~130 nm) and investigate their mechanism(s) of interaction with collagen. The addition of collagen to the Hap suspensions induces different effects that strongly depend on the nanoparticle type. Interestingly, the use of small rods, typically with ~30 nm of length (R30), leads to the formation of assembled collagen fibrils decorated with Hap nanocrystals which, in turn, self-assemble progressively to form larger fibrillar HapCol composite. The crystals decorating collagen provides “intrinsic” negative charges to the fibrillar objects that allows their incorporation in three-dimensional structure using layer-by-layer (LbL) assembly. This offers a straightforward way to construct a collagen-based hybrid material with well-defined hierarchy under near-physiological conditions. In situ, QCM-D monitoring revealed the build-up of soft and highly hydrated hybrid (PAH/R30Col)n multilayers, for which the mechanism of growth was very different from that observed for polyelectrolytes and nanoparticles without collagen (PAH/R30). The LbL assembly of crystal-decorated collagen yields a hierarchical nanostructured film whose thickness and roughness can be modulated by the addition of salt, and incorporate fibrillar objects of about 400 nm in width and few µm in length, as probed by AFM. The approach described in this work provides a relevant way to better control the (supra)molecular assembly of Col and Hap NPs with the perspective of developing hierarchical HapCol nanocomposites with tuned properties for various biomedical applications.