Plasmonic coupling between adjacent metallic nanoparticles can be exploited for acousto-plasmonics, single-molecule sensing, and photochemistry. Light absorption or electron probes can be used to study plasmons and their interactions, but their use is challenging for disordered systems and colloids dispersed in insulating matrices. Here, we investigate the effect of plasmonic coupling on optomechanics with Brillouin light spectroscopy (BLS) in a prototypical metal–polymer nanocomposite, gold nanorods (Au NRs) in polyvinyl alcohol. The intensity of the light inelastically scattered on thermal phonons captured by BLS is strongly affected by the wavelength of the probing light. When light is resonant with the transverse plasmons, BLS reveals mostly the normal vibrational modes of single NRs. For lower energy off-resonant light, BLS is dominated by coupled bending modes of NR dimers. The experimental results, supported by optomechanical calculations, document plasmonically enhanced BLS and reveal energy-dependent confinement of coupled plasmons close to the tips of NR dimers, generating BLS hot-spots. Our work establishes BLS as an optomechanical probe of plasmons and promotes nanorod–soft matter nanocomposites for acousto-plasmonic applications.