Particle vibrational spectroscopy has emerged as a new tool for the measurement of elasticity, glass transition, and interactions at a nanoscale. For colloid-based materials, however, the weakly localized particle resonances in a fluid or solid medium renders their detection difficult. The strong amplification of the inelastic light scattering near surface plasmon resonance of metallic nanoparticles (NPs) allowed not only the detection of single NP eigenvibrations but also the interparticle interaction effects on the acoustic vibrations of NPs mediated by strong optomechanical coupling. The "rattling" and quadrupolar modes of Ag/polymer and polymer-grafted Ag NPs with different diameters in their assemblies are probed by Brillouin light spectroscopy (BLS). We present thorough theoretical 3D calculations for anisotropic Ag elasticity to quantify the frequency and intensity of the "rattling" mode and hence its BLS activity for different interparticle separations and matrix rigidity. Theoretically, a liquidlike environment, e.g., poly(isobutylene) (PIB) does not support rattling vibration of Ag dimers but unexpectedly hardening of the extremely confined graft melt renders both activation of the former and a frequency blue shift of the fundamental quadrupolar mode in the grafted nanoparticle Ag@PIB film.