We address the possibility of atomic-scale control of the plasmon modes of graphene nanostructures. Using the time-dependent many-body approach we show that for the zigzag and armchair nanoribbons, the single carbon atom vacancy results in "on" and "off" switching of the longitudinal plasmon modes or in a change of their frequency. The effect stems from the robust underlying physical mechanism based on the strong scattering of the two-dimensional (2D) electrons on the vacancy defects in graphene lattice. Thus our findings establish a platform for optical response engineering or sensing in 2D materials.