In this work we study a local adaptive smoothing algorithm for a-posteriori-steered $p$-robust multigrid methods. The solver tackles a linear system which is generated by the discretization of a second-order elliptic diffusion problem using conforming finite elements of polynomial order $p \ge 1$. After one V-cycle (“full-smoothing” substep) of the solver of [HAL Preprint 02494538, 2020], we dispose of a reliable, efficient, and localized estimation of the algebraic error. We use this existing result to develop our new adaptive algorithm: thanks to the information of the estimator and based on a bulk- chasing criterion, cf. Dörfler [SIAM J. Numer. Anal., 33 (1996), pp. 1106–1124], we mark patches of elements and levels with increased estimated error. Then, we proceed by a modified and cheaper V-cycle (“adaptive-smoothing” substep), which only applies smoothing in the marked regions. The proposed adaptive multigrid solver picks autonomously and adaptively the optimal step-size per level as in our previous work but also the type of smoothing per level (weighted restricted additive or additive Schwarz) and concentrates smoothing to marked regions with high error. We prove that each substep (full and adaptive) contracts the error $p$-robustly, which is confirmed by numerical experiments. Moreover, the proposed algorithm behaves numerically robustly with respect to the number of levels as well as to the diffusion coefficient jump.