In this work, we study a local adaptive smoothing algorithm for a-posteriori-steered $p$-robust multigrid methods. The solver tackles a linear system which is generated by the discretization of a second-order elliptic diffusion problem using conforming finite elements of polynomial order $p$ ≥ 1. After one V-cycle (“full-smoothing” substep) of the solver of Miraçi, Papež, and Vohralík [SIAM J. Sci. Comput. Accepted for publication], we dispose of a reliable, efficient, and localized estimation of the algebraic error. We use this existing result to develop our new adaptive algorithm: thanks to the information of the estimator and based on a bulk-chasing criterion, cf. Dörfler [SIAM J. Numer. Anal., 33 (1996), pp. 1106–1124], we mark patches of elements with increased estimated error on all levels. Then, we proceed by a modified and cheaper V-cycle (“adaptive-smoothing” substep), which only applies smoothing in the marked regions. The proposed adaptive multigrid solver picks autonomously and adaptively the optimal step-size per level as in our previous work but also the type of smoothing per level (weighted restricted additive or additive Schwarz) and concentrates smoothing to marked regions with high error. We prove that, under a numerical condition that we verify in the algorithm, each substep (full and adaptive) contracts the error $p$-robustly, which is confirmed by numerical experiments. Moreover, the proposed algorithm behaves numerically robustly with respect to the number of levels as well as to the diffusion coefficient jump for a uniformly-refined hierarchy of meshes.