We study the fluctuations associated to the a.s. convergence of the outliers established by Belinschi–Bercovici–Capitaine of an Hermitian polynomial in a complex Wigner matrix and a spiked deterministic real diagonal matrix. Thus, we extend the nonuniversality phenomenon established by Capitaine–Donati-Martin–Féral for additive deformations of complex Wigner matrices, to any Hermitian polynomial. The result is described using the operator-valued subordination functions of free probability theory.