In this paper, we analyse the long-time behavior of solutions to a coupled system describing the motion of a rigid disk in a 2D viscous incompressible fluid. Following previous approaches, we look at the problem in the system of coordinates associated with the center of mass of the disk. Doing so, we introduce a further nonlinearity to the classical Navier Stokes equations. In comparison with the classical nonlinearities, this new term lacks time and space integrability, thus complicating strongly the analysis of the long-time behavior of solutions. We provide herein two refined tools : a refined analysis of the Gagliardo-Nirenberg inequalities and a thorough description of fractional powers of the so-called fluid-structure operator. On the basis of these two tools we extend previous decay estimates to arbitrary initial data and show local stability of the Lamb-Oseen vortex.