In this paper, a data-adaptive method, namely Principal Modes (PM) method, based on the spatially averaged temporal covariance of a time series of InSAR displacement measurement obtained from consecutive SAR acquisitions is proposed to retrieve the displacement signal for decorrelating targets. On wrapped interferogram time series, the PM method can highlight and restore coherent fringe patterns where they are more or less significantly hindered by decorrelation noise, while on unwrapped interferogram time series, the PM method provides a satisfactory separation of the displacement signal from the spatially correlated perturbations. A two stage application of the PM method to both wrapped and unwrapped interferogram time series can significantly improve the retrieval of the displacement signal. Synthetic simulations are first performed to investigate the impact of the choice of the appropriate number of modes to retain in the EOF decomposition and of the time series size on the performance of the PM method, as well as to highlight the efficiency of the PM method. Then, the PM method is applied to time series of wrapped and unwrapped Sentinel 1 A/B interferograms over the Gorner glacier between October 2016 and April 2017. The main characteristics of the PM method, such as realistic assumptions, ease of implementation and high efficiency, are highlighted.