We investigate transmitter (Tx) IQimbalance compensation based on the blind adaptivesource separation (BASS) method in a dualpolarization M−QAM optical coherent system. Therobustness of the BASS method against the residualcarrier frequency offset (CFO) is numericallyinvestigated and compared to that of the Gram-Schmidt orthogonalization procedure (GSOP). Wefurther validate experimentally the proposed methodwith 10−Gbaud optical 4−QAM and 16−QAM signals at30° and 10° phase imbalance, respectively, withsimulated impairments. More specifically, in thepresence of 5×10−6 residual CFO (normalized to thesample rate), the optical signal-to-noise ratio (OSNR)penalty reduction of the BASS method compared tothe GSOP method is 1 dB for 4−QAM at a bit-errorratio(BER) of 2×10−3 and 2 dB for 16−QAM at a BER of10−3. In contrast to the GSOP that requires anindependent block, the BASS method can beintegrated into an equalizer, simplifying theoperation and allowing parallel implementation.