Polarized light scanning microscopy is a non-invasive and contrast-enhancing technique to investigate anisotropic specimens and chiral organizations. However, such arrangements suffer from insensitivity to confined blend of structures at sub-diffraction level. Here for the first time, we present that the pixel-by-pixel polarization modulation converted to an image phasor approach issues an insightful view of cells to distinguish anomalous subcellular organizations. To this target, we propose an innovative robust way for identifying changes in the chromatin compaction and distortion of nucleus morphology induced by the activation of the lamin-A gene from Hutchinson-Gilford progeria syndrome that induces a strong polarization response. The phasor mapping is evaluated based on the modulation and phase image acquired from a scanning microscope compared to a confocal fluorescence modality of normal cell opposed to the progeria. The method is validated by characterizing polarization response of starch crystalline granules. Additionally, we show that the conversion of the polarization-resolved images into the phasor could further utilized for segmenting specific structures presenting various optical properties under the polarized light. In summary, image phasor analysis offers a distinctly sensitive fast and easy representation of the polarimetric contrast that can pave the way for remote diagnosis of pathological tissues in real-time.