The Polarimetric Synthetic Aperture Radar (PolSAR) covariance matrix is generally modeled by a complex Wishart distribution. For textured scenes, the product model is used and the texture component is often modeled by a Gamma distribution. In this paper, authors propose to use the Fisher distribution for texture modeling. From a Fisher distributed texture component, we derive the distribution of the complex covariance matrix and we propose to implement the KummerU distribution in a hierarchical segmentation and a hierarchical clustering algorithm. Segmentation and classification results are shown on synthetic images and on ESAR L-band PolSAR data over the Oberpfaffenhofen test-site.