Background and Objective: The prostate cancer interventions, which need an accurate prostate segmentation, are performed under ultrasound imaging guidance. However, prostate ultrasound segmentation is facing two challenges. The first is the low signal-to-noise ratio and inhomogeneity of the ultrasound image. The second is the non-standardized shape and size of the prostate. Methods: For prostate ultrasound image segmentation, this paper proposed an accurate and efficient method of Active Shape Model (ASM) with Rayleigh Mixture Model Clustering (ASM-RMMC). Firstly, Rayleigh Mixture Model (RMM) is adopted for clustering the image regions which present similar speckle distributions. These content-based clustered images are then used to initialize and guide the deformation of an ASM model. Results: The performance of the proposed method is assessed on 30 prostate ultrasound images using four metrics as Mean Average Distance (MAD), Dice Similarity Coefficient (DSC), False Positive Error (FPE) and False Negative Error (FNE). The proposed ASM-RMMC reaches high seg-mentation accuracy with 95 ± 1% for DSC, 1.87 ± 0.51 pixels for MAD, 2.10% ± 0.36% for FPE and 2.78% ± 0.7% for FNE, respectively. Moreover, 1 the average segmentation time is less than 8 s when treating a single prostate ultrasound image through ASM-RMMC. Conclusions: This paper presents a method for prostate ultrasound image segmentation, which achieves high accuracy with less computational complexity and meets the clinical requirements.