Background: Speech disorders are amongst the first symptoms to appear in Parkinson's disease (PD). Objectives: We aimed to characterize PD voice signature from the prodromal stage (isolated rapid eye movement sleep behavior disorder, iRBD) to early PD using an automated acoustic analysis and compare male and female patients. We carried out supervised learning classifications to automatically detect patients using voice only. Methods: Speech samples were acquired in 256 French speakers (117 participants with early PD, 41 with iRBD, and 98 healthy controls), with a professional quality microphone, a computer microphone and their own telephone. High-level features related to prosody, phonation, speech fluency and rhythm abilities were extracted. Group analyses were performed to determine the most discriminant features, as well as the impact of sex, vocal tasks, and microphone type. These speech features were used as inputs of a support vector machine and were combined with classifiers using low-level features. Results: PD related impairments were found in prosody, pause durations and rhythmic abilities, from the prodromal stage. These alterations were more pronounced in men than in women. Early PD detection was achieved with a balanced accuracy of 89% in males and 70% in females. Participants with iRBD were detected with a balanced accuracy of 63% (reaching 70% in the subgroup with mild motor symptoms). Conclusion: This study provides new insight in the characterization of sex-dependent early PD speech impairments, and demonstrates the valuable benefit of including automated voice analysis in future diagnostic procedures of prodromal PD.