In this paper, we propose a method to test a probabilistic FSM. The testing process consists of two parts. First, we check if there are any output faults or transfer faults in transitions. In order to identify a state of a PFSM, the characterization set is extended such that states are identified not only by observing output sequences but also by comparing probabilities. Second, we test whether the transition probabilities are correctly implemented. Interval estimation is used to assert the correctness of transition probabilities where a test verdict is assigned with a given confidence level. From a given confidence level and confidence interval length, a method is presented to determine the test sequence repetition numbers for testing probabilities. Fault coverage evaluation is carried out based on extended fault types where probabilities are changed. As an application, we apply the proposed method to a probabilistic non-repudiation protocol