The cardiovascular diseases are one of the main causes of death around the world. Automatic detection and classification of electrocardiogram (ECG) signals are important for diagnosis of cardiac irregularities. This paper proposes to apply the Support Vector Machines (SVM) classification, to diagnose heartbeat abnormalities, after performing feature extraction on the ECG signals. The experiments were conducted on the ECG signals from the MIT-BIH arrhythmia database [1] to classify two different abnormalities and normal beats. Kernel Principal Component Analysis (KPCA) is used for feature extraction since it performs better than PCA on ECG signals due to their nonlinear structures. This is demonstrated in a previous work [2]. Two multi-SVM classification schemes are used, One-Against-One (OAO) and One-Against-All (OAA), to classify the ECG signals into different disease categories. The experiments conducted show that SVM combined with KPCA performs better than that without feature extraction. Moreover, our results show a better performance in Gaussian KPCA feature extraction with respect to other kernels. Furthermore, the performance of Gaussian OAA-SVM combined with KPCA has higher average accuracy than Gaussian OAA-SVM in ECG classification.