Cardiovascular (CV) disorders and obstructive sleep apnea (OSA) are very prevalent diseases worldwide. Multiple studies have demonstrated that OSA is associated with increased CV risk. Clinicians need to assess the CV risk to select the proper OSA treatment. A growing number of research have employed machine learning to predict CV risk by integrating clinical and sleep features. In this paper, a multiple input deep learning model was proposed to directly use sleep signals combined with clinical features. Data from 5,506 patients from the Pays de la Loire Sleep Cohort, without a history of major adverse cardiovascular events (MACE), investigated for OSA, were used. After a median follow-up of 6.0 years, 613 patients had been diagnosed with MACE according to the French national health system. Following an architecture selection, deep survival convolutional neural networks were computed to assess the MACE risk score. A custom loss function was integrated to consider the follow-up time of each patient. Based on the weights of each model input, a method for interpreting the model was also proposed to show the contribution of signals compared to clinical features. Sleep signals were extracted from a home sleep apnea test. The best results were obtained with the autonomic manifestation signal. An area under the ROC curve of 0.823 was reached. After interpretation of the models, consideration of sleep appeared to be more important in women and in those under 60. This method may help improve OSA patient care by estimating their risk of MACE during sleep diagnosis.