Introduction: Large real-life databases (DB) of MS patients usually consist of clinical data, including limited imaging metrics. The possibility of re-analyzing images to extract a wide range of metrics is now possible through the use of AI based methods. The presence of new lesions on longitudinal MRIs for exemple is used to assess the effectiveness of treatments in real-life studies. However, the automated tools that currently identify these new lesions are designed as an aid for radiologists, potentially generating false positives. The possibility of transferring these methods to analyze DB without supervision should be assessed. Objectives/Aims: To compare the performance of an automated method to classify MS patients as “active” or “inactive” based on new lesions on FLAIR images in a large real-life multicentric DB with respect to the data provided in the clinical DB. Methods: We included 1412 pairs of brain MRI scans from 868 MS patients with both FLAIR images available in the French OFSEP HD cohort imaging DB at 2 time points, and the radiological comparison captured in the clinical DB. An automated tool based on a fully convolutional neural network (trained on 159 patients) was used to detect new lesions between the corresponding longitudinal FLAIR images. Then, 160 pairs of brain MRI scans for which the automated method output and the corresponding clinical DB comparison disagree were randomly selected and their MRI were reviewed by 2 experts to constitute a ground truth. Differences in sensitivity, specificity and accuracy between the automated method and the clinical DB were assessed. Results: Overall, 222 out of 1412 (16%) intervals were considered active from the clinical DB, compared to 467 (35%) from the automated method. Over the 160 cases of disagrement included in the ground truth, the automated method correctly classified patients in 66% of the cases and the clinical DB in 34%. More specifically, the automated method was more sensitive than the clinical DB (p<0.001), but the clinical DB was more specific (p<0.001). Under simplified assumptions, we extrapolate from these results a sensitivity, specificity and accuracy of about 0.95, 0.99 and 0.92 for the clinical DB and 0.99, 0.69 and 0.96 for the automated method. Conclusion: The automated analysis of images collected in large real-life databases allows to correctly classify MS patients as active or inactive in a large majority of cases, and offers the possibility to extract other metrics such as lesion number or volume to analyze the efficacy of treatments in real-life.