Purpose The cataract is the most common cause of severe vision impairment or blindness worldwide. It is essential to periodically diagnose the retina in order to prevent cataract severity, and so to enhance the life quality of cataract-affected patients. Cataract grading through a fundus image is feasible with higher accuracy. However, a delay of early cataract screening is registered caused by deficiency of ophthalmologists and imaging devices. The challenge is to propose a CAD system to grade the cataract from retinal images. Method In this paper, an ensemble learning framework for cataract grading is put forward, where three convolutional deep neural networks are stacked in order to provide higher performance grading. The main contributions of this work are given as follows: (1) Preprocessing and data augmentation of fundus images are performed to ensure the robustness of the cataract grading; (2) The well-known DL architectures (Inception-V3, MobileNet-V2 and NasNet-Mobile) are fine-tuned and learned as base classifiers; (3) A stacking method is propounded to combine the features of base classifiers. Results The evaluation is conducted using a dataset of 590 fundus images selected from two public databases. The suggested framework achieves 93.97% accuracy, 95.59% sensitivity, 91.67% specificity, 94.20% precision and 94.89% F-measure for cataract grading. Conclusion The proposed framework successfully grades fundus images into cataract severity. Moreover, stacking ensemble learning allows achieving a performance that significantly surpasses the ones realized by each DL architecture, applied separately.