Malaria is an infectious disease caused by Plasmodium parasites and is potentially human life-threatening. Children under 5 years old are the most vulnerable group with approximately one death every two minutes, accounting for more than 65% of all malaria deaths. The World Health Organization (WHO) encourages the research of appropriate methods to treat malaria through rapid and economical diagnostic. In this paper, we present a deep learning-based framework for diagnosing human malaria infection from microscopic images of thin blood smears. The framework is based on a direct segmentation and classification approach which relies on the analysis of the parasite itself. The framework permits to segment the Plasmodium parasite in the images and to predict its species among four dominant classes: P. Falciparum, P. Malaria, P. Ovale, and P. Vivax. A high potential of generalization with a competitive performance of our framework on inter-class data is demonstrated through an experimental study considering several datasets. Our source code is publicly available on https://github.com/Benhabiles-JUNIA/MalariaNet.