Classification of radar observations with machine learning tools is of primary importance for the identification of non-cooperative radar targets such as drones. These observations are made of complex-valued time series which possess a strong underlying structure. These signals can be processed through a time-frequency analysis, through their self-correlation (or covariance) matrices or directly as the raw signal. All representations are linked but distinct and it is known that the input representation is critical for the success of any machine learning method. In this article, we explore these three possible input representation spaces with the help of two kinds of neural networks: a temporal fully convolu-tional network and a Riemannian network working direcly on the manifold of covariances matrices. We show that all the considered input representations are a particular case of a generic machine learning pipeline which goes from the raw complex data to the final classification stage through con-volutional layers and Riemannian layers. This pipeline can be learnt end-to-end and is shown experimentally to give the best classification accuracy together with the best robustness to lack of data.