Cardiac computed tomography angiography (CCTA) provides a non invasive imaging solution that reliably depicts the anatomical features of coronary artery diseases (CAD). Despite many successes with deep learning applications to medical imaging, recently developed methods for the automated detection of coronary plaque only suggests its feasibility as results are not yet mature enough for application deployment. The task is mainly to localize plaques and to characterize their composition depending on their appearance. Plaque composition can be divided into three classes: calcified, mixed and non-calcified. This study proposes a novel architecture for plaque detection and characterization. Its performances are tested against 2 state-of-the-art methods for coronary analysis. All trained models are evaluated and tested in the same setting using a proprietary dataset of 205 manually annotated CCTA cases. Our proposed method addresses two issues: unlike related methods it seamlessly models bifurcations and secondly it does not rely on multi-planar reformatted (MPR) visualization techniques which are by construction sensitive to centerline detection.