In this contribution, we explore a machine learning approach for the concrete structure inspection using both surface and sub-surface imaging. For this purpose, we first propose and evaluate a deep learning based approach for the segmentation of rebar instances from ground penetrating radar images. The performance of a mask-R-CNN-based model show that the average precision is higher than 85% for reinforcement bar segmentation. We also evaluate the generalization capabilities of the model. In a second step, different criteria (reinforcement bars location and their normalized magnitudes) are computed from the extracted mask. These criteria are analysed in relation to the images of the structure surface that had been classified either in a healthy or damaged category (i.e. with cracks).