Building inspection is a vital task because infrastructure damage puts people at risk or causes economic losses. Thanks to the technological breakthroughs in regard to Unmanned Aerial Vehicles (UAVs) and intelligent systems, there is a real possibility to implement an inspection by means of these technologies. UAVs allow reaching difficult places and, depending on the hardware carried onboard, take data or compute algorithms to understand the environment. This paper proposes a real-time robust altitude control strategy for a quadrotor aircraft, also a convolutional neuronal network for crack recognition is developed. The main idea of this proposal is to lay the background for an autonomous system for the inspection of structures using a UAV. For the robust control, a combination of two control actions, one linear (PD) and another nonlinear (Sliding Mode) is used. The combination of these control actions allows increasing the system’s performance. To verify the satisfactory performance of proposed control law, simulations and experimental results with a quadrotor, in the presence of disturbances, are presented. For crack recognition in images, several experiments were carried out validating the proposed model. For CNN training, a database of cracks was built from images taken from the Internet.