Quadrotors are highly maneuverable light weight drones, which are prone to aerodynamic disturbances, vibrations and uncertainties. These factors stand for a problem that demands robust control laws. For position tracking, the control problem is exacerbated because the plant is underactuated in the coordinates of interest, requiring a high performance attitude tracking to resolve underactuation. In this paper, a novel fractional-order controller is proposed by considering a well-posed map that relates the position/yaw control to desired attitude references. The attitude control is continuous and enforces and sustains a sliding motion in finite-time for exponential convergence of the tracking errors to fulfill a "virtual" position controller. The resulting closed-loop system is robust against continuous disturbances that are not necessarily differentiable in the conventional sense. A numerical study based on simulations is presented to analyze the advantages of the fractional actions to design a physically realizable controller, and experiments are discussed to expose the reliability of the proposed fractional scheme implemented in an 'X' configuration quadrotor.