This paper presents a nonlinear controller for 3D trajectory tracking of an autonomous helicopter. The main idea consists of combining feedback linearization controller together with a novel nonlinear IMC control. This approach allows more robustness, fast and good trajectory tracking. It is applied to a small, eight-rotor, Square-Shaped Octo-Rotor and has shown satisfactory results using adequate control architecture. The controller effectiveness is shown through numerical simulations and confirmed using a software simulator and real tests.