This paper presents a skyline-based approach to enhance the visualization of a new construction project in augmented reality. We propose to process the video stream acquired with a mobile phone to register the real buildings with a 3D city model. We first combine the data acquired with the device’s instruments to estimate a rough user’s pose in the world coordinates system. Then, we use this estimated pose to generate a synthetic image of the user’s view from which we calculate a virtual skyline. In parallel, we extract a real skyline from the real-time video stream. Finally, we match these real and virtual skylines to correct the user’s pose (six degrees of freedom) and thus generate a more realistic augmented reality view. We evaluate the precision and the processing time of our approach using 2D and 3D registration algorithms, as well as with a novel double 2D strategy.