During the last decade, a number of highly successful visual servo control and real-time image processing algorithms have been proposed that use the homography between images of a planar scene as the primary measurement. The performance of the algorithms depends directly on the quality of the homography estimates obtained, and these estimates must be computed in real-time. In this chapter, we exploit the special linear Lie group structure of the set of all homographies to develop an on-line dynamic observer that provides smoothed estimates of a sequence of homographies and their relative velocities. The proposed observer is easy to implement and computationally undemanding. Furthermore, it is straightforward to tune the observer gains and excellent results are obtained for test sequences of simulation and real-world data.