This paper shows how to improve optical flow estimation by considering a neighborhood consensus strategy along with a reliable flow propagation method. Propagation takes advantages of reliability measures that are available from local low level image features. In this paper, we focus on color but our method could be easily generalized by considering also texture or gradient features. We investigate the conditions of estimating accurate optical flow and managing correctly flow discontinuities by proposing a variant of the well-known Kanade-Lucas-Tomasi (KLT) approach. Starting from this classical approach, a consensual flow is estimated locally while two additional criteria are proposed to evaluate its reliability. Propagation of reliable flow throughout the image is then performed using a specific distance criterion based on color and proximity. Experiments are conducted within the Middlebury database and show better results than classic KLT and even global methods like the well known Horn and Schunck or Black and Anandan approaches.