We propose a segmentation-based dense stereo algorithm within an energy minimization framework. The cost function includes a new consistency term to take into account an initial quasi-dense disparity map and handles occlusions explicitly. Based on quasi-dense matching and color segmentation, optimization is performed efficiently by assuming a constant disparity for each region. The assumption is made robust by over-segmentation and a dynamic region splitting method done by graph cut. The efficiency and accuracy of the algorithm are demonstrated on standard stereo data. Experiment results show that the algorithm compares favorably with other state-of-the-art stereo algorithms.