We address the problem of object tracking within image sequences through region-based energy minimization. A common underlying assumption in region tracking is that color statistics can be confidently estimated in a global manner over object and background regions. This can be a drawback for tracking in real scenes with cluttered backgrounds, where statistical color data is highly scattered, preventing the estimation of reliable color statistics for object/background discrimination. To overcome this limitation, we propose an approach based on a it narrow perception of background, which concentrates on the vicinity of tracked objects and thus extract more consistent statistical data for region separation. The benefits of our approach are demonstrated using two different statistical color models.