The ability to perform long-term robotic operations in dynamic environments remains a challenge in fields such as surveillance, agriculture and autonomous vehicles. For improved localization and monitoring over time, this paper proposes a novel model update framework using image-based 3D change localization and segmentation. Specifically, shallow image data is used to detect and locate significant geometric change areas in a pre-made 3D model. The main contribution of this paper is the ability to precisely segment and locate both new and missing objects from few observations, and to provide consistent model updates. The applied method for geometric change detection is robust to seasonal, viewpoint, and illumination differences that may occur between operations. Qualitative and quantitative tests with both our own and publicly available datasets show that the model update framework improves on previous methods and facilitates long-term localization.