Seeded region growing (SRG) algorithm is very attractive for semantic image segmentation by involving high-level knowledge of image components in the seed selection procedure. However, the SRG algorithm also suffers from the problems of pixel sorting orders for labeling and automatic seed selection. An obvious way to improve the SRG algorithm is to provide more effective pixel labeling technique and automate the process of seed selection. To provide such a framework, we design an automatic SRG algorithm, along with a boundary-oriented parallel pixel labeling technique and an automatic seed selection method. Moreover, a seed tracking algorithm is proposed for automatic moving object extraction. The region seeds, which are located inside the temporal change mask, are selected for generating the regions of moving objects. Experimental evaluation shows good performances of our technique on a relatively large variety of images without the need of adjusting parameters.