We study in this paper the problem of using multiple-instance semi-supervised learning to solve image Relevance feedback problem. Many multiple-instance learning algorithms have been proposed to tackle this problem; most of them only have a global representation of images. In this paper, we present a semi-supervised version of multiple instance learning. By taking into account both the multiple-instance and the semi-supervised properties simultaneously. A novel graph-based diffusion algorithm is developed, in which global and local information are used. Experimental results show promising results of the proposed method for a test database containing more than 2000 color seaweed images.