A theoretical modeling of the light leaving a scattering medium with a rough interface is presented. It combines the calculation of the reflection and the transmission from a random Gaussian rough surface and the radiative transfer equation solved by the auxiliary function method. Diffuse and collimated illumination is considered. Numerical simulations with various roughness parameters lead to quantitative comparisons between the surface and the bulk scattered fluxes. The validity domain of the modeling is deduced from flux conservation. An application for a real pigment shows that the surface roughness does not significantly modify its reflectance spectrum because of bulk scattering and induces a simple translation of the global spectrum because of the surface scattering. (C) 2010 Optical Society of America