Soil surface roughness is a key factor in affecting runoff generation and erosion. Nevertheless, how roughness affects runoff and erosion is not clear, because soil roughness can both converge and diverge surface flow, which can cause opposite effects on sediment production. In this research, we partitioned surface roughness into mounds and depressions and studied their effects on runoff and sediment production separately. The experiments were conducted in a dual soil box system with an up-and-down slope configuration such that erosion from the downslope study box can be quantified with and without receiving runon from the upslope soil box, simulating processes on a slope segment at different hillslope positions. Soil surface in the study box was prepared with and without mounds or depressions and the study was conducted on side by side rough vs. smooth surfaces. We also changed the near-surface hydraulic gradient from a downward drainage to upward seepage to create contrasting soil erodibility for the same soil. Our results showed that runoff was delayed from surfaces with depressions. Once at full runoff, there is no consistent trend in whether smooth or rough (with mounds or depressions) surface produced more or less sediment, even though surface with depressions produced more runoff water. Since increasing soil roughness has been prescribed as an effective soil and water conservation measure, our results indicated that the only benefit from surface roughening is in its runoff delay while rainwater was filling the depressional storage. Therefore, soil roughening may not be effective at all cases and its effectiveness depends on the antecede soil moisture condition and rainstorm pattern. Our results may also lead to changes in current erosion assessment tools, such as the Revised Universal Soil Loss Equation (RUSLE) and Water Erosion Prediction Project (WEPP), in how they handle the surface roughness factor.