This study focuses on the calibration of the effective vegetation scattering albedo (ω) and surface soil roughness parameters (HR, and NRp, p = H,V) in the Soil Moisture (SM) retrieval from L-band passive microwave observations using the L-band Microwave Emission of the Biosphere (L-MEB) model. In the current Soil Moisture and Ocean Salinity (SMOS) Level 2 (L2), v620, and Level 3 (L3), v300, SM retrieval algorithms, low vegetated areas are parameterized by ω = 0 and HR = 0.1, whereas values of ω = 0.06 − 0.08 and HR = 0.3 are used for forests. Several parameterizations of the vegetation and soil roughness parameters (ω, HR and NRp, p = H,V) were tested in this study, treating SMOS SM retrievals as homogeneous over each pixel instead of retrieving SM over a representative fraction of the pixel, as implemented in the operational SMOS L2 and L3 algorithms. Globally-constant values of ω = 0.10, HR = 0.4 and NRp = −1 (p = H,V) were found to yield SM retrievals that compared best with in situ SM data measured at many sites worldwide from the International Soil Moisture Network (ISMN). The calibration was repeated for collections of in situ sites classified in different land cover categories based on the International Geosphere-Biosphere Programme (IGBP) scheme. Depending on the IGBP land cover class, values of ω and HR varied, respectively, in the range 0.08–0.12 and 0.1–0.5. A validation exercise based on in situ measurements confirmed that using either a global or an IGBP-based calibration, there was an improvement in the accuracy of the SM retrievals compared to the SMOS L3 SM product considering all statistical metrics (R = 0.61, bias = −0.019 m3 m−3, ubRMSE = 0.062 m3 m−3 for the IGBP-based calibration; against R = 0.54, bias = −0.034 m3 m−3 and ubRMSE = 0.070 m3 m−3 for the SMOS L3 SM product). This result is a key step in the calibration of the roughness and vegetation parameters in the operational SMOS retrieval algorithm. The approach presented here is the core of a new forthcoming SMOS optimized SM product.