The uppermost mantle in back arc regions is the site of complex interactions between partial melting, melt percolation, and fluid migration. To constrain these interactions and evaluate their consequences on geochemical cycles, we carried out an in situ trace element and water study of a suite of spinel peridotite xenoliths from two regions of the Japan back arc system, Ichinomegata (NE Japan) and Oki-Dogo (SW Japan), using LA-ICPMS and FTIR spectrometry, respectively. This study provides the first full dataset of trace element and hydrogen compositions in peridotites including analyses of all their main constitutive silicate minerals: olivine, orthopyroxene and clinopyroxene. The Ichinomegata peridotites sample a LREE-depleted refractory mantle (Mg# olivine = 0.90; Cr# spinel = 0.07–0.23; Yb clinopyroxene = 7.8–13.3 × C1-chondrite, and La/Yb clinopyroxene = 0.003–0.086 × C1-chondrite), characterized by Th-U positive anomalies and constant values of Nb/Ta. The composition of the studied Ichinomegata samples is consistent with that of an oceanic mantle lithosphere affected by cryptic metasomatic interactions with hydrous/aqueous fluids (crypto-hydrous metasomatism). In contrast, the Oki-Dogo peridotites have low Mg# olivine (0.86–0.93) and a broad range of compositions with clinopyroxene showing “spoon-shaped” to flat, and LREE-enriched patterns. They are also characterized by their homogeneous compositions in the most incompatible LILE (e.g., Rb clinopyroxene = 0.01–0.05 × primitive mantle) and HFSE (e.g., Nb clinopyroxene = 0.01–2.16 × primitive mantle). This characteristic is interpreted as resulting from various degrees of melting and extensive melt-rock interactions. FTIR spectroscopy shows that olivine in both Ichinomegata and Oki-Dogo samples has low water contents ranging from 2 to 7 ppm wt. H2O. In contrast, the water contents of pyroxenes from Ichinomegata peridotites (113–271 ppm wt. H2O for orthopyroxene, and 292–347 ppm wt. H2O for clinopyroxene) are significantly higher than in Oki-Dogo peridotites (9–35 ppm wt. H2O for orthopyroxene, and 15–98 ppm wt. H2O for clinopyroxene). This indicates a relationship between melt-rock interaction and water concentrations in pyroxenes. Our study suggests that the water content of the Japan mantle wedge is controlled by the late melt/fluid/rock interactions evidenced by trace element geochemistry: a mechanism triggered by magma-rock interactions may have acted as an efficient dehydrating process in the Oki-Dogo region while the Ichinomegata mantle water content is controlled by slab-derived crypto-hydrous metasomatism.