Texture characterization from the metrological point of view is addressed in order to establish a physically relevant and directly interpretable feature. In this regard, a generic formulation is proposed to simultaneously capture the spectral and spatial complexity in hyperspectral images. The feature, named relative spectral difference occurrence matrix (RSDOM) is thus constructed in a multireference, multidirectional, and multiscale context. As validation, its performance is assessed in three versatile tasks. In texture classification on HyTexiLa, content-based image retrieval (CBIR) on ICONES-HSI, and land cover classification on Salinas, RSDOM registers 98.5% accuracy, 80.3% precision (for the top 10 retrieved images), and 96.0% accuracy (after post-processing) respectively, outcompeting GLCM, Gabor filter, LBP, SVM, CCF, CNN, and GCN. Analysis shows the advantage of RSDOM in terms of feature size (a mere 126, 30, and 20 scalars using GMM in order of the three tasks) as well as metrological validity in texture representation regardless of the spectral range, resolution, and number of bands.