The novelty of this paper is the alignment method of narrow field-of-view hyperspectral images to full-view RGB images. The interest is to locate hyperspectral measurements in an environment described by an equirectangular image. But the very different modalities (3 vs. hundreds of channels) and fields-of-view are challenges for accurate alignment. We solve these problems within a dense direct alignment framework that optimizes the warping parameters together with those of a global illumination difference model. Our alignment code is shared with an example dataset available at github.com/jrl-umi3218/hsrgbalign.