Recent advances in SPR-imaging detection in gas phase have led to the development of opto-electronic noses (opto-eNs) requiring the need for optical sensitivity characterization of such devices. Understanding of the optical contributions will have an implicit relevance on sensitivity enhancement of SPR-imaging in gas phase valuable to improve the performance of opto-eN and potentially open new applications as gas sensors. In this paper, we analyzed the optical contributions to the sensitivity of the SPR imaging prisms and potential insights into their contributing factors. We established a characterization method for the SPR prism sensitivity that is independent of the carrier gas considered. Then, by using this sensitivity parameter, the influence of two different kinds of adhesive layers, Cr and Ti, of the gold coated prisms were studied. Furthermore, we considered a theoretical model to rationalize our experimental results, which demonstrated the relevance of surface topography on the optical index 1 sensitivity. Those surface topographies were characterized experimentally and were implemented in the model free from any additional fitting parameters using a modified Maxwell-Garnet theory. Finally, the model was shown to predictively assess the experimentally measured effects of temperature on the prism sensitivity.