This paper presents a method for compensating temporal illumination variations in whisk-broom hyperspectral imaging. Whisk-broom imaging scans the scene sequentially, capturing a complete spectrum at each spatial coordinate pixel-by-pixel over time. The scanning process takes time, which is not problematic under constant illumination, but capturing cultural artefacts on-site often involves sunlight as the natural illumination source. While it may be considered beneficial due to its broad spectrum, sunlight fluctuates over time. Thus the resulting hyperspectral image suffers from temporal illumination variation, affecting the observed value and hindering scene analysis. A previous approach proposed using a quick extra single-vertical scan alongside the standard raster (horizontal) scan for compensation. However, it fails when the additional single-vertical scan is performed near or on a black frame. This work aims to overcome this issue by incorporating multiple columns or a full-vertical scan (column scan) to the horizontal scan image (row scan). Furthermore, we introduce a logarithm space and utilise the low-dimensional structures of the illumination and reflectance spectra. Experiments show that the proposed method eliminates the temporal illumination variations in the in-site captured hyperspectral images of stained-glass windows in the historic Amiens Cathedral, France.