In this paper, we propose a benchmarking of supervised machine learning techniques (neural networks, Gaussian processes and support vector machines) in order to forecast the Global Horizontal solar Irradiance (GHI). We also include in this benchmark a simple linear autoregressive (AR) model as well as two naive models based on persistence of the GHI and persistence of the clear sky index (denoted herein scaled persistence model). The models are calibrated and validated with data from three French islands: Corsica (41.91°N; 8.73°E), Guadeloupe (16.26°N; 61.51°W) and Reunion (21.34°S ; 55.49°E). The main findings of this work are, that for hour ahead solar forecasting, the machine learning techniques slightly improve the performances exhibited by the linear AR and the scaled persistence model. However, the improvement appears to be more pronounced in case of unstable sky conditions. These nonlinear techniques start to outperform their simple counterparts for forecasting horizons greater than one hour.