An accurate estimation of the measurement error in the hole drilling method is needed to choose an appropriate level of regularization and to perform a sensitivity analysis on the stress results. The latest release of ASTM E837 standard for the hole drilling method includes a procedure aimed at estimating the standard deviation of the random error component on strain measurements, proposed by Schajer. Nevertheless, strain measurements are also affected to some extent by systematic errors which are not included in the estimation and need to be compensated. For example, an error in the rosette gage factor or in the identification of the zero-depth datum systematically affects all strain measurements in a strongly correlated fashion. This paper describes a calibration bench, designed to superimpose a reference bending stress distribution on a given specimen while simultaneously performing a hole drilling measurement. Since the reference solution is known a priori and shares the measurement instrumentation, the hole geometry and the stepping process with the actual residual stress distribution, the bench provides the user with a direct validation of the obtained accuracy. In addition, strategies aimed at compensating systematic errors can be tested on the reference solution and then applied on the residual stress evaluation. The imperfect hole geometry and drilling alignment are proven to cause a significant underestimation of stresses near the surface, as they lead to an incorrect identification of the zero-depth datum. It is shown that this effect can be corrected through the proposed calibration bench.