Fabrication, characterization and comparison of gold and graphene micro- and nano-size Hall sensors for room temperature scanning magnetic field microscopy applications is presented. The Hall sensors with active areas from 5 $\mu$m down to 50 nm were fabricated by electron-beam lithography. The calibration of the Hall sensors in an external magnetic field revealed a sensitivity of 3.2 mV/(AT) $\pm$ 0.3 % for gold and 1615 V/(AT) $\pm$ 0.5 % for graphene at room temperature. The gold sensors were fabricated on silicon nitride cantilever chips suitable for integration into commercial scanning probe microscopes, allowing scanning Hall microscopy (SHM) under ambient conditions and controlled sensor-sample distance. The height dependent stray field distribution of a magnetic scale was characterized using a 5 $\mu$m gold Hall sensor. The uncertainty of the entire Hall sensor based scanning and data acquisition process was analyzed allowing traceably calibrated SHM measurements. The measurement results show good agreement with numerical simulations within the uncertainty budget.