In this study, we focus on addressing the challenge of measuring the quality of 3D reconstruction in large scale environments. Specifically, we consider scenarios where the map is built using a robot's 3D-Lidar point cloud observations, with potential uncertainty in the robot localization. When considering a large scale environment that is unstructured, such as a park or a forest, another difficulty arises: the data becomes extremely sparse. As a result, measuring the quality of a reconstruction becomes even more challenging. This study aims to compare the effectiveness of various metrics in measuring the quality of 3D reconstruction. Firstly, we evaluate these metrics in a controlled experimental setup, where the reconstruction is created by progressively degrading the ground-truth using different degradation models. Secondly, we compare their ability to measure reconstruction quality at a local level, across various simulated environments, ranging from structured to unstructured. Finally, we conduct a qualitative comparison to demonstrate the robustness of certain metrics to noise in the robot localization. Ultimately, we synthesize the properties of these metrics and provide practical recommendations for their selection.