While advances in autonomous robotics and earthwork automation are paving the way for the construction site of tomorrow by means of fleets of autonomous machinery, they also force the issue of the supervision of such machines when they get stuck or stop for safety. When traditional supervision approaches impose an operator to stay on site and physically reach into the paused machine to re-gain control, our method consists in controlling the machines remotely in VR by leveraging a digital twin of the machine in its environment, fulled by LiDAR mapping. Our approach also allows the supervisor to orchestrate the autonomous fleet by means of a high-level "bird-view" interface of the whole site. As an attempt to help designing the tools of future earthwork supervisors, our preliminary results demonstrate the feasibility of a VR digital-twin approach for single-machine control, coupled with a 2D fleet-level control interface.