In a recent work [11], we have introduced a pressure-robust Hybrid High-Order method for the numerical solution of the incompressible Navier-Stokes equations on matching simplicial meshes. Pressure-robust methods are characterized by error estimates for the velocity that are fully independent of the pressure. A crucial question was left open in that work, namely whether the proposed construction could be extended to general polytopal meshes. In this paper we provide a positive answer to this question. Specifically, we introduce a novel divergence-preserving velocity reconstruction that hinges on the solution inside each element of a mixed problem on a subtriangulation, then use it to design discretizations of the body force and convective terms that lead to pressure robustness. An in-depth theoretical study of the properties of this velocity reconstruction, and their reverberation on the scheme, is carried out for polynomial degrees $k \geq 0$ and meshes composed of general polytopes. The theoretical convergence estimates and the pressure robustness of the method are confirmed by an extensive panel of numerical examples.