The structure formation of the dense interstellar material and the fragmentation of clumps into cores is a fundamental step to understand how stars and stellar clusters form. We aim to establish a statistical view of clump fragmentation at sub-parsec scales based on a large sample of massive clumps selected from the ATLASGAL survey. We used the APEX/SABOCA camera at 350 $\mu$m to image clumps at a resolution of 8.$''$5. The majority of the sample consists of massive clumps that are weak or in absorption at 24 $\mu$m. We resolve rich filamentary structures and identify the population of compact sources. We use association with mid-infrared 22-24 $\mu$m and 70 $\mu$m point sources to pin down the star formation activity of the cores. We then statistically assess their physical properties, and the fragmentation characteristics of massive clumps. We find a moderate correlation between the clump fragmentation levels with the clump gas density and the predicted number of fragments with pure Jeans fragmentation scenario; we find a strong correlation between the mass of the most massive fragment and the total clump mass, suggesting that the self-gravity may play an important role in the clumps' small scale structure formation. We identify 27 massive quiescent cores with $M_{\rm core}>100$ M$_{\odot}$ within 5 kpc; these are massive enough to be self-gravitating but do not yet show any sign of star-formation. This sample comprises, therefore, promising candidates of massive pre-stellar cores, or deeply embedded high-mass protostars.