360-degree videos have gained widespread popularity due to their immersive experience. The massive amount of video data resulting from ultra-high resolution of 360 videos makes the coding process extremely complex and seriously limits their widespread applications. In this paper, we propose a new fast-Coding Unit (CU) partition algorithm for 360 videos based on Versatile Video Coding (VVC). First, we select a diverse range of video content to establish a large-scale database containing sufficient CU partitions. Second, we introduce a new multi-scale convolution kernel network structure based on the stretching factors of CUs across different latitudes. Finally, we design a loss function that considers both stretching factors and Rate-Distortion cost, and develop an unique multi-level threshold scheme to decide candidate split modes. Experimental results demonstrate that the proposed algorithm significantly improves encoding speed by 69.63%, while incurring only a 2.28% increase in Bjøntegaard Delta Bit Rate (BDBR), outperforming the performance of existing state-of-the-art method.