The fundamental frequency plays a primary role in the dynamic assessment of Cultural Heritage towers. Local and global features may impact its value: geometric, material features, interaction with the soil and adjacent buildings, aging, the construction phase, and repairs. A database is assembled to study the relationship between the fundamental frequency and the slender masonry structure features. Empirical and physics-based approaches were developed to assess the fundamental frequency from different sources of information. A Rayleigh–Ritz approach is proposed and compared with a 3D finite element model. A sensitivity analysis is then performed to quantify the contribution of each feature. As expected, it is shown that the height of the tower contributes the most to the fundamental frequency. The other tower features have a second-order impact on both the fundamental frequency and the mode shape. A comparison between the different approaches shows that the Rayleigh–Ritz drastically minimizes the difference between numerical and experimental frequencies when all information is available. Empirical relations are a good compromise when less information is available.