One of the most important technological challenges in the railway context nowadays is reducing the noise produced by the interaction between the train wheels and the track surface, that is the rolling noise. Introducing viscoelastic damping in the railway structure is one method to mitigate this rolling noise. In the paper, we investigate the impact of damping in the vibrational response of railway tracks using different viscoelastic models and parameters available in the literature. For this purpose, the main focus here is on developing an advanced numerical model in which viscoelastic models representing the rail foundation and/or pad are implemented in the framework of the Semi-Analytical Finite Element (SAFE) method. Linear or nonlinear viscoelastic models are considered, compared and effectively implemented for accelerance computation. The obtained numerical strategy leads to affordable and accurate predictions on dynamic features of the railway structure compared to classical methods.