Motivated by the recent development of terahertz pump-probe experiments, we investigate the short-time dynamics in superconductors with multiple attractive pairing channels. Studying a single-band square lattice model with spin-spin interaction as an example, we find the signatures of collective excitations of the pairing symmetries (known as Bardasis-Schrieffer modes) as well as the order parameter amplitude (Higgs mode) in the short-time dynamics of the spectral gap and quasiparticle distribution after an excitation by a pump pulse. We show that the polarization and intensity of the pulse can be used to control the symmetry of the non-equilibrium state as well as frequencies and relative intensities of the contributions of different collective modes. We find particularly strong signatures of the Bardasis-Schrieffer mode in the dynamics of the quasiparticle distribution function. Our work shows the potential of modern ultrafast experiments to address the collective excitations in unconventional superconductors and highlights the importance of sub-dominant interactions for the non-equilibrium dynamics in these systems.