The goal of this article is to report on the development of a study of the acoustic factors driving the short-term annoyance experienced by neighbors of high-speed train lines. In fact, annoyance caused by high-speed trains (> 250 km/h) is not completely modeled by current indicators (e.g., LDEN). For example, the suddenness, spectral content, temporal fluctuations of pass-by noises, and the density of train passages seem to significantly impact annoyance in a way that is not fully understood. To investigate these aspects, we first review different approaches used to study annoyance caused by transportation noise. For railway noise, two main approaches are reported in the literature: in-situ social surveys exploring the contributions of acoustic and non-acoustic factors to long-term annoyance; laboratory experiments, in which controlled stimuli (individual pass-by noises) are played back to participants who rate their annoyance. Another approach is also used in transportation noise studies, but not yet for railway noise: diary, or experience sampling methods, whereby neighbors report their annoyance, at home, at different times over a longer time span (usually several weeks) while noise exposure is simultaneously recorded. Such in situ approach allows experimenters to focus on the precise characteristics of each pass-by noise and consider annoyance in the context of participants’ real environment and activities. Thus, this article then proposes an adaptation of the experience sampling method (ESM) to study annoyance caused by the passage of high-speed trains.