Purpose - The objective of the study is to progress towards a comprehensive component-based Life Cycle Assessment model with clear and reusable Life Cycle Inventories (LCIs) for high-speed rail (HSR) infrastructure components, and to assess the main environmental impacts of HSR infrastructure over its lifespan, to finally determine environmental hotpots and good practices. Methods - A process-based LCA compliant with ISO 14040 and 14044 is performed. Construction-stage LCIs rely on data collection conducted with the concessionaire of the HSR line combined with EcoInvent 3.1 inventories. Use and End-of-Life stages LCIs rest on expert feedback scenarios and field data. A set of 13 midpoint indicators is proposed to capture the diversity of the environmental damage: climate change, consumptions of primary energy and non-renewable resources, human toxicity and ecotoxicities, eutrophication, acidification, radioactive and bulk wastes, stratospheric ozone depletion, and summer smog. Three characterization methods are used: the “Cumulative Energy Demand” method to quantify energy demand, the EDIP method for waste productions, and the CML method for the rest. Results and discussion - The study shows major contributions to environmental impact from rails (10–71%), roadbed (3–48%), and civil engineering structures (4–28%). More limited impact is noted from ballast (1–22%), building machines (0–17%), sleepers (4–11%), and power supply system (2–12%). The two last components, chairs and fasteners, have negligible impact (max. 1 and 3% of total contributions, respectively). Direct transportation can contribute up to 18% of total impact. The production and maintenance stages contribute roughly equally to environmental deterioration (respectively average of 62 and 59%). Because the End-of-Life (EoL) mainly includes recycling with environmental credit accounted for in our 100:100 approach, this stage has globally a positive impact (− 9 to − 98%) on all the impact categories except terrestrial ecotoxicity (58%), radioactive waste (11%), and ozone depletion (8%). Contribution analyses show that if concrete production is one of the important contributing processes over the construction stage, primary steel production is unquestionably the most important process on all the impact categories over the entire life cycle. Conclusions - These results are of interest for public authorities and the rail industry, in order to consider the full life cycle impacts of transportation infrastructure in a decision-making process with better understanding and inclusion of the environmental constraints. Suggestions are provided in this way for life cycle good practices—for instance as regards gravel recycling choices—and additional research to reduce the impact of current major contributors.