Part of the farmers have chosen to raise pigs with outdoor access. However, providing outdoor access to pigs is not a simple matter, and many farmers are hesitating or feel powerless to engage in this transition. A better understanding of their needs and challenges could facilitate the development of innovations that generate commitment. This survey aimed to identify the French pig farmers’ barriers to and drivers for providing outdoor access to pigs. A total of 36 farmers, aged 25–60, who worked in all types of pig farming systems (from full indoor to free-range) participated in a semi-structured interview that lasted 1.25–2.25 h. The topics covered included a historical overview, a description of the farm and practices, as well as opinions about the impact of outdoor access on farmers, animals, production and economic performance, environment, and society. Qualitative data were analysed using thematic analysis. Most of the participants agreed that rearing pigs indoors is a different job from that of rearing pigs with outdoor access and that it is above all a matter of choice, farmer work conception, and work comfort. Farmers generally agreed that working outdoors is particularly arduous, but this could be compensated by the satisfaction of being in contact with nature and seeing animals in a more complex environment. A large majority of farmers managing a system with outdoor access raised the issue of lack of support, highlighting the need for refinement and diffusion of guides of practices as well as day-to-day support. The impact of outdoor access on the health and welfare of pigs was discussed, especially regarding climatic hazards and the risk of zoonoses, and several outdoor farmers explained how their relationship with the animals changes when pigs are raised outside. Given that zootechnical performance may significantly decrease in farms with outdoor access, various strategies can be employed to maintain profitability, such as feed production, circularity, direct sales, or work diversification. They could be either motivating or demotivating factors depending on the individuals. Concerns about social criticism were prominent among many indoor farmers while farmers providing outdoor access generally felt more serene and proud. Overall, this study can serve as a basis to identify levers that could remove barriers, foster the adherence of more farmers, and facilitate the transition towards more pig farming systems with outdoor access, provided that those systems are viable and beneficial for the welfare and health of the animals and farmers.