The increasing resource pressure and the expanding amount of textile waste have been rising recycling as a clear priority for the fashion and apparel industry. However, textile recycling remains limited and is therefore a targeted issue in the forthcoming EU policies. As the fashion industry is embedded in complex value chains, enhancing textile recycling entails a comprehensive understanding of the existing challenges. Yet, the literature review suggests only limited empirical studies in the sector, and a dedicated state-of-the-art is still lacking. Filling this gap, a Delphi study was conducted supplemented by the Regnier’s Abacus technique. Through an iterative, anonymous, and controlled feedback process, the obstacles collected from the extant literature were collectively discussed with a representative panel of 28 experts, compared to the situation in Europe. After two rounds, the lack of eco-design practices, the absence of incentive policies, and the lack of available and accurate information on the product components emerged as the most consensual statements. Linking theory to practice, this paper aims to improve consistency in the understanding of the current state of textile recycling in Europe, while providing an encompassing outline of the current experts’ opinion on the priority challenges for the sector.