Business failure prediction (BFP) is an important instrument in assessing the risk of corporate failure. While a large body of research has focused on BFP, recent research in operations research and analytics acknowledges the beneficial effect of incorporating textual data for predictive modelling. However, extant BFP research that incorporates textual company information is very scarce. Based on a dataset containing 13,571 European companies provided by the largest European data aggregator, this study investigates the added value of extending traditional BFP models with textual website content. We further benchmark various feature extraction techniques in natural language processing (i.e. the vector-space approach, neural networks-based approaches and transformers) and assess the best way of representing and integrating textual website features for BFP modelling. The results confirm that including textual website data improves BFP predictive performance, and that textual features extracted by transformers add the most value to the BFP models in this benchmark setting.