The most popular method for measuring intelligibility of L2 pronunciation is orthographic transcription (Munro & Derwing, 2015). This paper discusses the construct of intelligibility underlying this method, as well as its strengths and weaknesses compared to other methods of assessing intelligibility, such as shadowing. It highlights issues arising when using orthographic transcription with large, linguistically diverse samples of listeners at different proficiency levels, as is often the case in English as a lingua franca (ELF) research. This is not to say that orthographic transcription should not be used for ELF intelligibility studies, in particular quantitative ones. Instead, a case is made for carefully considering its potential biases in particular research contexts, so that appropriate measures to counteract them can be taken to ensure a satisfactory level of validity and reliability. This argument is illustrated with examples from a large-scale study (N = 508) involving both native and non-native listeners from over 80 different L1 backgrounds at different proficiency levels.