Various research works have dealt with the comprehensibility of textual, audio, or audiovisual documents, and showed that factors related to text (e.g. linguistic complexity), sound (e.g. speech intelligibility), image (e.g. presence of visual context), or even to cognition and emotion can play a major role in the ability of humans to understand the semantic and pragmatic contents of a given document. However, to date, no reference human data is available that could help investigating the role of the linguistic and extralinguistic information present at these different levels (i.e., linguistic, audio/phonetic, and visual) in multimodal documents (e.g., movies). The present work aimed at building a corpus of human annotations that would help to study further how much and in which way the human perception of comprehensibility (i.e., of the difficulty of comprehension, referred in this paper as overall difficulty) of audiovisual documents is affected (1) by lexical complexity, grammatical complexity, and speech intelligibility, and (2) by the modality/ies (text, audio, video) available to the human recipient. To this end, a corpus of 55 short movie clips was created. Fifteen experts (language teachers) assessed the overall difficulty, the lexical difficulty, the grammatical difficulty and the speech intelligibility of the clips under different conditions in which one or more modality/ies was/were available. A study of the distribution of the experts’ ratings showed that the perceived difficulty of the 55 clips range from very easy to very difficult, in all the aspects studied except for the grammatical complexity, for which most of the clips were considered as easy or moderately difficult. The study reflected the relationship existing between lexical complexity and difficulty, grammatical complexity and difficulty and speech intelligibility and difficulty, as lexical complexity and speech intelligibility are strongly and positively correlated to difficulty and the grammatical difficulty is moderately and positively correlated to difficulty. A multiple linear regression with difficulty as the dependent variable and lexical complexity, grammatical complexity and intelligibility as the independent variable achieved an adjusted R2of 0.82, indicating that these three variables explain most of the variance associated with the overall perceived difficulty. The results also suggest that documents were considered as most difficult when only the audio modality was available, and that adding text and/or video modalities allowed to decrease the difficulty, the difficulty scores being minimized by the combination of text, audio and video modalities.