—Cultural heritage collections are being digitized and made available through online tools. Due to the large volume of documents being digitized, not enough manpower is available to provide useful annotations. Is this paper, we discuss the use of automatic tools to both automatically index the documents (i.e., provide labels) and search through the collections. We detail the challenges specific to these collections as well as research directions that have to be followed to answer the questions raised by these new data.