Deep metric learning for end-to-end document classification
Résumé
Document classification systems become more and more complex with the need to deal with new document formats or categories while obtaining a low error rate when classifying more and more documents. Such systems need to have important features including 1) the ability to eliminate ambiguity to improve precision or reduce the error rate, 2) the capability to detect and reject documents belonging to new categories or new variations. Previous studies often focused on closed datasets or solely on the problem of novelty detection, without evaluating the ability to reject ambiguous results after the novelty detection. In this paper, we propose an end-to-end document classification algorithm including both novelty and ambiguity rejection. The proposed algorithm utilizes deep metric learning to compact the knowledge space, and then uses the last hidden layer's features as input for an unsupervised KNN-based method for novelty and ambiguity rejection. Extensive experiments and analysis on private and public benchmark datasets demonstrate the effectiveness of our proposed algorithm. The algorithm provides the capability to handle new documents while effectively rejecting ambiguity to enhance the precision, recall of known categories, and coverage rate of the end-to-end document classification system.
| Origine | Publication financée par une institution |
|---|---|
| Licence |