Duplicate Detection with Efficient Language Models for Automatic Bibliographic Heterogeneous Data Integration - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2015

Duplicate Detection with Efficient Language Models for Automatic Bibliographic Heterogeneous Data Integration

Résumé

We present a new method to detect duplicates used to merge different bibliographic record corpora with the help of lexical and social information. As we show, a trivial key is not available to delete useless documents. Merging heteregeneous document databases to get a maximum of information can be of interest. In our case we try to build a document corpus about the TOR molecule so as to extract relationships with other gene components from PubMed and WebOfScience document databases. Our approach makes key fingerprints based on n-grams. We made two documents gold standards using this corpus to make an evaluation. Comparison with other well-known methods in deduplication gives best scores of recall (95\%) and precision (100\%).

Dates et versions

hal-03373972 , version 1 (11-10-2021)

Identifiants

Citer

Nicolas Turenne. Duplicate Detection with Efficient Language Models for Automatic Bibliographic Heterogeneous Data Integration. 2015. ⟨hal-03373972⟩

Collections

INRAE
15 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More