New metrics for assessing linkage quality in deterministic record linkage of health databases - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2022

New metrics for assessing linkage quality in deterministic record linkage of health databases

Erwan Drézen
  • Fonction : Auteur
  • PersonId : 1129710
Sandrine Kerbrat

Résumé

Linking several datasets is becoming increasingly important for epidemiological research. However, assessing linkage quality can be challenging. This paper introduces a deterministic record linkage strategy that focuses on assessing linkage quality using new quality metrics. We developed a deterministic linkage strategy that systematically considers all combinations of individual identifiers. An exhaustive exploration of all variable combinations makes it possible to compute a new metric, referred to as robustness, and to generate a linkage cartography that precisely summarizes the linked pair characteristics. This cartography is central to our approach and makes it possible for the expert to easily accept/reject groups of linked pairs. The approach was tested on synthetic datasets staging a variety of possible linkage scenarios, and on two real-world studies (a registry database and a clinical trial). Dataset simulations demonstrated very good accuracy with a limited impact of different factors (datasets size/ratio, overlap, and errors), scalability, and encouraging runtimes. Minima were greater than 0.95 for recall and greater than 0.99 for precision, whatever the scenario. Feasibility on real datasets was verified with good results: among 3985 patients from the registry, the algorithm found 3850 single linked pairs and 135 proposals with multiple candidates out of 504,795 candidates. After reviewing the linkage cartography, the expert validated 3783 linked pairs and a manual review of multiple candidates added 20 pairs, reaching a linkage rate of 95.4%. For the trial, only 2 records out of 129 were not linked among 22,426 candidates, as a result of early withdrawal (no information in the trial database), giving a linkage rate of 98.4%. The novelty of our approach is twofold: first, the linkage cartography provides a new way of classifying and comparing deterministic rules from the set of all possible rules and second, the approach is by design resilient to data corruption and can reach better recall than standard deterministic linkage strategies. Finally, good performance and scalability open the door to the linkage of very large datasets.
Fichier principal
Vignette du fichier
NewMetricsDeterministicLinkage.pdf (742.66 Ko) Télécharger le fichier
supplementaryfiles.pdf (384.91 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03601245 , version 1 (08-03-2022)

Identifiants

  • HAL Id : hal-03601245 , version 1

Citer

Erwan Drézen, André Happe, Sandrine Kerbrat, Frédéric Balusson, Emmanuel Oger. New metrics for assessing linkage quality in deterministic record linkage of health databases. 2022. ⟨hal-03601245⟩
236 Consultations
167 Téléchargements

Partager

Gmail Facebook X LinkedIn More