Delayed labelling evaluation for data streams - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Data Mining and Knowledge Discovery Année : 2020

Delayed labelling evaluation for data streams

Résumé

A large portion of the stream mining studies on classification rely on the availability of true labels immediately after making predictions. This approach is well exemplified by the test-then-train evaluation, where predictions immediately precede true label arrival. However, in many real scenarios, labels arrive with non-negligible latency. This raises the question of how to evaluate classifiers trained in such circumstances. This question is of particular importance when stream mining models are expected to refine their predictions between acquiring instance data and receiving its true label. In this work, we propose a novel evaluation methodology for data streams when verification latency takes place, namely continuous re-evaluation. It is applied to reference data streams and it is used to differentiate between stream mining techniques in terms of their ability to refine predictions based on newly arriving instances. Our study points out, discusses and shows empirically the importance of considering the delay of instance labels when evaluating classifiers for data streams.

Dates et versions

hal-04468430 , version 1 (20-02-2024)

Identifiants

Citer

Maciej Grzenda, Heitor Murilo Gomes, Albert Bifet. Delayed labelling evaluation for data streams. Data Mining and Knowledge Discovery, 2020, 34 (5), pp.1237--1266. ⟨10.1007/S10618-019-00654-Y⟩. ⟨hal-04468430⟩
7 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Mastodon Facebook X LinkedIn More