Some thoughts from IRIT about the scenario A of the TREC RTS 2016 and 2017 tracks - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2017

Some thoughts from IRIT about the scenario A of the TREC RTS 2016 and 2017 tracks

Résumé

The TREC Real-Time Summarization (RTS) track provides a framework for evaluating systems monitoring the Twitter stream and pushing tweets to users according to given profiles. It includes metrics, files, settings and hypothesis provided by the organizers. In this work, we perform a thorough analysis of each component of the framework used for batch evaluation of scenario A in 2016 and 2017. We found some weaknesses of the metrics and took advantage of these limitations to submit our official run in the 2017 edition. The good evaluation results validate our findings. This paper also gives clear recommendations to fairly reuse the collection.
Fichier principal
Vignette du fichier
hubert_22384.pdf (359.38 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02319737 , version 1 (18-10-2019)

Identifiants

  • HAL Id : hal-02319737 , version 1
  • OATAO : 22384

Citer

Gilles Hubert, Jose G. Moreno, Karen Pinel-Sauvagnat, Yoann Pitarch. Some thoughts from IRIT about the scenario A of the TREC RTS 2016 and 2017 tracks. 26th Text REtrieval Conference (TREC 2017), Nov 2017, Gaithersburg, Maryland, United States. pp.1-12. ⟨hal-02319737⟩
18 Consultations
8 Téléchargements

Partager

Gmail Facebook X LinkedIn More