Using CoST on self-assessment domain expertise in complex search tasks - Archive ouverte HAL
Communication Dans Un Congrès Année : 2022

Using CoST on self-assessment domain expertise in complex search tasks

Résumé

While great progress is made in the area of information access, there are still open issues that involve designing intelligent systems supporting task-based search. Despite the importance of task-based search, the information retrieval and information science communities still feel the lack of open-ended and annotated datasets that enable the evaluation of a number of related facets of search tasks in downstream applications. Existing datasets are either sampled from large-scale logs but provide poor annotations, or sampled from lower-scale user studies but focus on ranked list evaluation. In this work, we briefly present CoST 1 : a novel richly annotated dataset for evaluating complex search tasks, collaboratively designed by researchers from the computer science and cognitive psychology domains, and intended to answer a wide range of research questions dealing with task-based search. CoST collection has been entirely detailed in a previous paper [1]. We report here its main design methodology, characteristics of the data provided and illustrative evaluations showing its importance to the IR community, among which a new evaluation (in comparison to [1]) related to the impact of user's domain expertise on his self-assessment about task complexity.

Fichier principal
Vignette du fichier
CIRCLE_2022_paper_24.pdf (1.09 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04728753 , version 1 (09-10-2024)

Identifiants

  • HAL Id : hal-04728753 , version 1

Citer

Cheyenne Dosso, Jose G Moreno, Aline Chevalier, Lynda Tamine. Using CoST on self-assessment domain expertise in complex search tasks. CIRCLE’22: Joint Conference of the Information Retrieval Communities in Europe, Jul 2022, Samatan, France. ⟨hal-04728753⟩
0 Consultations
0 Téléchargements

Partager

More