Integrating rigorous qualitative methods into the design and evaluation of safety-critical systems - Archive ouverte HAL Access content directly
Conference Papers Year : 2024

Integrating rigorous qualitative methods into the design and evaluation of safety-critical systems

Abstract

The traditional approach to studying aviation systems applies Human Factors (HF) methods to generate task models, specify relevant functionality and measure system and user performance. Typical experiments establish protocols that test whether or not users perform tasks correctly. However, this approach largely ignores problems that arise under real-world conditions, such as when overloaded pilots are confused and switch off cockpit instruments. We argue that aviation researchers can benefit from a mixed-method approach that combine rigorous qualitative HCI methods to assess design variants in realistic situations. For example, the comparative structured observation method presents users with directly comparable tasks with selected design variants and asks them to reflect deeply on the tradeoffs associated with each. These studies follow best practices from controlled-experiment design, e.g. counter-balancing for order, but emphasize the collection of rich user insights over quantitative performance measures. When properly controlled, qualitative studies offer an important complement to theoretical task models and quantitative experiments to better understand how users will interact with proposed system designs. This paper demonstrates how to incorporate rigorous qualitative methods, with examples, into the process of designing and evaluating safety-critical systems.
No file

Dates and versions

hal-04596988 , version 1 (01-06-2024)

Identifiers

  • HAL Id : hal-04596988 , version 1

Cite

Romane Dubus, Anke Brock, Wendy E. Mackay. Integrating rigorous qualitative methods into the design and evaluation of safety-critical systems. ICCAS 2024 - the international conference on cognitive aircraft systems, May 2024, Toulouse, France. ⟨hal-04596988⟩
9 View
0 Download

Share

Gmail Mastodon Facebook X LinkedIn More