Operators of critical interactive systems are trained and qualified before being allowed to operate critical systems in “real” contexts. However, during operation, things might happen differently from during training sessions as system failures may occur and operators may make errors when interacting with the system. Both events may also be cross-related as a misunderstanding of a system failure can lead to an erroneous subsequent operation.The proposed approach focuses on assessing the impact that potential failures and/or human errors may have on human performance. This analysis targets the design and development phases of the system, when user tasks are analyzed in order to build the right system (i.e. corresponding to the users’ needs and activities they have to perform on the system). We use a task modeling notation for describing precisely operators’ activities as well as information, knowledge and objects required for performing these activities. These task models are then augmented into several variants through integration of potential system failure patterns (with associated recovery tasks) and human error patterns. The produced deviated task models are used to assess the impact of the task deviation on the operators’ performance.