Expression Recognition for Severely Demented Patients in Music Reminiscence-Therapy - Archive ouverte HAL Access content directly
Conference Papers Year : 2017

Expression Recognition for Severely Demented Patients in Music Reminiscence-Therapy


Recognizing expressions in severely demented Alzheimer's disease (AD) patients is essential, since such patients have lost a substantial amount of their cognitive capacity, and some even their verbal communication ability (e.g., aphasia). This leaves patients dependent on clinical staff to assess their verbal and non-verbal language, in order to communicate important messages, as of the discomfort associated to potential complications of the AD. Such assessment classically requires the patients' presence in a clinic, and time consuming examination involving medical personnel. Thus, expression monitoring is costly and logistically inconvenient for patients and clinical staff, which hinders among others large-scale monitoring. In this work we present a novel approach for automated recognition of facial activities and expressions of severely demented patients, where we distinguish between four activity and expression states, namely talking, singing, neutral and smiling. Our approach caters to the challenging setting of current medical recordings of music-therapy sessions, which include continuous pose variations, occlusions, camera-movements, camera-artifacts, as well as changing illumination. Additionally and importantly, the (elderly) patients exhibit generally less profound facial activities and expressions in a range of intensities and predominantly occurring in combinations (e.g., talking and smiling). Our proposed approach is based on the extension of the Improved Fisher Vectors (IFV) for videos, representing a video-sequence using both, local, as well as the related spatio-temporal features. We test our algorithm on a dataset of over 229 video sequences, acquired from 10 AD patients, with promising results, which have sparked substantial interest in the medical community. The proposed approach can play a key role in assessment of different therapy treatments, as well as in remote large-scale healthcare-frameworks.
Fichier principal
Vignette du fichier
EUSIPCO_ALZH_24.pdf (708.73 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-01543231 , version 1 (20-06-2017)


  • HAL Id : hal-01543231 , version 1


Antitza Dantcheva, Piotr Bilinski, Hung Thanh Nguyen, Jean-Claude Broutart, Francois Bremond. Expression Recognition for Severely Demented Patients in Music Reminiscence-Therapy. European Signal Processing Conference (EUSIPCO), Aug 2017, Kos island, Greece. pp.5. ⟨hal-01543231⟩
223 View
300 Download


Gmail Mastodon Facebook X LinkedIn More