Automatic facial expression recognition (FER) has a lot of potential applications. However, even if it can be beneficial for some areas, e.g. security and healthcare, several legal and ethical challenges arise. In this article, we first present such challenges related to the deployment of FER. Then, we introduce the conduct of a focus group which allowed to highlight interesting points regarding the use of FER in a medical context. Particularly, transparency, data management, diagnoses, liability, best endeavours obligation, and non-discrimination principle are debated. We finally discuss on our study's limitations and directions for future work. CCS Concepts: • Applied computing → Computers in other domains; • Social and professional topics → Intellectual property.