This paper describes our approach for studying and prototyping human-environment interaction (HEI) within a pervasive space applied to ambient assistive living context. The objective of our approach consists in developing and implementing an HEI framework to modelling the human-machine interaction. This framework allows a customization facility for designers, developers and even end-users in defining and processing multimodal interaction. We underline the fact that the design of interaction for users with special needs do not have to be considered as orthogonal to the application but rather as a validation context which is the heart of our research laboratory activities. We have integrated our framework in demonstrator dedicated to people with disabilities to validate the concept. This paper will present the framework, the developed components of our HEI model and the prototype realised. The outcome of this research activity is to provide a multimodal processing framework to favour natural human pervasive environment by decreasing the cognitive workload necessary in a complex environment. Thus, this will contribute to achieve ubiquitous accessible space.