Indoor tracking people activities with sensors networks is of high importance in number of domains, such as ambient assisted living. Home sensors have seen strong development over the last few years, especially due to the emergence of Internet of Things. A wide range of sensors are today available to be installed at home : video cameras, RGB-D Kinect, binary proximity sensors, thermometers, accelerometers, etc. An important issue in deploying sensors is to make them work in a common reference frame (extrinsic calibration issue), in order to jointly exploit the data they retrieve. Determining the perception areas that are covered by each sensor is also an issue that is not so easy to solve in practice. In this paper we address both calibration and coverage isssues within in a common framework, based on Hidden Mar-kov Models (HMMs) and clustering techniques. The proposed solution requires a map of the environment, as well as the ground truth of a tracked moving object/person, which are both provided by an external system (e.g. a robot that performs telemetric mapping). The objective of the paper is twofold. On one hand, we propose an extended framework of the classical HMM in order to (a) handle contextual observations and (b) solve general classification problems. In the other, we demonstrate the relevancy of the approach by tracking a person with 4 Kinects in an apartment. A sensing floor allows the implicit calibration and mapping during an initial learning phase.