Context-aware middleware is considered as an efficient solution to develop context-aware application, which provides a feasible development platform integrating various sensors and new technologies. With the development of sensors, the research work on the context shift from “location” to “activity” gradually. Then it puts forward a new requirement for context-aware middleware: activity context recognition. In this paper, we first survey the related literature in premier conferences over the past decade, review the main methods of activity context recognition, and conclude its three main facets: basic activity inference, dynamic activity analysis and future activity recommendation. We then propose an intelligent inference engine integrating three popular methods solving three facets mentioned above respectively, which is based on our context-aware middleware. Finally, to evaluate the feasibility of our proposal, we implement two scenarios: bus stop scenario and domestic activity application.