Context-aware middleware is a solution to challenges associated with service discovery, mobility, environmental changes, and context retrieval. Context inference plays the role of the brain in context-aware middleware. With the emergence of a variety of location sensors, “location” is no longer the sole context of researchers or that which concerns them most, but instead has been gradually replaced by “activity”. To meet the new requirement: activity, namely context recognition, context-aware middleware needs to acquire a “smart brain”. Existing context-aware middleware usually employ the rule-based method to solve this problem, which is easy to build and also intuitive to work with. However, this method is fragile, not flexible enough, and is inadequate to support diverse types of tasks. In this paper, we surveyed the related literature in premier conferences over the past decade, reviewed the main activity context recognition methods, and summarized their three main facets: basic activity inference, dynamic activity analysis, and future activity recommendation. Based on our previous work, we then proposed an intelligent inference engine for our context-aware middleware. Besides satisfying requirements for checking context consistency, our inference engine integrates the three most popular methods for activity context recognition: Rules, the Decision Tree, and the Hidden Markov Model. This engine provides a solution for all facets of activity context recognition based on our context-aware middleware.