Handoff management is one of the main research challenges for the realization of the envisioned mobile and wireless Internet. This is mainly due to the latency delay and energy consumption introduced during handoff, which are of major concern for real-time applications and battery-constrained mobile terminals. In this paper, we explore whether handoff management can benefit from the pervasiveness of future communication networks. The key idea is to follow the ambient intelligence paradigm for the purpose of context-aware handoff. More precisely, we focus on the Radio Frequency Identification (RFID) and Wireless Sensor and Actuator Networks (WSANs) pervasive technologies, and propose two schemes for handoff prediction at the network layer or at both link and network layers, respectively. Analytical models for their time response and energy consumption are firstly derived. Then simulation-based results show that both of our schemes can achieve 38% to 93% reduction in time response while the second scheme provides 88% energy savings, validating their superiority over the standard solutions.