Human Activity Recognition plays a crucial role in various fields, including healthcare and smart homes. With the increasing prevalence of smart homes equipped with ambient sensors, there is a growing interest in leveraging artificial intelligence techniques to understand and recognize human activities within these environments. However, the irregular and noisy nature of data collected by ambient sensors presents unique challenges. To address these challenges, we propose the use of a pre-trained embedding trained on sensor activation sequences, specifically an algorithm based on an architecture similar to GPT, to improve the classification performance of activities of daily living in smart homes. Additionally, we investigate the concept of transfer learning between smart homes, utilizing the knowledge gained from one environment to enhance activity recognition in another. The results demonstrate that the GPT-like-based approach outperforms other algorithms in terms of accuracy and balanced accuracy across multiple datasets. The findings also highlight the potential of transfer learning, with GPT-like pre-trained embeddings from a clean and large dataset showing promising results in various scenarios.