The 2024 Nobel Prize in Physics has been awarded to John J. Hopfield and Geoffrey E. Hinton for their pioneering contributions to harnessing principles from physics to establish foundational methods in machine learning. Their work catalysed ground-breaking computing concepts, establishing the basis for unconventional computing architectures that directly exploit the physics of their underlying hardware. Today, these novel paradigms promise to drive next-generation hardware with enhanced performance and efficiency, while advanced neural network architectures open doors to transformative scientific discoveries. In this review, we outline the broader context of their contributions to unconventional computing and emphasize the integration of physics-based concepts in modern machine learning architectures.