In this paper, we outline a diffractive practice of machine learning (ML) in the frame of material-centered interaction design. To this aim, we review related work in ML, HCI, design, new interfaces for musical expression, and computational art, and introduce two practice-based studies of music performance and robotic art based on interactive machine learning tools, with the hope of revealing the computational materiality of ML, and the potential of embodiment to craft prototypes of ML that reconfigure conceptual or technical approaches to ML. We derive five interference conditions for such art-based ML prototypes—situational whole, small data, shallow model, learnable algorithm, and somaesthetic behaviour—and describe their widening of design and engineering practices of ML prototyping. Finally, we sketch how a process of intra-active machine learning could complement that of interactive machine learning to take materiality as an entry point for ML design within HCI.