Magnetic responsive materials are the topic of intense research due to their potential breakthrough applications in the biomedical, coatings, microfluidics and microelectronics fields. By merging magnetic and polymer materials one can obtain composites with exceptional magnetic responsive features. Magnetic actuation provides unique capabilities as it can be spatially and temporally controlled, and can additionally be operated externally to the system, providing a non-invasive approach to remote control. We identified three classes of magnetic responsive composite materials, according to their activation mode and intended applications, which can be defined by the following aspects. (A) Their ability to be deformed (stretching, bending, rotation) upon exposure to a magnetic field. (B) The possibility of remotely dragging them to a targeted area, called magnetic guidance, which is particularly interesting for biomedical applications, including cell and biomolecule guidance and separation. (C) The opportunity to use magnetic induction for thermoresponsive polymer materials actuation, which has shown promising results for controlled drug release and shape memory devices. For each category, essential design parameters that allow fine-tuning of the properties of these magnetic responsive composites are presented using key examples.