Floating point arithmetic is one of the most commonly used units in nowadays computing systems and is deployed for a wide range of domains and applications. While floating point operators offer high precision calculations, a plethora of applications such as multimedia processing and machine learning tolerate errors and computation imprecision. In a context of limited power budget embedded systems, saving resources and energy with an acceptable precision loss is a challenging design task. Approximate computing is an emerging systems design paradigm that offers promising balance between accuracy on the one hand and power consumption and resource utilization on the other hand. While state of the art approximate techniques offer a wide design space at the operator level, few are the works that consider exploring different techniques to build a heterogeneous comprehensive approximate design. In this paper, we propose HEAP: a heterogeneous approximate floating point multiplier. Based on a design space exploration process, we present an approximation at the transistor level that reduces energy consumption of up to 68%. Experimental study on a set of machine learning applications shows promising results with comparable accuracy to exact multiplier based systems. © 2019 Association for Computing Machinery.