As technology improves, new kind of devices are given the ability to interconnect to others, thereby shaping today’s networks. Sensors, smartphones, tablets, laptops, desktops, servers, be they organized or not, altogether form the Internet of Things (IoT), the cloud, and the glue between them. All these devices are heterogeneous in terms of hardware (CPUs, memory type and size, network interfaces, etc), software (Unix, Windows, Android, etc) and behavior (mobility, reliability). However they all have in common the ability to perform computations. But in this context of extreme heterogeneity, distributed computing has become even harder, and the tools we use for clusters, grids and clouds cannot be used. A few middleware solutions to distributed computing this is difficult environment have been proposed recently, but these tools address only a limited number of issues, and they can hardly be coupled together to a single usable solution. As a consequence, Researchers most often opt for developing their own solution, or resorting to simulation. At I3S we also did that. Through several Research projects which implied many aspects of cluster computing and IoT, we built a codebase of successful solutions to many problems we faced along the years. Revamped and generalized into the most consistent, efficient and cleanest possible middleware, we now expose it to the Research communities as Idawi: an Open Source Java message/queue/service/object-oriented decentralized overlay network of components designed to meet the conceptual and practical needs of Researchers working on distributed algorithms.