Human-in-the-loop learning is a key aspect of ensuring a positive future for the interactions between AI systems and humans. Despite that, the tooling for this line of research is often incomplete or inaccessible, creating a significant obstacle in this field. In this work, we introduce Cogment Lab, a researcher's toolkit for reinforcement learning experiments with the involvement of humans. It is a layer of abstraction on top of the already existing Cogment, which proves to be powerful, but difficult to use. In contrast, Cogment Lab preserves most of Cogment's flexibility, but making it significantly easier to use for practical research and development. We describe the design philosophy of Cogment Lab, some elements of its implementation, as well as research directions that it enables. We hope that this library will accelerate human-in-the-loop research by drastically reducing the barrier to entry of this field.