Tree-like tableaux are objects in bijection with alternative or permutation tableaux. They have been the subject of a fruitful combinatorial study for the past few years. In the present work, we define and study a new subclass of tree-like tableaux enumerated by Baxter numbers. We exhibit simple bijective links between these objects and three other combinatorial classes: (packed or mosaic) floorplans, twisted Baxter permutations and triples of non-intersecting lattice paths. From several (and unrelated) works, these last objects are already known to be enumerated by Baxter numbers, and our main contribution is to provide a unifying approach to bijections between Baxter objects, where Baxter tree-like tableaux play the key role. We moreover get new enumerative results about alternating twisted Baxter permutations. Finally, we define a new subfamily of floorplans, which we call alternating floorplans, and we enumerate these combinatorial objects.