The GBOT group is in charge of the Ground Based Optical Tracking ofthe Gaia satellite. In concrete terms, since the launch of Gaia, our task is to take every night, using ground based medium-class telescopes, short sequences of 10 or 20 images of the Gaia satellite close to its meridian transit. For this purpose, we mainly use the VLT Survey Telescope and the Liverpool Telescope. In these images, taken close to the Sun's opposition - since Gaia is in L2 - we observe many asteroids: between 30 and 100 every night, up to magnitude 22. In order to extract the astrometric positions as well as the magnitudes of these asteroids, we have developed semi-automatic methods, strategies and tools tailored explicitly for this daily task. In only three and a half years of operation, this system has allowed us to send to the Minor Planet Center the position and the photometry of about 20,000 asteroids, amongst which 9,000 are new objects. Here we describe all the aspects of the GBOT asteroid survey.