Sagittarius A*, the supermassive black hole at the center of our galaxy, exhibits episodic near-infrared flares. The recent monitoring of three such events by the GRAVITY instrument has shown that some flares are associated with orbital motions in the close environment of the black hole with super Keplerian velocity. We develop a semi-analytic model of Sagittarius~A* flares based on an ejected large plasmoid, inspired by recent particle-in-cell global simulations of black hole magnetospheres. We model the infrared astrometric and photometric signatures associated to this model. We consider a spherical large plasmoid ejected along a conical orbit around the black hole. This plasmoid is assumed to be formed by successive mergers of smaller plasmoids produced through magnetic reconnection. Non-thermal electrons are injected in the plasmoid. We compute the evolution of the electron-distribution under the influence of synchrotron cooling. We solve the radiative transfer problem and transport the radiation along null geodesics of the Schwarzschild spacetime. We also take into account the quiescent radiation of the accretion flow, on top of which the flare evolves. For the first time, we successfully account for the astrometric and flux variations of the GRAVITY data with a flare model that incorporates an explicit modeling of the emission mechanism. We find good agreement between the prediction of our model and the recent data. In particular, the azimuthal velocity is set by the magnetic field line it belongs to, which is anchored in the inner parts of the accretion flow, hence the super-Keplerian motion. The astrometric track is also shifted with respect to the center of mass due to the quiescent radiation, in agreement with the difference measured with the GRAVITY data. These results support the picture of magnetic reconnection as a viable model for Sagittarius~A* infrared flares.