Fourier Ptychography Microscopy enables reconstructing both intensity and phase high-resolution wide-field images from multiple captures under varying illumination directions. The capture process is classically modeled using a neural network. The reconstructed object is iteratively optimized by gradient descent so the network output matches the captures. Although, this process hinges on a precise estimation of the system geometry. While previous works alternate object image refinement and LEDs positional misalignment correction, we show that geometry estimation can be efficiently integrated into the object reconstruction process, so achieving system self-calibration, and enhancing the quality of reconstructed images.