Intensity diffraction tomography (IDT) is a 3D phase imaging technique that enables the reconstruction of the refractive indexes (RIs) and absorption of a sample. IDT targets biological imaging in a label-free manner using the optical variation within the sample and multiple tilted imaging to reconstruct the 3D map of RIs. However, standard IDT techniques reveal several drawbacks in terms of limited field of view and feasibility of imaging living samples in time-lapse conditions. We focused on time-lapse imaging of large sample (>100µm) without the need of large NA objective or immersion oil. The challenge created by the absence of the phase information (intensity only measurements) as well as the limited illumination angle (low NA due to low magnification) has been solved using a Beam Propagation Method (BPM) embedded inside a deep leaning framework, that we call “physical neural network”. This network layers are encoding the 3D optical representation of the sample. Besides, we included in the forward model the effect of the spherical aberration introduced by the optical interfaces, which gave a strong impact on measurements under oblique illumination in terms of 3D spatial resolution. Using this framework, we achieved 3D reconstructions of mouse embryos (>100µm) in time-lapse conditions over 7 days, observing the intrinsic embryonic development from single cell (low-scattering sample) to the blastocyst level (highly scattering sample). Such time-lapse yields quantitative information on the development and viability of embryos in view of the sub-cellular imaging capacities. Our technology opens up novel opportunities for 3D live cell imaging of whole organoids in time-lapse.