The reconstruction of environments from omnidirectional images has two steps. The first one estimates the geometry and is described in other chapters. It includes structure-from motion (SfM), calibration or self-calibration. It estimates all parameters of the camera(s) including 6 degrees-of-freedom (DoF) poses, intrinsic parameters and radial distortions. SfM also provides a sparse cloud of 3D points reconstructed from features (e. g. points and lines) detected and matched in the images. The second step provides an approximation of the environment that is more complete than the sparse cloud: a dense cloud of points or a triangulated surface in 3D. This chapter surveys the second step in the previous works. In most cases, they assume that the camera is moving in a rigid scene, or similarly, that several cameras take images of a non-rigid scene at a same time. We start by prerequisites on reconstruction using perspective cameras (Sec. 1) and a discussion on omnidirectional cameras for reconstruction (Sec. 2). Then the previous works are classified in several groups: dense stereo adapted to omnidirectional cameras (Sec. 3), reconstruction from only one central image (Sec. 4), reconstruction of a non-rigid scene by using a stationary non-central camera (Sec. 5), and reconstruction by a moving camera (Sec. 6). Last we conclude in Sec. 7.