Joint Fourier Disparity Layers unrolling with learned view synthesis for light field reconstruction from few-shots focal stacks - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue IEEE Access Année : 2023

Joint Fourier Disparity Layers unrolling with learned view synthesis for light field reconstruction from few-shots focal stacks

Résumé

This paper addresses the problem of capturing a light field using a single traditional camera, by solving the inverse problem of dense light field reconstruction from a focal stack containing only very few images captured at different focus distances. An end-to-end joint optimization framework is presented, where a novel unrolled optimization method is jointly optimized with a view synthesis deep neural network. The proposed unrolled optimization method constructs Fourier Disparity Layers (FDL), a compact representation of light fields which samples Lambertian non-occluded scenes in the depth dimension and from which all the light field viewpoints can be computed. Solving the optimization problem in the FDL domain allows us to derive a closed-form expression of the data-fit term of the inverse problem. Furthermore, unrolling the FDL optimization allows to learn a prior directly in the FDL domain. In order to widen the FDL representation to more complex scenes, a Deep Convolutional Neural Network (DCNN) is trained to synthesize novel views from the optimized FDL. We show that this joint optimization framework reduces occlusion issues of the FDL model, and outperforms recent state-of-the-art methods for light field reconstruction from focal stack measurements.
Fichier principal
Vignette du fichier
Joint_Fourier_Disparity_Layers_Unrolling_With_Learned_View_Synthesis_for_Light_Field_Reconstruction_From_Few-Shots_Focal_Stacks(8).pdf (1.74 Mo) Télécharger le fichier
Origine : Publication financée par une institution

Dates et versions

hal-04292432 , version 1 (20-11-2023)

Licence

Paternité

Identifiants

Citer

Brandon Le Bon, Mikaël Le Pendu, Christine Guillemot. Joint Fourier Disparity Layers unrolling with learned view synthesis for light field reconstruction from few-shots focal stacks. IEEE Access, 2023, 11, pp.123350 - 123360. ⟨10.1109/access.2023.3329328⟩. ⟨hal-04292432⟩
35 Consultations
13 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More