Learning Kernel-Modulated Neural Representation for Efficient Light Field Compression - Archive ouverte HAL
Article Dans Une Revue IEEE Transactions on Image Processing Année : 2024

Learning Kernel-Modulated Neural Representation for Efficient Light Field Compression

Résumé

Light field is a type of image data that captures the 3D scene information by recording light rays emitted from a scene at various orientations. It offers a more immersive perception than classic 2D images but at the cost of huge data volume. In this paper, we draw inspiration from the visual characteristics of Sub-Aperture Images (SAIs) of light field and design a compact neural network representation for the light field compression task. The network backbone takes randomly initialized noise as input and is supervised on the SAIs of the target light field. It is composed of two types of complementary kernels: descriptive kernels (descriptors) that store scene description information learned during training, and modulatory kernels (modulators) that control the rendering of different SAIs from the queried perspectives. To further enhance compactness of the network meanwhile retain high quality of the decoded light field, we accordingly introduce modulator allocation and kernel tensor decomposition mechanisms, followed by non-uniform quantization and lossless entropy coding techniques, to finally form an efficient compression pipeline. Extensive experiments demonstrate that our method outperforms other state-of-the-art (SOTA) methods by a significant margin in the light field compression task. Moreover, after aligning descriptors, the modulators learned from one light field can be transferred to new light fields for rendering dense views, indicating a potential solution for view synthesis task.
Fichier principal
Vignette du fichier
Learning_Kernel_Modulated_Neural_Representation_for_Efficient_Light_Field_Compression_TIP24-3.pdf (44.99 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04627349 , version 1 (27-06-2024)

Licence

Identifiants

Citer

Jinglei Shi, Yihong Xu, Christine Guillemot Fellow. Learning Kernel-Modulated Neural Representation for Efficient Light Field Compression. IEEE Transactions on Image Processing, 2024, 33, pp.4060-4074. ⟨hal-04627349⟩
23 Consultations
10 Téléchargements

Altmetric

Partager

More