Communication Dans Un Congrès Année : 2025

DepthLight: a Single Image Lighting Pipeline for Seamless Integration of Virtual Objects into Real Scenes

Résumé

We present DepthLight, a method to estimate spatial lighting for photorealistic Visual Effects (VFX) using a single image as input. Previous techniques rely either on estimated or captured light representations that fail to account for localized lighting effects, or use simplified lights that do not fully capture the complexity of the illumination process. DepthLight addresses these limitations by using a single LDR image with a limited field of view (LFOV) as an input to compute an emissive texture mesh around the image (a mesh which generates spatial lighting in the scene), producing a simple and lightweight 3D representation for photorealistic object relighting. First, an LDR panorama is generated around the input image using a photorealistic diffusion-based inpainting technique, conditioned on the input image. An LDR to HDR network then reconstructs the full HDR panorama, while an off-the-shelf depth estimation technique generates a mesh representation to finally build a 3D emissive mesh. This emissive mesh approximates the bidirectional light interactions between the scene and the virtual objects that is used to relight virtual objects placed in the scene. We also exploit this mesh to cast shadows from the virtual objects on the emissive mesh, and add these shadows to the original LDR image. This flexible pipeline can be easily integrated into different VFX production workflows. In our experiments, DepthLight shows that virtual objects are seamlessly integrated into real scenes with a visually plausible estimation of the lighting. We compared our results to the ground truth lighting using Unreal Engine, as well as to state-of-the-art approaches that use pure HDRi lighting techniques (see Figure 1). Finally, we validated our approach conducting a user evaluation over 52 participants as well as a comparison to existing techniques.

Fichier principal
Vignette du fichier
Eurographics2025_DepthLight.pdf (18.84 Mo) Télécharger le fichier
thumbnail_WICED2025_DepthLight.jpg (120.24 Ko) Télécharger le fichier
Origine Fichiers éditeurs autorisés sur une archive ouverte
Licence

Dates et versions

hal-05091478 , version 1 (31-05-2025)

Licence

Identifiants

Citer

Raphael Manus, Marc Christie, Samuel Boivin, Pascal Guehl. DepthLight: a Single Image Lighting Pipeline for Seamless Integration of Virtual Objects into Real Scenes. Expressive+WICED 2025, May 2025, London, United Kingdom. ⟨10.2312/exw.20251060⟩. ⟨hal-05091478⟩
236 Consultations
140 Téléchargements

Altmetric

Partager

  • More