Skip to content

About

Light Direction Estimation from Depth and Shadow

Topics

Resources

Stars

2 stars

Watchers

0 watching

Forks

Latest commit

 

History

43 Commits

Folders and files

Repository files navigation

Light Direction Estimation from Depth (MoGe) and Shadow (SSIS)

This project leverages MoGe for depth estimation and SSIS for object-shadow association to accurately estimate the direction of the light source present at the time of image capture.

Such light direction estimation can be valuable for applications including realistic scene relighting, photo editing, and inferring the time of day based on lighting conditions in the image.

Example Results

The following example shows the object-shadow pairs detected using SSIS, the depth estimation generated by MoGe, and the final 3D scene showing the estimated light direction.

Intermediate Results

SSIS – Object-Shadow Pairs MoGe – Depth Estimation
Object-shadow pairs using SSIS Depth estimation using MoGe

Final Result: 3D Scene and Light Direction

The final result combines the depth estimation and object-shadow associations to estimate the direction of the light source and generate a 3D scene representation.

⬇ Download 3D Scene (.ply)

Visualization: Open the downloaded .ply file with CloudCompare or MeshLab.

Five sample results are provided in the result folder.

Installation and Setup

To run this code, clone all of the following Git repositories and install the required data and dependencies for each project, following the instructions provided on their official websites.

MoGe:

git clone https://github.com/microsoft/MoGe.git

SSIS:

git clone https://github.com/stevewongv/SSIS.git

LightDir:

git clone https://github.com/kidat/LightDirectionEstimation.git

Project structure

├── MoGe/
├── SSIS/
│   └── demo/
└── LightDirectionEstimation/
    ├── config/
    ├── models/
    .....
    └── main.py

Before running this project, make sure MoGe and SSIS are correctly installed, and all data and dependencies are setup according to the instructions provided on their official websites. Test them separately.

Demo

To evaluate the results, try the command example:

cd LightDirectionEstimation

pip install -r requirement.txt

python main.py --input ./samples

This project receives a folder of images and generates depth maps, object-shadow associations, light direction estimations in 2D, and 3D environment mappings in .ply format for each image.

All the codes in a single file is also provided in the onefile folder, and it can be run from the SSIS/demo directory.

Environment

The code was tested on CUDA 12.1 (V12.1.66) with Python 3.8.20 environment.

License

MIT License - See LICENSE file for details.

About

Light Direction Estimation from Depth and Shadow

Topics

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages