This project leverages MoGe for depth estimation and SSIS for object-shadow association to accurately estimate the direction of the light source present at the time of image capture.
Such light direction estimation can be valuable for applications including realistic scene relighting, photo editing, and inferring the time of day based on lighting conditions in the image.
The following example shows the object-shadow pairs detected using SSIS, the depth estimation generated by MoGe, and the final 3D scene showing the estimated light direction.
| SSIS – Object-Shadow Pairs | MoGe – Depth Estimation |
|---|---|
|
|
The final result combines the depth estimation and object-shadow associations to estimate the direction of the light source and generate a 3D scene representation.
Visualization: Open the downloaded .ply file with CloudCompare or MeshLab.
Five sample results are provided in the result folder.
To run this code, clone all of the following Git repositories and install the required data and dependencies for each project, following the instructions provided on their official websites.
MoGe:
git clone https://github.com/microsoft/MoGe.gitSSIS:
git clone https://github.com/stevewongv/SSIS.gitLightDir:
git clone https://github.com/kidat/LightDirectionEstimation.git├── MoGe/
├── SSIS/
│ └── demo/
└── LightDirectionEstimation/
├── config/
├── models/
.....
└── main.py
Before running this project, make sure MoGe and SSIS are correctly installed, and all data and dependencies are setup according to the instructions provided on their official websites. Test them separately.
To evaluate the results, try the command example:
cd LightDirectionEstimation
pip install -r requirement.txt
python main.py --input ./samplesThis project receives a folder of images and generates depth maps, object-shadow associations, light direction estimations in 2D, and 3D environment mappings in .ply format for each image.
All the codes in a single file is also provided in the onefile folder, and it can be run from the SSIS/demo directory.
The code was tested on CUDA 12.1 (V12.1.66) with Python 3.8.20 environment.
MIT License - See LICENSE file for details.

