Seeing Through Darkness Using Thermal Geometry without Daylight Tax

---

Minseong Kweon · Wenyuan Zhao · Nuo Chen · Lulin Liu
Huiwen Han · Zihao Zhu · Srinivas Shakkottai · Chao Tian · Zhiwen Fan

Overview

DarkVGGT is an RGB-Thermal feed-forward framework for robust 3D geometry estimation in low-visibility environments. Our model leverages complementary thermal pathways and selective thermal-to-RGB routing to recover reliable geometric cues under degraded RGB conditions. This improves dark-scene reconstruction while largely preserving VGGT’s well-lit performance.

Quick Start

Installation

git clone git@github.com:phai-lab/DarkVGGT.git
cd DarkVGGT

conda create -n darkvggt python=3.10 -y
conda activate darkvggt

pip install -r requirements.txt

Download the VGGT-1B and pretrained DarkVGGT checkpoints. Create a checkpoints/ folder in the repository root and place both files under it.

Use DarkVGGT on sample sequence

We provide sample RGB-Thermal sequences for demo inference. The following command launches a viser viewer to visualize the point cloud and camera poses predicted by DarkVGGT:

python demo_viser.py --scene_dir ./samples

Our demo viewer enables direct comparison of reconstruction quality with the VGGT baseline.

Use DarkVGGT on custom sequence

Lay out your own RGB-T pair the same way as samples/ and then run the model directly with the following script:

import torch
from inference import build_model, list_images, load_thermal_images
from darkvggt.utils.load_fn import load_and_preprocess_images

device = "cuda" if torch.cuda.is_available() else "cpu"
model = build_model("checkpoints/vggt_1b.pt", "checkpoints/darkvggt.pt", device)

rgb_path, thermal_path = "path/to/rgb", "path/to/thermal"
rgb = load_and_preprocess_images(list_images(rgb_path)).to(device)
thermal = load_thermal_images(thermal_path, list_images(rgb_path)).to(device)

with torch.no_grad():
    predictions = model(rgb, thermal)

"""
Outputs:
    - predictions['depth']: predicted depth map
    - predictions['pose_enc']: predicted camera pose encoding
    - predictions['world_points']: 3D point cloud from the point head
"""

Export COLMAP format

The following script converts a paired RGB-T sequence to COLMAP format:

python demo_colmap.py --scene_dir=/YOUR/SCENE_DIR/

Please ensure that images exist under --scene_dir's rgb and thermal paths. The reconstruction result is saved under --output_dir (defaults to /YOUR/SCENE_DIR/colmap).

What to expect

• Pretrained weights
• Inference code
• Training & evaluation code

Acknowledgements

This codebase builds on the VGGT. We thank the authors for their great work!

Citations

@article{kweon2026darkvggt,
  title={DarkVGGT: Seeing Through Darkness Using Thermal Geometry without Daylight Tax},
  author={Kweon, Minseong and Zhao, Wenyuan and Chen, Nuo and Liu, Lulin and Han, Huiwen and Zhu, Zihao and Shakkottai, Srinivas and Tian, Chao and Fan, Zhiwen},
  journal={arXiv preprint arXiv:2606.11326},
  year={2026}
}