DIRS

KO | EN

gitlite — search

#python #hacktoberfest #typescript #llm #golang #android #rust #deep-learning #linux #java #windows #awesome #awesome-list #security

(★ 9)

Official implementation for "Single Image Reflection Separation via Deep Feature Interaction"

DIRS 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# DIRS: Principled Reflection Separation via Nonlinear Superposition and Feature Interaction

<a href="https://openreview.net/pdf?id=Shwtw8uV8l">📄 Paper</a> |
 <a href="https://mingcv.github.io/DIRS-Page">🌐 Project Page</a> |
 <a href="https://mingcv.github.io/DIRS-Page">🖼️ Demo</a> |
 <a href="https://huggingface.co/huqiming513/DIRS-Models">🤗 Models & Data</a>

<a href="https://scholar.google.com/citations?user=4zasPbwAAAAJ">Qiming Hu</a>,
 <a href="https://mingjia.li/">Mingjia Li</a>,
 <a href="https://scholar.google.com/citations?&user=Pu7-JBEAAAAJ">Yuntong Li</a>,
 <a href="https://sites.google.com/view/xjguo/homepage">Xiaojie Guo</a>
 
 College of Intelligence and Computing, Tianjin University

DIRS is a unified reflection separation framework that models real sRGB layer
formation with learnable nonlinear superposition and separates transmission and
reflection through explicit dual-stream feature interaction. The release
contains three public variants, from efficient CNNs to a high-quality
Transformer, all sharing the same xreflection interface and returning
`T`, `R`, `Phi`, and `Psi`.

## Method Overview

DIRS is built around two tightly coupled ideas:

- **Learnable Offset-Residual Superposition (LORS)** models sRGB reflection
  formation with nonlinear residuals `Phi` and zero-order offsets `Psi`, instead
  of forcing all layer coupling into a linear mixture.
- **Dual-Stream Interactive Reflection Separation (DIRS)** keeps transmission
  and reflection in dedicated streams while exchanging discriminative features
  through a Selection-Interaction-Fusion design.

The released variants instantiate this design at different operating points:

- **DIRS-YTMT** recycles features suppressed by one branch into the other branch
  through paired activations, giving a compact and efficient CNN baseline.
- **DIRS-MuGI** replaces hard activation exchange with mutual gating, improving
  robustness under spatially varying nonlinear coupling.
- **DIRS-PAIR** uses Dual-Stream Joint Attention and parallel self-attention, so
  cross-stream information can be selected rather than blindly aggregated.

## Visual Results

<img src="assets/fig_realworld.jpg" width="96%">
 
 Transmission predictions in challenging real-world scenarios.

<img src="assets/fig_refscene.jpg" width="96%">
 
 Reflection separation and reflection scene reconstruction.

<img src="assets/fig_polar.jpg" width="96%">
 
 DIRS adapted to polarized multi-image reflection separation.

## Model Zoo

The model table follows the paper protocol. FLOPs and latency are measured on a
single NVIDIA RTX 3090 at 384 x 384 input resolution. PSNR/SSIM are averaged
over Real20 and SIR2.

| Model | Type | Params | FLOPs | Time | PSNR | SSIM | Checkpoint |
| --- | --- | ---: | ---: | ---: | ---: | ---: | --- |
| DIRS-YTMT | CNN, activation interaction | 32.42M | 102.91G | 31.35 ms | 24.94 | 0.902 | [ckpt](https://huggingface.co/huqiming513/DIRS-Models/blob/main/checkpoints/dirs_ytmt_lors.ckpt) |
| DIRS-MuGI | CNN, mutual gating | 84.47M | 153.98G | 49.95 ms | 25.63 | 0.913 | [ckpt](https://huggingface.co/huqiming513/DIRS-Models/blob/main/checkpoints/dirs_mugi_lors.ckpt) |
| DIRS-PAIR | Transformer, joint attention | 48.80M | 200.22G | 75.36 ms | 26.37 | 0.918 | [ckpt](https://huggingface.co/huqiming513/DIRS-Models/blob/main/checkpoints/dirs_pair_lors.ckpt) |
| DIRS-PAIR + Nature | Transformer, joint attention | 48.80M | 200.22G | 75.36 ms | 26.95 | 0.926 | [ckpt](https://huggingface.co/huqiming513/DIRS-Models/blob/main/checkpoints/dirs_pair_lors_nature.ckpt) |

Supplementary checkpoints:

| Task | Checkpoint | Notes |
| --- | --- | --- |
| Reflection Scene Reconstruction | [rsr_supplement.ckpt](https://huggingface.co/huqiming513/DIRS-Models/blob/main/checkpoints/rsr_supplement.ckpt) | Uses local SDXL base model files. |
| Polarized Reflection Separation | [polarized_dirs_pair_lors.ckpt](https://huggingface.co/huqiming513/DIRS-Models/blob/main/checkpoints/polarized_dirs_pair_lors.ckpt) | DIRS-PAIR adapted to PolaRGB inputs. |
| Swin prior | [swin_large_o365.pth](https://huggingface.co/huqiming513/DIRS-Models/blob/main/checkpoints/pretrained/swin_large_o365.pth) | Shared by DIRS-PAIR and polarized DIRS. |

## Installation

```bash
conda create -n dirs python=3.8
conda activate dirs
pip install -r requirements.txt
```

The supplementary SDXL/ControlNet experiments require extra packages:

```bash
pip install -r requirements-supplement.txt
```

## Checkpoints and Data

Download the released assets from
[DIRS-Models](https://huggingface.co/huqiming513/DIRS-Models) and
place the directories at the repository root:

```text
DIRS/
  checkpoints/
    dirs_ytmt_lors.ckpt
    dirs_mugi_lors.ckpt
    dirs_pair_lors.ckpt
    dirs_pair_lors_nature.ckpt
    polarized_dirs_pair_lors.ckpt
    rsr_supplement.ckpt
    pretrained/swin_large_o365.pth
  datasets/
    TJReflection/
```

## Evaluation

Run the Lightning/YAML evaluation path:

```bash
python -m xreflection.train \
  --config options/test_dirs_pair_lors.yml \
  --test_only \
  --resume checkpoints/dirs_pair_lors.ckpt
```

Use the matching config and checkpoint for other variants:

| Variant | Config | Checkpoint |
| --- | --- | --- |
| DIRS-YTMT | `options/test_dirs_ytmt_lors.yml` | `checkpoints/dirs_ytmt_lors.ckpt` |
| DIRS-MuGI | `options/test_dirs_mugi_lors.yml` | `checkpoints/dirs_mugi_lors.ckpt` |
| DIRS-PAIR | `options/test_dirs_pair_lors.yml` | `checkpoints/dirs_pair_lors.ckpt` |
| DIRS-PAIR + Nature | `options/test_dirs_pair_lors.yml` | `checkpoints/dirs_pair_lors_nature.ckpt` |

For a lightweight direct evaluation entry:

```bash
python scripts/eval_dirs_variants.py --variant all --limit 10
```

## Training

DIRS provides two training settings for each main variant. `wo_nature` trains
with synthetic VOC and Real20 pairs; `w_nature` additionally uses the Nature
split.

```bash
python -m xreflection.train --config options/train_dirs_pair_lors_wo_nature.yml
```

Available training configs:

| Variant | Setting I | Setting II |
| --- | --- | --- |
| DIRS-YTMT | `options/train_dirs_ytmt_lors_wo_nature.yml` | `options/train_dirs_ytmt_lors_w_nature.yml` |
| DIRS-MuGI | `options/train_dirs_mugi_lors_wo_nature.yml` | `options/train_dirs_mugi_lors_w_nature.yml` |
| DIRS-PAIR | `options/train_dirs_pair_lors_wo_nature.yml` | `options/train_dirs_pair_lors_w_nature.yml` |

Supplementary experiments:

```bash
python -m xreflection.train --config options/train_supp_rsr.yml
python -m xreflection.train --config options/train_supp_polarized_dirs_pair_lors.yml
```

Supplementary evaluation scripts:

```bash
python scripts/test_supp_rsr.py \
  --data_root datasets/supp_rsr/tjreflection_mid_eval/separation_768 \
  --limit 1

python scripts/test_supp_pola.py \
  --data_root datasets/PolaRGB \
  --limit 1
```

## Repository Map

| Path | Description |
| --- | --- |
| `xreflection/archs/dirs_lors_arch.py` | DIRS-LORS base class and three released variants. |
| `xreflection/archs/dirs_ytmt_arch.py` | YTMT interaction blocks. |
| `xreflection/archs/dirs_mugi_arch.py` | MuGI interaction blocks. |
| `xreflection/archs/dirs_pair_arch.py` | PAIR attention interaction blocks. |
| `xreflection/models/dirs_model.py` | Lightning model wrapper with grouped DIRS losses. |
| `xreflection/models/rsr_model.py` | Reflection scene reconstruction training wrapper. |
| `xreflection/models/dirs_polar_model.py` | Polarized multi-image DIRS wrapper. |
| `xreflection/data/dirs_dataset.py` | DIRS training and evaluation datasets. |
| `options/` | Training and evaluation YAML configs. |
| `scripts/` | Direct evaluation and supplementary testing scripts. |

## Citation

If DIRS is useful for your research, please cite:

```bibtex
@article{hu2026dirs,
  title={Principled Reflection Separation via Nonlinear Superposition and Feature Interaction},
  author={Hu, Qiming and Li, Mingjia and Li, Yuntong and Guo, Xiaojie},
  journal={arXiv preprint},
  year={2026}
}
```

## Acknowledgement

This codebase builds on the [xreflection training interface](https://github.com/hainuo-wang/XReflection) used in our previous
reflection separation projects. We also thank the maintainers of the public
reflection removal benchmarks used in the paper.

DIRS

DIRS 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

DIRS

DIRS 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README