DSERT-RoLL-Dataset

KO | EN

gitlite — search

#python #java #python3 #arduino #golang #machine-learning #rust #html #flask #javascript #seismology #nodejs

DSERT-RoLL-Dataset

★ 20 Open GitHub ↗

[CVPR 2026] DSERT-RoLL: Robust Multi-Modal Perception for Diverse Driving Conditions with Stereo Event-RGB-Thermal Cameras, 4D Radar, and Dual-LiDAR

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# DSERT-RoLL-Dataset
[CVPR 2026] DSERT-RoLL: Robust Multi-Modal Perception for Diverse Driving Conditions with Stereo Event-RGB-Thermal Cameras, 4D Radar, and Dual-LiDAR

**Project Page:** https://jeongyh98.github.io/dsert-roll/

## Authors

**[Hoonhee Cho](https://chohoonhee.github.io/hoonheecho/)***, **[Jae-Young Kang](https://mickeykang16.github.io/)***, **[Yuhwan Jeong](https://jeongyh98.github.io/)***, **[Yunseo Yang](https://jeongyh98.github.io/dsert-roll/)**, **[Wonyoung Lee](https://jeongyh98.github.io/dsert-roll/)**, **[Youngho Kim](https://jeongyh98.github.io/dsert-roll/)**, **[Kuk-Jin Yoon](https://jeongyh98.github.io/dsert-roll/)**

[VILAB](https://vi.kaist.ac.kr/), KAIST

* Equal contribution

## Dataset

### 🤗 Download on Hugging Face

[![Hugging Face Dataset](https://img.shields.io/badge/Hugging%20Face-Dataset-FFD21E?logo=huggingface&logoColor=black)](https://huggingface.co/datasets/jeongyh98/DSERT-RoLL)

You can download the DSERT-RoLL dataset here:

👉 https://huggingface.co/datasets/jeongyh98/DSERT-RoLL

**Total Size:** 269 GB

### Environmental Conditions

- **Weather:** Clear / Fog / Light Rain / Heavy Rain / Light Snow / Heavy Snow
- **Lighting:** Normal / Low Light / Over Expose / HDR

### Train/Validation Split

| Split | Sequences |
|------:|----------:|
| Train | 132 |
| Val   | 58  |
| Total | 190 |

### Directory Structure

```text
DSERT-RoLL/
│
├── split/
│   ├── train.txt
│   └── val.txt
│
├── {weather}/                        # Clear | Fog | Heavy_Rain | Heavy_Snow | Light_Rain | Light_Snow
│   └── {sequence_name}/              # e.g. 2025_07_16_21_36_14
│       ├── label.pkl                 # annotations & calibration for the sequence
│       │
│       ├── rectified_crop_RGB_L/     # Left RGB images         (.jpg)
│       ├── rectified_crop_RGB_R/     # Right RGB images        (.jpg)
│       ├── rectified_EVENT_L/        # Left event frames       (.npz)
│       ├── rectified_EVENT_R/        # Right event frames      (.npz)
│       ├── rectified_THERMAL_L/      # Left thermal images     (.npz)
│       ├── rectified_THERMAL_R/      # Right thermal images    (.npz)
│       ├── LIDAR_LIVOX_Tilted/       # Livox LiDAR point cloud (.npy)
│       ├── LIDAR_OUSTER_Tilted_90_degree/  # Ouster LiDAR point cloud (.npy)
│       └── RADAR_Tilted/             # Radar point cloud       (.npy)
```

### Statistics

```text
Clear:       102 sequences / 11,877 frames
Fog:          32 sequences /  3,282 frames
Heavy_Rain:   18 sequences /  1,898 frames
Light_Rain:   17 sequences /  2,214 frames
Light_Snow:   14 sequences /  1,428 frames
Heavy_Snow:    7 sequences /    980 frames
─────────────────────────────────────────
Total:       190 sequences / 21,679 frames
```

### `label.pkl` Structure

```text
label.pkl
├── meta
│   ├── weather          # weather condition
│   ├── light            # lighting condition (e.g. Low_Light)
│   ├── sequence_len     # number of frames
│   └── calibration      # sensor-wise intrinsic/extrinsic matrices
│       ├── Livox, Ouster, Radar
│       └── RGB_L/R, Thermal_L/R, Event_L/R
│
└── info                 # per-frame list
  ├── time_stamp
  ├── sample_idx / frame_idx
  ├── sequence_name
  ├── pose
  ├── sensor           # relative file paths for each sensor
  │   ├── rgb_left_path / rgb_right_path
  │   ├── event_left_path / event_right_path
  │   ├── thermal_left_path / thermal_right_path
  │   ├── livox_path / ouster_path
  │   └── radar_path
  └── annos            # 3D bounding box annotations
    ├── name, obj_ids
    ├── dimensions, location, heading_angles
    └── gt_boxes_livox / gt_boxes_rgb / gt_boxes_event / gt_boxes_thermal
```

## Visualization

`viz.py` renders all sensor combinations in a 3×6 grid (rows: RGB / Event / Thermal, columns: Livox / Ouster / Radar × Left / Right) and saves each frame as a JPG.

### Installation

```bash
pip install opencv-python numpy matplotlib
```

### Usage

```bash
# All sessions
python viz.py --root_path /path/to/DSERT-RoLL

# Single session
python viz.py --root_path /path/to/DSERT-RoLL --session 2025_04_01_16_15_57

# With 3D bounding boxes
python viz.py --root_path /path/to/DSERT-RoLL --bbox

# Additional options
python viz.py --root_path /path/to/DSERT-RoLL \
              --session 2025_04_01_16_15_57 \
              --bbox \
              --num_frames 50 \
              --dpi 150 \
              --out_subdir viz_output
```

| Argument | Default | Description |
|---|---|---|
| `--root_path` | required | Root dataset path (contains weather-named subdirectories) |
| `--session` | all | Process only the specified session name |
| `--bbox` | off | Overlay 3D bounding boxes on Livox panels |
| `--num_frames` | all | Maximum number of frames to render per session |
| `--dpi` | 80 | DPI of saved JPG images |
| `--out_subdir` | `viz_frames` | Output subdirectory name inside each session folder |

Output images are saved as `{session_folder}/{out_subdir}/frame_XXXX.jpg`.

## Citation

If you use this work, please cite:

```bibtex
@inproceedings{cho2026dsertroll,
  title={DSERT-RoLL: Robust Multi-Modal Perception for Diverse Driving Conditions with Stereo Event-RGB-Thermal Cameras, 4D Radar, and Dual-LiDAR},
  author={Cho, Hoonhee and Kang, Jae-Young and Jeong, Yuhwan and Yang, Yunseo and Lee, Wonyoung and Kim, Youngho and Yoon, Kuk-Jin},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2026}
}

DSERT-RoLL-Dataset

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README