TALoS

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #golang #machine-learning #macos #windows #awesome #kubernetes #vue #awesome-list

(★ 38)

[NeurIPS 2024] TALoS: Enhancing Semantic Scene Completion via Test-time Adaptation on the Line of Sight

TALoS 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# [NeurIPS 2024] TALoS: Enhancing Semantic Scene Completion via Test-time Adaptation on the Line of Sight

This repository contains the official PyTorch implementation of the paper "TALoS: Enhancing Semantic Scene Completion via Test-time Adaptation on the Line of Sight" paper (NeurIPS 2024) by [Hyun-Kurl Jang*](https://blue-531.github.io/
) , [Jihun Kim*](https://jihun1998.github.io/
) and [Hyeokjun Kweon*](https://sangrockeg.github.io/
).

(* denotes equal contribution.)

[[Paper]](https://arxiv.org/abs/2410.15674)
<img src="./assets/SemKITTI.gif"  />
## News
<ul>
  <li> TALoS is accepted at NeurIPS 2024 🎉🎉.</li>
  <li> Official code and Paper will be released soon! </li>
  <li> Official code is released on 2024.10.17</li>
  <li> Paper is released on 2024.10.22</li>
	
</ul>

## Introduction
<img src='/assets/kitti_qual.png'>

Our main idea is simple yet effective: 
**an observation made at one moment could serve as supervision for the SSC prediction at another moment.** 
While traveling through an environment, an autonomous vehicle can continuously observe the overall scene structures, including objects that were previously occluded (or will be occluded later), which are concrete guidances for the adaptation of scene completion. Given the characteristics of the LiDAR sensor, an observation of a point at a specific spatial location at a specific moment confirms not only the occupation at that location itself but also the absence of obstacles along the line of sight from the sensor to that location.
The proposed method, named 
**Test-time Adaptation via Line of Sight (TALoS)**
, is designed to explicitly leverage these characteristics, obtaining self-supervision for geometric completion.
Additionally, we extend the TALoS framework for semantic recognition, another key goal of SSC, by collecting the reliable regions only among the semantic segmentation results predicted at each moment.
Further, to leverage valuable future information that is not accessible at the time of the current update, we devise a novel dual optimization scheme involving the model gradually updating across the temporal dimension.
## Installation

- PyTorch >= 1.10 
- pyyaml
- Cython
- tqdm
- numba
- Numpy-indexed
- [torch-scatter](https://github.com/rusty1s/pytorch_scatter)
- [spconv](https://github.com/tyjiang1997/spconv1.0) (tested with spconv==1.0 and cuda==11.3)

## Data Preparation

## Test-Time Adaptation
1. **Download SCPNet Pre-trained Models**: Download the weights from this [[link]](https://drive.google.com/file/d/12jYauPbVodnSA-faBjFucUNgxeGU0pmP/view?usp=drive_link) and place them in the `./model_load_dir` directory.
2. **(Optional) Baseline Results**: Download the comparison results from this [[link]](https://drive.google.com/file/d/1gt65t7hkdnnax2v7BALgUsunTaGHRVkh/view?usp=drive_link) and place them in `./experiments/baseline`.
3. **Generate Predictions**: Run the script to generate predictions on the dataset.

### Validation set
```
python run_tta_val.py --do_adapt --do_cont --use_los --use_pgt 
```
### Test set
```
python run_tta_test.py --do_adapt --do_cont --use_los --use_pgt --sq_num={sequence number} 
```
## Evaluation
To evaluate test sequences in SemanticKITTI, you should submit the generated predictions to [link](https://codalab.lisn.upsaclay.fr/competitions/7170).
After generate predictions, prepare your submission in the designated format, as described in the competition page.
Use the validation script from the [semantic-kitti-api](https://github.com/PRBonn/semantic-kitti-api) to ensure that the folder structure and number of label files in the zip file is correct.

## Acknowledgements
We thanks for the open source project [SCPNet](https://github.com/SCPNet/Codes-for-SCPNet).

## Citation

If you use the codes, please cite the following publication:
```
@article{jang2024talos,
  title={TALoS: Enhancing semantic scene completion via test-time adaptation on the line of sight},
  author={Jang, Hyun-Kurl and Kim, Jihun and Kweon, Hyeokjun and Yoon, Kuk-Jin},
  journal={Advances in Neural Information Processing Systems},
  volume={37},
  pages={74211--74232},
  year={2024}
}
```

TALoS

TALoS 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

TALoS

TALoS 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README