The-Blessings-of-Unlabeled-Background-in-Untrimmed-Videos

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

The-Blessings-of-Unlabeled-Background-in-Untrimmed-Videos

(★ 14)

No description available.

The-Blessings-of-Unlabeled-Background-in-Untrimmed-Videos Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

## Causal inference framework for Weakly supervised temporal action localization
Our paper [The Blessings of Unlabeled Background in Untrimmed Videos](https://arxiv.org/abs/2103.13183) has been accepted by CVPR 2021.

Due to the excellent performance of [WUM](https://arxiv.org/abs/2006.07006), the released code is TS-PCA+WUM on the THUMOS dataset.

## Recommended Environment
* Pytorch 1.4
* numpy, scipy among others

## Data Preparation
- Prepare [THUMOS-14](https://www.crcv.ucf.edu/THUMOS14/) dataset.
    - Three test videos (270, 1292, 1496) are excluded as WUM did.
- We recommend re-extracting the I3D features yourself using the repo:
    - [I3D Features](https://github.com/Finspire13/pytorch-i3d-feature-extraction)
    - For convenience, we use features provided by the author of WUM directly. The features are [here](https://drive.google.com/file/d/1NqaDRo782bGZKo662I0rI_cvpDT67VQU/view?usp=sharing).
- Place the features inside the `dataset` folder.
    - The data structure is the same as WUM, which is shown  below.

~~~~
├── dataset
   └── THUMOS14
       ├── gt.json
       ├── split_train.txt
       ├── split_test.txt
       └── features
           ├── train
               ├── rgb
                   ├── video_validation_0000051.npy
                   ├── video_validation_0000052.npy
                   └── ...
               └── flow
                   ├── video_validation_0000051.npy
                   ├── video_validation_0000052.npy
                   └── ...
           └── test
               ├── rgb
                   ├── video_test_0000004.npy
                   ├── video_test_0000006.npy
                   └── ...
               └── flow
                   ├── video_test_0000004.npy
                   ├── video_test_0000006.npy
                   └── ...
~~~~

- Place CAS files generated by the WUM model in  the `WTAL_result_numpy` folder. You can train your own WUM model or use the pre-trained model provided by the author of WUM. Please refer to the official [WUM repo](https://github.com/Pilhyeon/WTAL-Uncertainty-Modeling) for more detailed information. The data structure is shown below.

~~~
├── WTAL_result_numpy
   ├── video-id_cas.npy
   ├── video-id_feat_act.npy
   ├── video-id_feat_bkg.npy
   ├── video-id_features.npy
   ├── video-id_score.npy
   └── ...
~~~

## Usage

### Running
You could optionally specify your preferred parameters with `options.py`.

Train and evaluate the TS-PCA confounder:

~~~~
$ sh run.sh
~~~~

### Evaulation
Test with a pretrained model:

~~~~
$ sh run_eval.sh
~~~~

## References
We referenced the repos below for the code.

* [STPN](https://github.com/bellos1203/STPN)
* [BasNet](https://github.com/Pilhyeon/BaSNet-pytorch)
* [WUM](https://github.com/Pilhyeon/WTAL-Uncertainty-Modeling)

## Citation
Please cite the following paper if you feel our paper useful to your research.
~~~
@inproceedings{wtal_blessing,
  author    = {Yuan Liu and
               Jingyuan Chen and
               Zhenfang Chen and
               Bing Deng and
               Jianqiang Huang and 
               Hanwang Zhang},
  title     = {The Blessings of Unlabeled Background in Untrimmed Videos},
  booktitle = {CVPR},
  year      = {2021},
}
~~~

The-Blessings-of-Unlabeled-Background-in-Untrimmed-Videos

The-Blessings-of-Unlabeled-Background-in-Untrimmed-Videos Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

The-Blessings-of-Unlabeled-Background-in-Untrimmed-Videos

The-Blessings-of-Unlabeled-Background-in-Untrimmed-Videos Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README