Action4D

KO | EN

gitlite — search

#javascript #hacktoberfest #typescript #llm #golang #go #android #machine-learning #rust #windows #awesome #cli #ios #open-source

(★ 13)

Framewise online action recognition using 4D data

Action4D Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Action4D

by [Quanzeng You](http://cs.rochester.edu/u/qyou/), [Hao Jiang](http://hao-jiang.net)

This is the code and data repository for our work **Action4D: Online Action Recognition in the Crowd and Clutter** published in **CVPR** 2019.

### Introduction
We propose to tackle action recognition using a holistic 4D “scan” of a cluttered scene to include every detail
about the people and environment. In this work, we tackle a new
problem, i.e., recognizing multiple people’s actions in the
cluttered 4D representation.
Our method is invariant to
camera view angles, resistant to clutter and able to handle crowd.

### Methods

#### 1. Holistic "scan" of a clustered scene from multiple depth cameras.
We capture the whole scene using multiple depth cameras. Then, we reconstruct a holistic scene by fusing multiple calibrated depth images. The following figure shows an example.

#### 2. Multiple people tracking using top-down view

Our tracker works on the top-down view of the scene. The following figure shows some of our tracking results. We show different colors for different tracked person.

![Tracking results](https://raw.githubusercontent.com/qyouurcs/Action4D/master/figures/tracking.png)

#### 3. Overall view of our online action recognition pipeline

![Overview of our approach](https://raw.githubusercontent.com/qyouurcs/Action4D/master/figures/overview.png)

### Datasets

The cropped volumes of each subject has been shared in **data** folder.

For test1 and test3, we also shared the training and validating file lists.

All data files are saved using compressed npz format. The dataloader is going to load directly from the tar file into system memory. Please check the data_loader.py for details.

### Experiments

#### 1. Example snapshots of different actions

#### 2. Example of tracking multiple people and recognizing their actions
![Multiple Person Example](https://raw.githubusercontent.com/qyouurcs/Action4D/master/figures/gifs/demo_label2_514_5.gif)
### Requirements
```
PyTorch >= 0.4.1
```

### Training and Testing
1. We use [Horovod](https://github.com/horovod/horovod) to enable distributed training. However, the code can also be running on a single machine.

```
python train_dist.py \
    --data_dir_val test3_cross_env/data_shell_f9_val.tar \
    --data_dir test3_cross_env/train.tar \
    --list_fn test3_cross_env/train.lst \
    --list_fn_val test3_cross_env/val.lst \
    --batch_size 24
```

### Citation

@InProceedings{action4d,
      author = {You, Quanzeng and Jiang, Hao},
      title = {Action4D: Online Action Recognition in the Crowd and Clutter},
      booktitle = {CVPR},
      year = {2019}
    }

Action4D

Action4D Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Action4D

Action4D Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README