OpenApePose

KO | EN

gitlite — search

#hacktoberfest #react #typescript #llm #golang #go #android #rust #deep-learning #linux #macos #awesome #security #open-source

(★ 13)

Official repo of the OpenApePose dataset

OpenApePose 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<h1 align="left">OpenApePose: a database of annotated ape photographs for pose estimation
</h1>

## Introduction

This repository contians the <a href='https://elifesciences.org/articles/86873'>OpenApePose</a> dataset. OpenApePose contains 71,868 photographs of six ape species in naturalistic contexts. Because of their close relationship with humans, non-human apes (chimpanzees, bonobos, gorillas, orangutans, and gibbons, including siamangs) are of great scientific interest. The goal of understanding their complex behavior would be greatly advanced by the ability to perform video-based behavioral tracking. Tracking, however, requires high-quality annotated datasets of ape photographs. Here we present a new public dataset of 71,868 photographs, annotated with 16 body landmarks, of six ape species in naturalistic contexts.

## Overview

The dataset includes species from the ape (Hominoidea) superfamily. This includes the great apes (Hominidae family) and the lesser apes, gibbons and siamangs (Hylobatidae family). Among the Hominidae, we include four species: bonobos (Pan paniscus), chimpanzees (Pan troglodytes), gorillas (Gorilla gorilla), and orangutans (Pongo sp.). Among the gibbons we separate the siamangs (Symphalangus syndactylus) but include other species under the "gibbon" label. Below is the number of images from each of these species.

### Number of images per species

<table div align=center>
 
<tbody>
 <tr>
 <td>Species</td>
 <td>Bonobos</td>
 <td>Chimpanzees</td>
 <td>Gibbons</td>
 <td>Gorillas</td>
 <td>Orangutans</td>
 <td>Siamangs</td>
 </tr>
 <tr>
 <td>Images</td>
 <td>11,685</td>
 <td>18,010</td>
 <td>9,274</td>
 <td>12,905</td>
 <td>12,722</td>
 <td>7,272</td>
</tbody>
</table>

### Keypoint definition

We include 16 landmarks in the following order: Nose, Left eye, Right eye, Head, Neck, Left shoulder, Left elbow, Left wrist, Right shoulder, Right elbow, Right wrist, Hip/Sacrum, Left knee, Left foot, Right knee, Right foot 

<img src="overview_images/keypoints.jpg" width="500">

## Download

The dataset can be downloaded from
<a href='https://drive.google.com/drive/folders/1UF2Wys2JwA4W1kGIiJIvvAZXZWrnzcBk?usp=share_link'>[Google Drive]</a> 
<a href='https://datadryad.org/stash/share/yIccvg7QBQ_UeRMZEE3lyHasDNBiBg61y8oblJDOnEI'>[Dryad]</a>

The downloaded folder looks as follows:

```text
|── OpenApePose
    │-- annotations
    │   │-- oap_all.json
    │   |-- oap_test.json
    │   |-- oap_train.json
    │   |-- oap_val.json
    │-- images
    │   │-- 00001_oap.JPG
    │   │-- 00002_oap.jpg
    │   │-- ...
    │-- models
    │   │-- hrnet_w48_oap_256x192_full.pth
    │-- code
    │   │-- TopDownOAPDataset.py
    │   │-- animal_oap_dataset.py
    │   |-- __init__.py
    │   |-- hrnet_w48_oap_256x192_full.py
```

## Annotation format

Our annotation format is inspired from other similar datasets such as <a href='https://idp.springer.com/authorize/casa?redirect_uri=https://link.springer.com/article/10.1007/s11263-022-01698-2&casa_token=kBSmHxBJToAAAAAA:mKpt4xKGExOp6471wnf2AB-bGka2o1A_rdih4VHg0yffT1sllepqI5c5R0nwtT1vt4bqiLKQdic5XnM'>OpenMonkeyPose</a>, <a href='https://cocodataset.org/#keypoints-2017'>COCO</a> etc. They are provided as separate JSON files for the full dataset, as well as train, validation, and test set splits used in the original <a href='https://arxiv.org/abs/2212.00741'>OpenApePose</a> paper. The annotation JSONs have the following format:

```text
{
 "data": [
 
 { 
 "file": str, 
 "species": str, 
 "bbox": [x, y, width, height], 
 "landmarks": [x1,y1, ...,x16,y16],
 "visibility": [v1,v2,...,v16]
 },
 
 { 
 "file": str, 
 "species": str, 
 "bbox": [x, y, width, height], 
 "landmarks": [x1,y1, ...,x16,y16],
 "visibility": [v1,v2,...,v16]
 },
 
 ....
 
 ]
}
```
Within each JSON file, all the annotations are included in the "data" tag. Each annotation contains the following tags: "file", "species", "bbox", "landmarks", and "visibility". "file" specifies the name of the file in the images folder. "species" specifies the species of the individual ape in the bounding box that is annotated. "bbox" specifies a bounding box in the image that contains the annotated ape. The bounding box format used is [x,y,width,height] where [x,y] specifies the x and y corridnate of the top-left of the bounding box, width specifies the width of the bounding box in pixels, and height specifies the height of the bounding box in pixels. "landmarks" specifies the x and y coordinates of the 16 landmarks in the order specified above. "visibility" specifies whether a given landmark is occluded (0) or visible (1).

## Usage instructions

The included HRNet-W48 PyTorch model is trained using the pipelines and tools available in <a href='https://mmpose.readthedocs.io/en/latest/'>MMPose</a>.

### Installation

For using the provided model, you may use the included yml file or the requirements.txt file to match the specific versions of MMPose and other packages used. For example, on linux:

````bash
conda env create -f OpenApePose.yml
````

On other platforms, you would need to refer to the included requirements.txt and install the corresponding packages.

Or you can download and install mmpose v0.26.0 using the <a href='https://mmpose.readthedocs.io/en/latest/installation.html'>installation instructions</a> to be able to use the package.

Refer to the documentation, in particular <a href='https://mmpose.readthedocs.io/en/latest/user_guides/configs.html'>about configs</a> and <a href='https://mmpose.readthedocs.io/en/latest/user_guides/prepare_datasets.html'>about working with datasets</a> to get a sense of how to use the tools in MMPose for training/inference using PyTorch models on a custom dataset. For using the model for inference on your photo/video, you would need to register the OpenApePose dataset class (refer to the usage guide on custom datasets for more information).

### Using the included code with MMPose

We have included the necessary code for defining and registering a custom dataset class for the OpenApePose dataset.

Paste the included file (TopDownOAPDataset.py) in the following location in the MMPose directory on your computer: (.../mmpose/configs/_base_/datasets/TopDownOAPDataset.py)

Then create a NEW folder, named 'oap', and paste the animal_oap_dataset.py file in the following location: (.../mmpose/mmpose/datasets/datasets/oap/animal_oap_dataset.py)

Update the __init__.py in ../mmpose/mmpose/datasets/datasets/ by replacing it with the included __init__.py

Create a new __init__.py file in the following location: .../mmpose/mmpose/datasets/datasets/oap/. Then, add the following code in this __init__.py file to ensure that the dataset class is registered properly for training and inference:

```python
from .animal_oap_dataset import TopDownOAPDataset
__all__ = [ 'TopDownOAPDataset']
```

### Inference with the included model

Paste the config file in (.../mmpose/configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/macaque/hrnet_w48_oap_256x192_full.py)

Once the dataset class is registered and the above files pasted in respective locations, you could use the trained model (hrnet_w48_oap_256x192_full.pth) for inference on your own photos/videos. Refer to <a href='https://mmpose.readthedocs.io/en/latest/demos.html'>the documentation on running demos</a> to get a sense of how to do this.

### Training new models with OpenApePose dataset

You can train new models using the OAP dataset with MMPose once the dataset class is registered, and you have downloaded the images and stored them in the (.../mmpose/data/oap/images) folder, and annotations in the (.../mmpose/data/oap/annotations) folder. Refer to <a href='https://mmpose.readthedocs.io/en/latest/user_guides/train_and_test.html'>the documentation</a> for training models on a custom dataset.

However, you are not limited to using any specific versions of MMPose as the OpenApePose dataset and annotations can be transformed to work with many different packages.

## License

The dataset follows MIT license.

## Cite as
Desai Nisarg, Bala Praneet, Richardson Rebecca, Raper Jessica, Zimmermann Jan, Hayden Benjamin (2023) OpenApePose: a database of annotated ape photographs for pose estimation eLife 12:RP86873 
https://doi.org/10.7554/eLife.86873.2

OpenApePose

OpenApePose 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

OpenApePose

OpenApePose 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README