aws-cv-task2vec - GitLite

KO | EN

gitlite — search

#javascript #python #ai #golang #machine-learning #deep-learning #docker #macos #java #windows #kubernetes #pytorch #ios #open-source

aws-cv-task2vec

(★ 126)

Official code for the paper "Task2Vec: Task Embedding for Meta-Learning" (https://arxiv.org/abs/1902.03545, ICCV 2019)

File Explorer

Download Latest Version (.zip)

- - montecarlo.yaml
  - variational.yaml
- config.yaml
- __init__.py
- cifar.py
- cub.py
- dataset.py
- expansion.py
- imat.py
- inat.py
- mnist.py
- download_cub.sh
- download_inat2018.sh
- distance_matrix.png
- taxonomical distance.png
- - final_tasks_map.json
- - classes_tasks.json
  - final_tasks_map.json
  - tasks.json
CODE_OF_CONDUCT.md
CONTRIBUTING.md
datasets.py
LICENSE
main.py
models.py
NOTICE
plot_distance_cub_inat.py
README.md
requirements.txt
small_datasets_example.ipynb
task2vec.py
task_similarity.py
utils.py
variational.py

# Installation Guide

1. Get the code

git clone https://github.com/awslabs/aws-cv-task2vec

Downloads the entire project code from GitHub to your computer.

cd aws-cv-task2vec

Moves into the project folder you just downloaded.

2. Python

Easy Recommended

Prerequisites

Git Needed to download the project code from GitHub.
Python 3 On Windows, be sure to check 'Add Python to PATH' during installation.

pip install -r requirements.txt

Installs the Python libraries listed in requirements.txt (or similar).

python main.py

Runs the Python script (or module).

✅ If it runs without errors and prints output in the terminal, it worked.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Task2Vec

This is an implementation of the Task2Vec method described in the paper [Task2Vec: Task Embedding for Meta-Learning](https://arxiv.org/abs/1902.03545).

Task2Vec provides vectorial representations of learning tasks (datasets) which can be used to reason about the nature of
those tasks and their relations.
In particular, it provides a fixed-dimensional embedding of the task that is independent of details such as the number of
classes and does not require any understanding of the class label semantics. The distance between embeddings
matches our intuition about semantic and taxonomic relations between different visual tasks
(e.g., tasks based on classifying different types of plants are similar). The resulting vector can be used to
represent a dataset in meta-learning applicatins, and allows for example to select the best feature extractor for a task
without an expensive brute force search.

## Quick start

To compute and embedding using task2vec, you just need to provide a dataset and a probe network, for example:
```python
from task2vec import Task2Vec
from models import get_model
from datasets import get_dataset

dataset = get_dataset('cifar10')
probe_network = get_model('resnet34', pretrained=True, num_classes=10)
embedding =  Task2Vec(probe_network).embed(dataset)
```
Task2Vec uses the diagonal of the Fisher Information Matrix to compute an embedding of the task. In this implementation
we provide two methods, `montecarlo` and `variational`. The first is the fastest and is the default, but `variational`
may be more robust in some situations (in particular it is the one used in the paper). You can try it using:
```python
task2vec.embed(dataset, probe_network, method='variational')
```  
Now, let's try computing several embedding and plot the distance matrix between the tasks:
```python
from task2vec import Task2Vec
from models import get_model
import datasets
import task_similarity

dataset_names = ('mnist', 'cifar10', 'cifar100', 'letters', 'kmnist')
dataset_list = [datasets.__dict__[name]('./data')[0] for name in dataset_names]

embeddings = []
for name, dataset in zip(dataset_names, dataset_list):
    print(f"Embedding {name}")
    probe_network = get_model('resnet34', pretrained=True, num_classes=int(max(dataset.targets)+1)).cuda()
    embeddings.append( Task2Vec(probe_network, max_samples=1000, skip_layers=6).embed(dataset) )
task_similarity.plot_distance_matrix(embeddings, dataset_names)
```
You can also look at the notebook `small_datasets_example.ipynb` for a runnable implementation of this code snippet.

## Experiments on iNaturalist and CUB

### Downloading the data
First, decide where you will store all the data. For example: 
```
export DATA_ROOT=./data
```
To download [CUB-200](http://www.vision.caltech.edu/visipedia/CUB-200.html), 
from the repository root run:
```sh
./scripts/download_cub.sh $DATA_ROOT
```

To download [iNaturalist 2018](https://github.com/visipedia/inat_comp/tree/master/2018),
from the repository root run:
```sh
./scripts/download_inat2018.sh $DATA_ROOT
```
**WARNING:** iNaturalist needs ~319Gb for download and extraction.
Consider downloading and extracting it manually following the instructions
[here](https://github.com/visipedia/inat_comp/tree/master/2018).

### Computing the embedding of all tasks
To compute the embedding on a single task of CUB + iNat2018, run:
```sh
python main.py task2vec.method=montecarlo dataset.root=$DATA_ROOT dataset.name=cub_inat2018  dataset.task_id=$TASK_ID -m
```
This will use the `montecarlo` Fisher approximation to compute the embedding of the task number `$TASK_ID` in the CUB + iNAT2018 meta-task.
The result is stored in a pickle file inside `outputs`.
 
To compute all embeddings at once, we can use Hydra's multi-run mode as follow: 
```sh
python main.py task2vec.method=montecarlo dataset.root=$DATA_ROOT dataset.name=cub_inat2018  dataset.task_id=`seq -s , 0 50` -m
```
This will compute the embeddings of the first 50 tasks in the CUB + iNat2018 meta-task.
To plot the 50x50 distance matrix between these tasks, first download all the `iconic_taxa`
[image files](https://github.com/inaturalist/inaturalist/tree/master/app/assets/images/iconic_taxa)
to `./static/iconic_taxa`, and then run:
```sh
python plot_distance_cub_inat.py --data-root $DATA_ROOT ./multirun/montecarlo
``` 
The result should look like the following. Note that task regarding classification of similar life forms
(e.g, different types of birds, plants, mammals) cluster together.

![task2vec distance matrix](https://raw.githubusercontent.com/awslabs/aws-cv-task2vec/master/static/distance_matrix.png)