aws-cv-task2vec - GitLite

KO | EN

gitlite — search

#javascript #python #typescript #golang #android #linux #docker #macos #nodejs #java #awesome #kubernetes #mcp #openai

aws-cv-task2vec

(★ 126)

Official code for the paper "Task2Vec: Task Embedding for Meta-Learning" (https://arxiv.org/abs/1902.03545, ICCV 2019)

파일 탐색기

최종 버전 다운로드 (.zip)

- - montecarlo.yaml
  - variational.yaml
- config.yaml
- __init__.py
- cifar.py
- cub.py
- dataset.py
- expansion.py
- imat.py
- inat.py
- mnist.py
- download_cub.sh
- download_inat2018.sh
- distance_matrix.png
- taxonomical distance.png
- - final_tasks_map.json
- - classes_tasks.json
  - final_tasks_map.json
  - tasks.json
CODE_OF_CONDUCT.md
CONTRIBUTING.md
datasets.py
LICENSE
main.py
models.py
NOTICE
plot_distance_cub_inat.py
README.md
requirements.txt
small_datasets_example.ipynb
task2vec.py
task_similarity.py
utils.py
variational.py

# 설치 가이드

1. 코드 내려받기

git clone https://github.com/awslabs/aws-cv-task2vec

깃허브에서 프로젝트 코드 전체를 내 컴퓨터로 내려받습니다.

cd aws-cv-task2vec

방금 내려받은 프로젝트 폴더 안으로 이동합니다.

2. Python

쉬움 추천

사전 준비물

Git GitHub에서 프로젝트 코드를 내려받으려면 필요합니다.
Python 3 설치 시 'Add Python to PATH' 옵션을 꼭 체크하세요 (Windows).

pip install -r requirements.txt

requirements.txt 등에 명시된 파이썬 라이브러리를 설치합니다.

python main.py

파이썬 스크립트(또는 모듈)를 실행합니다.

✅ 에러 메시지 없이 실행되고 터미널에 안내 문구가 출력되면 정상입니다.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Task2Vec

This is an implementation of the Task2Vec method described in the paper [Task2Vec: Task Embedding for Meta-Learning](https://arxiv.org/abs/1902.03545).

Task2Vec provides vectorial representations of learning tasks (datasets) which can be used to reason about the nature of
those tasks and their relations.
In particular, it provides a fixed-dimensional embedding of the task that is independent of details such as the number of
classes and does not require any understanding of the class label semantics. The distance between embeddings
matches our intuition about semantic and taxonomic relations between different visual tasks
(e.g., tasks based on classifying different types of plants are similar). The resulting vector can be used to
represent a dataset in meta-learning applicatins, and allows for example to select the best feature extractor for a task
without an expensive brute force search.

## Quick start

To compute and embedding using task2vec, you just need to provide a dataset and a probe network, for example:
```python
from task2vec import Task2Vec
from models import get_model
from datasets import get_dataset

dataset = get_dataset('cifar10')
probe_network = get_model('resnet34', pretrained=True, num_classes=10)
embedding =  Task2Vec(probe_network).embed(dataset)
```
Task2Vec uses the diagonal of the Fisher Information Matrix to compute an embedding of the task. In this implementation
we provide two methods, `montecarlo` and `variational`. The first is the fastest and is the default, but `variational`
may be more robust in some situations (in particular it is the one used in the paper). You can try it using:
```python
task2vec.embed(dataset, probe_network, method='variational')
```  
Now, let's try computing several embedding and plot the distance matrix between the tasks:
```python
from task2vec import Task2Vec
from models import get_model
import datasets
import task_similarity

dataset_names = ('mnist', 'cifar10', 'cifar100', 'letters', 'kmnist')
dataset_list = [datasets.__dict__[name]('./data')[0] for name in dataset_names]

embeddings = []
for name, dataset in zip(dataset_names, dataset_list):
    print(f"Embedding {name}")
    probe_network = get_model('resnet34', pretrained=True, num_classes=int(max(dataset.targets)+1)).cuda()
    embeddings.append( Task2Vec(probe_network, max_samples=1000, skip_layers=6).embed(dataset) )
task_similarity.plot_distance_matrix(embeddings, dataset_names)
```
You can also look at the notebook `small_datasets_example.ipynb` for a runnable implementation of this code snippet.

## Experiments on iNaturalist and CUB

### Downloading the data
First, decide where you will store all the data. For example: 
```
export DATA_ROOT=./data
```
To download [CUB-200](http://www.vision.caltech.edu/visipedia/CUB-200.html), 
from the repository root run:
```sh
./scripts/download_cub.sh $DATA_ROOT
```

To download [iNaturalist 2018](https://github.com/visipedia/inat_comp/tree/master/2018),
from the repository root run:
```sh
./scripts/download_inat2018.sh $DATA_ROOT
```
**WARNING:** iNaturalist needs ~319Gb for download and extraction.
Consider downloading and extracting it manually following the instructions
[here](https://github.com/visipedia/inat_comp/tree/master/2018).

### Computing the embedding of all tasks
To compute the embedding on a single task of CUB + iNat2018, run:
```sh
python main.py task2vec.method=montecarlo dataset.root=$DATA_ROOT dataset.name=cub_inat2018  dataset.task_id=$TASK_ID -m
```
This will use the `montecarlo` Fisher approximation to compute the embedding of the task number `$TASK_ID` in the CUB + iNAT2018 meta-task.
The result is stored in a pickle file inside `outputs`.
 
To compute all embeddings at once, we can use Hydra's multi-run mode as follow: 
```sh
python main.py task2vec.method=montecarlo dataset.root=$DATA_ROOT dataset.name=cub_inat2018  dataset.task_id=`seq -s , 0 50` -m
```
This will compute the embeddings of the first 50 tasks in the CUB + iNat2018 meta-task.
To plot the 50x50 distance matrix between these tasks, first download all the `iconic_taxa`
[image files](https://github.com/inaturalist/inaturalist/tree/master/app/assets/images/iconic_taxa)
to `./static/iconic_taxa`, and then run:
```sh
python plot_distance_cub_inat.py --data-root $DATA_ROOT ./multirun/montecarlo
``` 
The result should look like the following. Note that task regarding classification of similar life forms
(e.g, different types of birds, plants, mammals) cluster together.

![task2vec distance matrix](https://raw.githubusercontent.com/awslabs/aws-cv-task2vec/master/static/distance_matrix.png)