Vision-Transformers - GitLite

KO | EN

gitlite — search

#python #hacktoberfest #react #llm #golang #android #linux #docker #java #windows #cli #kubernetes #vue #security

Vision-Transformers

(★ 227)

This repo has all the basic things you'll need in-order to understand complete vision transformer architecture and its various implementations.

File Explorer

Download Latest Version (.zip)

- - base.py
  - data.py
  - requirements.txt
  - trainer.py
  - utils.py
  - ViT.py
  - vit_with_lora.py
- - vis.py
- readme.md
.DS_Store
.gitignore
LICENSE
README.md
requirements.txt

# Installation Guide

1. Get the code

git clone https://github.com/0xD4rky/Vision-Transformers

Downloads the entire project code from GitHub to your computer.

cd Vision-Transformers

Moves into the project folder you just downloaded.

2. Python

Easy Recommended

Prerequisites

Git Needed to download the project code from GitHub.
Python 3 On Windows, be sure to check 'Add Python to PATH' during installation.

pip install -r requirements.txt

Installs the Python libraries listed in requirements.txt (or similar).

python <실행할 파일명>.py # README에서 정확한 실행 파일명을 확인하세요

Runs the Python script (or module).

✅ If it runs without errors and prints output in the terminal, it worked.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Zero-to-Hero: ViT🚀

I have tried to cover all the bases for understanding and implementing Vision Transformers (ViT) and their evolution into Video Vision Transformers (ViViT).
The main focus is on dealing with the spatio-temporal relations using visual transformers.

![image](https://github.com/user-attachments/assets/bc8a2727-b33a-4681-aee6-c6b617e7ad81)

## 1. Vision Transformer (ViT) Fundamentals

### Surveys and Overviews:

* [Transformers in Vision: A Survey](https://arxiv.org/abs/2101.01169)
* [A Survey of Visual Transformers](https://arxiv.org/abs/2111.06091)
* [Transformers in Vision](https://arxiv.org/abs/2101.01169)

### Key Papers:

* An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale: [Paper](https://arxiv.org/abs/2010.11929) | [Code](https://github.com/google-research/vision_transformer)
* Training data-efficient image transformers & distillation through attention (DeiT): [Paper](https://arxiv.org/abs/2012.12877) | [Code](https://github.com/facebookresearch/deit)

### Concepts and Tutorials:

* "Attention Is All You Need": [Paper](https://arxiv.org/abs/1706.03762)
* "The Illustrated Transformers": [Blog Post](http://jalammar.github.io/illustrated-transformer/)
* "Vision Transformer Explained" [Blog Post](https://theaisummer.com/vision-transformer/)

## 2. Convolutional ViT and Hybrid Models:

* CvT: Introducing Convolutions to Vision Transformers: [Paper](https://arxiv.org/abs/2103.15808) | [Code](https://github.com/microsoft/CvT)
* CoAtNet: Marrying Convolution and Attention for All Data Sizes: [Paper](https://arxiv.org/abs/2106.04803)
* ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases: [Paper](https://arxiv.org/abs/2103.10697) | [Code](https://github.com/facebookresearch/convit)

## 3. Efficient Transformers and Swin Transformer:

* Swin Transformer: Hierarchical Vision Transformer using Shifted Windows: [Paper](https://arxiv.org/abs/2103.14030) | [Code](https://github.com/microsoft/Swin-Transformer)
* Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions: [Paper](https://arxiv.org/abs/2102.12122) | [Code](https://github.com/whai362/PVT)
* Efficient Transformers: A Survey: [Paper](https://arxiv.org/abs/2009.06732)

## 4. Space-Time Attention and Video Transformers:

* TimeSformer: Is Space-Time Attention All You Need for Video Understanding? [Paper](https://arxiv.org/abs/2102.05095) | [Code](https://github.com/facebookresearch/TimeSformer)
* Space-Time Mixing Attention for Video Transformer: [Paper](https://arxiv.org/abs/2106.05968)
* MViT: Multiscale Vision Transformers: [Paper](https://arxiv.org/abs/2104.11227) | [Code](https://github.com/facebookresearch/SlowFast)

## 5. Video Vision Transformer (ViViT):

*  ViViT: A Video Vision Transformer: [Paper](https://arxiv.org/abs/2103.15691) | [Code](https://github.com/google-research/scenic/tree/main/scenic/projects/vivit)
*  Video Transformer Network: [Paper](https://arxiv.org/abs/2102.00719) | [Code](https://github.com/mx-mark/VideoTransformer-pytorch)

## How to use this Repo?

* Start by reading the survey papers to get a broad understanding of the field.
* For each key paper, read the abstract and introduction, then skim through the methodology and results sections.
* Implement key concepts using the provided GitHub repositories or your own code.
* Experiment with different architectures and datasets to solidify your understanding.
* Use the additional resources to dive deeper into specific topics or applications.