KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#developer-tools #typescript #codex #ai-agents #claude-code #dsh #dsh-plugin #cli #macos #ai-agent #deepseek-harness #open-source

VisionTransformer-MNIST

★ 37 Open GitHub ↗

This notebook is designed to plot the attention maps of a vision transformer trained on MNIST digits.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# VisionTransformer (ViT) Attention Maps using MNIST

## Code walkthrough ([YouTube Video](https://youtu.be/y1ZmMcMYjkY?feature=shared))
[<img src="https://github.com/mashaan14/YouTube-channel/blob/main/imgs/2023-11-29-VisionTransformer-MNIST.avif" />](https://youtu.be/y1ZmMcMYjkY?feature=shared)

## An attention map for a test image ([code](https://mashaan14.github.io/YouTube-channel/vision_transformers/2023_11_29_VisionTransformer_MNIST))

## An attention map for query and key images ([code](https://mashaan14.github.io/YouTube-channel/vision_transformers/2023_08_23_VisionTransformer_MNIST_query_key))

---

# Videos you might be interested in

## Vision Transformer (ViT) in JAX
[<img src="https://github.com/mashaan14/YouTube-channel/blob/main/imgs/2024_03_28_jax_ViT.avif" />](https://youtu.be/LDwA31hARrA)

## Swin Transformer Code
[<img src="https://github.com/mashaan14/YouTube-channel/blob/main/imgs/2024_08_19_Swin_Transformer.avif" />](https://youtu.be/LspWysWparE)

## Swin Transformer Attention
[<img src="https://github.com/mashaan14/YouTube-channel/blob/main/imgs/2024_09_16_Swin_Transformer_Attention.avif" />](https://youtu.be/mtCTIGgzfbc)

## References
```bibtex
@misc{dosovitskiy2021image,
  title         = {An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale},
  author        = {Alexey Dosovitskiy and Lucas Beyer and Alexander Kolesnikov and Dirk Weissenborn and Xiaohua Zhai and Thomas Unterthiner and Mostafa Dehghani and Matthias Minderer and Georg Heigold and Sylvain Gelly and Jakob Uszkoreit and Neil Houlsby},
  year          ={2021},
  eprint        = {2010.11929},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV}
}
```

VisionTransformer-MNIST

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README