SViTE

โ˜… 87 Open GitHub โ†—

[NeurIPS'21] "Chasing Sparsity in Vision Transformers: An End-to-End Exploration" by Tianlong Chen, Yu Cheng, Zhe Gan, Lu Yuan, Lei Zhang, Zhangyang Wang

Explore Similar Repositories

LLaMA-MoE-v2 : ๐Ÿš€ LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training
tgan2 : The official implementation of "Train Sparsely, Generate Densely: Memory-efficient Unsupervised Training of High-resolution Temporal GAN"
aarambh-studio : ๐Ÿฆ€ Decoder-only LLM built from scratch in pure Rust using Candle โ€” no Python, no PyTorch. Gated DeltaNet + sparse attention, fine-grained MoE, native video/document understanding, long-horizon tool agents, quantization-aware training. Scales: Tiny (25M) to Large (1.3B).
LLM : LightningLM 0.1V โ€” Reference training pipeline for the LightningLM family. 2B dense seed โ†’ 5B MoE โ†’ 9B MoE โ†’ 120B sparse MoE through TurboQuant-PreTraining on a single eight-GPU node. Companion code for *Reversible Foundations*.
BatchTopK : Implementation of the BatchTopK activation function for training sparse autoencoders (SAEs)
// repository documentation