KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#typescript #ai-agents #developer-tools #cli #codex #claude-code #react #mcp #open-source #deepseek-harness #dsh-plugin #ai

DistributedDeepLearningReads

★ 96 Open GitHub ↗

Papers and blogs related to distributed deep learning

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Distributed Deep Learning Reads

Compilation of literature related to distributed deep learning.  Pull requests welcome :)

* [100-epoch ImageNet Training with AlexNet in 24 Minutes](https://arxiv.org/abs/1709.05011)
* [Accumulated Gradient Normalization](https://arxiv.org/abs/1710.02368)
* [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://arxiv.org/pdf/1706.02677.pdf)
* [Asynchronous Parallel Stochastic Gradient for Nonconvex Optimization](http://papers.nips.cc/paper/5751-asynchronous-parallel-stochastic-gradient-for-nonconvex-optimization.pdf)
* [Asynchrony begets Momentum, with an Application to Deep Learning](https://arxiv.org/abs/1605.09774)
* [Bandwidth Optimal All-reduce Algorithms for Clusters of Workstations](http://www.cs.fsu.edu/~xyuan/paper/09jpdc.pdf)
* [Bringing HPC Techniques to Deep Learning](http://research.baidu.com/bringing-hpc-techniques-deep-learning/)
* [Deep learning with Elastic Averaging SGD](https://arxiv.org/abs/1412.6651)
* [Distributed Delayed Stochastic Optimization](https://arxiv.org/abs/1104.5525)
* [Don't Decay the Learning Rate, Increase the Batch Size](https://arxiv.org/abs/1711.00489)
* [FireCaffe: near-linear acceleration of deep neural network training on compute clusters](https://arxiv.org/abs/1511.00175)
* [Heterogeneity-aware Distributed Parameter Servers](https://ds3lab.org/wp-content/uploads/2017/07/sigmod2017_jiang.pdf)
* [Hogwild!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent](https://people.eecs.berkeley.edu/~brecht/papers/hogwildTR.pdf)
* [How to scale distributed deep learning?](https://arxiv.org/abs/1611.04581)
* [ImageNet Training in Minutes](https://arxiv.org/abs/1709.05011)
* [Joeri Hermans ADAG Blog](http://joerihermans.com/ramblings/distributed-deep-learning-part-1-an-introduction/)
* [Large Scale Distributed Deep Networks](https://static.googleusercontent.com/media/research.google.com/en//archive/large_deep_networks_nips2012.pdf)
* [Meet Horovod: Uber’s Open Source Distributed Deep Learning Framework for TensorFlow](https://eng.uber.com/horovod/)
* [More Effective Distributed ML via a Stale
Synchronous Parallel Parameter Server](http://repository.cmu.edu/cgi/viewcontent.cgi?article=1163&context=machine_learning)
* [Omnivore: An Optimizer for Multi-device Deep Learning on CPUs and GPUs](https://arxiv.org/abs/1606.04487)
* [On Parallelizability of Stochastic Gradient Descent for Speech DNNs](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/ParallelSGD-ICASSP2014-published.pdf)
* [On Scalable Deep Learning and Parallelizing Gradient Descent](https://github.com/JoeriHermans/master-thesis/tree/master/thesis)
* [One weird trick for parallelizing convolutional neural networks](https://arxiv.org/abs/1404.5997)
* [Parallel training of DNNs with Natural Gradient and Parameter Averaging](https://arxiv.org/abs/1410.7455)
* [Poseidon: A System Architecture for Efficient GPU-based Deep Learning on Multiple Machines](https://arxiv.org/abs/1512.06216)
* [PowerAI DDL](https://arxiv.org/abs/1708.02188)
* [Revisiting Distributed Synchronous SGD](https://arxiv.org/pdf/1604.00981.pdf)
* [Scalable Distributed DNN Training Using Commodity GPU Cloud Computing](https://s3-us-west-2.amazonaws.com/amazon.jobs-public-documents/strom_interspeech2015.pdf)
* [SparkNet: Training Deep Networks in Spark](https://arxiv.org/abs/1511.06051)
* [Staleness-aware Async-SGD for Distributed Deep Learning](https://arxiv.org/abs/1511.05950)
* [TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning](https://arxiv.org/abs/1705.07878)

DistributedDeepLearningReads

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README