optimizer-visualization

KO | EN

gitlite — search

#javascript #python #hacktoberfest #typescript #llm #go #android #deep-learning #linux #windows #awesome #cli #kubernetes #vue

optimizer-visualization

(★ 402)

Visualize Tensorflow's optimizers.

파일 탐색기

최종 버전 다운로드 (.zip)

- AdadeltaOp.png
- AdadeltaOp_2.png
- AdagradOp.png
- AdagradOp_2.png
- AdamOp.png
- AdamOp_2.png
- FtrlOp.png
- FtrlOp_2.png
- GDOp.png
- GDOp_2.png
- MomentumOp.png
- MomentumOp_2.png
- movie.gif
- movie10.gif
- movie11.gif
- movie12.gif
- movie2.gif
- movie3.gif
- movie5.gif
- movie6.gif
- movie7.gif
- movie8.gif
- movie9.gif
- ProximalAdagradOp.png
- ProximalGDOp.png
- RMSPropOp.png
- RMSPropOp_2.png
gif_maker.py
LICENSE
optimizer_visualization.py
README.md

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# optimizer-visualization

## Visualize gradient descent optimization algorithms in Tensorflow.

All methods start at the same location, specified by two variables. Both x and y variables are improved by the following Optimizers:

[Adadelta documentation](https://www.tensorflow.org/api_docs/python/tf/train/AdadeltaOptimizer)

[Adagrad documentation](https://www.tensorflow.org/api_docs/python/tf/train/AdagradOptimizer)

[Adam documentation](https://www.tensorflow.org/api_docs/python/tf/train/AdamOptimizer)

[Ftrl documentation](https://www.tensorflow.org/api_docs/python/tf/train/FtrlOptimizer)

[GD documentation](https://www.tensorflow.org/api_docs/python/tf/train/GradientDescentOptimizer)

[Momentum documentation](https://www.tensorflow.org/api_docs/python/tf/train/MomentumOptimizer)

[RMSProp documentation](https://www.tensorflow.org/api_docs/python/tf/train/RMSPropOptimizer)

For an overview of each gradient descent optimization algorithms, visit [this helpful resource](http://ruder.io/optimizing-gradient-descent/).

#### Numbers in figure legend indicate learning rate, specific to each Optimizer.
![](https://github.com/Jaewan-Yun/optimizer-visualization/blob/master/figures/movie11.gif)
![](https://github.com/Jaewan-Yun/optimizer-visualization/blob/master/figures/movie5.gif)

#### Note the optimizers' behavior when gradient is steep.
![](https://github.com/Jaewan-Yun/optimizer-visualization/blob/master/figures/movie9.gif)
![](https://github.com/Jaewan-Yun/optimizer-visualization/blob/master/figures/movie10.gif)
![](https://github.com/Jaewan-Yun/optimizer-visualization/blob/master/figures/movie7.gif)
![](https://github.com/Jaewan-Yun/optimizer-visualization/blob/master/figures/movie6.gif)

#### Note the optimizers' behavior when initial gradient is miniscule.
![](https://github.com/Jaewan-Yun/optimizer-visualization/blob/master/figures/movie12.gif)
![](https://github.com/Jaewan-Yun/optimizer-visualization/blob/master/figures/movie8.gif)

<!-- ## Additional Figures

![](https://github.com/Jaewan-Yun/optimizer-visualization/blob/master/figures/movie3.gif)

![](https://github.com/Jaewan-Yun/optimizer-visualization/blob/master/figures/movie2.gif)

![](https://github.com/Jaewan-Yun/optimizer-visualization/blob/master/figures/movie.gif)

#### AdadeltaOptimizer(learning_rate=50):
![](https://github.com/Jaewan-Yun/optimizer-visualization/blob/master/figures/AdadeltaOp_2.png)

#### AdagradOptimizer(learning_rate=0.05):
![](https://github.com/Jaewan-Yun/optimizer-visualization/blob/master/figures/AdagradOp_2.png)

#### AdamOptimizer(learning_rate=0.05):
![](https://github.com/Jaewan-Yun/optimizer-visualization/blob/master/figures/AdamOp_2.png)

#### FtrlOptimizer(learning_rate=0.05):
![](https://github.com/Jaewan-Yun/optimizer-visualization/blob/master/figures/FtrlOp_2.png)

#### GradientDescentOptimizer(learning_rate=0.05):
![](https://github.com/Jaewan-Yun/optimizer-visualization/blob/master/figures/GDOp_2.png)

#### MomentumOptimizer(learning_rate=0.05, momentum=0.9)
![](https://github.com/Jaewan-Yun/optimizer-visualization/blob/master/figures/MomentumOp_2.png)

#### RMSPropOptimizer(learning_rate=0.05)
![](https://github.com/Jaewan-Yun/optimizer-visualization/blob/master/figures/RMSPropOp_2.png)

#### AdadeltaOptimizer(learning_rate=1000):
![](https://github.com/Jaewan-Yun/optimizer-visualization/blob/master/figures/AdadeltaOp.png)

#### AdagradOptimizer(learning_rate=0.5):
![](https://github.com/Jaewan-Yun/optimizer-visualization/blob/master/figures/AdagradOp.png)

#### AdamOptimizer(learning_rate=0.5):
![](https://github.com/Jaewan-Yun/optimizer-visualization/blob/master/figures/AdamOp.png)

#### FtrlOptimizer(learning_rate=0.5):
![](https://github.com/Jaewan-Yun/optimizer-visualization/blob/master/figures/FtrlOp.png)

#### GradientDescentOptimizer(learning_rate=0.05):
![](https://github.com/Jaewan-Yun/optimizer-visualization/blob/master/figures/GDOp.png)

#### MomentumOptimizer(learning_rate=0.05, momentum=0.9)
![](https://github.com/Jaewan-Yun/optimizer-visualization/blob/master/figures/MomentumOp.png)

#### ProximalAdagradOptimizer(learning_rate=0.5):
![](https://github.com/Jaewan-Yun/optimizer-visualization/blob/master/figures/ProximalAdagradOp.png)

#### ProximalGradientDescentOptimizer(learning_rate=0.05):
![](https://github.com/Jaewan-Yun/optimizer-visualization/blob/master/figures/ProximalGDOp.png)

#### RMSPropOptimizer(learning_rate=0.5)
![](https://github.com/Jaewan-Yun/optimizer-visualization/blob/master/figures/RMSPropOp.png) -->

#### Inspired by the following GIFs:
![](https://i.stack.imgur.com/qAx2i.gif)
![](https://i.stack.imgur.com/1obtV.gif)
[From here](http://ruder.io/optimizing-gradient-descent/)

optimizer-visualization

파일 탐색기

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

optimizer-visualization

파일 탐색기

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README