cyclic-learning-schedulers-pytorch

KO | EN

gitlite — search

#javascript #hacktoberfest #typescript #llm #machine-learning #docker #java #windows #awesome #cli #kubernetes #ios #openai #open-source

cyclic-learning-schedulers-pytorch

(★ 35)

A PyTorch Implementation of popular cyclic learning rate schedules

cyclic-learning-schedulers-pytorch Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Cyclic learning rate schedulers -PyTorch

### Implementation
Cyclic learning rate schedules -
- cyclic cosine annealing - CycilcCosAnnealingLR()
- cyclic linear decay - CyclicLinearLR()

### Requirements
- numpy 
- python >= 2.7
- PyTorch >= 0.4.0

### Reference
<a href= https://arxiv.org/pdf/1608.03983.pdf> *SGDR: Stochastic Gradient Descent with Warm Restarts* </a>

### Usage
Sample - (follow similarly for CyclicLinearLR)
```
from cyclicLR import CyclicCosAnnealingLR
import torch

optimizer = torch.optim.SGD(lr=1e-3)
scheduler = CyclicCosAnnealingLR(optimizer,milestones=[30,80],eta_min=1e-6)
for epoch in range(100):
  scheduler.step()
  train(..)
  validate(..)
```
>Note: scheduler.step() shown is called at every epoch. It can be called even in every batch. Remember to specify milestones in number of batches (and not number of epochs) in such as case.

### Visualization
Cyclic Cosine Annealing Learning Rate Schedule
![Cosine LR](https://raw.githubusercontent.com/Harshvardhan1/cyclic-learning-schedulers-pytorch/master/cosine.png)

Cyclic Linear Learning Rate Schedule
![Linear LR](https://raw.githubusercontent.com/Harshvardhan1/cyclic-learning-schedulers-pytorch/master/linear.png)

cyclic-learning-schedulers-pytorch

cyclic-learning-schedulers-pytorch Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

cyclic-learning-schedulers-pytorch

cyclic-learning-schedulers-pytorch Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README