KO
|
EN
gitlite — search
Search
#python
#react
#typescript
#javascript
#nodejs
#rust
#dart
#flutter
#golang
#template
#cli
#nextjs
autograd-hacks
★ 159
Open GitHub ↗
No description available.
Download README (.md)
Explore Similar Repositories
ngx-digit-only
:
An Angular directive to only allow [0-9] in the input box when typing, pasting or drag/dropping.
GBDT-PL
:
Gradient Boosting With Piece-Wise Linear Trees
create-react-redux-app-structure
:
Create React + Redux app structure with build configurations ✨
cs193p-Fall-2017
:
These are the lectures, slides, reading assignments, and problem sets for the Developing Apps for iOS 11 with Swift 4 CS193p course offered at the Stanford School of Engineering and available on iTunes U.
react-tensorflow
:
Tensorflow hooks for React.js
// repository documentation
Was this content helpful?
★ 0
(0 ratings)
Select Rating:
★
★
★
★
★
Submit Feedback
Recent Feedback
×
Download README
Do you want to download the
README.md
file for
autograd-hacks
?
Download (.md)
# autograd-hacks Extract useful quantities from PyTorch autograd ## Per-example gradients ``` autograd_hacks.add_hooks(model) output = model(data) loss_fn(output, targets).backward() autograd_hacks.compute_grad1() # param.grad: gradient averaged over the batch # param.grad1[i]: gradient with respect to example i for param in model.parameters(): assert(torch.allclose(param.grad1.mean(dim=0), param.grad)) ``` ## Hessians (assuming ReLU activations, oherwise produces Gauss-Newton matrix) ``` autograd_hacks.backprop_hess(model(data), hess_type='CrossEntropy') autograd_hacks.compute_hess(model) print(param.hess) # print Hessian of param ```
