pwlu-pytorch

KO | EN

gitlite — search

#react #typescript #llm #golang #go #deep-learning #macos #windows #kubernetes #vue #security #pytorch #mcp #ios

(★ 18)

Unofficial pytorch implementation of Piecewise Linear Unit dynamic activation function

pwlu-pytorch 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Piecewise Linear Unit
Proposed by Yucong Zhou,Zezhou Zhu,Zhao Zhong in a paper:[Learning specialized activation functions with the Piecewise Linear Unit](https://arxiv.org/abs/2104.03693), Piecewise Linear Unit is a variant of dynamic activation function, that has several good properties:
- Most of the flexibilities lie in a bounded region, which
can maximize the utilization of learnable parameters
- As a universal approximator, PWLU can closely approximate any contiguous, bounded scalar functions
- Thanks to the uniform division of intervals, PWLU is
efficient in computation, especially for inference

# WARNING
Pay attention to the _mode_ argument, you need to set it to __1__ to get a dynamic behaviour, otherwise PWLA will act as ReLU. In theory, you can always pass one, but authors got their best results with __heavily modified__ training pipeline, check the paper.

### Notes on this project
This is an implementation that I am going to use in my own projects, which means several things:
- Project will be upgrading
- No extensive testing
- **Contributions and feedback are welcome**
## Roadmap
- [ ] Tested on several real datasets
- [ ] Efficiency improved
- [ ] Replicate in JAX-based libraries?

pwlu-pytorch

pwlu-pytorch 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

pwlu-pytorch

pwlu-pytorch 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README