KO
|
EN
gitlite — search
Search
#javascript
#python
#hacktoberfest
#react
#ai
#typescript
#llm
#go
#golang
#android
#machine-learning
#rust
#deep-learning
#linux
rope-flax
★ 9
Open GitHub ↗
Rotary Position Embedding for Flax
Download README (.md)
Explore Similar Repositories
cloudflare-template-gpt
:
A dead-simple example of setting up a simple GPT bot API.
ChatGLM2_Lora
:
This repo was a simple way to implement Lora to fine-tuning ChatGLM2.这个项目是用LORA微调chatglm2的简单实现。
ComfyUI-Viewer
:
No description available.
Translator_discord_bot
:
Discord Bot for auto translate in selected channel
Siddhesh_Portfolio
:
No description available.
// repository documentation
Was this content helpful?
★ 0
(0 ratings)
Select Rating:
★
★
★
★
★
Submit Feedback
Recent Feedback
×
Download README
Do you want to download the
README.md
file for
rope-flax
?
Download (.md)
# Rotary Position Embedding for Flax. Implements the position embeddings from [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://arxiv.org/abs/2104.09864) for [Flax](https://flax.readthedocs.io/en/latest/). [rotary-embedding-torch](https://github.com/lucidrains/rotary-embedding-torch) was used as a reference implementation. Features: * 1D (for sequence models) and 2D axial (for ViT) rotary embeddings. * Learnable frequencies, including separate learnable frequencies per attention head. ## Usage Coming soon
