KO
|
EN
gitlite — search
Search
#python
#java
#python3
#arduino
#golang
#machine-learning
#rust
#html
#flask
#javascript
#seismology
#nodejs
nanogpt-speedrun
★ 20
Open GitHub ↗
NanoGPT (124M) as fast as possible
Download README (.md)
Explore Similar Repositories
WeatherForecast-4622
:
A cool open-source project
motion_retarget
:
Retargetting motion from one robot to another.
building-with-llms-made-simple
:
A tutorial on how to build stuff with large language models, made simple
MDI-configurator
:
Guidance scripts related to Defender For Identity: Prereqs, configuration etc
text_improvement
:
No description available.
// repository documentation
Was this content helpful?
★ 0
(0 ratings)
Select Rating:
★
★
★
★
★
Submit Feedback
Recent Feedback
×
Download README
Do you want to download the
README.md
file for
nanogpt-speedrun
?
Download (.md)
# nanogpt-speedrun Reproducing GPT-2 (124M) as fast as possible on an RTX 4090. karparthy: > The 124M model is the smallest model in the GPT-2 series released by OpenAI in 2019, and is actually quite accessible today, even for the GPU poor...You can train the model with a single GPU too, it would just take proportionally longer (e.g. ~4-24 hours depending on the GPU). This repo is heavily influced by https://github.com/KellerJordan/modded-nanogpt. The initial baseline here was taken directly from the initial commit of that repo, with minor modifications. See also: https://github.com/karpathy/llm.c/discussions/481 and https://github.com/tysam-code/hlb-gpt ## setup and run Install dependencies, download data, and start training with the following commands: ```bash uv sync --all-extras uv run python src/data/cached_fineweb10B.py ./run.sh ```
