ng-video-lecture

KO | EN

gitlite — search

#javascript #hacktoberfest #react #ai #typescript #golang #rust #docker #windows #awesome #cli #kubernetes #vue #security

ng-video-lecture

(★ 4,922)

No description available.

파일 탐색기

최종 버전 다운로드 (.zip)

bigram.py
gpt.py
input.txt
more.txt
README.md

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# nanogpt-lecture

Code created in the [Neural Networks: Zero To Hero](https://karpathy.ai/zero-to-hero.html) video lecture series, specifically on the first lecture on nanoGPT. Publishing here as a Github repo so people can easily hack it, walk through the `git log` history of it, etc.

NOTE: sadly I did not go too much into model initialization in the video lecture, but it is quite important for good performance. The current code will train and work fine, but its convergence is slower because it starts off in a not great spot in the weight space. Please see [nanoGPT model.py](https://github.com/karpathy/nanoGPT/blob/master/model.py) for `# init all weights` comment, and especially how it calls the `_init_weights` function. Even more sadly, the code in this repo is a bit different in how it names and stores the various modules, so it's not possible to directly copy paste this code here. My current plan is to publish a supplementary video lecture and cover these parts, then I will also push the exact code changes to this repo. For now I'm keeping it as is so it is almost exactly what we actually covered in the video.

### License

MIT

ng-video-lecture

파일 탐색기

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

ng-video-lecture

파일 탐색기

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README