ml-from-scratch - GitLite

KO | EN

gitlite — search

#python #ai #go #rust #deep-learning #linux #nodejs #macos #windows #vue #awesome-list #pytorch #security #mcp

ml-from-scratch

(★ 224)

All the ML algorithms, ML models are coded from scratch by pure Python/Numpy with the Math under the hood. It works well on CPU.

파일 탐색기

최종 버전 다운로드 (.zip)

- - Attention_Mechanism-checkpoint.ipynb
- Attention_Mechanism.ipynb
- main.cpp
- matrix.cpp
- matrix.h
- vector.cpp
- vector.h
- - effective-convolution-checkpoint.ipynb
- __init__.py
- convolutional_neural_network.py
- effective-convolution.ipynb
- - 1vs2.all
  - recipes.all
  - sentiment.all
  - titanic_test.csv
  - titanic_train.csv
- decision_tree.py
- gan.py
- - test.csv
  - testing.csv
  - train.csv
  - trainDirection.csv
- knn.py
- cifar10_lib.py
- mnist_lib.py
- utils.py
- - Linear Regression-checkpoint.ipynb
- Linear Regression.ipynb
- linear_regression.gif
- linear_regression.py
- linear_regression_1.gif
- main.py
- prostate.data.txt
- README.md
- visualize.py
- - amazon_baby_subset.csv
  - important_words.json
- logistic_regression.gif
- logistic_regression.py
- README.md
- visualize.py
- __init__.py
- neural_network.py
- nn_weights.pkl
- test
- activations.py
- initializers.py
- layers.py
- losses.py
- optimizers.py
- main_rnn.py
- names
- recurrent_neural_network.py
- rnn_keras.py
- softmax_regression.py
- - convex-optim.ipynb
  - qp_ex.png
  - SVM-programming.ipynb
  - svm_ex.png
- emailSample1.txt
- emailSample2.txt
- ex6data1.mat
- ex6data2.mat
- ex6data3.mat
- main.py
- spamTest.mat
- spamTrain.mat
- svm.py
- test_cross_entropy.py
- utils.py
- vocab.txt
- - Transformer-checkpoint.ipynb
  - Transformer_Pytorch-checkpoint.ipynb
  - Transformer_TF-checkpoint.ipynb
  - Untitled-checkpoint.ipynb
- nmt_utils.py
- Transformer_Pytorch.ipynb
- Transformer_TF.ipynb
.gitignore
__init__.py
LICENSE
README.md
requirements.txt

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Machine Learning from scratch

## About
This ML repository is all about coding Machine Learning algorithms from scratch by Numpy with the math under the hood without Auto-Differentiation frameworks like Tensorflow, Pytorch, etc. Some advanced models in Computer Vision, NLP require Tensorflow to quickly get the idea written in paper.

## Repository structure
As a software engineer, I follow the principle of OOP to construct the repository. You can see that `NeuralNetwork` class will use `FCLayer`, `BatchNormLayer`, `ActivationLayer` class and `CNN` class will use `ConvLayer`, `PoolingLayer`, `FCLayer`, `ActivationLayer`,... This helps me easily reuse every piece of code I wrote as well as for readable code.

## Dependencies: 
- tqdm, numpy, sklearn, matplotlib

## Table of contents
- Machine Learning models:
  * [Linear Regression](https://github.com/giangtranml/ml-from-scratch/blob/master/linear_regression/linear_regression.py)
  * [Logistic Regression](https://github.com/giangtranml/ml-from-scratch/blob/master/logistic_regression/logistic_regression.py)
  * [Softmax Regression](https://github.com/giangtranml/ml-from-scratch/blob/master/softmax_regression/softmax_regression.py)
  * [Neural Network](https://github.com/giangtranml/ml-from-scratch/blob/master/neural_network/neural_network.py)
  * [Convolutional Neural Network](https://github.com/giangtranml/ml-from-scratch/blob/master/convolutional_neural_network/convolutional_neural_network.py)
  * [Generative Adversarial Network](https://github.com/giangtranml/ml-from-scratch/blob/master/gan/gan.py)
  * [Support Vector Machine](https://github.com/giangtranml/ml-from-scratch/blob/master/svm/svm.py)
  
- Deep Learning layers:
  * [Fully-Connected Layer](https://github.com/giangtranml/ml-from-scratch/blob/master/nn_components/layers.py#L43)
  * [Convolutional Layer](https://github.com/giangtranml/ml-from-scratch/blob/master/nn_components/layers.py#L107)
  * [Pooling Layer](https://github.com/giangtranml/ml-from-scratch/blob/master/nn_components/layers.py#L245)
  * [Activation Layer](https://github.com/giangtranml/ml-from-scratch/blob/master/nn_components/layers.py#L372)
  * [BatchNorm Layer](https://github.com/giangtranml/ml-from-scratch/blob/master/nn_components/layers.py#L436)
  * [Dropout Layer](https://github.com/giangtranml/ml-from-scratch/blob/master/nn_components/layers.py#L407)
  
- Optimization algorithms:
  * [SGD](https://github.com/giangtranml/ml-from-scratch/blob/master/optimizations_algorithms/optimizers.py#L16)
  * [SGD with Momentum](https://github.com/giangtranml/ml-from-scratch/blob/master/optimizations_algorithms/optimizers.py#L24)
  * [RMSProp](https://github.com/giangtranml/ml-from-scratch/blob/master/optimizations_algorithms/optimizers.py#L37)
  * [Adam](https://github.com/giangtranml/ml-from-scratch/blob/master/optimizations_algorithms/optimizers.py#L51)
- Weights initialization:
  * [He initialization](https://github.com/giangtranml/ml-from-scratch/blob/master/nn_components/initializers.py#L3)
  * [Xavier/Glorot initialization](https://github.com/giangtranml/ml-from-scratch/blob/master/nn_components/initializers.py#L24)
- Advanced models:
  * [Attention Mechanism (Bahdanau and Luong Attention)](https://github.com/giangtranml/ml-from-scratch/blob/master/attention_mechanism/Attention_Mechanism.ipynb)
  * [Transformer](https://github.com/giangtranml/ml-from-scratch/blob/master/transformer/Transformer_Pytorch.ipynb)