100_Days_of_ML_Code

KO | EN

gitlite — search

#javascript #python #hacktoberfest #go #rust #macos #awesome #vue #pytorch #security #mcp #ios #openai #open-source

100_Days_of_ML_Code

(★ 49)

100 Days of Machine Learning Code

100_Days_of_ML_Code Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# 100 Days of Machine Learning Code

100 Days of Machine Learning Code. Full instructions about the challenge [here](https://github.com/george-studenko/100_Days_of_ML_Code/blob/master/100_Days_of_ML_Code_Instructions.MD)

Here you can find a very useful [Machine Learning Glossary](https://developers.google.com/machine-learning/glossary/)

Here you can find the [Amazon Machine Learning paths](https://aws.amazon.com/training/learning-paths/machine-learning/)

And the always useful [Cheatsheets!](https://github.com/george-studenko/ds-cheatsheets)

## 100DaysOfMLCode Index

[Attention mechanisms](content/Attention-mechanisms.md)  
[Batch size](content/Batch-size.md)   
[CNNs](content/CNN.md)   
[Computer Vision](content/Computer-Vision.md)   
[Conda](content/Conda.md)   
[Data Augmentation](content/Data-augmentation.md)  
[Data Science Cheatsheets](https://github.com/george-studenko/ds-cheatsheets)  
[Debug Jupyper Notebooks](content/Debug-jupyter.md)  
[Defining a network structure](content/Defining-network-structure.md)   
[Downloading Datasets](content/Downloading-Datasets.md)   
[Dropout](content/Dropout.md)  
[Embeddings](content/Embeddings.md)  
[FastAI](content/fast-ai.md)   
[Feather](content/Feather.md)  
[Filtered Images](resources/kernels.png)  
[Facia-Keypoints-Detector notes](content/Day-10.md)   
[Fully Convolutional Neural Networks](content/FCNNs.md)  
[Generate requirements.txt file](content/Generate-requirements.md)  
[GPU States](content/GPU.md)  
[High bias & high variance](content/High-bias-and-high-variance.md)   
[Homemade Machine Learning](https://github.com/trekhleb/homemade-machine-learning)
[Hyperparameters](content/Hyperparameters.md)   
[Image Captioning Project Notes](content/Image-Captioning.md)  
[Intro to Pandas Lab](code/Intro_to_pandas.ipynb)  
[Jobs in Computer Vision](content/Jobs-in-Computer-Vision.md)  
[Jupyter notebooks extensions](content/Jupyter-extensions.md)    
[Jupyter notbook server](content/Jupyter-server.md)  
[KNN with Lodash](code/knn-lodash.js)  
[Layer Shapes](content/Layer-shapes.md)  
[Learning Rates](content/Learning-rate.md)   
[Localization](content/Localization.md)   
[LSTM cells](content/LSTM.md)   
[Momentum](content/Momentum.md)   
[Machine Learning Concepts](content/Machine-learning-concepts.md)   
[NLTK](content/NLTK.md)  
[OpenCV](content/OpenCV.md)    
[Probability](content/Probability.md)   
[PyTorch Anatomy](https://freecontent.manning.com/pytorch-crash-course-part-1/)   
[PyTorch notes](content/PyTorch.md)   
[Pandas DataFrame Cheatsheet](cheatsheets/Pandas-DataFrame-Notes.pdf)  
[Region Proposal Algorithms](content/Region-Proposal-Algorithms.md)    
[Regularization](content/Regularization.md)   
[Reinforcement Learning (DQN) Notes](content/Reinforcement-Learning.md)  
[Reinforcement Learning: An Introduction (second edition) Book](https://s3-us-west-1.amazonaws.com/udacity-drlnd/bookdraft2018.pdf)  
[Reinforcement Learning Cheatsheet](cheatsheets/rl-cheatsheet.pdf)  
[RNNs](content/RNN.md)   
[Training Dev and Testing sets](content/Training-Dev-and-Testing-Sets.md)   
[Visualizing Feature Maps](content/Visualizing-Feature-Maps.md)   
[Weighted Loss Functions](content/Weighted-Loss-Functions.md)   
[YOLO](content/YOLO.md)   
  
### Code  
[Architecture](code/cv/models.py)  
[Contour detection and features](code/cv/Contour%20detection%20and%20features.ipynb)  
[Finding Edges and Custom Kernels](code/cv/Finding%20Edges%20and%20Custom%20Kernels.ipynb)   
[Guide to Mathematical Notation](code/Linear_Algebra/9uide_to_mathematical_notation.ipynb)  
[Hough Lines](code/cv/Hough_lines.ipynb)   
[Matrices](code/Linear_Algebra/2_matrices_in_python.ipynb)   
[Matrix Addition](code/Linear_Algebra/3_matrix_addition.ipynb)   
[Matrix Multiplication](code/Linear_Algebra/4_matrix_multiplication.ipynb)   
[Robot Localization](code/Robot_Localization.ipynb)   
[Vectors](code/Linear_Algebra/1_vector_coding.ipynb)   
  
### Repositories  
[Detecting edges live with webcam repo](https://github.com/george-studenko/Live-Sketch-with-Computer-Vision)  
[Facial Keypoints Detector Project](https://github.com/george-studenko/Facia-Keypoints-Detector)  
[Landmard Detection and Tracking Project](https://github.com/george-studenko/landmark-detection-and-tracking)  
[Hotdog Not Hotdog](https://github.com/george-studenko/Hotdog-not-Hotdog)

### Resources 
[32.81TB of research data](http://academictorrents.com/)  
[Amazon Machine Learning paths](https://aws.amazon.com/training/learning-paths/machine-learning/)

## Day 160: May 31 2019

**Today's Progress**: Applying cosine similarities to find recommended articles and recommended artists and completed the Unsupervised Learning with Python course @datacamp where I learned how to cluster, transform, visualize, and extract insights from unlabeled datasets.

**Thoughts:** I know the basics now, about unsupervised learning but there is a long way to go, I will be practicing with datacamps project and should also go back to kaggle to grab some projects from there as well, busy days.

## Day 159: May 30 2019

**Today's Progress**: Completed the Generating Keywords for Google Ads project @datacamp, learning how to create recommendation systems using NMF with word arrays and cosine similarities.

## Day 158: May 29 2019

**Today's Progress**: Applying NMF Non-Negative Matrix Factorization to images to check how it learns about segments of the image, comparing to PCA which is not able to learn about those segments.

## Day 157: May 28 2019

**Today's Progress**: Learning about NMF Non-Negative Matrix Factorization for matrix decomposition and applying it to cluster and find topics in wikipedia articles. It can also be used for images to recognize parts of images by flattening the grayscale image.

## Day 156: May 27 2019

**Today's Progress**: Continued with the Usupervised learning datacamp course, finished the Decorrelating your data and dimension reduction chapter, learning about tf-idf for word frequency arrays

## Day 155: May 23 2019

**Today's Progress**: Continued on the Unsupervised learning course, learning about PCA (Principal Component Analysis), to decorrelate features and reduce dimensions by checking the variance of each feature, also learning about intrinsic dimensions in data.

**Thoughts:** Not sure how it works in the background but PCA seems to be doing a great job

## Day 154: May 22 2019

**Today's Progress**: Learning about t-SNE for high dimension data visualizations, plotting t-sne to see how they correlate and how the clusters look like, learning about the t-sne learning_rate and what happens when the lr is not correct

**Thoughts:** it is a lot easier to understand the visualizations with t-sne for me

## Day 153: May 21 2019

**Today's Progress**: Learning about hierarchical clustering for data visualization with linkage() and dendrogram to plot mergings

**Thoughts:** A bit hard to understand the mergings plot at first, but it is interensting how items in the same group do have things in common

## Day 152: May 20 2019

**Today's Progress**: Creating pipelines with make_pipeline so include Scalers or Normalizers to the pipeline before training the kmeans model in order to improve results by scaling or normalizing values

**Thoughts:** the make_pipeline method is very simple to use making the use of pipelines a snap

## Day 151: May 19 2019

**Today's Progress**: Learning how to evaluate a cluster with metrics such as inertia_ and and pandas crosstab, plotting inetia for models with different values of k to find the best k value for a kmeans model

**Thoughts:** Having fun with clustering and kmeans

## Day 150: May 18 2019

**Today's Progress**: Started with the unsupervised learning with python course @DataCamp, experimenting with KMeans for clustering, plotting clusters and their corresponding centroids with pyplot scatter plots to visualize them

**Thoughts:** I have always been working most of the time with supervised learning methods, although I did try doing some clustering in the past I'm learning new things with this

## Day 149: May 17 2019

**Today's Progress**: Implementing pipelines for classification and regression models with that finished with the Supervised Learning with scikit-learning datacamp course, was a lot of fun, looking forward to continue with more ML courses @datacamp

## Day 148: May 16 2019

**Today's Progress**: Learning about data imputation with imputers, creating a simple pipelines in sklearn, data standarization by centering and scaling to improve model performance.

**Links:** [Preprocessing in Data Science (Part 1): Centering, Scaling, and KNN](https://www.datacamp.com/community/tutorials/preprocessing-in-data-science-part-1-centering-scaling-and-knn)

## Day 147: May 15 2019

**Today's Progress**: Learning aboud data preprocesing like dummy variables and how to obtain them with pandas and scikit learn, get_dummies() and OneHotEncoder() and how we can drop one of the columns, dealing with missing data functions. Experimenting around with #docker deep learning #containers.

**Thoughts:** Had to install differnet docker images until I found one that seems to be fine for what I need, will keep experimenting with it, so far so good.

## Day 146: May 14 2019

**Today's Progress**: Learning about GridSearchCv and RandomizedSearchCv to find and tune hyperparameters, applying them with cross-validation, accessing the best values with the best_score_ and best_params_ properties

## Day 145: May 13 2019

**Today's Progress**: Learning about Logistic Regression for classification problems in sklearn. For metrics learning about the ROC curve and AUC (area under curve), applying AUC with cross-validation (cross_val_score)

## Day 144: May 12 2019

**Today's Progress**: Playing around with Pandas exploring Catalunya open data, not much so far, my pandas's skills are a bit rusty

**Thoughts:** Was nice to find open datasources from Barcelona and Badalona, looking forward to make some data exploration in depht and see what insights I can find.

**Links:** [Open Data BCN](https://opendata-ajuntament.barcelona.cat/en/) | [Badalona Open Data](http://opendata.badalona.cat/od/)

## Day 143: May 11 2019

**Today's Progress**: Learning about classification_report and confussion matrix in sklearn to evaluate models, metrics such as accuracy, precission, recall and F1 score can be calculated from the confusion matrix.

**Thoughts:** sklearn makes it quite easy to evaluate models, but it is important to know how to read those metrics

## Day 142: May 10 2019

**Today's Progress**: learning about Ridge regression and Lasso Regression, Lasso is quite interesting for feature selection using the coefficients, it can help filter out the good features to use when dealing with a lot of them, just plot the coefficients.

**Thoughts:** love the idea of getting features selection simplified with Lasso coefs!

## Day 141: May 9 2019

**Today's Progress**: continued with the Linear Regression lessons on the Sklearn course, performing 5-fold cross-validation

**Thoughts:** sklearn is quite simple to use so far, I'm really enjoying it!

## Day 140: May 8 2019

**Today's Progress**: finished chapter 1 and now going through chapter 2 of Datacamp Supervised Learning with scikit-learn course on Regression

**Thoughts:** so far simple stuff, but I always learn some things I didn't know

## Day 139: May 7 2019

**Today's Progress**: going through chapter 1 of Datacamp Supervised Learning with scikit-learn course, enrolled to datacamp, and having a lot of fun!

**Thoughts:** took me a while to decide myself to enrol to datacamp, but finally did it, and so far so good!

## Day 138: May 6 2019

**Today's Progress**: exploring the Datacamp course catalog and Kaggle free courses, looking forward to start working with DL and ML at work.

**Thoughts:**  Kaggle offers some interesting free courses, and datacamp you can get some good courses and projects quite cheap if you buy yearly with eventual offers, also saw dataquest in the past as a different alternative to datacamp

## Day 137: May 5 2019

**Today's Progress**: Re-learning about Generative Adversarial Networks and DCGANs, checking implementations in Tensorflow and Pytorch, researching about ML and DL for aviation.

**Thoughts:** I had a really hard time learning about GANs in the past, I'm now just trying to learn about it again,

**Links:**[Genearting images with GANs Siraj Raval](https://www.youtube.com/watch?v=yz6dNf7X7SA) | [Generative Adversarial Network(Pytorch)](https://www.youtube.com/watch?v=8l6NzV4Eryo) | [Forecasting airline passengers using designer machine learning](https://www.youtube.com/watch?v=A-azJ8qmwug) | [Artificial Intelligence for Real Airplanes](https://www.eurocontrol.int/sites/default/files/events/presentation/art-ws-ml-ai-10-ai-for-real-airplanes-boeing.pdf)

## Day 136: May 4 2019

**Today's Progress**: Working on the Drone project With the team, starting to integrate the pieces we've been developing so far, getting ready for the demo day, learning about kafka as we might use it for the drone project.

**Links:** [Kafka Tutorial](https://www.youtube.com/watch?v=U4y2R3v9tlY)

## Day 135: May 3 2019

**Today's Progress**: Going through the Tensorflow course on more advanced topics about Convolutional Neural Networks with multiple filters for RGB images.

## Day 134: May 2 2019

**Today's Progress**: Reviewing CNN's concepts, convolutional layers and max pooling layers for downsampling images, learned that I can enable TF2 features in TF 1.13.1 by executing tf.enable_eager_execution()

**Thoughts:** It is always nice to review basic concepts, things that 2 years ago were so confusing now are quite clear, considering redoing many projects I did in the past in either PyTorch or TF 2.

## Day 133: May 1 2019

**Today's Progress**: Kept learning about Tensorflow, implemented a very simple network with a single dense layer to solve the typical housing price problem (with one feature as input), learning how to implement callbacks when training with tensorflow

**Thoughts:** nice to learn about callbacks when training, I never though of that before, as far as I know PyTorch does not implement that out of the box.

## Day 132: Apr 30 2019

**Today's Progress**: continued watching the Tensorflow videos and testing notebooks in Colab training a simple model to convert degrees Celsius to Fahrenheit is quite interesting how the networks sets the weights and bias to almost the same as in the official conversion formula.

**Thoughts:** The formula is f = 1.8c + 32 while the network came as close as  1.798 and 31.905, learning a bit about keras as well while doing it, since now Keras is part of Tensorflow.

## Day 131: Apr 29 2019

**Today's Progress**: Watching some videos from the Intro to TensorFlow for Deep Learning which teaches Tensorflow 2.0, reading Math for Programmers (3D graphics, machine learning, and simulations with Python) MEAP chapter 1 and a nice article A Recipe for Training Neural Networks.

**Links:** [Article: A Recipe for Training Neural Networks](http://karpathy.github.io/2019/04/25/recipe/) | [Intro to TensorFlow for Deep Learning](https://eu.udacity.com/course/intro-to-tensorflow-for-deep-learning--ud187)

## Day 130: Apr 28 2019

**Today's Progress**: Learning about Tensorflow 2.0 and the new features it supports, it takes out a lot of the previous complexity and me making it a lot easier to learn and use. Learning about TF Serving to deploy and maintain models in production.

**Links:** [Serving Models in Production with TensorFlow Serving](https://www.youtube.com/watch?v=q_IkJcPyNl0) |  [Siraj Raval Tensorflow 2.0 overview])(https://www.youtube.com/watch?v=WS9Nckd2kq0)

## Day 129: Apr 27 2019

**Today's Progress**: Presenting a talk about Deep Neural Networks and Convolutional Neural Networks at Software Crafters Canarias.

**Links:** [Software Crafters Canarias](https://socracan.com/)

## Day 128: Apr 26 2019

**Today's Progress**: Watching some lessons on how to train and use a Mask R-CNN with openCV using cv2.dnn.readNetFromTensorflow for inference.

**Thoughts:** I would really like to be able to implement a Mask R-CNN myself, although it seems to be harder than any other DNN architecture.

**Links:** [Object Detection Custom Training of Image Mask RCNN Deep Learning](https://www.youtube.com/watch?v=uUYR6IEm5VM) |[Train Mask-RCNN repo](https://github.com/AISangam/Mask-RCNN-bottle-training) | [Tool to label images](http://www.robots.ox.ac.uk/~vgg/software/via/via-1.0.6.html)

## Day 127: Apr 25 2019

**Today's Progress**: Learning about Mask R-CNN for image segmentation,the original paper is from 2017, how come nobody told me about it yet!

**Thoughts:** I'm fascinated by this, I believe that Image segmentation is one of the coolest things in computer vision!

**Links:** [facebookresearch/maskrcnn-benchmark](https://github.com/facebookresearch/maskrcnn-benchmark) | [Mask R-CNN for Ship Detection & Segmentation](https://towardsdatascience.com/mask-r-cnn-for-ship-detection-segmentation-a1108b5a083) | [Mask R-CNN for Object Detection and Segmentation](https://github.com/matterport/Mask_RCNN) | [Building a Custom Mask RCNN model with Tensorflow Object Detection
](https://towardsdatascience.com/building-a-custom-mask-rcnn-model-with-tensorflow-object-detection-952f5b0c7ab4) | [Mask R-CNN with OpenCV
](https://www.pyimagesearch.com/2018/11/19/mask-r-cnn-with-opencv/)

## Day 126: Apr 24 2019

**Today's Progress**: Published my framework to create image classifiers in a simple and quick way, called Image Classifier Creator, I know, too original right?, I'm open to name suggestions.

**Thoughts:** I wrote this code long ago but didn't realize it could be reused so easily, so just decided to refactor it and make it public

**Links:** [Image Classifier Creator site](https://george-studenko.github.io/image-classifier-creator/) |  [Image Classifier Creator repo](https://github.com/george-studenko/image-classifier-creator)

## Day 125: Apr 23 2019

**Today's Progress**: Following my research from yesterday created my own implementation of the Hotdog Not Hotdog Sillicon Valley image classifier which can be trained with either Densenet or Resnet at the moment, would like to implement other architectures as well soon.

**Thoughts:** It was fun to create the hotdog not hotdog classifier, I would like to deploy it to a web app.

**Links:** [Hotdog Not Hotdog](https://github.com/george-studenko/Hotdog-not-Hotdog) | [Generate requirements.txt file](content/Generate-requirements.md)

## Day 124: Apr 22 2019

**Today's Progress**: Researching on how to create an unary image classifier, the easier way is to cheat and create a binary classifier as we are already used to but to refactor the classes into a positive and negative class, similar to the hotdog not hotdog from Sillicon Valley.

**Thoughts:** It is interesting how it can be harder to create a single class classifier rather than a binary one.

**Links:**[Santa, not Santa classifier](https://www.pyimagesearch.com/2017/12/11/image-classification-with-keras-and-deep-learning/) | [One-class SVM](http://scikit-learn.org/stable/modules/generated/sklearn.svm.OneClassSVM.html) | [Isolation Forest](http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.IsolationForest.html) | [One Class Classification for Images with Deep features](https://hackernoon.com/one-class-classification-for-images-with-deep-features-be890c43455d)

## Day 123: Apr 21 2019

**Today's Progress**: Importing YOLO V3 and Tiny YOLO models with openCV.dnn.readNetFromDarknet and testing its performance, for some reason it doesn't work as well as the previous implementation we already have in the drone project.

**Thoughts:** Good to know I can import Deep Neural Networks with openCV!

**Links:**[YOLO with openCV](https://github.com/george-studenko/Drone-Madness/blob/develop/src/Implementation/Tiny_Object_Detection.py)

## Day 122: Apr 20 2019

**Today's Progress**: Debugging the Drone sensors data function, refactoring the object_detection class to optimize it and be able to include it as part of the pipeline, trying to get image frames from the Drone, watching some videos on Calculus and Differential Equations

**Thoughts:** Considering to start a 100 days of math challenge to go through Algebra, Calculus, Differential Equations, Probability and Statistics

## Day 121: Apr 19 2019

**Today's Progress**: Watching Netwonian Motion physics videos from math tutor and solving some problems to practice,  checking  differences between YOLOV3 and TinyYOLOv3, found an interesting repo that implements it with PyTorch, going through the Deep Learning Nanodegree lessons again to see refresh concepts.

**Thoughts:** 2 years ago I graduated from the Deep Learning Nanodegree, back then it was quite hard for me, I got my classroom videos updated to a newer version of the Nanodegree with Luis Serrano as the curriculum lead this time, I'm rewatching the videos now with this new content to see if this time I can grab more information than the first time while also refreshing concepts

**Links:**[yolov3_pytorch repo](https://github.com/holli/yolov3_pytorch)

## Day 120: Apr 18 2019

**Today's Progress**: Continued working on the bicycle model implementation, stepping back to refresh some physics and differential equations concepts, asking for help on the forums, so far nothing. Thinking on the Drone project next steps, maybe search patterns and object tracking, but first we need to integrate all the pieces together.

**Thoughts:** it is frustrating when you get to the point to not know how to continue, but are those moments when you just need to keep on going, All things are difficult before they are easy (Thomas Fuller), am not giving up, not yet...

## Day 119: Apr 17 2019

**Today's Progress**: Working on the kinematic bicycle model implementation, did the first test, unsurprisingly it did not work as expected, checking the forums for help, looks like I'm not the only one with problems, everyone is struggling to get it right

**Thoughts:** Although I'm still stuck, today I was revisiting the slides and it does make a bit more of sense now, but still need to figure out what is wrong with my formulas implementations. I cant post code as for the coursera policies

## Day 118: Apr 16 2019

**Today's Progress**: Watching the Lateral Dynamics of Bicycle Model and Vehicle actuation lessons of the intro to self driving cars course, now working on implementing a kinematic bicycle model.

**Thoughts:** I'm having a really hard time, it is really hard to follow at the moment, it is a very advance course and way too many formulas to understand everything, but will keep trying.

## Day 117: Apr 15 2019

**Today's Progress**: On week 4 of the Intro to self-driving cars course, watching the videos aboutu kinematic modeling in 2D, Kinematic bicycle model, Dynamic modeling in 3D and Longitudinal vehicle modeling, quite hard stuff, hardcore physics

**Thoughts:** Starting to scare me off, but will keep pushing, too many formulas and pretty hard to understand them all.

Links: [Vehicle Dynamics](http://publications.lib.chalmers.se/records/fulltext/244369/244369.pdf) | [Classical Dynamics](http://www.damtp.cam.ac.uk/user/tong/dynamics/clas.pdf) | [The Kinematic Bicycle Model](https://www.springer.com/cda/content/document/cda_downloaddocument/9781461414322-c1.pdf?SGWID=0-0-45-1265143-p174267791)

## Day 116: Apr 14 2019

**Today's Progress**: Finished week 3 of Intro to Self-driving cars, on Safety Assurance for Autonomous Vehicles, reviewing some basic physics and probability concepts and formulas, this week will be working on the next module about Vehicle Dynamic Modeling.

**Thoughts:** It's interesting to see the framework that is proposed for autonomous driving safety and see how GM and Waymo apply them or even come up with their own ones.

Links: [NHTSA - Automated Driving Systems: A Vision for Safety 2.0 report](https://www.nhtsa.gov/sites/nhtsa.dot.gov/files/documents/13069a-ads2.0_090617_v9a_tag.pdf) | [RAND Corporation Report on Driving to Safety](https://www.rand.org/pubs/research_reports/RR1478.html)

## Day 115: Apr 13 2019

**Today's Progress**: Working all day on the drone project with the team, managed to get sensor readings while the drone is flying and implementing callbacks to trigger with sensor information, finished week 2 of intro to self-driving cars

**Thoughts:** the team is making good progress with the project, we did good progress in the first 15 days of the project

## Day 114: Apr 12 2019

**Today's Progress**: Managed to get the Tello sensor states, now trying to incorporate that in the code, need to refactor the previous event driven code I did with the udacidrone API to use plain python or more standard packages. Working on week 2 of Intro to Self Driving cars, learning about the software architecture and components of a self driving car.

**Thoughts:** Working with network packages is not that simple, but starting to get used to it.

## Day 113: Apr 11 2019

**Today's Progress**: Experimenting with different Tello APIs, doing some basic tests executing some commands with the drone, organizing the code and trying to implement events to get it properly done, trying to retrieve sensor states, reading a paper about path planning for MAVs and another about wall segmentation, finished the week 1 of Intro to self-driving cars course by the University of Toronto.

**Thoughts:** I want to go back to autonomous tech, the drone project is progressing little by little, lets see what we get by Saturday.

**Links:** [Real-Time Segmentation, Tracking, and Coloring of
Walls Using iOS](https://stacks.stanford.edu/file/druid:yj296hj2790/Yeung_Piersol_Liu_Project_Imaggle.pdf) | [Path Planning for Non-Circular Micro Aerial Vehicles in Constrained
Environments](https://www.cs.cmu.edu/~maxim/files/pathplanforMAV_icra13.pdf)

## Day 112: Apr 10 2019

**Today's Progress**: Configuring an AP to connect different Tellos to form a Swarm, sending UDP packages with commands to the drone to connect to the AP, apparently the standard version of Tello does not support the command and can only connect directly to a device, researching on how can it be done found some ways to do it although they are not the best way to do it.

**Thoughts:** The hardware is making things harder and harder, too many limitations, trying to overcome them and continue with what is important.

**Links:** [Controlling a Tello Swarm](https://medium.com/@henrymound/adventures-with-dji-ryze-tello-controlling-a-tello-swarm-1bce7d4e045d) | [Tello SDK 2.0](https://dl-cdn.ryzerobotics.com/downloads/Tello/Tello%20SDK%202.0%20User%20Guide.pdf) | [Tello Video](https://github.com/f41ardu/Tello-Python/tree/master/PY3_Tello_Videop)

## Day 111: Apr 9 2019

**Today's Progress**: Meeting with the team, deciding on the next phases of work in the drone project, researching Tello APIs and how to command swarms of Tellos, working on the pipeline for object detection.

**Thoughts:** We will need to focus on working with what we have for now, while we are still researching about PX4 and Mavlink compatible drones we do need to keep on moving so we will be working with the Tellos for now.

**Links:** [Tello Programming with Python](https://github.com/dbaldwin/DroneBlocks-Tello-Python) | [Microlinux Tello](https://github.com/microlinux/tello) | [PyTello](https://bitbucket.org/PingguSoft/pytello/src) | [How to add Person Tracking to a Drone using Deep Learning and NanoNets](https://medium.com/nanonets/how-i-built-a-self-flying-drone-to-track-people-in-under-50-lines-of-code-7485de7f828e)

## Day 110: Apr 8 2019

**Today's Progress**: Learning about the Bresenham's algorithm, moving the object detection with Yolo to a method that can be called into a pipeline, looking for a course to learn more about drone programming

**Thoughts:** Quite busy days, definitely need more time to keep on track

## Day 109: Apr 7 2019

**Today's Progress**: Researching about the different SDKs available to program drones with python to see if to replace the Udacidrone one, the main ones I found are Dronecode and Dronekit, cheking the WTL Machine Learning infographic.

**Thoughts:** Dronecode seems to be more straigh forward, at least while checking the example code and comparing both

**Link to code:** [Dronecode](https://github.com/Dronecode/DronecodeSDK-Python) | [Dronekit](https://github.com/dronekit/dronekit-python) | [Udacidrone](https://github.com/udacity/udacidrone)

## Day 108: Apr 6 2019

**Today's Progress**: Modified the  backyard flyer script to work with real drones through Mavlink and Px4, then tried to run it with a real drone, but didn't work apparently DJI does not support mavlink, doing some research on drones that might support it.  Learning about waypoint extraction.

**Thoughts:** We are making progress on the project, we just need to keep the good pace!

**Link to code:** [Drone-backyard-flyer repository](https://github.com/george-studenko/drone-backyard-flyer)

## Day 107: Apr 5 2019

**Today's Progress**: Finished programming the backyard flyer python script to fly a square pattern using event driven programming, the hard part was to handle position uncertainty since the the coordinates are not exact, checking when the drone hits a waypoint was a challenge.

**Thoughts:** The code can still be improved a lot, looks like I can check for waypoints with  `np.linalg.norm` by substracting the target_position from the local_position and allowing some tolerance.

**Link to code:** [Drone-backyard-flyer repository](https://github.com/george-studenko/drone-backyard-flyer)

## Day 106: Apr 4 2019

**Today's Progress**: Kept researching about infering depth from single images, found some interesting papers, also  kept learning about motion planning, going back to  the slam lessons I took some months ago.

**Links:** [Inferring Depth from Single Images in Natural Scenes](https://pdfs.semanticscholar.org/6fcc/3b49c49be7fc5741a892d18319eb65898094.pdf) | [Make3D: Depth Perception from a Single Still Image](http://www.cs.cornell.edu/~asaxena/reconstruction3d/Saxena_depthperception_aaai08.pdf) | [Inferring Depth from Single Images in Natural
Scenes](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.95.1296&rep=rep1&type=pdf) | [Perceiving 3D from 2D Images](https://courses.cs.washington.edu/courses/cse576/book/ch12.pdf)

## Day 105: Apr 3 2019

**Today's Progress**: Learning about motion planning, researching how to see  the depth of an image to be able to recognize what is close and what is far in an image, since we do not have stereo images it becomes harder to do it, still researching if it can be done.

**Thoughts:** There is a lot to do and too little time to do it...

**Links:**[Motion Planning Videos](https://classroom.udacity.com/courses/cs373/lessons/48646841/concepts/486837620923) | [Depth Perception with a Single Camera](https://pdfs.semanticscholar.org/bd1d/1918307e8cedef12f8406f16044ace9f0ef2.pdf) | [A Single Camera Stereo System for Obstacle Detection](https://www.researchgate.net/publication/2647247_A_Single_Camera_Stereo_System_for_Obstacle_Detection)

## Day 104: Apr 2 2019

**Today's Progress**: Learning about event driven  programming, working on the backyard flyer project, implementing the callback methods, so far I got the code to connect arm the drone and takeoff. Tonight we have our first meeting with the team to evaluate current progress on the drone project.

**Thoughts:** Learning new programming paradigms is always fun and a bit confusing at first until it clicks!

**Link to code:** [Drone Backyard Flyer repo](https://github.com/george-studenko/drone-backyard-flyer)

## Day 103: Apr 1 2019

**Today's Progress**: Got the udacidrone environment set up and code to control the drone from python, the code is in the repo, next will be to program the flight computer with events, so now looking into event driven programming.

**Thoughts:** The udacidrone simulator seems to be quite simple to use.

**Link to code:** [Udacidrone Notebook](https://github.com/george-studenko/Drone-Madness/tree/master/src/Notebooks/UdaciDrone)

## Day 102: Mar 31 2019

**Today's Progress**: Implemented the YOLO notebook to start with phase one of the Drone Madness project, we are also considering RetinaNet, watching some lessons from the Flying Car and Autonomous Flight Engineer Nanodegree.

**Thoughts:** Looking forward to next Saturday to see how far we can get with this in one week.

**Link to code:** [YOLO notebook implementation](https://github.com/george-studenko/Drone-Madness/tree/master/src/Notebooks/YOLO)

## Day 101: Mar 30 2019

**Today's Progress**: The Drone Madness #DeepLearning Project, today @AISaturdaysEs we created a group of 4 people to work on our next project, we're on the early stages of planning and choosing the stack, more details comming soon, stay tunned, could this be round 2 of  #100DaysOfML?

**Thoughts:** The project is quite ambitious and we have 50 days at most to present it, so lets see how far we can get.

**Link to code:** [Drone Madness Repo](https://github.com/george-studenko/Drone-Madness)

## Day 100: Jan 07, 2019

**Today's Progress**: Learning more about reinforcement learning and Deep Q Learning which processes images using convolutional layers (CNNs) without pooling layers to keep information about the location of the objects in the image as that is crucial in this case.

**Thoughts:** This is the day 100 of my #100DaysOfMLCode journey, definitely enjoyed it but I won't stop here, even though I might not post my daily updates as often I will continue learning, there is still a lot to learn about machine learning and all its related areas.

**Link to code:** [Reinforcement Learning Cheatsheet](cheatsheets/rl-cheatsheet.pdf) | [Reinforcement Learning (DQN) Notes](content/Reinforcement-Learning.md)

## Day 99: Jan 06, 2019

**Today's Progress**: Introducing myself a bit into the world of Deep Reinforcement Learning, almost getting to the end of this 100 days journey and there are still so many topics to learn about in Machine Learning.

**Thoughts:** So far during my 100 days of ML code journey I had the opportunity to see and learn so many things, yet there are still a world of things to learn like for example Reinforcement Learning, which I haven't even started to look into until today.

**Link to code:** [Reinforcement Learning: An Introduction (second edition) Book](https://s3-us-west-1.amazonaws.com/udacity-drlnd/bookdraft2018.pdf) | [Reinforcement Learning (DQN) with Pytoch](https://pytorch.org/tutorials/intermediate/reinforcement_q_learning.html) | [Open AI Gym](https://gym.openai.com/) | [Open AI Gym Repo](https://github.com/openai/gym)

## Day 98: Jan 05, 2019

**Today's Progress**: Reading about how to work with Volumetric Data like medical images (for example a CT scan) which will use a 5D tensor of shape Batch x Channels x Depth x Height x Width with Pytorch

**Link to code:** [Load volumetric data with Pytorch](code/load_volumetric_data.py)

## Day 97: Jan 04, 2019

**Today's Progress**: Continued learning about Lodash, implemented a function to split the dataset into test and training sets, another function to get the accuracy of the knn and improving the KNN implementation in javascript

**Links:** [KNN with Lodash](code/knn-lodash.js)

## Day 96: Jan 03, 2019

**Today's Progress**: Re-Learning about K-Nearest Neighbor from a different perspective and how to implement a KNN in Javascript with Lodash

**Links:** [KNN with Lodash](code/knn-lodash.js)

## Day 95: Jan 02, 2019

**Today's Progress**: Learning how to generate confusion matrices, and tables of precision an recall scores with scikit-learn to analyze misclassified images to find out the why they are beign misclassified and improve accuracy

## Day 94: Jan 01, 2019

**Today's Progress**: Testing some code with opencv to separate words and lines with handwritten text, still cant overcome the problem of 2 lines overlapping by a char in the top line, which is a common case in real world scenarios, checking a tutorial to implement the whole OCR only with opencv and KNNs

**Links:** [OCR of Hand-written Data using kNN](https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_ml/py_knn/py_knn_opencv/py_knn_opencv.html) | [opencv - cropping handwritten lines (line segmentation)](https://stackoverflow.com/questions/46282691/opencv-cropping-handwritten-lines-line-segmentation/53996835#53996835)

## Day 93: Dec 31, 2018

**Today's Progress**: Learning about Convolutional Neural Network Visualizations, Filters, Heatmaps of class activations, found a nice repo with code to create those visualizations with PyTorch, very important when trying to see why images were misclassified

**Links:** [Pytorch Convolutional Neural Network Visualizations](https://github.com/utkuozbulak/pytorch-cnn-visualizations#gradient-visualization)

## Day 92: Dec 30, 2018

**Today's Progress**: Learning about DarkFlow and Darknet to use YOLO in Python, different ways to create the data annotations, and SIFT (Scale-Invariant Feature Transform)

**Links:** [Official YOLOv3](https://pjreddie.com/darknet/yolo/) | [Start Training YOLOv3 Using Darknet to Detect Custom Objects](https://www.youtube.com/watch?v=TP67icLSt1Y&t=1206s) | [Master Deep Learning Computer Vision™ CNN, SSD, YOLO & GANs](https://www.udemy.com/master-deep-learning-computer-visiontm-cnn-ssd-yolo-gans/) | [SIFT](https://www.youtube.com/watch?v=oKAnOzIu66c)

## Day 91: Dec 29, 2018

**Today's Progress**: Back to learn about YOLO, completed the course on YOLO v3 - Robust Deep Learning Object Detection, they use supervise.ly to annotate datasets and train models with YOLO, now checking how to train YOLO locally on my own.

**Links:** [YOLO v3 - Robust Deep Learning Object Detection in 1 hour](http://ude.my/UC-UXD67VXG)

## Day 90: Dec 28, 2018

**Today's Progress**: Learning about HAAR Cascade Classifiers with openCV, learning more about YOLOv3, added 2 new resources to the index, Homemade Machine Learning and Data Science Cheatsheets

**Links:** [Homemade Machine Learning](https://github.com/trekhleb/homemade-machine-learning) | [Data Science Cheatsheets](https://github.com/george-studenko/ds-cheatsheets)

## Day 89: Dec 27, 2018

**Today's Progress**: I just found while True: learn(), it is a "Simulator of a machine learning specialist", an educational game with visual programming and links to full tutorials on different topics of ML, a fun way to learn Machine learning!

**Links:** [while True: learn()](https://luden.io/wtl/)

## Day 88: Dec 26, 2018

**Today's Progress**: Learning more about HSV color space. H= Hue (Wavelength of the color 0-360), S= Shade of the color (0-100), S= Saturation (intensity of the brightness 0-100) HSV separates luma, or the image intensity, from chroma or the color information, this can be very useful in computer vision

## Day 87: Dec 25, 2018

**Today's Progress**: Wrote my first image classifier with ml5js and MobileNet following the ml5 examples, uploaded it, it is pretty straight forward with ml5, it uses callbacks as it works asynchronously.

**Links:** [ml5js Image classifier](http://georgestudenko.com/ml5js/)

## Day 86: Dec 24, 2018

**Today's Progress**: Starting to learn about ml5js, ml5 is the equivalent of keras for tensorflowjs. Trying to run some tfjs web samples locally, having problems with files in typescript looks like I need to run it with npm. Checking some nice ml5 samples image classifier with mobilenet in real time.

**Links:** [tfjs examples](https://github.com/tensorflow/tfjs-examples/) | [MNIST training and inference with tensorflowJS example](https://storage.googleapis.com/tfjs-examples/mnist/dist/index.html)

## Day 85: Dec 23, 2018

**Today's Progress**: Practicing to export Pytorch models with ONNX, having problems to expot models where the forward function expects more than one param, managed to export the HWR project model, learning how to import it in tensorflowJS with ```tfjs-onnx```.

**Links:** [Code to Export to ONNX](code/Export_Model_to_ONNX.ipynb) | [Tensorflow.js Onnx Runner](https://www.npmjs.com/package/tfjs-onnx)

## Day 84: Dec 22, 2018

**Today's Progress**: Learning about tensorflowjs converter to export a pre-trained model to tfjs, I could only find how to export keras models so far ```tfjs.converters.save_keras_model(model, tfjs_target_dir)```. For pytorch models can be exported with ONNX.

**Link to repository:** [Export Pytorch with ONNX](https://github.com/onnx/tutorials/blob/master/tutorials/PytorchOnnxExport.ipynb) | [Export keras model](https://js.tensorflow.org/tutorials/import-keras.html)

## Day 83: Dec 21, 2018

**Today's Progress**: implemented and compiled a basic model with tensorflowjs, now learning how to train and make predictions with tensorflowjs, will implement a MNIST classifier as soon as I get to learn all the basic syntax of tensorflowjs.

**Link to repository:** [TensorflowJS Basics](https://github.com/george-studenko/tensorflowjs-basics)

## Day 82: Dec 20, 2018

**Today's Progress**: Continued learning about tensorflowjs, variables, tensors operations, memory management with tf.tidy, and creating and compiling models. Enrolled to the course "30 Days of Python" looking forward to start with ml5js soon.

**Thoughts**: ml5.js is a high level interface to TensorFlow.js, so you could compare it with Keras

**Link to repository:** [TensorflowJS Basics](https://github.com/george-studenko/tensorflowjs-basics)

## Day 81: Dec 19, 2018

**Today's Progress**: Writing the predict_topk method to return a top of classes with its probs, researching about tensorflowjs and ml5js for training and deploying ML model in a browser, going through some tutorials about tensorflowjs, importing tensorflowJS from cdn, creating and printing a tensor.

**Thoughts**: Combining lightweight frameworks with javascript is interesting, wondering how powerful this can be, if it works well (which seems to be the case) then this can go a long way and might be worth it spending time in learning such frameworks

**Link to repository:** [TensorflowJS Basics](https://github.com/george-studenko/tensorflowjs-basics)

## Day 80: Dec 18, 2018

**Today's Progress**: Extracting the probability of the recognized char and exploring  different examples to find a reasonable threshold to count the prediction as confident, exploring characters from the train dataset, reading papers on documents segmentation with computer vision.

**Thoughts**: Had to normalize the values of the extracted probabilities to find the probability percentage, thinking about training the model again with a balanced dataset to avoid bias as at the moment there are a lot more of samples for digits than for chars.

## Day 79: Dec 17, 2018

**Today's Progress**: Implementing some code in OpenCVSharo to practice, reading papers on one class classification, reading different approach, I will try to implement mine with a simple approach having a threshold in the probability of a class and maybe combine it with the total of white pixels having a min and max to consider it a char.

**Thoughts**: One class classifiers are used as outlier detectors, seems like an interesting application of deep learning.

**Links:** [Does Your Model Know the Digit 6 Is Not a Cat? A Less Biased Evaluation of "Outlier" Detectors](https://arxiv.org/abs/1809.04729) | [Towards Open Set Deep Networks](https://arxiv.org/abs/1511.06233) | [OpenCVSharp Basics Repository](https://github.com/george-studenko/OpenCVSharp-Basics/tree/master/OpenCVSharp-Basics)

## Day 78: Dec 16, 2018

**Today's Progress**: Added padding to the extacted characters on the HWR project, seems to be helping, wrote a 2nd version of the image pre-processing pipeline that uses otsu  for thresholding, researching how to write a one-class classificator model. Researching how to deploy a model either to web or mobile.

**Thoughts**: The one-class classificator  is something I never thought of, training a model with only positive labels is a bit weird, and challenging to do, trying to figure.

## Day 77: Dec 15, 2018

**Today's Progress**: making the first presentations of the Handwriting Recognition project to our fellows of AI Saturdays Barcelona at DepotLab, getting feedback to improve it for the official presentation at Alpha Telefónica in January 2019, preparing the slides, experimenting with Canny edges and different blur kernels.

**Thoughts**: Got great feedback from my peers today, and looking forward to improve the project and slides for next month.

## Day 76: Dec 14, 2018

**Today's Progress**: continued learning about OpenCVSharp, learning about transformation matrices like the RotationMatrix, the translation matrix to shift an image by x and y pixels on the desired directions ```[1, 0, tx], [0, 1, ty]```. Bitwise operations ```and``` ```or```  ```xor``` and ```not``` and reviewing ```thresholding``` I will be doing some tests with Otsu thresholding soon. Preparing the HWR project for demonstration

**Thoughts**: it is interesting to see how Linear Algebra takes an important role in image processing. There is still a lot of work to be done in the HWR project.

## Day 75: Dec 13, 2018

**Today's Progress**: Checking OpenCV libraries for C# found Emgu and OpenCVSharp, learning about OpenCVSharp, it has quite the exact syntax as the python version. Learning about the basic data structures like ```Mat``` that represents an N-Dimensional dense numerical single channel or multichannel array. More basic types like ```CV_8UC3``` which represents an 8 bit unsigned integer with 3 channels (color image).

**Thoughts**: While learning about openCV in C# I did learn more in depth about how the information is stored and now I understand how we can plot images with matplotlib.pyplot.imshow as well, also learning what are the data types that until today did sound weird like the CV_8UC3 type, learning something from a different perspective always helps.

## Day 74: Dec 12, 2018

**Today's Progress**: Continued with Computer vision lessons on affine transforms ```getAffineTransform``` and not-affine transforms with ```getPerspectiveTransform``` which I might need to implement in the HWR project in the future for rotating pictures when they are not in the right orientation, matching contours by shapes with ```matchShapes``` and approximating contours with ```approxPolyDP``` and ```convexHull```

**Thoughts**: Being able to get the perspective of an image and transform it to put it on the right orientation is crucial for real world data where the pictures are not perfect and might be rotated or on an angle.

## Day 73 : Dec 11, 2018

**Today's Progress**: Managed to implement the 2-parts characters extraction by applying a vertical rectangle kernel, this also fixed problems with I letters when the 3 sections were a bit apart, experimenting with Canny edges, it seems promising but will break the current code as is now since Canny already threshes the images and inverts it so it wont work with the code that I wrote to split image lines.

**Thoughts**: Using version control with Jupyter notebooks can be a hassle specially when checking for changes in code with ```git diff```, today I found this script to make things easier with GIT, pending to implement it.

**Links to work:** [Jupyter Notebooks & GIT Article](http://pascalbugnion.net/blog/ipython-notebooks-and-git.html) | [Jupyter Notebooks & GIT Code](https://gist.github.com/george-studenko/f91f62227e406aa208f4616432dd3a60)

## Day 72 : Dec 10, 2018

**Today's Progress**: Back to image segmentations lessons, will try to use canny edges first before findContours to reduce noise, researching on how to get bounding boxes on letters with 2 parts like an `i` or `j` doing some tests now, at the moment things like 'jijiji' are the worse nightmare of the project

**Thoughts**: I could see it is possible to fix this problem, but still working on it, there is still a lot to learn about image pre-processing and definitely a good pre-processing of the image will do a huge difference in getting more accurate results

## Day 71 : Dec 9, 2018

**Today's Progress**: working all day on the HWR project, realized that when using real images the image preprocessing was not working well so it wont detect any characters, now using adaptiveThreshold with blur and dilation to improve the quality of the extracted chars, testing with real images and seems to be working fine.

**Thoughts**: redefined the whole preprocessing pipeline, now back to the function to detect spaces between words, the results after testing were not good enough

## Day 70 : Dec 8, 2018

**Today's Progress**: wrote a function to detect spaces between words, refactoring some methods in the HWR project, trying to use ```adaptiveThreshold```, trying to apply dilation to images to see if it helps to enhance image quality.

**Thoughts**: when working with real photos lighting can be a problem trying to overcome that problem.

## Day 69 : Dec 7, 2018

**Today's Progress**: Started using feather to store and load data in the HW Recognition project, with numpy it took around 5 minuts to load from csv, with pandas 2 minutes and with feather you can have it in seconds, nice optimization on speed at the cost of more space to store the data.

**Thoughts**: I heard about feather about a month ago watching a video from Jeremy (FastAI), was having memory issues and long loading times to decided I will try it.

**Link to work:** [Feather](content/Feather.md)

## Day 68 : Dec 6, 2018

**Today's Progress**: Working on the HW Recognition project, wrote a function to split the lines of text in an image, a method to encapsulate the whole image pre-processing pipeline, found and fixed a bug on the make_square method, tried to apply gaussian blur on the images to see if it helps, but just made it worse, organizing the whole project code now.

**Thoughts**: To recognize and split lines in an image I found many code snippets but they were too complex, I implemented my own algorithm  which is so far working pretty well.

## Day 67 : Dec 5, 2018

**Today's Progress**: Joined the Humpback Whale Identification Kaggle Competition, downloading, preparing and understanding the dataset, been watching some videos on PyTorch finally understood why we don't need a softmax activation in the last layer of a classifier when using crossentropy loss, I've been a bit confused with that in the past.

**Thoughts**: The challenge on this project is the little amount of images that there are per whale so I guess it is time to go for data augmentation, also while checking the images they are not perfect, some images contains text, some are blured, or they might even contain something else in the background.

**Link to competition:** [Humpback whale identification competition](https://www.kaggle.com/c/humpback-whale-identification)

## Day 66 : Dec 4, 2018

**Today's Progress**: Wrote an algorithm to resize the characters depending on the longest side keeping the aspect ratio, and another function to fill the image with borders to make the image the correct size. Testing different interpolation modes to see the difference in quality

**Thoughts**: I found that for down-sampling an image the recommended interpolation is INTER_AREA but it was too thing in this case at least visually we will have to test how the model reacts to different interpolations when predicting.

## Day 65 : Dec 3, 2018

**Today's Progress**: Working on a function to make all images the same size while keeping the aspect ratio for the Handwriting Recognition project, already have a function to do it but it is not keeping the aspect ratio.

**Thoughts**: OpenCV is giving some errors, probably because the two images I want to add together are not the same shape. Looks like I need to work with a cv2 function called copyMakeBorder

## Day 64 : Dec 2, 2018

**Today's Progress**: Continuing with Fully-Convolutional Neural Networks. Yesterday graduated from the Computer Vision Nanodegree. Working on the handwriting recognition project, pre-processing images to extract characters and find where the next word starts.

**Thoughts**: Pre-processing images to extract chars is interesting, I faced different challenges such as: each text might have a different separation between words so the threshold is not always the same, problems with finding contours twice like in a "o" character where there is an external and internal contour, or the fact that the contours are not sorted so even after extracting the characters the text will not be correct if not sorted in the right order.

**Link to work:**  [Fully Convolutional Neural Networks](content/FCNNs.md)

## Day 63 : Dec 1, 2018

**Today's Progress**: Working on the handwriting recognition project, testing different model architectures so far the best model I could get had a 87.5% accuracy, trying to improve it. Trying to get word boundaries with openCV, checking YOLO to see if I can use it to get the boundaries of words or at least characters. Testing new approaches with lr_scheduler.

**Thoughts**: One of the things I did to try to improve the model was to cut off layers in my model, to my surprise it performs as well as the bigger net, so bigger is not always better.

## Day 62 : Nov 30, 2018

**Today's Progress**: Learning about Fully-Convolutional Neural Networks. Got feedback from the Landmark detection project made the changes an re-submitted it, I was not adding noise to the measurements before.

**Thoughts**: It was a bit challenging to find the right way to add noise to all measurements for omega and xi but I think I finally made it, my final estimated measurements looks correct.

**Link to work:**  [Fully Convolutional Neural Networks](content/FCNNs.md) | [Landmark Detection and Tracking Project](https://github.com/george-studenko/landmark-detection-and-tracking)

## Day 61 : Nov 29, 2018

**Today's Progress**: Finally submitted the Landmark detection and Tracking project, managed to fix the singular matrix problem, the landmarks positions in the matrix were not being calculated correctly, waiting for the project review now.

**Thoughts**: It took me a while to complete this project, I got most of the code done in little time but trying to figure this bug took me a long time, I was on the right track for days, I saw that the landmarks positions were not correct but logically my code did seem correct so it was very hard to find the problem.

**Link to work:**  [Bad Omega constraint matrix](content/bad-constraint-matrix.md) | [Landmark Detection and Tracking Project](https://github.com/george-studenko/landmark-detection-and-tracking)

## Day 60 : Nov 28, 2018

**Today's Progress**: always wondered how to show all values of a numpy array when printing, today I needed to actually inspect all values on a big array and since it always gets truncated I found the setting that will make it print the full set of values ```np.set_printoptions(threshold=np.nan)```. Seeing the full matrix is helping me to debug the problem. Uploaded the project to GitHub. Added [Amazon Machine Learning paths](https://aws.amazon.com/training/learning-paths/machine-learning/) to the resources index above.

**Thoughts**: I got to start indexing more resources, lots of them are being shared everyday need to start gathering them and also keep getting more cheat sheets

**Link to work:** [Landmark Detection and Tracking Project](https://github.com/george-studenko/landmark-detection-and-tracking)

## Day 59 : Nov 27, 2018

**Today's Progress**: back to pencil and paper to get the right index of the constraints in a 2d world, realized that I need to refactor again my code on the Landmark Detection project from 2 matrices in 1D to 1 matrix in 2D to make it work with the helpers methods to check the final status of the code, either that or to refactor all the helper methods which I'm not exactly sure how they work.

## Day 58 : Nov 26, 2018

**Today's Progress**: installing and configuring extensions for Jupyter Notebooks, added [32.81TB of research data](http://academictorrents.com/) to the resources list in the index, trying the variable inspector extension and PixieDebugger with the Slam project. Managed to get over the singular matrix error.

**Link to work:**  [Jupyter notebooks extensions](content/Jupyter-extensions.md) | [Debug Jupyper Notebooks](content/Debug-jupyter.md)

## Day 57 : Nov 25, 2018

**Today's Progress**: having some out of memory errors when trying to make the split for the training set, checking alternatives to make it with PyTorch SubsetRandomSampler. Fixing some problems with the dataset (TensorDataset) by loading data with numpy.loadtxt instead of pandas.read_csv. Trying to fix the singular matrix problem in the Slam project but still working on it.

## Day 56 : Nov 24, 2018

**Today's Progress**: we made progress in the Handwriting Recognition project, investigating the EMNIST dataset, creating the classes index, training, validation and test sets, learning about PyTorch TensorDataset to load data directly from numpy arrays. Preparing all the methods to train and validate the model.

## Day 55 : Nov 23, 2018

**Today's Progress**: executing the first tests with the Slam method, getting a ```LinAlgError: Singular matrix error```, it also happened to me before while working on the lessons exercises, but cant figure what am i doing wrong, it happens when I try to do the inverse of the matrix ```np.linalg.inv(np.matrix(omega_x)) * xi_x``` trying to figure, looks like it is quite normal in robot localization,  reading some papers that talk about ways to avoid it.

## Day 54 : Nov 22, 2018

**Today's Progress**: refactoring the functions on the slam project to make them work with 2 matrices instead of 1, working on the update function to set and update all values in the constraint matrix according to the world observations, some samples below, each one is one robot movement:

```
0 - Measurement:  [0, 14.041141265751651, -14.681929570535768]
0 - Measurement:  [1, 43.8790296430265, 49.47795079424027]
0 - Measurement:  [2, 32.58593495634767, 8.733395657214311]
0 - Measurement:  [3, 41.391463862883604, -42.81033082282804]
0 - Measurement:  [4, 48.16958980823874, 30.77201714559928]
0 - Motion:       [19.797473966744985, 2.8390182345407036]

1 - Measurement:  [0, 35.972246403584755, -13.737033304213115]
1 - Measurement:  [1, 24.25932330061779, 49.886769607894166]
1 - Measurement:  [3, 17.858095906012498, -40.78942619313321]
1 - Motion:       [19.797473966744985, 2.8390182345407036]
```

**Thoughts**: Refactoring it with multiple matrices it is making more sense now, I was still getting confused with the world grid and the constraint matrix, they are not the same but somehow in my mind I was mixing the concepts.

## Day 53 : Nov 21, 2018

**Today's Progress**: Implementing the Slam function to update the constraint matrix, working with x and y on the same matrix is a bit confusing for the constraint matrix, I'm changing the implementation to use 2 matrices instead one for x and one for y for both omega and xi. Checking the robot generated data to understand better how to calculate the constraint matrix.

**Thoughts**: better to refactor the code earlier before this gets more complex and confusing. Trying to get a hold of the mentors of the project for 3 days now, but looks like nobody is around.

## Day 52 : Nov 20, 2018

**Today's Progress**: Working on the Landmark detection and tracking project, implementing the constraint initialization function, at first I was confused thinking in terms of the world and not in terms of the constraints. Watching the SLAM lessons for the second time it is making more sense now. To get the solutions of all equations (the mu vector) 
mu = omega<sup>-1</sup> * xi

**Thoughts**: the omega matrix follows a pattern to add the numbers, and the constraints are simple to implement if you look at it like simple lineal equations and then clear the value from both sides of the equation so in one equation the value will be negative and in the other the value will be positive for example: ```x0 - x1 = -5``` and ```x1 - x0 = 5```

## Day 51 : Nov 19, 2018

**Today's Progress**: Working on the Landmark detection and tracking project, implemented the Sense function to sense landmarks in the 2D world with added uncertainty for dx and dy, now trying to implement the constraint matrix. Been also experimenting a bit with the EMNIST dataset, in pytorch looks like there are some bugs as not all classes are in the classes dictionary as I can only see the MNIST ones (0-9)

**Thoughts**: Implementing the constraint matrix in 2D is quite confusing and still trying to understand it

## Day 50 : Nov 18, 2018

**Today's Progress**: Half way there. Today I've been going through all my notes during the first half and re-organizing them, created an [index](https://github.com/george-studenko/100_Days_of_ML_Code#100daysofmlcode-index) of all content so far. I've also been doing some tests on the QuickDraw model to see if I can improve it.

**Thoughts**: Half way there, I'm learning a lot of things doing this challenge.

## Day 49 : Nov 17, 2018

**Today's Progress**: We finally sent the first submission to Kaggle to see how is our model doing, we are still getting a low Map@3 score of 0.770 we are now working on improving the model. Planning now our next tam project, to recognize handwritten text, we will use the Extended MNIST dataset to train a model and we will try to use it later to digitize hand written notes like with OCR but with hand written text, preparing the new notebook with PyTorch and downloading the EMNIST datasets for training and test. Reading about OOP in Python.

**Thoughts**: Looking forward to improve the quickdraw model and work on the next project!

## Day 48 : Nov 16, 2018

**Today's Progress**: Finished the lessons on SLAM, added sensor measurements after motion to the matrix calculations and will start now working on the Landmark Detection and Tracking with SLAM project. The project consists in implementing SLAM (Simultaneous Localization and Mapping) for a 2 dimensional world. combining robot sensor measurements and movement to create a map of an environment from only sensor and motion data gathered by a robot, over time.

**Thoughts**: I'm starting to get how SLAM works after doing some coding but I will probably have to review the whole section on SLAM again to fully get it, also the project seems simple but will be challenging and I'm sure I will learn a lot while doing it.

## Day 47 : Nov 15, 2018

**Today's Progress**: Continuing with SLAM today I implemented the constraint matrix in python ```omega``` and ```xi```. To solve for all the poses and landmark positions, we can use linear algebra; all the positional values are in the vector ```mu``` which can be calculated as a ```product of the inverse of omega times xi```. The model kept training, I've restarted the training many times so far, and while the loss keeps getting lower the accuracy is increasing very slow and I'm not sure it will increase much more but will leave it training at least one more day to see if it gets better.

**Thoughts**: I got a LinAlgError: Singular matrix error today trying to find the inverse of a matrix that apparently has a determinant of zero, this is the definition of a Singular matrix (one for which an inverse does not exist).
```
[1, -1, 0]  
[1, 0, -1]   
[0, 1, -1]  
```

## Day 46 : Nov 14, 2018

**Today's Progress**: Loading different checkpoints of the trained model and checking how it behaves when trying to resume training, when using adam as the optimizer it is important to set the same parameters again, also learned that it is possible to save the state_dict of the adam optimizer to load it again later. Continuing with SLAM.

**Thoughts**:  It was quite a relief to be able to get the last model checkpoint back to the same state .

## Day 45 : Nov 13, 2018

**Today's Progress**: Learning about SLAM for 2 dimensional worlds, a way to track the location of a robot in the world in real-time and identify the locations of landmarks such as buildings, trees, rocks, and other world features. Also learning about constraint matrices

**Thoughts**:  Still trying to get a better understanding about how to implement the constraint matrices.

## Day 44 : Nov 12, 2018

**Today's Progress**: Continued reviewing linear algebra, coding vectors and matrices operations, while I would normally use Numpy for such tasks it is a good review to implement all of the vector and matrices operations with plain Python from scratch.

**Thoughts**: Implementing dot product, matrix multiplications, and transpose is not hard but still glad we have numpy to do that.

**Link to work:** [Vectors](code/Linear_Algebra/1_vector_coding.ipynb) | [Matrices](code/Linear_Algebra/2_matrices_in_python.ipynb) | [Matrix Addition](code/Linear_Algebra/3_matrix_addition.ipynb) | [Matrix Multiplication](code/Linear_Algebra/4_matrix_multiplication.ipynb) | [Guide to Mathematical Notation](code/Linear_Algebra/guide_to_mathematical_notation.ipynb)

## Day 43 : Nov 11, 2018

**Today's Progress**: Training the model now with the Map@3 score as well to see how well it can do in the competition. Learning about Kalman filter and transformation of states. Review of Linear Algebra to apply it to Kalman Filters states.

**Thoughts**: Was interesting to learn how to simplify formulas to make them easier to read.

## Day 42 : Nov 10, 2018

**Today's Progress**: Training the quickdraw model, still on the 1st epoch, with around 55% accuracy so far, understanding better the relation between epoch, step and batch size and different types of batch sizes. Continued to learn about object tracking and localization, this time storing state in state vectors (and operations with those vectors)

**Thoughts**: It was good to review and understand these basic concepts better

**Link to work:**  [Batch size](content/Batch-size.md)

## Day 41 : Nov 9, 2018

**Today's Progress**: Prepared 2 datasets for the QuickDraw Kaggle competition, and been  testing it, the architecture performs very well with MNIST 97.5% with 2 epochs, started doing tests with a toy dataset of quickdraw with 2 epochs got around 20% accuracy on the validation set, now will train it on a bigger dataset to see how it performs. Also learning about GPU states P0-P12

**Thoughts**: having big fully connected layers in a CNN network do add a lot of memory, with images at 256px x 256px it was taking long to load the network itself and I was running out of CUDA memory, now trying to find the perfect batch size by monitoring GPU Memory with nvidia-smi

**Link to work:**  [GPU States](content/GPU.md)

## Day 40 : Nov 8, 2018

**Today's Progress**: Learning more about Kalman Filters, state, motion and kinematics, Kinematics is the study of the motion of objects. For predicting state we can use Object Oriented Programming and Linear Algebra (vectors and matrices) to keep track of the states. For the kaggle competition I made sure to use only one channel in the image since they are grayscale.

**Thoughts**: We now have all 3 sets of images to work with, Train, Dev and Test sets, and we will start training the model soon!

## Day 39 : Nov 7, 2018

**Today's Progress**: Learning about Gaussian Variance, and shifting the mean, combining gaussians to get a new improved mean. Implemented the getitem method for the dataset to return  extra information about the images, preparing the train method.

**Thoughts**: It is interesting to see how combining two gaussians can improve the certainty (similar to bayes!)

## Day 38 : Nov 6, 2018

**Today's Progress**: Working on the Computer Vision Nanodegree, did  mini-project to implement a 2d histogram for localization. Starting to learn about Kalman Filters. Finally got the first set of training images for the Kaggle competition, will start with 3400000 images and double or triple it as needed. Will continue to calculate the layers shapes of the network for the Kaggle competition.

**Thoughts**: Kalman filters is a different  approach for tracking, differnet from Monte Carlo, which uses histograms with discrete values. Kalman filters use continues values with Gaussian.

## Day 37 : Nov 5, 2018

**Today's Progress**: Working on the Computer Vision Nanodegree, learning about localization with Sense (measuring sensor data) and Move. Working on the Kaggle competition, working on the model architecture and forward function, fixing minor problems with too many images to process and planning the next steps.

**Thoughts**: With over 40 million images and more than 600 GBs of images for the competition we were having problems to generate all the images. Decided to take an iterative approach.

**Link to work:**  [Localization](content/Localization.md) | [Robot Localization](code/Robot_Localization.ipynb)

## Day 36 : Nov 4, 2018

**Today's Progress**: Continued on the Computer Vision Nanodegree, learning about probability distributions and implementing simple functions for robot localization applying probability distributions. Working on the data-loader class for the Kaggle competition, basic project structure and writing images to to disk, dealing with memory problems due to huge csv files and data to process.

**Thoughts**: Data preparation is known to be what takes most of the time!, hopefully we will be able to continue on the competition soon (when finally getting all the data processed to start working on the network.

**Link to work:**  [Probability](content/Probability.md) | [Quick, Draw! Doodle Recognition Challenge](https://www.kaggle.com/c/quickdraw-doodle-recognition)

## Day 35 : Nov 3, 2018

**Today's Progress** : I've been testing the image captioning model with the coco dataset and then implementing my own class to test the model with my own images and it is performing pretty well. So today I  submitted the Images Captioning project and got it reviewed. Then kept going with object tracking and localization this time reviewing probability concepts. And today @ AI Saturdays 4 of us created a team to work on a Kaggle competition ,lets see how it goes!

**Thoughts**: The Image captioning project was very challenging and it was quite curious that it actually took me longer to find the right hyperparameters  and of course the training took many hours as well. While I've been a Kaggle member for a while I never participated on a competition so far, it is going to be a new experience (and hopefully a great one!)

**Link to work:**  [Probability](content/Probability.md) | [Quick, Draw! Doodle Recognition Challenge](https://www.kaggle.com/c/quickdraw-doodle-recognition)

## Day 34 : Nov 2, 2018

**Today's Progress** : reading about the PyTorch Anatomy, it turnos out that PyTorch itself is just and Python API that sits on top of many other components that are written in C++ and C, and that is why  it is so powerfull,  which is nice to know.  Kept trying different hyperparameters for my Image Captioning project and getting now a smaller loss and lower Perplexity score as well, which is great! Started to learn about Motion in Computer Vision, with Optical Flow and motion vectors.

**Thoughts**: I should be able to deliver the Image Captioning project soon, now starting to learn about Object Tracking and Localization.

**Link to work:** [PyTorch Anatomy](https://freecontent.manning.com/pytorch-crash-course-part-1/)

## Day 33 : Nov 1, 2018

**Today's Progress** : I exported all my jupyter notebook code files to .py files and running them in PyCharm, debugging so I can see how the variables are being set and see what is going on with my sample function, I also found that with PyCharm is also possible to run "cells" just like if it was a Jupyter notebook and keep state in between them. Started reading [Deep Learning with PyTorch](https://www.manning.com/books/deep-learning-with-pytorch)

**Thoughts**: After downloading and installing all the images and captions for the cocodatase and the pycocotools  I got it to work on my local, lets see if debugging helps troubleshooting it, so far I can see that the outputs of my LSTM cells are not good.

## Day 32 : Oct 31, 2018

**Today's Progress** : Today I found a very interesting trick for __jupyter notebooks__ and that is how to save the notebook with a line of code, so imagine you are running a long process and want to save it after it finishes while being unattended just add this to the cell below and run it, once the cell above finishes it will save the notebook. Training the image caption model again with new parameters, increasing hidden units on the lstm and increasing batch size as well.

``` 
%%javascript
IPython.notebook.save_notebook()
```

**Thoughts**: The first iterations with the new parameters seems to be showing better results so far, lets see if it improves.

```
Epoch [1/3], Step [100/6471], Loss: 4.6383, Perplexity: 103.3649
Epoch [1/3], Step [200/6471], Loss: 4.3353, Perplexity: 76.35148
Epoch [1/3], Step [300/6471], Loss: 3.9353, Perplexity: 51.17674
Epoch [1/3], Step [400/6471], Loss: 3.7730, Perplexity: 43.5094
Epoch [1/3], Step [500/6471], Loss: 4.1964, Perplexity: 66.4450
Epoch [1/3], Step [600/6471], Loss: 3.6579, Perplexity: 38.7780
Epoch [1/3], Step [700/6471], Loss: 3.5322, Perplexity: 34.1984
Epoch [1/3], Step [800/6471], Loss: 3.4114, Perplexity: 30.3068
Epoch [1/3], Step [900/6471], Loss: 3.4160, Perplexity: 30.44853
Epoch [1/3], Step [1000/6471], Loss: 3.4420, Perplexity: 31.2493
Epoch [1/3], Step [1027/6471], Loss: 3.3547, Perplexity: 28.6383
```

## Day 31 : Oct 30, 2018

**Today's Progress** : Still working on the Image caption project, implementing a function for inference, since this is a bit of a special network I have to implement something different than the ```forward``` function.

**Thoughts**: I would normally use the forward function to make inference, but this time seems to be a bit different since it should only take the image features as input and not the captions this time.

## Day 30 : Oct 29, 2018

**Today's Progress** : Kept working on the Image caption project, tunning hyperparameters for the network and training the first epochs on the network, I've also been reading more about activation functions and Adam optimizer.

**Thoughts**: Finally started to train the network! With a good GPU it might take around 12 hours to train, I've been training for more than 2 hours now and it has covered 66% of the 1st epoch.

```
Epoch [1/3], Step [1/12942], Loss: 8.1496, Perplexity: 3462.1592
Epoch [1/3], Step [2/12942], Loss: 8.0925, Perplexity: 3269.7607
Epoch [1/3], Step [3/12942], Loss: 8.1527, Perplexity: 3472.8072
Epoch [1/3], Step [4/12942], Loss: 8.1414, Perplexity: 3433.8146
Epoch [1/3], Step [8100/12942], Loss: 4.5722, Perplexity: 96.75917
Epoch [1/3], Step [8200/12942], Loss: 4.3483, Perplexity: 77.34675
Epoch [1/3], Step [8300/12942], Loss: 4.5639, Perplexity: 95.96166
Epoch [1/3], Step [8400/12942], Loss: 4.5306, Perplexity: 92.81236
Epoch [1/3], Step [8500/12942], Loss: 4.5245, Perplexity: 92.25309
Epoch [1/3], Step [8595/12942], Loss: 4.5756, Perplexity: 97.08521
Epoch [3/3], Step [12939/12942], Loss: 3.9346, Perplexity: 51.1442
Epoch [3/3], Step [12940/12942], Loss: 3.7537, Perplexity: 42.6778
Epoch [3/3], Step [12941/12942], Loss: 3.5892, Perplexity: 36.2059
Epoch [3/3], Step [12942/12942], Loss: 3.9569, Perplexity: 52.2962
```

## Day 29 : Oct 28, 2018

**Today's Progress** : Trying to understand how to feed my RNN with the Image Feature maps and Captions at the same time, so far what I understood is that first I need to feed my LSTM with the feature maps and then feed the next LSTM with the captions, but it is still giving me some errors, working on a solution.  
Update: Apparently I need to concatenate the embeddings with the features unsqueezed like this before sending it to the LSTM cell ```torch.cat((features.unsqueeze(1), embeddings), 1)```

**Thoughts**: I'm getting some CUDNN errors at the moment, even with fewer batches something might be wrong in my network architecture. Now trying to figure out why concatenating the inputs is the way to go.

## Day 28 : Oct 27, 2018

**Today's Progress** : Went through the Lesson 5 of FAST.AI about collaborative filtering, learning more about PyTorch modules and what pytorch squeeze does and how to properly use reshape, and got some practice with scikit learn

**Thoughts**: While I've been working with PyTorch for a while there still a lot to learn, I will get in more depth about PyTorch in the coming days.

## Day 27 : Oct 26, 2018

**Today's Progress** : Was reading the paper [Show and Tell: A Neural Image Caption Generator](https://arxiv.org/pdf/1411.4555.pdf) to understand better the project I'm working on. I was also working on the decoder network.

**Thoughts**: Still a bit lost with this project, it is not going to be easy, but nothing is when it comes to deep learning!

**Link to work:** [Show and Tell: A Neural Image Caption Generator Paper](https://arxiv.org/pdf/1411.4555.pdf)

## Day 26 : Oct 25, 2018

**Today's Progress** : Kept working on the Image Captioning project, starting to define the CNN and RNN models that I will start using, for the CNN encoder I will be using the pre-trained ResNet-50 as for the RNN I'm still working on it but I will be using LSTMs cells. I also found an interesting Pandas Dataframes cheat sheet that I'm adding to the collection, check the link below.

**Thoughts**: I need more practice with RNNs, I feel more confident with CNNs at the moment but I should be able to pull it out.

**Link to work:** [Pandas DataFrame Cheatsheet](cheatsheets/Pandas-DataFrame-Notes.pdf)

## Day 25 : Oct 24, 2018

**Today's Progress** : Working on the Image Captioning project that uses the [Microsoft COCO Dataset](http://cocodataset.org/#home), pre-processing images tokenizing captions.

**Thoughts**: I'm just starting on the project, it is the first time I will use different architectures on the same project and feeding a network the output of the previous network, will be an interesting project to do.

**Link to work:** [Image Captioning Project Notes](content/Image-Captioning.md)

## Day 24 : Oct 23, 2018

**Today's Progress** : Learning about Image Captioning and Tokenization with NLTK, exploring the [Microsoft COCO Dataset](http://cocodataset.org/#home) and starting to work on the Image Captioning project.

**Thoughts**: NLTK is an interesting package that has some extra functionalities to tokenize words, sentences or even tweets. The Image captioning project seems quite interesting and will help me to learn  more about RNNs, LSTMs and CNNs, and not only that but to combine different networks to get even more interesting stuff.

**Link to work:** [NLTK](content/NLTK.md)

## Day 23 : Oct 22, 2018

**Today's Progress** : Learning about more about attention mechanisms, a way in which the neural network focuses its attention on the most relevant parts of the input data. Now studying about Image Captioning.

**Thoughts**: Attention mechanisms are quite interesting, shifting attention to different parts of an image or different words in a text is something I wouldn't have expected to be able to do with a neural network

**Link to work:** [Attention mechanisms](content/Attention-mechanisms.md)

## Day 22 : Oct 21, 2018

**Today's Progress** : Learning about embeddings and starting to learn about attention mechanisms

**Thoughts**: Embeddings are a bit hard to understand at the beginning, but they are just some hidden units that represent a categorical value that is translated to a vector of continuous values and are considered just another parameter of the network with their own weights so they will be trained and will take part of back-propagation as all other nodes in the network.

**Link to work:** [Embeddings](content/Embeddings.md)

## Day 21 : Oct 20, 2018

**Today's Progress** : Learning about Model Hyper parameters, a bit more on dropout and continued learning about embeddings.

**Thoughts**: It is good to have a rough idea on starting values for model hyperparameters, got to read more about embeddings while I've been using embeddings in the past I didn't really understand what they are until now, but still would like to get into more details to understand it better.

**Link to work:** [Hyperparameters](content/Hyperparameters.md) | [Dropout](content/Dropout.md)

## Day 20 : Oct 19, 2018

**Today's Progress** : Yet more on LSTMs, added code examples in the LSTM notes file, and back to basics on hyperparameters, today I studied about Optimizer Hyperparameters, next I will also add information about Model Hyperparameters

**Thoughts**: Hyperparameters are all about intuition and experience, there is no one fit for all so you will have to experiment most of the time, however you will be able find the best hyperparameters quicker and quicker over time!

**Link to work:** [Hyperparameters](content/Hyperparameters.md) | [LSTM cells](content/LSTM.md)

## Day 19 : Oct 18, 2018

**Today's Progress** : Day four of the Fast.AI marathon working on lesson 4, still in progress!, continued learning about LSTMs and how to implement basic LSTM cells and get the outputs and hidden state with PyTorch.

**Thoughts**: LSTMs are a bit complex to understand at first, I need a bit more of practice with them to really understand them, I already understand the concept and how the cell works so that is a good start.

**Link to work:** [LSTM cells](content/LSTM.md)

## Day 18 : Oct 17, 2018

**Today's Progress** : Day three of the Fast.AI marathon completed the video of lesson 3!, interesting things on different activation functions, like softmax for non-binary classifications and sigmoid for multiple classes classification (predicting multiple correct outputs), also checking where and how to get the datasets. Also finally got the Kaggle Fastai kernels to work! Also learning more about what is inside an LSTM cell which in short are 4 gates: Learn, Forget, Remember and Use gates.

**Thoughts**: It is interesting to lean that activation functions have their own "personality"

**Link to work:** [LSTM cells](content/LSTM.md) | [Downloading Datasets](content/Downloading-Datasets.md) | [FastAI](content/fast-ai.md)

## Day 17 : Oct 16, 2018

**Today's Progress** : Day two of the Fast.AI marathon completed the video of lesson 2!, there are some really cool stuff in this lesson, unfortunately for people just starting on deep learning it won't be much appreciated (if this was for real their 2nd lesson on DL) like for example some neat tricks to find great Learning Rates like the Cosine Annealing with warm restarts and cycles. I then kept working on the computer vision program, I've been learning about more about Recurrent Neural Networks (RNNs) Backpropagation Through Time, and also Long Short-Term Memory cells (LSTMs)

**Thoughts**: BPTT is quite complex, too many terms to work with, but well it is just backpropagation but with many more terms involved. I'm learning a lot of neat tricks with Jeremy @ fastai, and now starting to understand all the complexity below the LSTMs cells.

**Link to work:** [Learning Rates](content/Learning-rate.md) | [RNNs](content/RNN.md)

## Day 16 : Oct 15, 2018

**Today's Progress** : Day one of the Fast.AI marathon completed the video of lesson 1!, and also worked on the computer vision program, I've been learning about Recurrent Neural Networks (RNNs), how the folded and unfolded models are represented, back to basics, Feedforward and Backpropagation and started learning about Backpropagation Through Time (BPTT)

**Thoughts**: Recurrent neural networks will add memory to a neural network, so each time we feed an input we will also feed the activation of the previous hidden layer.

**Link to work:** [RNNs](content/RNN.md)

## Day 15 : Oct 14, 2018

**Today's Progress** : I've been learning about YOLO (You Only Look Once) algorithm, I've also been documenting how to troubleshoot Conda environments problems with Jupyter notebooks and trying to make OpenCV work properly in Ubuntu.

**Thoughts**: YOLO seems to be amazing, you do need a GPU for real time recognition, with CPU it can take 2 to 3 seconds to analyze an image and return the outputs, looking forward to start using it!

**Link to work:** [YOLO](content/YOLO.md) | [Conda](content/Conda.md)

## Day 14 : Oct 13, 2018

**Today's Progress** : Back to basics day!, on training, dev and testing sets, recognizing and fixing high variance and high bias problems and a bit of regularization and weight decay.

**Thoughts** : Having a clear idea of how to split your data is important. Knowing what to do when you have problems of high bias and/or high variance is important, there are suggested steps to follow to fix those!

**Link to work:** [Training Dev and Testing sets](content/Training-Dev-and-Testing-Sets.md) | [High bias & high variance](content/High-bias-and-high-variance.md) | [Regularization](content/Regularization.md)

## Day 13 : Oct 12, 2018

**Today's Progress** : I submitted the Facial Keypoints Project and it was approved, now stating to learn about more advanced features to work with multiple objects in a scene, with algorithms like R-CNNs

**Thoughts** : I had lots of challenging moments while working on the project which were great to learn, I will try to implement more things on the notebook 4 of the project.

**Link to work:** [Weighted Loss Functions](content/Weighted-Loss-Functions.md) | [Region Proposal Algorithms](content/Region-Proposal-Algorithms.md) | [Facial Keypoints Detector Project](https://github.com/george-studenko/Facia-Keypoints-Detector)

## Day 12 : Oct 11, 2018

**Today's Progress** : Kept working on the Facial Keypoints Project, I changed the code to make it run on my local GPU, and that did help a lot, I could test many different architectures very quick, and now I finally have a decent network. Was also working on show the Feature maps visualizations, by applying the kernels that the network learned during training to the images we can see exactly what the network sees, check the notes for a screen shot

**Thoughts** : If you can run it on a GPU, do it, it is just hundreds of times faster, your time is worth it. it is really cool to see the feature maps that the network created and used to learn!

**Link to work:** [Visualizing Feature Maps](content/Visualizing-Feature-Maps.md) | [Filtered Images](resources/kernels.png)

## Day 11 : Oct 10, 2018

**Today's Progress** : Kept trying different architectures on the Facial Keypoints Project, Loss functions, and optimizers, I changed my input images from 96x96 to 224x224 and the training time increased by 5 or 10 at least, also the loss increased so it is performing worse than before.

**Thoughts** : I got to keep working on it, will try to make it run on GPU to be able to experiment more.

## Day 10 : Oct 9, 2018

**Today's Progress** : Working on the Facial Keypoints Project, reading a paper about Naimishnet to have an idea of the implemented architecture, implemented my architecture although I'm still experimenting with it.

**Thoughts** : As always there is a lot of experimentation when it comes to the architecture and hyperparameters tunning, so I'm still testing different sets of layers and activation functions, loss functions and other hyperparameters.

**Link to work:** [Day 10 notes](content/Day-10.md) | [Architecture](code/cv/models.py)

## Day 9 : Oct 8, 2018

**Today's Progress** : I've been working on the CV nanodegree, learning more about ways to improve a network with Dropout and Momentum and also how to decide what layers to use when developing a network (hint: it is all about intuitions).

**Thoughts** : Momentum seems to be a good technique to use when local minimas might be a problem, and dropout will help a lot to make all nodes of a network better and avoid having some nodes that might be doing all the job opn their own. While deciding on what layers to include is all about intuitions there are some things that you can try out to see if the model improves check the list in the notes of day 9.

**Link to work:** [Defining a network structure](content/Defining-network-structure.md) | [Momentum](content/Momentum.md) | [Dropout](content/Dropout.md)

## Day 8 : Oct 7, 2018

**Today's Progress** : I've been working on the CV nanodegree, although I didn't advance much I took time to really understand how the shape of layers change after each convolutional layer and after each pooling layer, since that is one of the main confusions I had with CNNs for a long time.

**Thoughts** : it is pretty simple to understand once you do it a couple of times, I have also been testing CNNs architectures adding more layers and finishing with Convolutions or pooling layers before the last fully connected layer and when trying that I figured out that when having a Convolutional layer before the fully connected the accuracy was of 4.6% but if instead I used a max pooling layer before the FC layer the accuracy went up to 10%.

**Link to work:** [Layer Shapes](content/Layer-shapes.md)

## Day 7 : Oct 6, 2018

**Today's Progress** : Working on the Fast.AI Deeplearning course on data augmentation and ways to find the best learning rate

**Thoughts** : It's interesting to realize that data augmentation is not always beneficial and to understand when it is, also it is interesting to see that are some crucial concepts like the lr_find function that is based on the part of a not really known paper, that is just great to find the optimal learning rate.

**Link to work:** [Learning rates](content/Learning-rate.md) | [Data Augmentation](content/Data-augmentation.md)

## Day 6 : Oct 5, 2018

**Today's Progress** : learning about CNNs types of layers and how to define the network and feedforward with PyTorch

**Thoughts** : Still a bit confused when it comes to define the actual size (shape) of the network after a couple of layers.

**Link to work:** [CNNs](content/CNN.md)

## Day 5 : Oct 4, 2018

**Today's Progress** : Progress on the Computer Vision ND, learning about ORB and HOG algorithms, and starting with Convolutional Neural Networks

**Thoughts** : ORB is easier to understand than HOG, both serve a different purpose, I've some previous experience with CNNs but I was struggling with it, this time I'm getting a better grab of it.

**Link to work:** [Computer Vision notes](content/Computer-Vision.md)

## Day 4 : Oct 3, 2018

**Today's Progress** : I decided to go a bit less technical today and research about computer vision jobs, it is a very interesting field of study and it can be applied to so many things.

**Thoughts** : From medicine to self-driving cars, it is just amazing and inspiring at the same time the things that can be accomplished with computer vision.

**Link to work:** [Jobs in Computer Vision](content/Jobs-in-Computer-Vision.md)

## Day 3 : Oct 2, 2018

**Today's Progress** : I finished the Types of Features and Image Segmentation material of the Computer Vision Nanodegree and going back to Tensorflow after many months with the Google ML Crash course

**Thoughts** : Looks like detecting corners is a quite useful technique in Computer Vision, also image contouring seems to be a great way to detect images. K-means clustering can group part of the image together to help with image segmentation! I have learned about high features in Tensorflow, I'm not sure if those are new features or I just didn't get the chance to use them before.

**Link to work:** [Computer Vision](content/Computer-Vision.md) | [Contour detection and features](https://github.com/george-studenko/100_Days_of_ML_Code/blob/master/code/cv/Contour%20detection%20and%20features.ipynb)

## Day 2 : Oct 1, 2018
0
**Today's Progress** : Finished the lessons: Image Representation and Classification, Convolutional Filters and Edge Detection from the computer Vision Nanodegree, only 3 lessons to go to start with the first project: Facial Keypoint Detection.

**Thoughts** : Edge detection can be quite useful to detect images, Hough transformations are also quite useful to detect lines that correspond to the same object, lots of parameters to adjust and tune. Bluring images before applying Canny edge detection is quite useful to get rid of unwanted noise in images.

**Link to work:** [Finding Edges and Custom Kernels](code/cv/Finding%20Edges%20and%20Custom%20Kernels.ipynb) | [Hough Lines](code/cv/Hough_lines.ipynb) | [Detecting edges live with webcam repo](https://github.com/george-studenko/Live-Sketch-with-Computer-Vision)

## Day 1 : Sep 30 , 2018

**Today's Progress** : I've started to work on the Machine Learning Crash Course with TensorFlow APIs Google's fast-paced, practical introduction to machine learning today.
  
**Thoughts** : This is a good beginners course, useful to refresh some concepts and get to practice tensorflow.

**Link to course:** [Machine Learning Crash Course](https://developers.google.com/machine-learning/crash-course/ml-intro)

**Link to work:** [Machine Learning Concepts](content/Machine-learning-concepts.md) | [Intro to Pandas Lab](code/Intro_to_pandas.ipynb)

## Day 0 : Sep 29 , 2018
 
**Today's Progress** : I have joined and attended [AI Saturdays](https://nurture.ai/ai-saturdays) in Barcelona today, I completed a couple of chapters on the Computer Vision Nanodegree I'm working on right now. I've been working on creating masks to do some image edition with CV2 (OpenCV). Learned about color spaces (RBG, HSI and HSL) and worked on a simple image classifier, creating a basic ML pipeline to load images, pre-process images and predict and image label.

**Thoughts** : Getting to work and know people who is interested in AI, ML and DL is great to share knowledge and encourage each other, a great way to learn. Computer vision is a daunting part of AI, but with the right material, teachers and projects I think I will manage to pull it over, got time until the end of December 2018 to get it done.

**Link to course:** [Computer Vision Nanodegree](https://eu.udacity.com/course/computer-vision-nanodegree--nd891) you can find a [Free Preview here](https://www2.udacity.com/course/ud891-preview)

100_Days_of_ML_Code

100_Days_of_ML_Code Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

100_Days_of_ML_Code

100_Days_of_ML_Code Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README