KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#developer-tools #typescript #claude-code #codex #dsh #dsh-plugin #ai-agent #ai-agents #deepseek-harness #agent-skills #macos #cli

DeepLearning

★ 108 Open GitHub ↗

Lecture slides for study about "Deep Learning" written by Ian Goodfellow, Yoshua Bengio and Aaron Courville

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

Lecture Slides for Deeplearning book
===================================
This repo contains lecture slides for [Deeplearning book](http://www.deeplearningbook.org/). This project is maintained by [_InfoLab_ @ DGIST](https://infolab.dgist.ac.kr/) (_Large-scale Deep Learning Team_), and have been made for _InfoSeminar_. It is freely available only if the source is marked.

> The slides contain additional materials which have not detailed in [the book](http://www.deeplearningbook.org/).<br/>
> Also, some materials in the book have been omitted.
<br/>

## Maintained by
**[_InfoLab_](https://infolab.dgist.ac.kr/)** @ DGIST(Daegu Gyeongbuk Institute of Science & Technology)<br/>

## Coverage
This repo covers Chapter 5 to 20 in the book.

## Credits
Name | Chapters
------------ | -------------
[Jinwook Kim](https://infolab.dgist.ac.kr/~bm010515/) | 5-1, 8, 16, 17
[Heechul Lim](https://infolab.dgist.ac.kr/~hclim/) | 9-2, 11, 12, 15, 19
[Hyun-Lim Yang](https://infolab.dgist.ac.kr/~hlyang/) | 5-2, 7-2, 10-2
[Keonwoo Noh](https://infolab.dgist.ac.kr/~kwnoh/) | 7-1, 9-1, 14, 20
[Eunjeong Yi](https://infolab.dgist.ac.kr/~ejyi/) | 6, 10-1, 13, 18

## The contents of lecture slides

### Part 1. Applied Math and Machine Learning Basics
> ### 5-1. Machine Learning Basics
> - Learning algorithms
> - Capacity, overfitting and underfitting
> - Hyperparameters and validation sets
> - Estimators, bias and variance
> - Maximum likelihood estimation

> ### 5-2. Machine Learning Basics
> - Bayesian statistics
> - Supervised learning algorithms
> - Unsupervised learning algorithms
> - Stochastic gradient descent

### Part 2. Deep Networks: Modern Practices
> ### 6. Deep Feedforward Networks
> - Example: Learning XOR
> - Gradient-Based Learning
> - Hidden Units
> - Architecture Design
> - Back-Propagation and Other Differentiation

> ### 7-1 Regularization for Deep Learning
> - Parameter Norm Penalties
> - Norm Penalties as Constrained Optimization
> - Regularization and Under-Constrained Problems
> - Dataset Augmentation
> - Noise Robustness
> - Semi-Supervised Learning
> - Multitask Learning
> - Early stopping

> ### 7-2 Regularization for Deep Learning
> - Parameter Tying and Parameter Sharing
> - Bagging and Other Ensemble Methods
> - Dropout
> - Adversarial Training

> ### 8 Optimization for Training Deep Models
> - How Learning Differs from Pure Optimization
> - Challenges in Neural Networks 
> - Basic Algorithms
> - Algorithms with Adaptive Learning Rates
> - Parameter Initialization Strategies
> - Approximate Second-order Methods
> - Optimization Strategies and Meta-algorithms

> ### 9-1 Convolutional Networks
> - The Convolution Operation
> - Motivation
> - Pooling
> - Convolution and Pooling as an Infinitely Strong Prior
> - Variants of the Basic Convolution Function

> ### 9-2 Convolutional Networks
> - Structured Outputs
> - Data Types
> - Efficient Convolution Algorithms
> - Random or Unsupervised Features
> - The Neuroscientific Basis for Convolutional Networks

> ### 10-1 Sequence modeling: Recurrent and Recursive Nets
> - Unfolding Computational Graphs
> - Recurrent Neural Networks
> - Bidirectional RNNs
> - Encoder-Decoder Sequence-to-Sequence Architectures
> - Deep Recurrent Networks
> - Recursive Neural Networks

> ### 10-2 Sequence modeling: Recurrent and Recursive Nets
> - The challenge of Long-term 
> - Echo State Networks
> - Leaky Units and Other strategies for Multiple Time Scales
> - The Long Short-Term Memory and Other Gated RNNs
> - Optimization for Long-Term Dependencies
> - Explicit Memory

> ### 11, 12 Practical Methodology and Applications
> - Performance Metrics
> - Default Baseline Models
> - Determining Whether to Gather More Data
> - Selecting Hyperparameters
> - Debugging Strategies
> - Computer Vision

### Part 3. Deep Learning Research
> ### 13 Linear Factor Models
> - Probabilistic PCA and Factor Analysis
> - Independent Component Analysis
> - Sparse Coding

> ### 14 Autoencoders
> - Introduction
> - Stochastic Encoders and Decoders
> - Regularized autoencoders
> - Representational Power, Layer Size and Depth
> 
> ### 15 Representation Learning
> - Unsupervised pre-training
> - Introduction of supervised(SL) and unsupervised learning(UL)
> - Representation
> - Clustering
> - K-means
> - Gaussian Mixture Model
> - EM algorithm
> - Practical example
> 
> ### 16, 17 Structured Probabilistic Models for Deep Learning and Monte Carlo Methods
> - The Challenge of Unstructured Modeling
> - Using Graphs to Describe Model Structure
> - Sampling from Graphical Models
> - The Deep Learning Approach to Structured Probabilistic Models
> - Sampling and Monte Carlo Methods
> - Markov Chain Monte Carlo Methods
> - Gibbs Sampling
> 
> ### 18 Confronting the Partition Function
> - The Log-Likelihood Gradient
> - Stochastic Maximum Likelihood and Contrastive Divergence
> - Estimating the Partition Function
> 
> ### 19 Approximate Inference
> - Approximation
> - Maximum Likelihood(MLE) and Maximum A Posteriori(MAP)
> - Inference
> - Taxonomy of deep generative models
> - KL-Divergence
> - Variational Inference
> 
> ### 20 Deep Generative Models
> - Generative models
> - Boltzmann Machines
> - Restricted Boltzmann Machines
> - Deep Belief Networks

***
License: [CC-BY](https://creativecommons.org/licenses/by/3.0/)

DeepLearning

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README