KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#developer-tools #codex #typescript #ai-agents #ai-agent #cli #claude-code #open-source #deepseek-harness #dsh-plugin #python #mcp

AdvBayesLearnCourse

★ 68 Open GitHub ↗

Course material for the PhD course in Advanced Bayesian Learning

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Advanced Bayesian Learning - PhD course, 8 credits

### Course information

**The typical participant** is a PhD student in Statistics or related fields (Mathematical Statistics, Engineering Science, Quantitative Finance, Computer Science, ...). The participants are expected to have taken a basic course in Bayesian methods, for example [Bayesian Learning](https://github.com/mattiasvillani/BayesLearnCourse) at Stockholm University, and to have some experience with programming.

**Examination and Grades**: The course is graded Pass or Fail. Examination is through individual reports on distributed problems for each topic. Many of the problems will require computer implementations of Bayesian learning algorithms.

**Course organization**
The course is organized in four topics, each containing four lecture hours. Course participants will spend most of their study time by solving the problem sets for each topic on their own computers without supervision.

All lectures are given physically at Campus Albano at Stockholm University. Lecture rooms will be announced later on.

Welcome!

[**Mattias Villani**](https://www.mattiasvillani.com/)  
Professor of Statistics, Stockholm University

---

### Topic 1 - Gaussian processes regression and classification

Reading:  [Gaussian Processes for Machine Learning](http://www.gaussianprocess.org/gpml/chapters/RW.pdf) - Chapters 1, 2.1-2.5, 3.1-3.4, 3.7, 4.1-4.3. Alt: [Bayesian Learning](https://github.com/mattiasvillani/BayesianLearningBook/raw/main/pdf/BayesBook.pdf) Chapter 15

Code: [GPML for Matlab](http://mlg.eng.cam.ac.uk/carl/gpml/) | [GPy for Python](https://sheffieldml.github.io/GPy/) | [Gausspr in R](https://rdrr.io/cran/kernlab/man/gausspr.html) | [Gaussianprocesses.jl in Julia](https://github.com/STOR-i/GaussianProcesses.jl) | [GPyTorch - GPs in PyTorch](https://gpytorch.ai/)

Other material: [Visualize GP kernels](https://smlbook.org/GP/)

**Lecture 1 - April 16, hours 10-12** in [Lärosal 13](https://www.su.se/download/18.7b9fb02419936f8648810abc/1758613821031/Undervisningssalar,%20Albano.pdf)\
[slides](/Slides/ABL1.pdf)  
**Lecture 2 - April 16, hours 13-15** in [Lärosal 29](https://www.su.se/download/18.7b9fb02419936f8648810abc/1758613821031/Undervisningssalar,%20Albano.pdf)\
[slides](/Slides/ABL2.pdf)

**Lab Topic 1**  
[Problems](/Labs/Lab1.pdf) | [Lidar data](/Labs/LidarData.dat)

---

### Topic 2 - Mixture models and Bayesian nonparametrics

Reading: [Bayesian Data Analysis](http://www.stat.columbia.edu/~gelman/book/BDA3.pdf) - Chapter 23 or [Bayesian Learning](https://github.com/mattiasvillani/BayesianLearningBook/raw/main/pdf/BayesBook.pdf) Chapter 13 and 16 | [The Neal (2000) article on MCMC for Dirichlet Process Mixtures](https://www.cs.princeton.edu/courses/archive/fall11/cos597C/reading/Neal2000a.pdf)

Widgets: [Dirichlet distribution](https://observablehq.com/@mattiasvillani/dirichlet-distribution) | [Bayes for multinomial data](https://observablehq.com/@mattiasvillani/multinomial-dirichlet) | [mixture of normals](https://observablehq.com/@mattiasvillani/normal-mixture) | [mixture of Poissons](https://observablehq.com/@mattiasvillani/mixture-of-poissons)

**Lecture 3 - April 29, hours 10-12**  in [Lärosal 28](https://www.su.se/download/18.7b9fb02419936f8648810abc/1758613821031/Undervisningssalar,%20Albano.pdf)\
[slides](/Slides/ABL3.pdf)  
**Lecture 4 - April 29, hours 13-15** in  in [Lärosal 26](https://www.su.se/download/18.7b9fb02419936f8648810abc/1758613821031/Undervisningssalar,%20Albano.pdf)\
[slides](/Slides/ABL4.pdf) | [derivation marginal Gibbs](/Notes/MarginalGibbsDerivation.pdf)

**Lab Topic 2**  
[Problems](/Labs/Lab2_v2024.pdf) | [Galaxy data](/Labs/GalaxyData.dat)

---

### Topic 3 -  Variational inference

Reading: [Blei et al JASA](https://amstat.tandfonline.com/doi/abs/10.1080/01621459.2017.1285773#.XraDPXUzaLI) | [Tran's VI Notes](/Material/VBnotesMNT.pdf) \
Other material: [Kullback-Leibler widget](https://observablehq.com/@mattiasvillani/kl-divergence-continuous) | 
[Talk about a paper that uses VI in Stan](https://github.com/mattiasvillani/Talks/raw/master/VillaniStatSUMarch2024.pdf) | 
[Natural gradients](https://andrewcharlesjones.github.io/journal/natural-gradients.html)\
Autodiff:

- [RTMB - autodiff in R](https://cran.r-project.org/web/packages/RTMB/) and [toy example](Code/RTMBExample.R)
- [JAX - autodiff in Python](https://docs.jax.dev/en/latest/notebooks/thinking_in_jax.html) and [toy example](Code/JAXExample.py)
- [ForwardDiff in Julia](https://github.com/JuliaDiff/ForwardDiff.jl) and [toy example](Code/JuliaForwardDiffExample.jl)

**Lecture 5 - May 13, hours 10-12** in [Lärosal 27](https://www.su.se/download/18.7b9fb02419936f8648810abc/1758613821031/Undervisningssalar,%20Albano.pdf)\
[slides](/Slides/ABL5.pdf)  
**Lecture 6 - May 13, hours 13-15** in [Lärosal 13](https://www.su.se/download/18.7b9fb02419936f8648810abc/1758613821031/Undervisningssalar,%20Albano.pdf)\
[slides](/Slides/ABL6.pdf)

**Lab Topic 3**  
[Problems](/Labs/Lab3.pdf) | [Time series data](/Labs/timeseries.csv)

---

### Topic 4 - Bayesian regularization and variable selection

Reading: Sections 12.2-12.5 and 14.6 of [Bayesian Learning book](https://github.com/mattiasvillani/BayesianLearningBook/raw/main/pdf/BayesBook.pdf) | [Handbook chapter on variable selection](https://bpb-us-e1.wpmucdn.com/blogs.rice.edu/dist/0/5714/files/2022/06/book_chapter.pdf) | [Article on Bayesian regularization](https://wires.onlinelibrary.wiley.com/doi/full/10.1002/wics.1463?casa_token=TRIRvJPG7fAAAAAA%3AErexRs49LFd9QFMR5ZlGJ9RyWs9r7Q_Fc5Hird767zNVPf2CRfKC5O91y2isnLAGJub1yArfEt-RfJhH)

**Lecture 7 - May 25, hours 10-12** in [TBA](https://www.su.se/download/18.7b9fb02419936f8648810abc/1758613821031/Undervisningssalar,%20Albano.pdf)\
[slides](/Slides/ABL7.pdf)  
**Lecture 8 - May 25, hours 13-15** in [TBA](https://www.su.se/download/18.7b9fb02419936f8648810abc/1758613821031/Undervisningssalar,%20Albano.pdf)\
[slides](/Slides/ABL8.pdf)

**Lab Topic 4**  
[Problems](/Labs/Lab4.pdf)

AdvBayesLearnCourse

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README