Hello, and welcome to my page for my “Deep Probabilistic Models” mini course. This page contains resources from my week-long series on Deep Probabilistic Models (and an introduction to Pyro) at the Australian Mathematical Sciences Institute (AMSI) Winter School in 2021. This is an annual Winter School attended by graduate students, early career researchers, and people from industry.

The content was quite well-received so I have made all the lecture slide and tutorial materials available here on this page for the general public who may also find it interesting. The material consists of approximately eight 50 minute lectures and two one-hour tutorials (practicals).  (with examples implemented in PyTorch and Pyro).

My goal for the course was present the material in a “unified” framework and a manner that suits people who know neural network basics but not necessarily a lot of probabilistic machine learning / statistics, using some minimal coded examples (in PyTorch and Pyro) throughout to help aid teaching.

The course is comprised of four parts:

I.  Flow-Based Models

II. Generative Adversarial Networks and Stochastic Backpropagation

III.  Graphical Models,  Deep Latent Variable Models, and Variational Learning

IV. Amortized Variational Inference, Variational Autoencoders, and an Introduction to Pyro

The lecture recordings are only available for AMSI Winter School participants. However, I hope anyone else finds the slides and tutorial worksheets sufficiently self-contained they can benefit from them.

I had a lot of fun teaching the course, and given the effort that went in to preparing the course, I am also not opposed to giving it (or a reduced/extended form of it) again in the future. Feel free to email robert.salomone (at) qut.edu.au if you would like to discuss such opportunities (or maybe just if you spot the inevitable occasional typo!)

Enjoy!

Rob Salomone

 

Lectures

Note that if you click on the embedded pdf files below, you can use your left and right arrows to change slides. There is also a fullscreen button if you would like a larger view.

Part I: Flow-Based  Models

 

Part II: Generative Adversarial Networks and Stochastic Backpropagation

Part III: Graphical Models,  Deep Latent Variable Models, and Variational Learning

Part IV: Amortized Variational Inference, Variational Autoencoders, and an Introduction to Pyro

 

Please note that the second part of this lecture (introduction to Pyro) is contained in the second tutorial sheet below.

Tutorials

Please note that the Jupyter Notebook files for the two tutorials below are available on my Github.

 

Tutorial 1: Flows in Pyro and GANs

Tutorial 2: Variational Learning and an Introduction to Pyro

 

Additional Content: Workshop on Automatic Differentiation

See also this link for lecture slides from a 3-hour workshop on the fundamentals of Automatic Differentiation that I gave in 2019.

The material discussed is often called “advanced” aspects of automatic differentiation. However, in my opinion, they are better viewed as fundamentals as they will enrich ones understanding of even doing basic things and help one better understand how to do things efficiently.

This was not part of the Winter School course but is related as automatic differentiation is typically used to implement all of the models discussed (so it is is good to know the fundamentals of how it works!).