• Instructor: Steven Holtzen, s.holtzen@northeastern.edu.
  Feel free to email me with any questions, but please include CS7480 in the subject line.
• Session: Fall 2021
• Student Office Hours: Tuesday 11AM – 12PM EST (Also available by appointment)
• Meeting Time: Tues/Fri 3:25PM – 5:05PM
• Location: International Village 022

Description

Probabilistic programming languages (PPLs) use the syntax and semantics of programming languages to define probabilistic models. PPLs enable a diverse audience – data scientists, systems designers, medical doctors, etc. – to design and reason about probabilistic systems. PPLs are becoming a central topic in artificial intelligence and programming languages research, with increasing interest from industry and academia in designing and applying PPLs.

The goal of this course is to introduce the core ideas of probabilistic programming languages: probabilistic inference, semantics, program analysis, language design, and applications. The course will consist of formal lectures as well as research presentations by students surveying the modern landscape of developments in probabilistic programming. There will be a minor project involving implementing a probabilistic programming language, and a self-directed term project that aims to deeply explore some of the core ideas of probabilistic programming.

Prerequisites

The target audience for this course is advanced master’s students and PhD. students in programming languages, artificial intelligence, and machine learning. There are no formal prerequisites, but students are expected to be comfortable programming and have mathematical maturity. In particular, students should be familiar with the notion of mathematical proof and comfortable programming in at least one major programming language.

Organization & Grading

The course will consist in part of (1) lectures delivered by the instructor; (2) student presentations on research projects and research papers; (3) a minor project; (4) a self-directed term-project. Grades will follow a standard letter-grade curve, and points will be apportioned as:

• 20%: Course Participation and Presentations
  • Participation is an important part of this class. Please inform the instructor if you need to miss class; no excuse is necessary.
• 30%: Minor Project (Link to syllabus)
• 50%: Final Project (Link to syllabus)

Each assignment will have an accompanying syllabus with a description of how the grade is apportioned within it.

Schedule

This schedule is tentative and is subject to change and re-ordering in topics.

Paper Presentations

An important part of the class is getting up to date and familiar with reading the modern literature in probabilistic programming. The reading will be chosen at least 1 week in advance and posted on the course website. Before each session, everyone is expected to fill out the following reading questionnaire:

• What is the main motivation for the paper? What problem is it solving?
• What is the paper’s main technical contribution? What tools does it use?
• Does the paper live up to its claims? Why or why not?
• How well does the paper situate itself in the broader literature? Did it miss anything important?
• Was the paper easy to understand? Which parts were confusing or well-explained?
• Did you have any questions while reading?

Please make sure your questionnaire is accessible to you during class. Each paper will have a discussant whose job it is to choose which paper to read. Everyone is expected to be a discussant at least once. In addition to the usual reading questionnaire, discussants should fill out the following discussant questionnaire:

• Why did you choose this paper?
• Where does this paper sit in the research landscape? Is it a significant milestone, or a more of an incremental improvement? Justify your answer, this can be subjective!
• What is one thing you would do to improve or extend upon the ideas in this paper? This can be an extension, a technical clarification, etc.
• Is there any follow-up work to the paper, or very closely related work that we should know about?

Please be prepared to speak in front of the class for approximately 15-20 minutes about your answers to these questions. If you have something else interesting that you want to bring up, please feel free to do so: this is a chance for you to be creative.

Selecting readings. A list of suggested papers can be found at this link.

Additional Materials

This course does not have a required textbook, but it does by necessity rely on a number of topics including programming languages, artificial intelligence, computational complexity, and machine learning. Some useful materials include:

• N. D. Goodman and A. Stuhlmueller. The Design and Implementation of Probabilistic Programming Languages.
  • Basic content on probabilistic programs, introduction to WebPPL, many examples.
• Adnan Darwiche. Modeling and Reasoning with Bayesian Networks.
  • Introduction to graphical models, foundations of tractable inference.
• Fabrizio Riguzzi. Foundations of Probabilistic Logic Programming: Languages, Semantics, Inference and Learning
  • Good introduction to probabilistic logic programming.
• Pierce, Benjamin C., and C. Benjamin. Types and programming languages. MIT press, 2002.
  • Foundations of programming languages.

Related courses taught at other universities:

• https://web.cs.ucla.edu/~guyvdb/teaching-service/cs267a/2018f/
• http://danroy.org/teaching/2015/STA4516/
• http://probcomp.csail.mit.edu/9.S915/
• https://github.com/hongseok-yang/probprog17
• https://www.cs.ubc.ca/~fwood/CS532W-539W/