CSCI 618: Trustworthy Machine Learning from an Optimization Lens

Instructors: Meisam Razaviyayn, Vatsal Sharan

Basic Information

Course Description and Objectives

Optimization techniques lie at the heart of how models are trained and developed. In this course, we will explore modern considerations such as privacy, robustness and fairness, particularly from the standpoint of optimization techniques. We will both discuss recent research work on formalizing these societal requirements, and algorithmic solutions for obtaining them. Optimization-based approaches such as differentially private optimization, minimax and constrained optimization are particularly useful toolboxes for these problems, and will be explored in this context.

Recommended Preparation

Machine learning knowledge (at the level of CSCI 567, CSCI 467, or ISE 529 is sufficient) using Python. Basic optimization knowledge, basic probability and linear algebra concepts. Mathematical maturity to read research papers.

Syllabus and Materials

The following is a tentative schedule. We will post lecture notes and assignments here. Additional related reading for all lectures will be posted on Ed Discussion after the lecture.
Lecture Topics Homework
1, Mon 08/24 Course introduction, ML basics Lecture slides, Optimization Colab
2, Wed 08/26 Adversarial examples, finding adversarial examples, adversarial training Lecture slides
3, Mon 08/31 Certified robustness, randomized smoothing Lecture slides
4, Wed 09/02 Data poisoning
Paper presentations:
(1) Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples
(2) Robustness Meets Algorithms 		Lecture slides
Mon 09/07 No class — Labor Day
5, Wed 09/09 Undetectable backdoors, tradeoffs in adversarial robustness
Paper presentations:
(1) Deliberative Alignment: Reasoning Enables Safer Language Models (briefly cover Adversarial Reasoning at Jailbreaking Time)
(2) Constitutional AI: Harmlessness from AI Feedback
6, Mon 09/14 Robust and non-robust features, distributional robustness
Paper presentations:
(1) Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training
(2) Discrimination in the Age of Algorithms 		HW1
7, Wed 09/16 Introduction to algorithmic fairness
Paper presentations:
(1) Goal Misgeneralization in Deep Reinforcement Learning
(2) Chapter 1 of AI Snake Oil
8, Mon 09/21 Fairness notions in classification, individual fairness
Paper presentations:
(1) Fairness-in-the-Workflow: How Machine Learning Practitioners at Big Tech Companies Approach Fairness in Recommender Systems
9, Wed 09/23 Group fairness, case study of fairness notions
Paper presentations:
(1) Performative Prediction
(2) Performative Prediction: Past and Future
10, Mon 09/28 Inherent tradeoffs between fairness notions
Paper presentations:
(1) Explicitly unbiased large language models still form biased associations
11, Wed 09/30 Individual fairness via uncertainty quantification, multicalibration
Paper presentations:
(1) Fairness through Difference Awareness: Measuring Desired Group Discrimination in LLMs
(2) Uncertainty Quantification in LLM Agents: Foundations, Emerging Challenges, and Opportunities
12, Mon 10/05 Review of iteration complexity analysis: smooth convex and strongly convex optimization
13, Wed 10/07 Review of iteration complexity analysis: nonconvex optimization Project proposal due
14, Mon 10/12 Privacy attacks and data privacy threats
15, Wed 10/14 Membership inference attacks
16, Mon 10/19 Differential privacy: definition and intuition
17, Wed 10/21 Basic properties of differential privacy HW1 due
18, Mon 10/26 Differential privacy mechanisms
19, Wed 10/28 Properties of differential privacy
20, Mon 11/02 DP optimization: output perturbation and objective perturbation
21, Wed 11/04 DP optimization: exponential mechanism
22, Mon 11/09 DP optimization: DP-SGD
Wed 11/11 No class — Veterans Day
23, Mon 11/16 Project presentations
24, Wed 11/18 Project presentations
25, Mon 11/23 Project presentations
Wed 11/25 No class — Thanksgiving recess
26, Mon 11/30 Project presentations
27, Wed 12/02 Project presentations HW2 due

Requirements and Grading

  1. 2 homeworks worth 10% each (20% total). Homeworks should be written in LaTeX and submitted via Gradescope. There will be an oral evaulation for the homeworks.
  2. Mini-homeworks worth 15% of the grade. You should read presented papers before class so that you can contribute and get the most out of the presentation and discussion. Part of the course grade is also based on this via mini-homeworks. We will drop your two lowest mini-homework score for the final grade. Submit responses through the mini-homework form.
  3. Class participation will be worth 5% of the grade.
  4. Each student will be required to present a paper in class, which will be worth 20% of the grade. See the paper presentation guide and grading rubric for preparation advice, sign-up sheets, and requirements.
  5. Scribing one lecture in Overleaf will be worth 5% of the grade.
  6. The research components will be a project proposal (5%), a project presentation (15%), and a project final report (15%). An overview of the requirements is given below; detailed instructions will be discussed later. The project will be in groups of two students.
    • The goal of the project is to give you experience in research on topics in trustworthy ML. You are free to pursue a purely theoretical project, a purely empirical project, or some combination of these. You can discuss project ideas with the instructors.
    • The project proposal is meant to finalize your project topic, and will be a short 1 page report.
    • The project presentations will be held in class on 11/16, 11/18, 11/23, 11/30, and 12/2. Your project need not be complete by this stage, but you should have made reasonable progress.
    • The project final report has to be written in Latex and should be 8-9 pages long, excluding references. Part of the report should discuss the related research landscape, and the rest of it should cover your original work. Please use the LaTex template based on the NeurIPS format. The project should be written in a way such that most students in the class should be able to understand the report.
    • You are free to use LLMs/generative AI tools to help you with your research and project in any way. Students still bear full responsibility for the contents of the report, including content generated by LLMs that could be construed as plagiarism or scientific misconduct (e.g., fabrication of facts, hallucinations).