EE61012: Convex Optimization in Control and Signal Processing

Welcome to this PG level subject.

Logistics of Grading

  • End Semester: 50
  • Mid Semester: 30
  • Class Tests: 10
  • Homework, Class Participation: 10

Course Content (Tentative)

Here is a basic overview of the topics that are planned to be covered.

  1. Background (1 week)
  2. Convex Sets and Convex Functions (2 weeks)
  3. Convex Optimization Problems (1 week)
  4. Lagrangian Duality (1 week)
  5. Necessary and Sufficient Optimality Conditions (1 week)
  6. Regression, Classification and Clustering Problems (1 week)
  7. ML Estimation, Hypothesis Testing, Optimal Detection (1 week)
  8. Algorithms for Convex Optimization, First Order Methods, Primal-Dual Algorithms, ADMM (2 weeks)
  9. LMIs and SDP Duality (1 week)
  10. Application of LMIs in Linear Control (1 week)
  11. Constrained Optimal Control, MPC, Application in System Identification (1 week)

Software Packages

There are several dedicated environments that enable us to easily encode and solve convex optimization problems. We will demonstrate some examples in class using PYOMO/YALMIP.

  1. CVXPY.
  2. YALMIP.
  3. PYOMO.
  4. MOSEK.
  5. Casadi.
  6. JuliaOPT.

Textbooks

There is no single textbook for this subject. We will discuss a variety of topics from different books. The first reference will be followed to a large extent. You are encouraged to refer the other texts below depending on your interests.

  1. Convex Optimization (freely available to download) by Boyd and Vandenberghe.
  2. Algorithms for Convex Optimization by Nisheeth K. Vishnoi.
  3. Optimization III: Convex and Nonlinear Programming by Ben-Tal and Nemirovski. Lecture Notes.

The following resources are excellent overviews of algorithmic aspects of convex optimization that are relevant in ML.

  1. Brief overview of SGD by Praneeth Netrapalli.
  2. SGD Tutorial by Prof. Ketan Rajawat. .
  3. Potential function approach to proving convergence results.