Nonlinear Optimization; G63.2031/G22.2750
Spring 2008; Mondays, 1:25-3:15pm; WWH Room 1302
Instructor: Frank E. Curtis
- Office: WWH 524
- Office Hours: Wednesdays, 1:00-3:00pm, but e-mail anytime for an appointment
- E-mail: see homepage
Course Description: Nonlinear optimization, through its theory and practical algorithms and software, is an important tool for scientists, engineers, and business people for the design and analysis of physical, social, and economic systems. Mathematically speaking, the goal is to minimize some function subject to a series of constraints. This course presents some of the central ideas behind the formulation, design, and analysis of nonlinear optimization algorithms. Topics include the fundamentals of unconstrained and constrained programming, line search and trust region techniques, and practical Newton, Sequential Quadratic Programming, and interior-point methods for optimization.
Prerequisites: A solid foundation in undergraduate-level linear algebra, real analysis, and multivariable calculus will be beneficial. Previous experience with Matlab will also be helpful.
Grading: This course will be graded as a regular course. Final grade will be based equally on homework, project, and oral exam grades.
Textbook: Numerical Optimization, Nocedal & Wright, Spring-Verlag, 2nd Ed. See website for information about ordering and errata.
Software: The homeworks will require the understanding and use of Matlab. A student version of Matlab is available for purchase here, but you will almost certainly be more inclined to use the version of Matlab available on the Courant network. Alternatively, please feel free to try Octave, though I cannot guarantee that all the code I will provide will be compatible with it. Other resources/software related to optimization are NEOS, AMPL, and CUTEr.
Mailing list: Please make sure you receive e-mails from me related to the course by e-mailing me your address (if I don't have it already).
Lectures: General outline of the lecture schedule:
- 1/28: Overview and introduction (Chapters 1 and 2)
- 2/4: Line search methods (Chapter 3)
- 2/11: Trust region methods (Chapter 4)
- 2/18: NO CLASS (President's Day)
- 2/25: Conjugate gradient techniques (Chapters 5 and 7)
- 3/3: Quasi-Newton techniques (Chapters 6-7)
- 3/10: Least-squares problems and theory of equality constrained optimization (Chapters 10-12)
- 3/17: NO CLASS (Spring break)
- 3/24: Theory of inequality constrained optimization (Chapter 12)
- 3/31: Linear programming: Interior-point methods (Chapter 14)
- 4/7: Quadratic programming (Chapter 16)
- 4/14: Fundamentals for algorithms for nonlinear constrained optimization (Chapter 15)
- 4/21: Penalty, augmented Lagrangian, and sequential quadratic programming (Chapters 17 and 18)
- 4/28: Nonlinear programming: Interior-point methods (Chapter 19)
- 5/5: Overview of contemporary optimization software
Homework: Listed as they become available:
- Homework #1: HW 1, due on Feb. 13 at 5pm.
- Homework #2: HW 2, due on Feb. 20 at 5pm.
- Homework #3: HW 3, due on Mar. 5 at 5pm.
- Homework #4: HW 4, due on Mar. 12 at 5pm.
- Homework #5: HW 5, due on Mar. 26 at 5pm.
- Homework #6: HW 6, due on Apr. 9 at 5pm.
- Homework #7: HW 7, due on Apr. 16 at 5pm.
- Homework #8: HW 8, due on Apr. 30 at 5pm.
- Homework #9: HW 9, due on May 7 at 5pm.
Project: The project description, problems, and write-up description for the project are posted below. The project is due on May 8 at 5pm (late submissions will NOT be accepted). Please submit a write-up along with a zipped file of all the output from your codes (see the write-up description below for more details).
- Project Description
- Problems: quadratic.m, regressor.m, loghairy.m, genhumps.m, sinquad.m
- Write-up Description
Oral Exam: The final will consist of an oral exam to be administered in my office. Each exam will last 30 minutes and will cover topics seen in the lectures and homeworks throughout the course. A schedule for the exams will be discussed towards the end of the semester.