Back to Teaching
ISE 530 · 4 Units

Optimization Methods for Analytics

Fundamentals of linear, integer, quadratic, and nonlinear programming for analytics and data science problems.

Course Description

This course covers the most common classes of optimization problems, their theory, and basic methods for finding their numerical solutions. The topics provide a set of fundamental tools for the understanding and solution of a variety of data-analytic problems including those arising from operations, statistics, machine learning, financial engineering, and data-driven modeling.

Learning Objectives

  • Describe common optimization problem classes (linear, quadratic, non-linear, and integer)
  • Identify their mathematical properties and basic algorithms for solving them
  • Develop a set of tools to be applied to a variety of problems

Topics Covered

  • Optimization and Modeling
    • What is optimization? Modeling and basic concepts
  • Linear Programming
    • Modeling; standard LP form; simplex method and its variations
    • Duality theory for LP
  • Unconstrained Optimization
    • Root finding, convexity, optimality conditions
    • Descent methods, coordinate descent, Newton’s method
  • Quadratic Programming
    • Least-squares regression, portfolio selection, LASSO
    • Algorithms and implementation
  • Nonlinear Programming: Foundation
    • Logistic regression, SVM; convex optimization
    • Duality: weak/strong duality, KKT conditions, constraint qualification
  • Nonlinear Programming: Algorithms
    • Projected gradient descent, coordinate descent, penalty methods
    • Applications in machine learning and data analytics
  • Integer Programming
    • IP examples: knapsack, set covering/packing, logical relations
    • Basic algorithms for IP

Textbooks

  • R. W. Cottle and M. N. Thapa, Linear and Nonlinear Optimization, Springer New York, 2017.
  • G. Cornuejols and R. Tütüncü, Optimization Methods in Finance, Cambridge University Press. (Free online)

Modeling language: Some homework assignments require programming with AMPL and Python. See the AMPL book and the NEOS server.

Prerequisites

This course makes heavy use of linear algebra and matrix operations and assumes comfort with college-level mathematical reasoning typical of an engineering curriculum. Programming in Python is required for homework assignments involving algorithm implementation.

Assignments & Policies

  • Biweekly (sometimes weekly) homework assignments submitted via Blackboard (PDF only); two lowest scores dropped.
  • Late submissions are not accepted under any circumstances.
  • Discussion with peers is encouraged; each student must submit their own work.
  • Technical questions should be posted to the course Slack channel; grade-related queries should be emailed to the TA.