Rémi Flamary

Professional website

Optimization for Machine Learning M1


  • Introduction to numerical optimization [PDF]
    • Optimization problem formulation and principles
    • Properties of optimization problems
    • Machine learning as an optimization problem
  • Constrained Optimization and Standard Optimization problems [PDF]
    • Constraints, Lagrangian and KKT
    • Linear Program (LP)
    • Quadratic Program (QP)
    • Other Classical problems (MIP,QCQP,SOCP,SDP)
  • Smooth Optimization [PDF]
    • Gradient descent
    • Newton, quasi-Newton and Limited memory
  • Non-smooth Optimization [PDF]
    • Proximal operator and proximal methods
    • Conditional gradient
  • Conclusion
    • Other approaches
    • Optimization problem decision tree
    • References and toolboxes

Practical sessions

Practical sessions will require a working Python environnement with te libraries Numpy/Scipy and Matplotlib installed. You can get such an environnement for Windows/Linux/MacOSX on Anaconda.

Here is a list of nice references for the Pythoon :