19. Proximal Netwon Method

이번 장에서는 Proximal Newton Method과 Proximal quasi-Newton method, Projected Newton method에 대해서 살펴보도록 하겠다.

참고 논문

Proximal Newton method:

• J. Friedman and T. Hastie and R. Tibshirani (2009), “Regularization paths for generalized linear models via coordinate descent”
• C.J. Hsiesh and M.A. Sustik and I. Dhillon and P. Ravikumar (2011), “Sparse inverse covariance matrix estimation using quadratic approximation”
• M. Patriksson (1998), “Cost approximation: a unified framework of descent algorithms for nonlinear programs”
• J. Lee and Y. Sun and M. Saunders (2014), “Proximal Newton-type methods for minimizing composite functions”
• P. Tseng and S. Yun (2009), “A coordinate gradient descent method for nonsmooth separable minimization”

Projected Newton method:

• A. Barbero and S. Sra (2011), “Fast Newton-type methods for total variation regularization”
• D. Bertsekas (1982), “Projected Newton methods for optimization problems with simple constraints”
• D. Kim and S. Sra. and I. Dhillon (2010), “Tackling box-constrained optimization via a new projected quasi-Newton approach”
• M. Schmidt and D. Kim and S. Sra (2011), “Projected Newton-type methods in machine learning”

• 19-01 Proximal Newton method
• 19-01-01 Reminder - proximal gradient descent
• 19-01-02 Proximal Newton method
• 19-01-03 Scaled proximal map
• 19-02 Backtracking line search
• 19-03 When would we use proximal Newton?
• 19-04 Convergence analysis
• 19-05 Notable examples
• 19-06 Proximal quasi-Newton methods
• 19-07 Projected Newton method