15-01-03 Log barrier calculus

Log barrier function의 gradient와 hessian은 다음과 같다.

$$\begin{array}{r}\varphi (x)=-\sum _{i=1}^m\log (-h_i(x))\end{array}$$

Gradient :

$$\begin{array}{r}\mathrm{\nabla }\varphi (x)=-\sum _{i=1}^m\frac{1}{h_i(x)}\mathrm{\nabla }{h}_i(x)\end{array}$$

Hessian :

$$\begin{array}{r}{\mathrm{\nabla }}^2\varphi (x)=\sum _{i=1}^m\frac{1}{h_i(x{)}^2}\mathrm{\nabla }{h}_i(x)\mathrm{\nabla }{h}_i(x{)}^T-\sum _{i=1}^m\frac{1}{h_i(x)}{\mathrm{\nabla }}^2{h}_i(x)\end{array}$$

Example : $\varphi (x)=-\sum _{i=1}^n\log (x_i)$

Barrier function $\varphi (x)=-\sum _{i=1}^n\log (-x_i)$에 대한 gradient와 hessian을 구해보면 다음과 같은 결과를 얻을 수 있다.

$$\begin{array}{r}\varphi (x)=-\sum _{i=1}^n\log (x_i)\end{array}$$ 따라서, $h_i(x)=-x_i$이고 $x_i\ge 0$이다.

Gradient :

$$\mathrm{\nabla }\varphi (x)=-\left[\begin{array}{c}1/x_1\\ ⋮\\ 1/x_n\\ \end{array}\right]=-X^{-1}\mathbb{1},X=\text{diag}(x)$$

Hessian :

$${\mathrm{\nabla }}^2\varphi (x)=\left[\begin{array}{ccc}1/x_1^2& \cdots & \\ ⋮& \ddots & ⋮\\ & \cdots & 1/x_n^2\\ \end{array}\right]=X^{-2}$$