Newton's method

Concept

$$x_{k+1}=x_k-H_f^{-1}(x_k)\nabla f(x_k)$$

where $H$ is Hessian matrix and $\nabla$ is gradient.
For the $H^{-1}$ here, we can solve it instead of inverting like we did in chapter 5:

$$H_f(x_k)s_k=-\nabla f(x_k)$$

and

$$x_{k+1}=x_k+s_k$$

Cost

it has quadratic convergence rate.

Variants

• Gauss Newton Method