Nonlinear Least Squares

Concept

Given data $(t_i,b_i)$
define residual function:

$$r_i=b_i-f(t_i,x)$$

objective function:

$$\varphi (x)=\frac{1}{2}{r}^T(x)r(x)$$

gradient vector:

$$\nabla \varphi (x)=J^T(x)r(x)$$

where $J$ is Jacobian matrix of $r$.
Hessian matrix:

$$H_{\varphi }=J^T(x)J(x)+\sum _{i=1}^m{r}_i(x)H_i(x)$$

However it’s expensive to expend such sum and since it multiple to $r_I$ which is very small that we can just ignore this whole sum term.