normal equation

Definition

Solution is

$$x=(A^{T}A{)}^{-1}{A}^{T}b$$

If $(A^{T}A{)}^{-1}$ such inverse not exists, we need to use $x\approx A^{+}b$ from pseudo inverse (Moore-Penrose Inverse) to solve it.

Example of solving polynomial

$$f(x)=c_0+c_{1}x+c_2{x}^2+c_3{x}^3$$

Given vector $x$ and $f$ as input and output, we want to find $c$ coefficients.

1. build Vandermonde matrix
2. Solve it by normal equation / pseudo inverse (Moore-Penrose Inverse)

# build Vandermonde matrix
A = np.vander(x, 4, True)
c = la.solve(A.T @ f, A.T @ A)