Eritque Arcus Math Notes

Singular value decomposition

4 min read

𝐴=π‘ˆΞ£π‘‰π‘‡
where

  • 𝐴 is π‘šΓ—π‘›
  • π‘ˆ: AnΒ π‘šΓ—π‘šΒ Orthogonal matrix matrix whose columns are the left singular vectors of 𝐴.
  • Ξ£: A diagonalΒ π‘šΓ—π‘›Β matrix containing the singular values of 𝐴 in descending order.
  • 𝑉𝑇: The transpose of an 𝑛×𝑛 Orthogonal matrix where the columns are the right singular vectors of 𝐴.

Steps

  1. calculate 𝐴𝐴𝑇
  2. calculate Eigenvalues of 𝐴𝐴𝑇 by solving 𝑑𝑒𝑑(π΄π΄π‘‡βˆ’πœ†πΌ)=0, where 𝐼 is identify matrix
  3. then Ξ£ is 𝛿 from those πœ†
  4. calculate 𝑉 right singular vectors(should be unit vectors), for each πœ†, (π΄π‘‡π΄βˆ’πœ†πΌ)𝑣=0
  5. calculate π‘ˆ left singular vectors(should be unit vectors), for each πœ†, (π΄π΄π‘‡βˆ’πœ†πΌ)𝑒=0

3x2 example

ref https://www.geeksforgeeks.org/singular-value-decomposition-svd/

𝐴=[132231] 𝐴𝐴𝑇=[132231][123321]=[1+4+93+4+33+4+39+4+1]=[14101014] det(π΄π΄π‘‡βˆ’πœ†πΌ)=[14βˆ’πœ†101014βˆ’πœ†]=(14βˆ’πœ†)2βˆ’102=142βˆ’28πœ†+πœ†2βˆ’102=πœ†2βˆ’28πœ†+96 (πœ†βˆ’24)(πœ†βˆ’4)=0

Then we can get some πœ† Eigenvalues, πœ†1=24 and πœ†2=4 and Singular value 𝛿1=√24 and 𝛿2=2.
we can say

Ξ£=[𝛿100𝛿200]=[√2400200]

For each of them, calculate eigenvectors (π΄π‘‡π΄βˆ’πœ†πΌ)𝑣=0
where

𝐴𝑇𝐴=[10868886810] (π΄π‘‡π΄βˆ’24𝐼)𝑣1=[βˆ’14868βˆ’16868βˆ’14]𝑣1=0

RREF it

[100010βˆ’1βˆ’10]𝑣1=0

so π‘₯βˆ’π‘§=0 and π‘¦βˆ’π‘§=0

𝑣1=[1√31√31√3]

so do πœ†2=4

(π΄π‘‡π΄βˆ’4𝐼)𝑣2=[686848686]𝑣2=0

RREF it

[100010100]

so π‘₯+𝑧=0 and 𝑦=0

𝑣2=[1√20βˆ’1√2]

Then for 𝑣3, since it must perpendicular to 𝑣1,𝑣2, we can say 𝑣𝑇1𝑣3=0 and 𝑣𝑇2𝑣3=0 is true. Solve these two equations to get 𝑣3.

𝑣3=[1√6βˆ’2√61√6]

Then

𝑉=[βˆ—π‘£1βˆ—π‘£2βˆ—π‘£3]=[1√31√31√31√20βˆ’1√21√6βˆ’2√61√6]

Lastly need to find π‘ˆ with similar steps.

(π΄π΄π‘‡βˆ’24𝐼)𝑒1=[βˆ’101010βˆ’10]𝑒1=0

RREF it

[βˆ’1010]

so 𝑒1=[1√21√2]

(π΄π΄π‘‡βˆ’4𝐼)𝑒2=[10101010]𝑒1=0

RREF it

[1010]

so 𝑒2=[1√2βˆ’1√2]
Then

π‘ˆ=[βˆ—π‘’1βˆ—π‘’2]=[1√21√21√2βˆ’1√2]

Therefore

𝐴=π‘ˆΞ£π‘‰π‘‡=[1√21√21√2βˆ’1√2][√2400200][1√31√31√31√20βˆ’1√21√6βˆ’2√61√6]𝑇

β–‘

Applications

Pseudo Inverse (Moore-Penrose Inverse)

𝑀=π‘ˆΞ£π‘‰π‘‡

Graph View

Backlinks