Matrix Decomposition Theorem

Let ${\hbox{\sf P}}$ be a Matrix of Eigenvectors of a given Matrix ${\hbox{\sf A}}$ and ${\hbox{\sf D}}$ a Matrix of the corresponding Eigenvalues. Then ${\hbox{\sf A}}$ can be written

$${\hbox{\sf A}}={\hbox{\sf P}}{\hbox{\sf D}}{\hbox{\sf P}}^{-1},$$ (1)

where ${\hbox{\sf D}}$ is a Diagonal Matrix and the columns of ${\hbox{\sf P}}$ are Orthogonal Vectors. If ${\hbox{\sf P}}$ is not a Square Matrix, then it cannot have a Matrix Inverse. However, if P is $m\times n$ (with $m>n$), then ${\hbox{\sf A}}$ can be written using a so-called Singular Value Decomposition of the form

$${\hbox{\sf A}}={\hbox{\sf U}}{\hbox{\sf D}}{\hbox{\sf V}}^{\rm T},$$ (2)

where ${\hbox{\sf U}}$ and ${\hbox{\sf V}}$ are $n\times n$ Square Matrices with Orthogonal columns,

$${\hbox{\sf U}}^{\rm T}{\hbox{\sf U}}={\hbox{\sf V}}^{\rm T}{\hbox{\sf V}}={\hbox{\sf I}}.$$ (3)

See also Singular Value Decomposition

References

Press, W. H.; Flannery, B. P.; Teukolsky, S. A.; and Vetterling, W. T. ``Singular Value Decomposition.'' §2.6 in Numerical Recipes in FORTRAN: The Art of Scientific Computing, 2nd ed. Cambridge, England: Cambridge University Press, pp. 51-63, 1992.