Singular value decomposition takes a rectangular matrix of gene expression data (defined as A, where A is a n x p matrix) in which the n rows represents the genes, and the p columns represents the experimental conditions. The SVD theorem states:
$$A_{n\times p}=U_{n\times n}S_{n\times p}V^T_{p\times p}$$
Where $U^TU=I_{n\times n}$
$V^TV=I_{p\times p} $ (i.e. U and V are orthogonal)
Where the columns of U are the left singular vectors (gene coefficient vectors); S (the same dimensions as A) has singular values and is diagonal (mode amplitudes); and $V^T$ has rows that are the right singular vectors (expression level vectors). The SVD represents an expansion of the original data in a coordinate system where the covariance matrix is diagonal.
Calculating the SVD consists of finding the eigen values and eigenvectors of $AA^T$ and$A^TA$. The eigen vectors of $A^TA$ make up the columns of V, the eigenvectors of $AA^T$ make up the columns of U. Also, the singular values in S are square roots of eigen values from $AA^T$ or$\ \ A^TA$. The singular values are the diagonal entries of the S matrix and are arranged in descending order. The singular values are always real numbers. If the matrix A is a real matrix, then U and V are also real.
Given $A=\left[ \begin{array}{c}
1 \\
1 \\
1 \end{array}
\ \begin{array}{c}
\ \ \ \ \ \ \ 1 \\
\ \ \ \ \ \ \ 1 \\
\ \ \ \ \ -1 \end{array}
\right]$
Step1: To find the eigen values of $AA^T$
$A^T=\left[ \begin{array}{c}
1 \\
1 \end{array}
\ \ \begin{array}{c}
1 \\
1 \end{array}
\ \ \begin{array}{c}
1 \\
-1 \end{array}
\right]$
$\therefore AA^T=\left[ \begin{array}{c}
1 \\
1 \\
1 \end{array}
\ \begin{array}{c}
\ \ \ \ \ \ \ 1 \\
\ \ \ \ \ \ \ 1 \\
\ \ \ \ \ -1 \end{array}
\right]\left[ \begin{array}{c}
1 \\
1 \end{array}
\ \ \begin{array}{c}
1 \\
1 \end{array}
\ \ \begin{array}{c}
1 \\
-1 \end{array}
\right]=\left[ \begin{array}{ccc}
2 & 2 & 0 \\
2 & 2 & 0 \\
0 & 0 & 2 \end{array}
\right]$
The characteristic roots of $AA^T$ are
$$\left|AA^T-\lambda I\right|=0$$
$${\left(-1\right)}^3{\lambda }^3+{\left(-1\right)}^2s_1{\lambda }^2+\left(-1\right)s_2\lambda +\left|A\right|=0.........(1)$$
Where $$s_1=trace\left(A\right)=\left(2+2+2\right)=6$$
$$\therefore s_{1}=6$$
$$s_2=\left| \begin{array}{cc}
2 & 0 \\
0 & 2 \end{array}
\right|+\left| \begin{array}{cc}
2 & 0 \\
0 & 2 \end{array}
\right|+\left| \begin{array}{cc}
2 & 2 \\
2 & 2 \end{array}
\right|$$
$$=4+4+0=8$$
$$\therefore s_{2}=8$$
$$\left|A\right|=8-8=0$$
$$\therefore |A|=0$$
Equation(1) becomes,
$${\lambda }^3+6{\lambda }^2+8\lambda +0=0$$
$$\lambda ({\lambda }^2-6\lambda +8)=0$$
Hence eigen values of A are
$$\lambda =0,2,4$$
Step2: To find eigen vectors of corresponding eigen values
To find eigen vectors corresponding to these eigen values, we use the system of homogenous equations $\left(\boldsymbol{A}{\boldsymbol{A}}^{\boldsymbol{T}}-\lambda \boldsymbol{I}\right)\boldsymbol{X}\boldsymbol{=}\boldsymbol{0}$
When $\boldsymbol{\lambda }\boldsymbol{=}\boldsymbol{0}$, corresponding system of homogenous equations is
$$\left[ \begin{array}{ccc}
2 & 2 & 0 \\
2 & 2 & 0 \\
0 & 0 & 2 \end{array}
\right]\left[ \begin{array}{c}
x \\
y \\
z \end{array}
\right]=\left[ \begin{array}{c}
0 \\
0 \\
0 \end{array}
\right]$$
$$R_2\to R_2-R_1$$
$$\left[ \begin{array}{ccc}
2 & 2 & 0 \\
0 & 0 & 0 \\
0 & 0 & 2 \end{array}
\right]\left[ \begin{array}{c}
x \\
y \\
z \end{array}
\right]=\left[ \begin{array}{c}
0 \\
0 \\
0 \end{array}
\right]$$
$$\therefore z=0 \\
2x+2y=0$$
Let $y=k\ \ \ \therefore x=-k$
Hence
$$\left[ \begin{array}{c}
\begin{array}{c}
-k \\
k \end{array}
\\
0 \end{array}
\right]=k\left[ \begin{array}{c}
-1 \\
1 \\
0 \end{array}
\right]$$
Hence setting $k=1$ the eigen vector corresponding to $\lambda =0$
$${\boldsymbol{X}}_{\boldsymbol{1}}\boldsymbol{=}\left[ \begin{array}{c}
\boldsymbol{-}\boldsymbol{1} \\
\boldsymbol{1} \\
\boldsymbol{0} \end{array}
\right] {\boldsymbol{X}}_{\boldsymbol{1}}\boldsymbol{=[-}\boldsymbol{1}\boldsymbol{\ }\boldsymbol{1}\boldsymbol{\ }\boldsymbol{0}\boldsymbol{]'}$$
When $\boldsymbol{\lambda }\boldsymbol{=}\boldsymbol{2}$, corresponding system of homogenous equations is
$$\left[ \begin{array}{ccc}
0 & 2 & 0 \\
2 & 0 & 0 \\
0 & 0 & 0 \end{array}
\right]\left[ \begin{array}{c}
x \\
y \\
z \end{array}
\right]=\left[ \begin{array}{c}
0 \\
0 \\
0 \end{array}
\right]$$
Considering any two linear equations and solving them by Cramer's rule, we get
$$0x+2y+0z=0$$
$$2x+0y+0z=0$$
$$\frac{x}{\left| \begin{array}{cc}
2 & 0 \\
0 & 0 \end{array}
\right|}=\frac{-y}{\left| \begin{array}{cc}
0 & 0 \\
2 & 0 \end{array}
\right|}=\frac{z}{\left| \begin{array}{cc}
0 & 2 \\
2 & 0 \end{array}
\right|}$$
$$\frac{x}{0}=-\frac{y}{0}=\frac{z}{-4}$$
$$\frac{x}{0}=\frac{y}{0}=\frac{z}{1}$$
Hence, the eigen vector corresponding to the eigen value $\lambda =2$ is
$${\boldsymbol{X}}_{\boldsymbol{2}}\boldsymbol{=}\left[ \begin{array}{c}
\boldsymbol{0} \\
\boldsymbol{0} \\
\boldsymbol{1} \end{array}
\right] \boldsymbol{i}.\boldsymbol{e}\boldsymbol{.\ \ }{\boldsymbol{X}}_{\boldsymbol{2}}\boldsymbol{=[}\boldsymbol{0}\boldsymbol{\ }\boldsymbol{0}\boldsymbol{\ }\boldsymbol{1}\boldsymbol{]'}$$
When $\boldsymbol{\lambda }\boldsymbol{=}\boldsymbol{4}$, corresponding system of homogenous equations is
$$\left[ \begin{array}{ccc}
-2 & 2 & 0 \\
2 & -2 & 0 \\
0 & 0 & -2 \end{array}
\right]\left[ \begin{array}{c}
x \\
y \\
z \end{array}
\right]=\left[ \begin{array}{c}
0 \\
0 \\
0 \end{array}
\right]$$
$$R_2\to R_2+R_1$$
$$\left[ \begin{array}{ccc}
-2 & 2 & 0 \\
0 & 0 & 0 \\
0 & 0 & -2 \end{array}
\right]\left[ \begin{array}{c}
x \\
y \\
z \end{array}
\right]=\left[ \begin{array}{c}
0 \\
0 \\
0 \end{array}
\right]$$
$$\therefore z=0$$
$$-2x+2y=0$$
Let $y=k\ \ \ \therefore x=k$
Hence
$$\left[ \begin{array}{c}
\begin{array}{c}
k \\
k \end{array}
\\
0 \end{array}
\right]=k\left[ \begin{array}{c}
1 \\
1 \\
0 \end{array}
\right]$$
Hence setting $k=1$ the eigen vector corresponding to $\lambda =4$
$${\boldsymbol{X}}_{\boldsymbol{3}}\boldsymbol{=}\left[ \begin{array}{c}
\boldsymbol{1} \\
\boldsymbol{1} \\
\boldsymbol{0} \end{array}
\right] \boldsymbol{i}.\boldsymbol{e}.{\boldsymbol{X}}_{\boldsymbol{3}}\boldsymbol{=[}\boldsymbol{1}\boldsymbol{\ }\boldsymbol{1}\boldsymbol{\ }\boldsymbol{0}\boldsymbol{]'}$$
Step3: To find orthogonal matrix U:
Arrange eigen vectors in columns in descending order of their corresponding eigen values
$$Let B=\left[ \begin{array}{ccc}
1 & 0 & -1 \\
1 & 0 & 1 \\
0 & 1 & 0 \end{array}
\right]$$
We now convert this matrix into an orthonormal matrix by using Gram-schmidt orthonormalization process
Let $u_1=\left(1, 1, 0\right), u_2=\left(0, 0, 1\right)\ \&\ u_3=(-1, 1, 0)$
Now $v_1=u_1=\left(1, 1, 0\right)$
$v_2=u_2-\frac{\lt u_2,v_1\gt}{{\left|\left|v_1\right|\right|}^2}.v_1$
$\lt u_2,v_1\gt0$
${\left|\left|v_1\right|\right|}^2=2$
$\therefore v_2=u_2=\left(0,\ 0,\ 1\right)$
$v_3=u_3-\frac{\lt u_3,v_1\gt}{{\left|\left|v_1\right|\right|}^2}.v_1-\frac{\lt u_3,v_2\gt}{{\left|\left|v_2\right|\right|}^2}.v_2$
${\left|\left|v_2\right|\right|}^2=1$
$\lt u_3,v_1\gt\ =0\ \ \ \ \&\ \ \ \lt u_3,v_2\gt\ =0$
$\therefore v_3=u_3=(-1,\ 1,\ 0)$
${\left|\left|v_3\right|\right|}^2=2$
Norms of these vectors are
$\left|\left|v_1\right|\right|=\sqrt{2}$
$\left|\left|v_2\right|\right|=1$
$\left|\left|v_3\right|\right|=\sqrt{2}$
$\therefore $ The normalized vectors are
$v_1=\frac{v_1}{||v_1||}=\left(\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}},\ 0\right)$
$v_2=\frac{v_2}{||v_2||}=\left(0,0,1\right)$
$v_3=\frac{v_3}{||v_3||}=\left(-\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}},\ 0\right)$
$$\therefore \boldsymbol{U}\boldsymbol{=}\left[ \begin{array}{ccc}
\frac{\boldsymbol{1}}{\sqrt{\boldsymbol{2}}} & \boldsymbol{0} & \boldsymbol{-}\frac{\boldsymbol{1}}{\sqrt{\boldsymbol{2}}} \\
\frac{\boldsymbol{1}}{\sqrt{\boldsymbol{2}}} & \boldsymbol{0} & \frac{\boldsymbol{1}}{\sqrt{\boldsymbol{2}}} \\
\boldsymbol{0} & \boldsymbol{1} & \boldsymbol{0} \end{array}
\right]$$
Step4: Now, we find orthonormal matrix ${\boldsymbol{V}}^{\boldsymbol{T}}$ which is obtained from ${\boldsymbol{A}}^{\boldsymbol{T}}\boldsymbol{A}$
$$A^TA=\left[ \begin{array}{c}
1 \\
1 \end{array}
\ \ \begin{array}{c}
1 \\
1 \end{array}
\ \ \begin{array}{c}
1 \\
-1 \end{array}
\right]\left[ \begin{array}{c}
1 \\
1 \\
1 \end{array}
\ \begin{array}{c}
\ \ \ \ \ \ \ 1 \\
\ \ \ \ \ \ \ 1 \\
\ \ \ \ \ -1 \end{array}
\right]=\left[ \begin{array}{cc}
\boldsymbol{3} & \boldsymbol{1} \\
\boldsymbol{1} & \boldsymbol{3} \end{array}
\right]$$
The characteristic roots of $A^TA$ are
$$\left|A^TA-\lambda I\right|=0$$
$${\lambda }^2+\left(-1\right)s_1\lambda +\left|A\right|=0............(1)$$
Where $$s_1=trace\left(A\right)=\left(3+3\right)=6$$
$$\therefore {\boldsymbol{s}}_{\boldsymbol{1}}\boldsymbol{=}\boldsymbol{6}$$
$$\left|A\right|=8$$
$$\therefore |\boldsymbol{A}\boldsymbol{|=}\boldsymbol{8}$$
Equation (1) becomes,
$$\ ({\lambda }^2-6\lambda +8)=0$$
Hence Eigen values of A are
$$\boldsymbol{\lambda }\boldsymbol{=\ }\boldsymbol{2},\boldsymbol{4}$$
Step5: To find eigen vectors of corresponding eigen values
To find eigen vectors corresponding to these eigen values, we use the system of homogenous equations $\left({\boldsymbol{A}}^{\boldsymbol{T}}\boldsymbol{A}-\lambda \boldsymbol{I}\right)\boldsymbol{X}\boldsymbol{=}\boldsymbol{0}$
When $\boldsymbol{\lambda }\boldsymbol{=}\boldsymbol{2}$, corresponding system of homogenous equations is
$\left[ \begin{array}{cc}
1 & 1 \\
1 & 1 \end{array}
\right]\left[ \begin{array}{c}
x \\
y \end{array}
\right]=\left[ \begin{array}{c}
0 \\
0 \end{array}
\right]$
$R_2\to R_2-R_1$
$\left[ \begin{array}{cc}
1 & 1 \\
0 & 0 \end{array}
\right]\left[ \begin{array}{c}
x \\
y \end{array}
\right]=\left[ \begin{array}{c}
0 \\
0 \end{array}
\right]$
$x+y=0$
Let $y=-k\ \ \ \therefore x=k$
$\left[ \begin{array}{c}
k \\
-k \end{array}
\right]=k\left[ \begin{array}{c}
1 \\
-1 \end{array}
\right]$
Hence setting $k=1$ the eigen vector corresponding to $\lambda =2$
$${\boldsymbol{X}}_{\boldsymbol{1}}\boldsymbol{=}\left[ \begin{array}{c}
\boldsymbol{1} \\
\boldsymbol{-}\boldsymbol{1} \end{array}
\right] \boldsymbol{i}.\boldsymbol{e}.{\boldsymbol{X}}_{\boldsymbol{1}}\boldsymbol{=[}\boldsymbol{1}\boldsymbol{\ -}\boldsymbol{1}\boldsymbol{\ ]'}$$
When $\boldsymbol{\lambda }\boldsymbol{=}\boldsymbol{4}$, corresponding system of homogenous equations is
$\left[ \begin{array}{cc}
-1 & 1 \\
1 & -1 \end{array}
\right]\left[ \begin{array}{c}
x \\
y \end{array}
\right]=\left[ \begin{array}{c}
0 \\
0 \end{array}
\right]$
$R_2\to R_2+R_1$
$\left[ \begin{array}{cc}
-1 & 1 \\
0 & 0 \end{array}
\right]\left[ \begin{array}{c}
x \\
y \end{array}
\right]=\left[ \begin{array}{c}
0 \\
0 \end{array}
\right]$
$-x+y=0$
Let $y=k\ \ \ \therefore x=k$
$\left[ \begin{array}{c}
k \\
k \end{array}
\right]=k\left[ \begin{array}{c}
1 \\
1 \end{array}
\right]$
Hence setting $k=1$ the eigen vectors corresponding to $\lambda =4$
$${\boldsymbol{X}}_{\boldsymbol{2}}\boldsymbol{=}\left[ \begin{array}{c}
\boldsymbol{1} \\
\boldsymbol{1} \end{array}
\right] \boldsymbol{i}.\boldsymbol{e}.{\boldsymbol{X}}_{\boldsymbol{2}}\boldsymbol{=[}\boldsymbol{1}\boldsymbol{\ }\boldsymbol{1}\boldsymbol{\ ]'}$$
Step6: To find ${\boldsymbol{V}}^{\boldsymbol{T}}$
Arrange eigen vectors in columns in descending order of corresponding eigen values
$$Let\ C=\left[ \begin{array}{cc}
1 & 1 \\
1 & -1 \end{array}
\right]$$
We now convert this matrix into an orthonormal matrix by using Gram-schmidt orthonormalization process
Let $u_1=\left(1,\ 1\right),\ u_2=\left(1,\ -1\right)\ $
Now $v_1=u_1=\left(1,\ -1\right)$
$v_2=u_2-\frac{\lt u_2,v_1\gt}{{\left|\left|v_1\right|\right|}^2}.v_1$
$\lt u_2,v_1\gt0$
${\left|\left|v_1\right|\right|}^2=2$
$\therefore v_2=u_2=\left(1,\ -1\right)$
Norms of these vectors are
$\left|\left|v_1\right|\right|=\sqrt{2}$
$\left|\left|v_2\right|\right|=\sqrt{2}$
$\therefore $ The normalized vectors are
$v_1=\frac{v_1}{||v_1||}=\left(\frac{1}{\sqrt{2}},\ \frac{1}{\sqrt{2}}\right)$
$v_2=\frac{v_2}{||v_2||}=\left(\frac{1}{\sqrt{2}},\frac{-1}{\sqrt{2}}\right)$
$$\therefore V=\left[ \begin{array}{cc}
\frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \\
\frac{1}{\sqrt{2}} & -\frac{1}{\sqrt{2}} \end{array}
\right]$$
$$\therefore V^T=\left[ \begin{array}{cc}
\frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \\
\frac{1}{\sqrt{2}} & -\frac{1}{\sqrt{2}} \end{array}
\right]$$
Step 7: To obtain S by taking square roots of non zero eigen values
$S=\left[ \begin{array}{c}
2 \\
0 \\
0 \end{array}
\ \ \begin{array}{c}
0 \\
\sqrt{2} \\
0 \end{array}
\ \ \right]$
$$A_{n\times p}=U_{n\times n}S_{n\times p}V^T_{p\times p}$$
$$A=\left[ \begin{array}{ccc}
\frac{1}{\sqrt{2}} & 0 & -\frac{1}{\sqrt{2}} \\
\frac{1}{\sqrt{2}} & 0 & \frac{1}{\sqrt{2}} \\
0 & 1 & 0 \end{array}
\right]\left[ \begin{array}{c}
2 \\
0 \\
0 \end{array}
\ \ \begin{array}{c}
0 \\
\sqrt{2} \\
0 \end{array}
\ \ \right]\left[ \begin{array}{cc}
\frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \\
\frac{1}{\sqrt{2}} & -\frac{1}{\sqrt{2}} \end{array}
\right]$$
$$A=\left[ \begin{array}{c}
\sqrt{2} \\
\sqrt{2} \\
0 \end{array}
\ \ \begin{array}{c}
0 \\
0 \\
\sqrt{2} \end{array}
\ \right]\left[ \begin{array}{cc}
\frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \\
\frac{1}{\sqrt{2}} & -\frac{1}{\sqrt{2}} \end{array}
\right]$$
$$\boldsymbol{A}\boldsymbol{=}\left[ \begin{array}{c}
\boldsymbol{1} \\
\boldsymbol{1} \\
\boldsymbol{1} \end{array}
\boldsymbol{\ } \begin{array}{c}
\boldsymbol{\ \ \ \ \ \ \ }\boldsymbol{1} \\
\boldsymbol{\ \ \ \ \ \ \ }\boldsymbol{1} \\
\boldsymbol{\ \ \ \ \ -}\boldsymbol{1} \end{array}
\right]$$
Thus we have decomposed A
and 2 others joined a min ago.
teamques10
★ 70k
krithikkm200
• 10