Orthogonal projections - Orthogonality and Least Squares

Notes:

The Orthogonal Decomposition Theorem
Let $S$ be a subspace in $\Bbb{R}^n$. Then each vector $y$ in $\Bbb{R}^n$ can be written as:

$y=\hat{y}+z$
where $\hat{y}$ is in $S$ and $z$ is in $S^{\perp}$. Note that $\hat{y}$ is the orthogonal projection of $y$ onto $S$

If {$v_1,\cdots ,v_p$} is an orthogonal basis of $S$, then

proj$_sy=\hat{y}=\frac{y \cdot v_1}{v_1 \cdot v_1}v_1 + \frac{y \cdot v_2}{v_2 \cdot v_2}v_2 + \cdots + \frac{y \cdot v_p}{v_p \cdot v_p}v_p$
However if {$v_1,\cdots ,v_p$} is an orthonormal basis of $S$, then

proj$_sy=\hat{y}=(y \cdot v_1)v_1+(y \cdot v_2)v_2 + \cdots + (y \cdot v_p)v_p$
Property of Orthogonal Projection
If {$v_1,\cdots ,v_p$} is an orthogonal basis for $S$ and if $y$ happens to be in $S$, then
proj$_sy=y$
In other words, if y is in $S=$Span{$v_1,\cdots ,v_p$}, then proj$_S y=y$.

The Best Approximation Theorem
Let $S$ be a subspace of $\Bbb{R}^n$. Also, let $y$ be a vector in $\Bbb{R}^n$, and $\hat{y}$ be the orthogonal projection of $y$ onto $S$. Then $y$ is the closest point in $S$, because

$\lVert y- \hat{y} \rVert$ < $\lVert y-u \rVert$
where $u$ are all vectors in $S$ that are distinct from $\hat{y}$.