Orthogonal projections

Orthogonal projections

Lessons

The Orthogonal Decomposition Theorem
Let SS be a subspace in Rn\Bbb{R}^n. Then each vector yy in Rn\Bbb{R}^n can be written as:

y=y^+zy=\hat{y}+z

where y^\hat{y} is in SS and zz is in SS^{\perp}. Note that y^\hat{y} is the orthogonal projection of yy onto SS

If {v1,,vpv_1,\cdots ,v_p } is an orthogonal basis of SS, then

projsy=y^=yv1v1v1v1+yv2v2v2v2++yvpvpvpvp_sy=\hat{y}=\frac{y \cdot v_1}{v_1 \cdot v_1}v_1 + \frac{y \cdot v_2}{v_2 \cdot v_2}v_2 + \cdots + \frac{y \cdot v_p}{v_p \cdot v_p}v_p

However if {v1,,vpv_1,\cdots ,v_p } is an orthonormal basis of SS, then

projsy=y^=(yv1)v1+(yv2)v2++(yvp)vp_sy=\hat{y}=(y \cdot v_1)v_1+(y \cdot v_2)v_2 + \cdots + (y \cdot v_p)v_p

Property of Orthogonal Projection
If {v1,,vpv_1,\cdots ,v_p } is an orthogonal basis for SS and if yy happens to be in SS, then
projsy=y_sy=y

In other words, if y is in S=S=Span{v1,,vpv_1,\cdots ,v_p}, then projSy=y_S y=y.

The Best Approximation Theorem
Let SS be a subspace of Rn\Bbb{R}^n. Also, let yy be a vector in Rn\Bbb{R}^n, and y^\hat{y} be the orthogonal projection of yy onto SS. Then yy is the closest point in SS, because

yy^\lVert y- \hat{y} \rVert < yu\lVert y-u \rVert

where uu are all vectors in SS that are distinct from y^\hat{y}.
  • 1.
    Orthogonal Projections Overview:
    a)
    The Orthogonal Decomposition Theorem
    • Make yy as the sum of two vectors y^\hat{y} and zz
    • Orthogonal basis → y^=yv1v1v1v1++yvpvpvpvp\hat{y}= \frac{y \cdot v_1}{v_1 \cdot v_1}v_1 + \cdots + \frac{y \cdot v_p}{v_p \cdot v_p}v_p
    • Orthonormal basis → y^=(yv1)v1++(yvp)vp\hat{y}=(y\cdot v_1)v_1+\cdots +(y\cdots v_p)v_p
    z=yy^z=y - \hat{y}

    b)
    Property of Orthogonal Projections
    • projsy=y_s y=y
    • Only works if yy is in SS

    c)
    The Best Approximation Theorem
    • What is the point closest to yy in SS? y^\hat{y}!
    • Reason why: yy^\lVert y - \hat{y} \rVert < yu\lVert y-u \rVert
    • The Distance between the yy and y^\hat{y}


  • 2.
    The Orthogonal Decomposition Theorem
    Assume that {v1,v2,v3v_1,v_2,v_3 } is an orthogonal basis for Rn\Bbb{R}^n. Write yy as the sum of two vectors, one in Span{v1v_1}, and one in Span{v2,v3v_2,v_3}. You are given that:
    vector 1, 2, 3, 4

  • 3.
    Verify that {v1,v2v_1,v_2 } is an orthonormal set, and then find the orthogonal projection of yy onto Span{v1,v2v_1,v_2}.
    Verify these vectors are an orthonormal set

  • 4.
    Best Approximation
    Find the best approximation of yy by vectors of the form c1v1+c2v2c_1 v_1+c_2 v_2, where best approximation, vector y, and best approximation, vector v_1, best approximation, vector v_3.

  • 5.
    Finding the Closest Point and Distance
    Find the closest point to yy in the subspace SS spanned by v1v_1 and v2v_2.
    Find the closest point to y between vector 1 and vector 2

  • 6.
    Find the closest distance from yy to S=S=Span{v1,v2v_1,v_2 } if
    Find the closest distance from y to s