Inner product, length, and orthogonality

Get the most by viewing this topic in your current grade. Pick your course now.

Introduction
Lessons
  1. Inner Product, Length, and Orthogonality Overview:
  2. The Length of a Vector
    • Also known as norm
    • Some properties with a length of a vector
    • Unit vectors
  3. Distance of two vectors/Orthogonal Vectors
    • Dist(u,v)=uv(u,v)=\lVert u-v \rVert
    • How to tell with two vectors are orthogonal
    • Orthogonal vectors with Pythagorean Theorem
Examples
Lessons
  1. Utilizing the Inner Product
    Given that vector u and vector v, compute:
    uuvvv\frac{u \cdot u}{v \cdot v}v
    1. Given that vector u and vector v, compute:
      (uv)u\lVert (u \cdot v)u \rVert
      1. Finding the Unit vector
        Find the unit vector in the direction of the vector Finding the Unit vector
        1. Calculating the Distance
          Find the distance between the vectors Calculating the Distance, vector u and Calculating the Distance, vector v.
          1. Showing Orthogonality
            For what value(s) of bb make vectors uu and vv orthogonal if
            what value b make this two vectors orthogonal
            1. Proof Questions
              Show that the parallelogram is true for vectors uu and vv in R^n. In other words, show that:
              u+v2+uv2=2u2+2v2\lVert u+v \rVert^2 + \lVert u-v \rVert^2 = 2 \lVert u \rVert^2 + 2 \lVert v \rVert^2
              1. Suppose vector uu is orthogonal to vectors xx and yy. Show that uu is also orthogonal to x+yx+y.
                Free to Join!
                StudyPug is a learning help platform covering math and science from grade 4 all the way to second year university. Our video tutorials, unlimited practice problems, and step-by-step explanations provide you or your child with all the help you need to master concepts. On top of that, it's fun - with achievements, customizable avatars, and awards to keep you motivated.
                • Easily See Your Progress

                  We track the progress you've made on a topic so you know what you've done. From the course view you can easily see what topics have what and the progress you've made on them. Fill the rings to completely master that section or mouse over the icon to see more details.
                • Make Use of Our Learning Aids

                  Last Viewed
                  Practice Accuracy
                  Suggested Tasks

                  Get quick access to the topic you're currently learning.

                  See how well your practice sessions are going over time.

                  Stay on track with our daily recommendations.

                • Earn Achievements as You Learn

                  Make the most of your time as you use StudyPug to help you achieve your goals. Earn fun little badges the more you watch, practice, and use our service.
                • Create and Customize Your Avatar

                  Play with our fun little avatar builder to create and customize your own avatar on StudyPug. Choose your face, eye colour, hair colour and style, and background. Unlock more options the more you use StudyPug.
                Topic Notes

                Inner product, length and orthogonality


                The time has come for us to work on vectors. As we know, a vector is a mathematical object with a magnitude and a direction. We know how to represent vectors geometrically in coordinate systems, and we know how to make a few calculations with them, let look at one of its main characteristics (length) and then continue with one of the operations that can be performed with them: the inner product.

                The length of a vector The length of a vector, which is also called its norm refers simply to the magnitude of the vector. So, what is that in simple words? Just as the word length defines it, the length is just that; for example: if you have a vector you know this is represented geometrically as an arrow, thus its length will be the quantity measured of the distance from where the vector tail starts to its arrow-tip. This is what we call its magnitude, its length or its norm (any of the three words can be used interchangeably, but the standard in mathematics textbooks is usually the word magnitude since it is intuitive to think this quantity gives us the size of a vector).

                The formula to calculate the magnitude of a vector vv goes as follows:

                v  =v12  +  v22  +  ...  +  vn2 \parallel v \parallel \; = \sqrt{v^{2}_{1} \;+\; v^{2}_{2} \;+\; ... \; + \; v^{2}_{n} }
                Equation 1: General formula for the length of a vector

                In this case   v12,v22,...,vn2 \; v^{2}_{1}, v^{2}_{2} , .. . , v^{2}_{n} \, represent the variables of the vector, in other words, if you have a vector vv in terms of xx and yy, then: x=v1x=v_1 \, and y=v2 \, y=v_2, therefore, its magnitude (length) formula would be:

                v  =x2  +  y2 \parallel v \parallel \; = \sqrt{x^2 \; + \; y^2 }
                Equation 2: Formula for the length of a vector v with terms on x and y only.

                And important identity to remember about the magnitude of a vector is that the multiplication of a vector by itself is equal to the square of the magnitude of the vector:

                v2=vv \parallel v \parallel^2 = v \cdot v
                Equation 3: Square of the magnitude of the vector

                This makes sense since multiplying a vector by itself means that you will multiply each term in the vector by itself. These multiplications by themselves will always produce positive results, because no matter if a term inside the vector is positive or negative: a positive times a positive produces a positive same as a negative times a negative produces a positive, thus, this is the same as obtaining the magnitude of the vector and just square it.

                Talking about magnitudes of vectors, there exist vectors that can have a magnitude of one, those vectors are called unit vectors.
                Knowing this, it is important to know the notations used to represent vector magnitudes, vectors and unit vectors. Notice that in equation 1 the magnitude (length or norm) is denoted with the letter coming from the name of the vector enclosed in between of two vertical lines, the magnitude can actually be represented in many different ways. In the next table we show you the most common ways vectors and magnitudes are represented:

                Inner product, length and orthogonality
                Figure 1: Standard notation for length, vectors and unit vectors

                And so, we can now understand the formula for a unit vector:

                v^=vˉv \large \hat{v} = \frac{\bar{v}}{\parallel v\parallel}
                Equation 4: Equation for a unit vector

                Therefore:
                Vector vv can be written as: vˉ\bar{v} or vv.
                The magnitude of vector vv can be written as: v,v  orvv, \mid v \;\mid \, or \, \parallel v\parallel.
                And the unit vector v is written as: v^\hat{v}.

                Note that during this lessons problem equations we will usually denote vectors just by their letter name instead of putting an arrow or bold caps (this is due to differences within software symbols and lack of the appropriate symbols in certain tools), for that reason, we will make sure to denote the magnitude of a vector as the letter name enclosed by two vertical lines in order to be able to differentiate them always.

                What is inner product?

                If we think of two column vectors being multiplied as vector spaces, the inner product happens to be the dot product which can be defined as:

                uv=uTvu \cdot v = u^{T}v
                Equation 5: Inner product algebraic definition.

                Where   uT\; u^{T} is the transpose of vector uu and vv is simply the vector vv.
                Therefore, if we have the two vectors uu and vv defined as follows:

                Inner product, length and orthogonality
                Equation 6: Vectors u and v

                Then the inner product of the two vectors is:

                Inner product, length and orthogonality
                Equation 7: Inner product of vectors u and v


                Inner product properties

                Just as the square of the magnitude of a vector identity shown in equation 3 is useful, we have a few inner product properties listed below. These properties can be of aid later in the lesson while solving problems.

                1. \quad uv=vu u \cdot v = v \cdot u
                The first property defines that the inner product, or dot product of two vectors is commutative, in other words, you can change the order in which two vectors are being operated on through an inner product and the result will not be affected.

                2. \quad w(v+v)=wu+wv w(v+v)=w \cdot u + w \cdot v
                This is what we call the distributive property which allow us to distribute the effect of the outer factor, in this case vector w, to be applied to both of the vectors inside the parentheses without affecting the operation happening in there (the addition). The distributive property works for both, addition and subtraction operations being performed inside of a parenthesis with a factor affecting both elements of the inside operation.

                3. \quad (cu)v=c(uv)=u(cv)(cu) \cdot v = c (u \cdot v)=u \cdot (cv)
                The associative property of the inner product says that when having such multiplication of more than two vectors, it does not matter which ones you associate and multiply first to then use the result of this first operation to multiplicate with the one that it is left. In other words, the associative property allows a longer multiplication containing a bigger quantity of vectors, to be broken down in pieces to be solved by steps without affecting the final result.

                4. \quad uu0,   u \cdot u \geq 0, \; and  uu=0   \; u \cdot u = 0 \; only when  u=0 \; u = 0
                This property complements equation 3 by saying the result of the inner product of a vector by itself will always result in a scalar higher than zero (as explained before) unless the vector in itself is zero, which would produce the multiplication of zero times zero, equal zero.

                What is dot product

                Mathematically speaking, the dot product definition is:

                uv=  u  vcosθu \cdot v = \; \parallel u\parallel \; \parallel v\parallel \cos\theta
                Equation 8: Dot product geometric definition.

                The inner product is a generalization which can also be defined as the dot product, or scalar product (due to the result it produces when vectors are being multiplied this way). The dot product formula as seen in equation 8 aids in the definition of orthogonality, which is the quality of two elements such as vectors of being perpendicular to each other.
                In this case, we can know if a pair of vectors are perpendicular (or orthogonal) to each other by computing its dot product since: The dot product of perpendicular vectors is equal to zero.

                The other way around also helps us in obtaining information from euclidean spaces and vectors, since one can obtain the angle between vectors by using dot product.

                Cross product vs dot product

                When multiplying vectors there are two different kind of basic multiplications, the dot product and the cross product. The main difference between them is that the result of a vector dot product is a scalar and the result of a cross product is a vector perpendicular to the plane of the two original vectors. We will go deeper into this topic in later lessons, but for now it is important you remember that the result of each operation gives these two multiplications their nicknames: the vector dot product is also called the scalar product, while the cross product is also called the vector product.

                Example problems


                For this section we will work out through various examples where we are to compute (or use) the inner product of two vectors to solve the given problems.

                Example 1

                Given the vectors u and v as shown below:

                Inner product, length and orthogonality
                Equation 9: Vectors u and v

                Compute the operation: uuvvv \large \frac{u \cdot u}{ v \cdot v}v
                We start by computing the dot product of vector uu by itself:

                Inner product, length and orthogonality
                Equation 10: Dot product of u \cdot u

                Now computing the dot product of vector vv by itself:

                Inner product, length and orthogonality
                Equation 11: Dot product of v \cdot v

                These two resulting numbers allow us to perform the division of the dot products quite easily (note how all of the elements entering this operation were vectors and the result is a scalar, this is why the inner product space is called the scalar product of vectors):

                Inner product, length and orthogonality
                Equation 12: Division of dot products \cdot v

                And so, we finish our calculations by multiplying by vector vv to obtain a final vector as follows:

                Inner product, length and orthogonality
                Equation 13: Final result for example problem 1


                Example 2

                Given the vectors uu and vv as shown below:

                Inner product, length and orthogonality
                Equation 14: Vectors u and v

                Compute the operation: (uv)  u \parallel ( u \cdot v) \; u\parallel
                Notice that since the multiplications we are to perform are all surrounded by double vertical lines, it means we are about to obtain a magnitude as a final solution for this operation. We start by computing the dot product of vectors uu and vv:

                Inner product, length and orthogonality
                Equation 15: Dot product of u \cdot v

                Multiplying this result by vector uu as asked:

                Inner product, length and orthogonality
                Equation 16: Multiplication of a scalar times a vector

                Now we just need to compute the absolute magnitude of this resulting vector in order to obtain our final result:

                (uv)  u  =(10)2  +  (20)2  +  (30)2  =100  +  400  +  900  =1400 \parallel ( u \cdot v) \;u \parallel \;= \sqrt{(10)^2 \; + \;(20)^2 \; + \; (30)^2 } \;= \sqrt{100 \; + \;400 \; + \; 900 } \;= \sqrt{1400}
                Equation 17: Final result for example problem 2

                And so, our final answer is a scalar (which makes sense since we are finding the length of the vector found in equation 16), this scalar is equal to the square root of 1,400. You could finish off and perform the square root on this number, but it is not necessary, to avoid the messiness of a number with decimals we can just keep the answer as it is.

                Example 3

                Find the unit vector in the direction of the vector vv as defined below:

                Inner product, length and orthogonality
                Equation 18: Vector v

                To solve this problem we make use of equation 4, the equation for a unit vector, thus, we have to find the magnitude of vector vv first:

                v  =(4)2  +  (2)2  +  (6)2  =16  +  4  +  36  =56\large \parallel v \parallel \;= \sqrt{(-4)^2 \; + \;(2)^2 \; + \; (6)^2 } \;= \sqrt{16 \; + \;4 \; + \; 36 } \;= \sqrt{56}
                Equation 19: Magnitude of v

                Given that we don't want to deal with decimal numbers, we leave the result as is with the square root sign, and now we perform the division to find the unit vector v^\hat{v} :

                Inner product, length and orthogonality
                Equation 20: Unit vector v^\hat{v}


                Example 4

                Find the distance between the vectors uu and vv as they are defined below:

                Inner product, length and orthogonality
                Equation 21: Vectors u and v

                In order to find the distance between two vectors we must subtract the vectors in question and then obtain the absolute magnitude of the resulting one, in other words, we must obtain the vector that would represent the distance between one and the other, and then the absolute magnitude is computed in order to know the length (in quantity) of such distance.
                The process can be summarized in the next operation:

                Inner product, length and orthogonality
                Equation 22: Distance between vectors u and v (part 1)

                Notice how the equation above can be read as: the distance between vectors u and v is equal to the absolute magnitude of the subtraction of those two vectors, equal to the absolute magnitude of the vector resulting from the subtraction of the two vectors. Then, we perform the magnitude calculation to obtain the final result:

                Inner product, length and orthogonality

                =(5)2  +  (4)2  +  (3)2  =25  +  16  +  9  =50\large = \sqrt{(5)^2 \; + \;(4)^2 \; + \; (-3)^2 } \;= \sqrt{25 \; + \;16 \; + \; 9 } \;= \sqrt{50}
                Equation 23: Distance between vectors u and v (part 1)

                And so, the distance between vectors u and v is equal to the square root of 50.
                Remember that subtracting means finding the difference between two quantities, in this case, between the two vectors. Since we will obtain the magnitude of the resulting vector from the subtraction, it does not matter the order in which you perform the subtraction of the vectors for this particular case. This is because the order of a vector subtraction affects the direction of the resulting vector, but it does not affect its magnitude and so, either you perform the subtraction as uv  u-v \; or   vu \; v-u will not matter for this case.
                You can prove this by performing the operations once more but now using the other order in the vector subtraction, we do encourage you to check it out!.

                Example 5

                What value(s) of b make vectors u and v orthogonal if they are defined as shown below:

                Inner product, length and orthogonality
                Equation 24: Vectors u and v

                Remember the orthogonality condition says that for two vectors to be orthogonal, their dot product must be zero, thus: uv=0 \, u \cdot v=0.
                And so, we can paraphrase the question to: what values of b will produce a dot product of 2 vectors uu and vv equal to zero? Well, let us find out!
                If we perform the dot product of u and v and set it equal to zero, we can then solve for the b value that will agree with that condition:

                uv=(1)(1)+(5)(3)+(7)(b)=0u \cdot v = (-1)(1) \,+ \,(5)(3) \,+ \, (-7)(b)=0

                1+157b=0   -1+15-7b=0 \;   147b=0 \; 14-7b=0

                7b=14 -7b=-14

                b=2 b=2
                Equation 25: Dot product of vectors u and v

                And we find that if bb is equal to 2, then the dot product of uu and vv will be zero, and so, they will be orthogonal.

                Example 6

                Show that the parallelogram is true for vectors uu and vv in RnR^n. In other words, show that:

                u  +  v2  +  u    v2  =2u2  +  2v2\parallel u \;+\; v\parallel^2 \; + \; \parallel u\;-\; v\parallel^2 \; = \,2 \parallel u\parallel^2 \; + \; 2\parallel v\parallel^2
                Equation 26: Prove this identity true

                This problem is all about playing with the terms around following the dot product rules, and so, we are to remember the dot product properties before we start:

                • v2=vv   \parallel v\parallel^2 = v \cdot v \;    \; square magnitude identity
                • w(v+v)=wu  +  wv  w(v+v)=w \cdot u \; + \; w \cdot v \;    \; distributive property
                • uv=vu   u \cdot v=v \cdot u \;    \; commutative property

                Knowing this, let us work through the left hand side of equation 26 in order to solve it until we arrive to the result on its right hand side. First we simplify by using the magnitude identity:

                u  +  v2  +  u    v2  =(u+v)(u+v)  +  (uv)(uv)\parallel u \;+\; v\parallel^2 \; + \; \parallel u\;-\; v\parallel^2 \; =(u+v) \cdot (u+v) \;+ \; (u-v) \cdot (u-v)
                Equation 27: Solving the left hand side of the identity (part 1)

                Then we use the distributive property to break down the expression even more:

                =(uu)+(uv)+(vu)+(vv)+(uu)+(u(v))+((v)u)+((v)(v)) = (u \cdot u) \, + \, (u \cdot v) \, + \, (v \cdot u) \, +\, (v \cdot v) \, + \, (u \cdot u) \, + \, (u \cdot (-v)) \, + \, ((-v) \cdot u) \, + \, ((-v) \cdot (-v))

                =(uu)  +  (uv)  +  (vu)  +  (vv)  +  (uu)(uv)(vu)  +  (vv) =(u \cdot u) \; + \; (u \cdot v) \; + \; (v \cdot u) \; + \;(v \cdot v) \; + \; (u \cdot u) - (u \cdot v)-(v \cdot u) \; + \; (v \cdot v)

                =2(uu)  +  (uv)  +  (vu)  +  2(vv)(uv)(vu) =2(u \cdot u) \; + \; (u \cdot v) \; + \; (v \cdot u) \; + \; 2 (v \cdot v)-(u \cdot v)-(v \cdot u)
                Equation 28: Solving the left hand side of the identity (part 2)

                And finally, using the commutative property:

                u+v2+uv2=2(uu)+2(uv)+2(vv)2(uv)=2(uu)+2(vv) \parallel u\,+ \, v \parallel^{2} \, + \, \parallel u - v \parallel^{2} = 2(u \cdot u) \,+\, 2(u \cdot v) \, + \, 2(v \cdot v) - 2(u \cdot v) = 2(u \cdot u) \, + \, 2(v \cdot v)
                Equation 29: Solving the left hand side of the identity (part 3)

                This is as simple as we can go when trying to break down the left hand side of equation 26, but the great thing about this is that if we use the magnitude identity once more, but backwards and we will obtain:

                u  +  v2  +  uv2  =2(uu)  +  2(vv)=2u2  +  2v2 \parallel u \; + \; v \parallel^{2} \; + \; \parallel u - v \parallel^{2} \; = 2(u \cdot u) \; + \; 2(v \cdot v) = 2 \parallel u \parallel^{2} \; + \; 2 \parallel v \parallel^{2}
                Equation 30: Solving the left hand side of the identity (part 4)

                Which proves that equation 26 holds true.

                Example 7

                Suppose vector uu is orthogonal to vectors xx and yy. Show that u is also orthogonal to x+yx+y.
                To solve this particular problem we need to realize that if vector uu is orthogonal to vectors xx and yy, then the next conditions hold true: ux=0u \cdot x = 0 \, and uy=0\, u \cdot y = 0 . And so, for uu to be orthogonal to x+yx+y, then we need to prove that u(x+y)=0u \cdot (x+y)=0. So let us compute that:

                u(x+y)=ux  +  uy=0  +  0=0 u \cdot (x+y)=u \cdot x \; + \; u \cdot y = 0 \; + \; 0 = 0
                Equation 31: Showing orthogonality

                As you can see above, due to the distributive property, we can distribute the dot inner product of uu with xx and yy separately. Since we know that both of those product are zero (is the initial condition of this question) then, the result is that the dot product u(x+y)u \cdot (x+y) is also equal to zero, making uu orthogonal to x+yx+y.

                ***

                To finalize this lesson we would like to recommend you to visit the next wonderful notes on the inner product, length and orthogonality and this handout on the same topic containing some extra example problems.
                This is it for our lesson of today, see you in the next one!
                Let vectors uu and vv be:
                vector u and vector v

                Then the inner product of the two vectors will be:
                inner product of vector u and vector v

                Let u,vu,v and ww be vectors in Rn\Bbb{R}^n, and let cc be a scalar. Then,
                1) uv=vuu \cdot v=v \cdot u
                2) w(u+v)=wu+wv w(u+v)=w \cdot u+w \cdot v
                3) (cu)v=c(uv)=u(cv) (cu) \cdot v=c(u \cdot v)=u \cdot (cv)
                4) uu0u \cdot u \geq 0, and uu=0u \cdot u=0 only if u=0u=0

                Suppose vector v. Then the length (or norm) of a vector vv is

                v=v12+v22++vn2  ,  v2=vv \lVert v \rVert = \sqrt{v_{1}^2+v_{2}^2+\cdots +v_{n}^2}\;,\; \lVert v \rVert ^2 = v \cdot v

                Suppose dist(u,v)(u,v) is the distance between the vectors uu and vv. To find the distance between the two vectors, we calculate
                dist(u,v)=uv(u,v)=\lVert u-v \rVert

                If two vectors uu and vv are orthogonal to each other, then it must be true that
                uv=0u \cdot v =0