Properties of matrix multiplication

Properties of matrix multiplication

A lot has been said and seen about matrix multiplication by now, at this time you already know how to multiply matrices (either among them, or with a scalar) and what type of results to expect; still, we would like to dedicate a complete lesson on the properties of it in order to show a few examples on how they can be used and how they reign this particular operations between two or more matrices.

How to multiply two matrices

Matrix multiplication is an operation descending from the dot product of vectors, since it actually consists in of a matrix dot product operation affecting an array of vectors performed in an orderly manner from a linear map described by the two (or more) matrices involved.

This can be seen clearly in the next diagram, which at this point, you must very familiarized with:


Just as explained in our lesson on matrix multiplication, the dot product occurring in this operation of matrices is the multiplication of each of the corresponding terms between a horizontal vector (row vectors from the rows of the first matrix) and a vertical vector (column vectors from the columns of the second matrix in the multiplication), and this same dot product occuring to produce each of the element entries of the resulting matrix.

A dot product is also called a scalar product because the result of two vectors in this manner is always a scalar number. For a matrix multiplication, the scalars resulting from each of the dot products described in the paragraph above happen to be the entries of the resulting matrix, hence, the way the multiplications of the row vectors with the columns vectors end up producing a matrix which may have a different set of dimensions as the original, but still a combination of them both, by always resulting with the same number of rows from the first matrix and the same number of columns as the second matrix (since the order in which the scalar products are performed is from the rows of the first matrix and the columns of the second).

In general, the dot products occurring while multiplying matrices can be seen in the right hand side of the equation below, producing the entries of the resulting matrix.


On equation 2 above we can see an example of a multiplication of two square matrices, notice the matrix properties of those involved will always affect the properties of the resulting matrix. For the case of multiplication of square matrices, the dimensions of the resulting matrix will be the same as those which produced it.

With that in mind, let us focus on learn about the specific properties of matrix multiplication in the next section.

Properties of matrix multiplication in linear algebra

Let $X$, $Y$, and $Z$ be matrices, $I_{n}$ be and identity matrix and $0_{n}$ be a zero matrix. Now let all of these five matrices to be square matrices of the same order (same dimensions), the next matrix multiplication properties hold true:
When multiplying two matrices the result is not always defined. Matrix multiplication rules tell us that in order for such multiplication to produce a defined resulting matrix, the first matrix must have the same amount of columns as the second matrix has rows. That being said, the dimensions of the resulting matrix will be defined by the amount of rows in the first matrix and the number of columns in the second matrix (check equation 1).

The dot product of matrices of the same size, which are square matrices such as in the case for the five defined matrices in this section, produces a matrix of the same size as those involved in the multiplication.

The associative property of matrix multiplication gives the opportunity to perform a multiple-matrix multiplication in steps by associating together two matrices, performing their multiplication, and then using its result as a new factor instead of the two which generated it.

At the same time, the matrix multiplication associative rule allow us to select which set of matrices to multiply together first. Using the matrices $X$, $Y$ and $Z$ defined on this section, this associative property says that:

$X \cdot Y \cdot Z = (X \cdot Y) \cdot Z = X \cdot (Y \cdot Z)$ Notice we did not change the order in which the matrices are multiplied we only associated two of them to multiply first, and then use its result to apply in the multiplication with the one that is left. This property is very useful when we have multiplications of matrices in larger quantities than two (matrix chain multiplication), since it allows us to take steps to simplify the large multiplication by associating matrices within themselves until you have a simpler matrix times matrix operation.

The distributive properties of matrix multiplication come to place when an operation of the matrix multiplication is combined with another arithmetic operation, such as an addition or subtraction. In other words, we use the distributive property to simplify problems in which one of the factors in the matrices multiplication is an addition or a subtraction of matrices.

Mathematically speaking the distributive property can be defined if two ways:
When a matrix is multiplying a factor containing either an addition of subtraction of two (or more) matrices. In which case the operation would look like:

$X (Y+Z) = XY + X$
Or, when a term containing an addition or subtraction of two (or more) matrices is multiplying a matrix:

$(Y+Z)X = YX + ZX$
Notice that in both cases the order in which the matrices are being multiplied is conserved, we will talk about this on the next section of our lesson, since is related to the lack of commutative properties of matrix multiplication. For now, a few details are worth mentioning: Since $X$, $Y$ and $Z$ are all square matrices of the same order (size), the resulting matrices will have the same dimensions as them, but if we were to use matrices of different dimensions, these multiplications would have to follow the property of dimension rules and the resulting matrices would have dimensions according to that.

This property dictates that as long as the matrix product is defined, meaning that the multiplication follows the dimension properties and a result can be obtained, such result will always be the zero matrix itself. This rule applies to both scalar multiplication and matrix multiplication with matrices of any dimension, since as long as you have a zero matrix as a factor, the result will always end up being the zero matrix as long as the dimension property allows a result.

If you multiply matrices by a zero matrix or vice versa, the logic behind the operation is very simple. Given that all of the elements in a zero matrix are zeros, no matter which matrix you multiply to them, if a result can be defined it will produce a series of dot products by zero which will produce element entries equal to zero in the resulting matrix, and thus, a zero matrix.
So, if you multiply matrix X by the zero matrix (using the matrices defined at the beginning of this section), the result goes as follows:

$0_{n} \cdot X = 0_{n} \,$ or $\, X \cdot 0_{n} = 0_{n}$

Similarly to the way how multiplying matrix X times the zero matrix works, the multiplication of a matrix times the identity matrix always gives the same result: the original matrix which is being multiplied to the identity.

In simple words, when multiplying a matrix times an identity matrix (making sure the product is defined by following the dimension property) the result will always be the original, non-identity, matrix, and so:

$I_{n} \cdot X = X \,$ or $\, X \cdot I_{n} = X$

Which properties do not hold true for matrix multiplication

After studying the matrix multiplication properties above you may have noticed there is one property we have seen in other matrix operations which has not appeared here, and so, you may be wondering: Is matrix multiplication commutative? The short answer is NO, in general.

Remember you can multiply matrices of different dimensions as long as the first matrix contains the same amount of columns as the number of rows in the second matrix. If you look at the multiplication shown in equation 1, you will clearly see how changing the order of the matrices in the operation will not be able to produce a result since the characteristic of the columns on first = rows on second is not met.

But the dimension rules are not the only reason why the factors in a matrix multiplication cannot be commuted, the same holds in general for square matrices, which would have no issue corresponding to their dimensions. The result of a matrix multiplication of square matrices will change, in general, if you exchange the order in which the matrices are being multiplied. The issue arises from the order in which the elements inside the matrices are being operated in dot products, since changing the order of the matrices, changes the order in which these multiplications take place (except for certain special cases) and thus, they produce different element entries in the resulting matrix. This is defined as:

$XY \,$ ≠ $\, YX$
More on this will be explained and verified during the example exercises in the last section of this lesson, for now, it is important for you to know that although the commutative property does not hold for matrix multiplication in general, there are two special cases in which it does: the matrix multiplication with the zero matrix, and the matrix multiplication with the identity matrix.
These two cases were explained in our last section, and will be verified in the exercises in the next section.

Properties of matrix multiplication PROOF

On this section we will work through a variety of example problems designed to verify the properties involved in how to do matrix multiplication.

Example 1

On this problem we will verify the properties of matrix to matrix multiplication using the matrices defined below:



Using these, verify the following expressions and name the properties which are being proved.

This expression is the same as Equation 3, and therefore, the question here is: is matrix multiplication associative? We know it is, so let us prove it using the matrices above. We start by calculating the left hand side: $(XY)Z$






Now to compute the right hand side: $X(YZ)$







And so, we have proved that the associative property holds true.

The multiplication of matrix $X$ with the addition of $Y$ and $Z$ is part of the distributive property of matrix multiplication, as shown in equation 4. Therefore, let us verify it, we start by computing the left hand side: $X(Y+Z)$




Now to compute the right hand side: $XY + XZ$





And so, we have proved that the distributive property holds true.


Once more, we are to verify the distributive property on this case, but now the associated with equation 5. So, calculating the left hand side: $(Y+Z)X$




Computing the right hand side: $YX + ZX$





And so, we have proved that the distributive property holds true again.

Performing the matrix multiplication 2x2 of the zero matrix $0$ and $X$, we can easily observe that the result is a zero matrix equal to $0$.




Thus, the matrix to matrix multiplication property for the zero matrix holds true as shown in equation 6.

Multiplying 2x2 matrices I and Y, we can easily observe that the result is equal to the non-identity matrix.




Thus, the matrix to matrix multiplication property for the identity matrix holds true as shown in equation 7.

Example 2

On this matrix multiplication example we will show how the commutative property fails in general for matrix multiplication operations. Once more having the five matrices $X$, $Y$, $Z$, $0$ and $I$ as shown below:


Verify the next expressions:

Notice that dimensionally speaking there is no impediment for these two sides of the algebraic equation to be the same since both $X$ and $Y$ are square matrices of the same size, but, the elements inside them are multiplied in different order when the matrices get exchange, and so, the results of the multiplications in each side of this expression as not the same:


$\downarrow \,$ ≠


Just as before, the same happens on this example where the commutative property fails:

Example 3

Let us work with the dimension property of matrix multiplication for this example. Here, you are given a 2x4 matrix named $X$, a 3x3 matrix named $Y$ and a 4x3 matrix named $Z$. Are the matrix multiplications in the expressions below defined? If defined, show the dimensions of the resulting matrix for each case.

Note: Remember that the dimension property follows the rule: $(A \times B )(B \times C) = (A \times C)$
Where A is equal to the amount of rows in the first matrix, $B$ is the amount of columns in the first matrix which is equal to the amount of rows in the second matrix, and $C$ is the amount of columns in the seconds matrix. Thus, the resulting matrix has the same number of rows as the first one, and the same number of columns as the second one.

The expression describing the dimension relationship on this matrix multiplication goes as follows:

$(2 \times 4) \cdot (3\times3)=undefined$
Notice the multiplication is undefined because $X$ does not have the same number of columns as $Y$ has rows.

The dimension relationship on this matrix multiplication is:

$(2 \times 4) \cdot (4\times3)= 2 \times 3$
Since $X$ has a matching number of columns with the rows in $Z$, the matrix can be performed with the dimensions in the resulting matrix being 2 × 3.

For this case, the dimension relationship for the matrices on this multiplication is:

$(4 \times 3) \cdot (2\times4)= undefined$
Which is undefined due $Z$ having a number of columns different than the number of rows in $X$.

Here, $Z$ and $Y$ have a matrix multiplication dimension relationship of:

$(4 \times 3) \cdot (3 \times 3)= 4 \times 3$
Since $Z$ has a matching number of columns with the rows in $Y$, the matrix can be performed with the dimensions in the resulting matrix being 4 × 3.

For this multiplication we will have to use the associative property to simplify into a 2-factor multiplication. The complete dimension relationship goes as:

$(2 \times 4) \cdot (4 \times 3) \cdot (3 \times 3) = (2 \times 3) \cdot (3 \times 3) = (2 \times 3)$
As you can see, given this order in which the matrices are being multiplied there is a defined result, which is a matrix of dimensions 2 × 3.

Using the associative property once more to simplify into a 2-factor multiplication. The complete dimension relationship of the matrix multiplication $ZYX$ goes as:

$(4 \times 3) \cdot (3 \times 3) \cdot (2 \times 4) = (4 \times 3) \cdot (2 \times 4) = undefined$
Where the result is NOT defined due to the unmatching number of columns from the matrix resulting from $ZY$ with the rows in matrix $X$.

***
As we finalize this lesson, we recommend you to visit the next link on the properties of matrix multiplication which contains some more problems for you to practice.

We hope you have enjoyed this lesson, see you in the next one!