Vectors  and  Matrices

                                             By:   Ali Mohamed Abu oam          asam295@hotmail.com

       Short History:

  The beginning of matrices and determinants goes back to the second century BC although traces can be seen back  to the fourth century BC. However it was not until near the end of 17th century that the ideas reappeared and development really got underway.

  It is not surprising that the beginning of the matrices and determinants should arise through the study of systems of linear equations.The Babylonians studied problems which lead to simultaneous linear equations and some of  these preserved in clay tables which survive.

  The Chinese, between 200 BC and 100 BC ,came much closer to matrices than the Babylonians.Indeed it is fair to say that the text "Nine Chapters of the Mathematical Art"  written during the Han Dynasty gives the first  known example of matrix method.

       Definition:

 What is a matrix?

 Matrices are every where, if you have used a spreadsheet such as exel or Lotus or written a table , you have used a  matrix. Matrices make presentation of numbers clearer and make calculations easier to program. In other words, a matrix is a rectangular array of elements. The elements can be symbolic expressions or numbers.

What is a vector?

A vector is a special type of a matrix that has one row or one column. So there are two types of vectors: row vectors and column vectors.

  The purpose of this chapter is to introduce Vectors and Matrices in computations to ease their arithmetic properties.

       1.1         Vectors and Planes:

     1.1.1  Vectors:

  How to construct a vector:

First one can know how to construct a vector:

To construct a row vector:

To construct a column vector:

Or, do the following:

Also, it is useful to know how to construct a row vector or a column vector if its entries are equal:

We have a row vector each of its entries is -4 with 3 columns. To find a column vector each of its entries forexample is 7 with 5 rows:

Also, one can construct a general vector:

         Vector Sum:

 Here, use VectorSumPlot command which displays the sum of collection of vectors. The sum is displayed graphically by connecting the head of each vector in the collection to the tail of another vector in the collection. as there are many ways to form these connections, the routine by default displays all connections, effectively producing a lattice whose edges can be followed from the origin point that corresponds to the sum. By default, the sum vector is drawn in black.( first  load the package and then its short commands ):

Where, 'infolevel[Student]: = 1: ' provides the user's informations like entries, functions, integers, animation...etc...

To see this sum in plot:

We have the same result because the addition of vectors is commutative ( and of course the vectors have the same dimension). To do for the 3-D vectors:

To animate the plot to know the possible paths one at a time can be requested, use the option ' output ':

     Cross Product:

Now,  to know the different between the  'dotproduct' and the 'crossproduct' for the vectors. Let's define the two following random vectors a and b:

Where " rand( ) " determines the range of the integers.

To see the dot product:

The results are the same (commutative). To find the cross product, so as to see the difference between the two products:

The results are differ one is minus an other.

Now use the " CrossProductPlot" command. By default, the result is colored magenta unless we used options for that.

In the" CrossProductPlot " the colors can be adjusted, that is:

The result is red.( the colors are respectively for v1,v2 and the result ). Also other controls are global in nature, such as orientation, title and axes( to use more options ):

Where " orientation = [-50,50] " specifies the theta and phi angles of the point in 3-dimensions from which the plot is to be viewed. The default is at a point that is out perpendicular from the screen ( negative z axis ) so that the entire surface can be seen. the point is described in spherical coordinates where theta and phi are angles in degrees. with default 45 degrees in each case.

" view " indicates minimum and maximum coordinates to be displayed.

   1.1.2   Planes:

 To visualize planes and their related vectors in three dimensions  use " PlanePlot " command which specifies a plane in different ways. It also displays the plane and, optionally, a normal vector, a pair of basis vectors, a vector to a point on the plane from the origin, and the vector from the origin to the closet point on the plane to the origin.

To see a pair of vectors that form a basis for the plane (when added to the specified point),  use "showbasis" option:

(1 ): One can specify the plane via a set or  list of vectors (the basis) as follows  :

( 2 ): Or one can specify the plane via a vector -valued function of two parameters as follows  :

( 3 ): And also, one can specify the plane via an equation or algebraic expression in three variables. In this case the variables' names must be specified unless there are exactly three such names in the first argument:

In either the normal vector form or the basis vectors form if a point in the plane is not specified , then the plane passes through the origin.

    1.1.3    Projections:

   projecting vectors onto lines and planes is a common and important operation. To visualize this operation, use " ProjectionPlot" command which displays the vector being projected, the line or plane onto which it is being projected,the projected vector, and the projection onto the orthogonal complement of the projection line or plane.

Furthermore, dashed lines from the head of the input vector to the head of its projections onto the line or plane and its projection onto the corresponding orthogonal complement are drawn:

One can use  " show... " options to suppress these components and optionally request that the vector or vectors used to define the projection line or plane be shown as follows :

You can see the difference between the two plots when  the " show... " options used.

Also a plane to be projected on is specified as a set of or a list of vectors :

           1.2      Matrices:

 1.2.1  How to create a matrix?

( 1 ) IF a matrix has equal entries ( whatever how many rows or columns it has ): suppose we want to create a 4 by 6 matrix of each entry equal 2:

The third argument is a function that generates the elements.

( 2 ) BY using functional operators as the third argument to represent a function in two variables, the first of which is the row index and the second of which is the column index, thus we obtain a matrix the elements of which are the same as the number of the row:

Where 3 and 6 indicate rows and columns.

To create more complicated matrix, let's make a matrix, the elements of which x to the power of row number times y to the power of the column number:

If x and y are integers ( forexample x = 2 , y = -5):

Here are other methods for typing matrices:

One, can type any symbolic matrix:

    Special Types of Matrices:

* Diagonal Matrices:

To make a function that can generate a Diagonal matrix " Di " (where "D" is protected by the program ), the symbol <> denotes not equal, and fi is the finishing of the statement of if :

   * Lower Triagular Matrices:

Also one can make a function that generates the general form of a lower triangular matrix:

   * Upper Triangular Matrices:

A function that uses an if test to generate a general form of an upper triangular matrix is considered:

  1.2.2      Matrices Addition and Multiplication:

  Let's first define the two following matrices A and B :

Where " matrixplot " command defines a three-dimensional graph where the x and y coordinates represent the rows and columns indices of A respectively. The z values are the corresponding entries of A.

To add A to B use the " evalm " command , to evaluate:

We have the same result which means that : matrix addition is commutative

The "  % " denotes resent result, that means plot ( B+A ).Also one can add options to the plot, and for addition the matrices must have the same dimension. For multiplication use the following:

Here one can observed that algebraic results for the multiplication of A and B are not equal, and so their plots which indicated that matrix multiplication is not commutative, So we used the sign ( &* ) to denote this kind of multiplication ( we can't use " * " only because it denotes commutative multiplication ).

To use some options for the plot:

Where "hieghts = histogram " will tally the data into bars of equal area, and creates for each data set, and they are plotted on the same graph, and makes the plot closest to the viewer.

" style = patch ": for colored surface patch rendering.

Also the plot command valids for hilbert and Toeplitz matrices and for all kinds of matrices (these two matrices defined in chapter three ):

We knew that the multiplication of two matrices can be obtained only when the number of the columns of the first matrix is equal to the number of the rows of the second matrix. To use options:

 1.2.3       The Identity for Matrix Multiplication:

  The Identity for matrix multiplication is a matrix of zeros, with ones on the diagonal. In order to make a functional operator which does this, we need an " if " statement. The syntax is as follows:

" if " statement must end with " fi " the backwards " if " .

To make 7 by 7 identity matrix :

This identity matrix can be obtained directly like this:

In "LinearAlgebra" package, also one can obtain the identity directly as follows:

Now let's make a 7 by 7 random matrix which integers from 1 to 7 (by functional operator ):

Also one can make a random matrix directly:

Now, let's check:

The result is the same, and so " I7 " is the identity matrix of matrix G . To see the plot of G and its identity S( I7 ) :

This plot is for the identity of any 7 by 7 matrix whatever the entries are.

        1.3  Gauss Reduction and the Gauss-Jordan Method

 1.3.1   Gauss Reduction, the fast way:

First let's define a matrix A which entries are the coefficients of the variables in a set of linear equations:

And define a vector that contains the constant terms in the set of linear equations:

Now we make the augmented matrix:

Where "augment" command joins two or more matrices or vectors together horizontally. Then use "gausselim" command which does Gauss Reduction :

 The Gauss-Jordan Method, the fast way:

 we have the augmented matrix which we made previously, then:

To introduce "gaussjord" command which performs the Gauss-Jordan Method:

It gives the result immediately.

Or use " rref " command to see the reduced row echelon form of (Ab) which equivalent to " gaussjord " command:

   rref(Ab);

         Gauss Reduction, the long way:

Using the same augmented matrix:

Now we do Gauss Reduction using elementary row operations. First swap rows 1 and 2 using the following command:

Next, add row 1, multiplied by 2, to row 3 and replace row 3 with the result as follows:

Then, add row 2, multiplied by -1 to row 3 and replace row 3 with the result:

This is the Row-Echelon form of the augmented mat. Let's see if it agrees with what " gausselim" gives us:

We have agreement!!!

However, we were just lucky. The method of Gauss reduction is non-unique. Another choice of row operations would lead to a different form of the augmented matrix. See the following:

     Gauss Reduction is Non-Unique:

Performing Gauss Reduction the long way for our previous  augmented Matrix. However, we will do things a little differently, and thereby end up with a different form of the Row-Echelon matrix. remember the augmented matrix:

Instead of swapping rows 1 and 2, swap rows 1 and 3:

Next: add row 1, multiplied by 1/2, to row 2, and replace row 2 with the result:

Multiply row 2, by -2 in order to get rid of the fraction using " mulrow " command:

Finally, add row 2 to row 3, and replace  row 3 with the result:

So, matrix  "L1" is in Row-Echelon form, but it is not the same as which obtained before (L). We can see that:

They are definitely not the same, that is Gauss Reduction is non-unique.

          The Gauss-Jordan Method, the long way:

Let's consider a 3 by 4 matrix, A, which represents the augmented matrix for a system of 3 equations in 3 variables, and to follow the procedure of the row operations to obtain the Reduced Row-Echelon form of this augmented matrix:

Now, this is the Reduced Row-Echelon form of the augmented matrix. To see if it agrees with "gaussjord" or not:

We have agreement.While "Gauss Reduction" is non-unique, we have the "Gauss-Jordan" is unique. The full Gauss-Jordan Method does lead to unique results (we can use similar way to that in case of Gauss Reduction).

       1.3.2   How to Tell if a System is Inconsistent:

A system of equations that has no solution is said to be inconsistent.

Now let C be a 4 by 5 matrix which represents the augmented matrix for a system of 4 equations in 4 variables:

To obtain a Row-Echelon form:

So,the system of equations is inconsistent, because it says: " 0 = 4 " which isn't true.

To have a similar case but it is consistent: Let M be a 4 by 5 matrix which represents the augmented matrix for a system of 4 equations in 4 variables:

To find a Row-Echelon form:

Here the system is consistent. Don't get confused! It is not saying " -10 = 0", but it is saying -10x_4 = 0,(i.e; -10 multiplied by x4 = 0). That means x_4 = 0, so the system can be solved completely. Using "Gauss-Jordan Method":

  1.3.3    How to Tell a System has Free Variables:

A system With One Free Variable:

Now, consider a 4 by 5 matrix , A, which represents the augmented matrix for a system of 4 equations in 4 variables:   

The Row-Echelon form of this matrix A:

There is no pivot in column 3, which indicates that the 3rd variable x_3 is free. If we use Gauss-Jordan Method, we find the other three variables in terms of the free variable:

       A system with two Free Variables:

Consider a 4 by 5 matrix, B, which represents the augmented matrix for a system of 4 equations in 4 variables:

The Row-Echelon form of a matrix B:

There are no pivots in columns 3 and 4, which indicates that the 3rd and the 4th variables (x_3,x_4), are free. If  we use Gauss-Jordan Method, we'll find the other two variables in term of the two free variables:

       1.3.4    Using Elementary Matrices In Computations:

The purpose of this section is to show how to use Elementary Matrices for performing elementary row operations on matrices in computations. This will be illustrated by using Elementary Matrices to perform Gauss Reduction on an augmented matrix:

 A Sample System To work with:

 Use the following system: (our matrix A and Vector b which used before in Gauss Reduction):

That is our system to be used.

 Gauss Reduction, Using Elementary Matrices:

First we call an elementary 3 by 3 matrix L which is an identity 3 by 3 matrix:

Then do this: ( multiply row 1 of L by 2 and add to row 3 ):

Finally we do the following:

      Gauss Reduction, Using Elementary Row Operations:

Use Gauss Reduction to reduce the matrix "Ab" to Row Reduction form.

Multiply row 1 by 2 and add the result to row 3:

Multiply row 2 by -1 and add the result to row 3:

Now  check this with the previous result obtained before:

The two methods agreed.

    Using Elementary Matrices to do Gauss Reduction in one Step:

One can ask what is the use of elementary matrices? It seems a rather inefficient way to do Gauss Reduction when we can apply the row operations directly to the augmented matrix. However, elementary matrices have theoretical interest, because they can be used to do Gauss Reduction in one step, as follows:

Which is the same as "gausselim" command:

      1.3.5        Finding the Inverse of a Matrix:

 The purpose of this section is to show how to find the inverse of a matrix, with "inverse" command, and using Gauss-Jordan Method.

Now make a 6 by 6 matrix to work with:

To find the inverse of  T using "inverse" command:

To make sure that Q is indeed the inverse of  T:

That is okay.

        Finding the Inverse Using Gauss-Jordan Method:

First let's find a 6 by 6 identity matrix ( of  T ), call it C:

Now obtain the augmented matrix TC:

Then use Gauss-Jordan Method, the inverse of  T will appear to the right of the partition:  

To extract the inverse from the above, use "submatrix" command as follows:

Tinv : denotes the inverse of the matrix T, (1..6) denotes the dimension of the matrix and (7..12) the columns of the matrix that required.  And one can see that the result is equal to the inverse Q which obtained before.

         1.3.6    The Subspaces of a Matrix and Their Bases:

The purpose of this section is to illustrate the matrix concepts of row space, column space, and nullspace.

Now define a random 5 by 5 matrix whose elements are integers between -6 and 6:

The probability that this matrix is singular is only about 3 in 400. However, we like to work with a singular matrix,  otherwise the dimension of the nullspace will be zero. Therefore, let's make the last two rows dependent on the first three as follows:

Now the matrix N will be a singular matrix.

      Finding The nullspace of N through row reduction:

What is the nullspace ? The nullspace is a set of solutions of the homogeneous equations. To find the solutions, augment the matrix N with the zero vector, and use the Gauss-Jordan Method:

One can see that there are two free variables, as expected, because we set up N so that the 4th and the 5th rows would be dependent on the 1st, 2nd and the 3rd rows. Let's call the free variables r and s, then we have: x1 = 13r/11 + 12s/11,   x2 = -5r/11 - 19s/11,  x3 = -13r/11- 35s/22,  x4 = r and x5 = s. Or, in column vector notation, we have:

Which are the vectors that span the nullspace of the matrix N. In other words, they form a basis for the nullspace.

     Finding the nullspace of N using the "nullspace" command:

We can directly find the nullspace of N by using "nullspace" command as shown below:

How that can be compared to what we obtained through row reduction?

This is not the same basis! However, we can show that the two basis are equivalent, if one can be expressed in terms of a linear combination of the other. To form that, make a matrix out of the 4 columns vectors and use the Gauss-Jordan Method:

We therefore, see that w1 = -11v1/5,   w2 = v2-19v1/5. Indeed, the basis are equivalent. In other words they span the same space.

         The Column Space Of N:

  First: To find the column space of the matrix N through row reduction, let's call our matrix N;

Now, we will use "gausselim" command to obtain Row-Echelon form:

Here columns 1,2 and 3 form a basis for column space of N ( there are pivots only in them), to present these vectors:

So, the column space is spanned by b1, b2 and b3.

Second: Let's find the column space of N by using "colspace" command:

           The "colspace" (rowspace) is a function returns a set of column (row) vectors that form a basis for the vector space that spanned by the columns (rows) of a matrix. The vectors are returned in canonical form with leading entries 1. The column space (rowspace) of a zero matrix is an empty set. To examine this do the following:

Its clear that the column space of matrix A is an empty set because it is a zero matrix.

Now to find the column space of matrix N using "colspace" command, say it 'csp':

It seemed not the same basis we obtained before, to compare that and to show the two basis are equivalent, we expressed one in terms of linear combinations of the other: that is:

Now, we call the augmented matrix of the two basis:

Then use "gaussjord" command

Here, we got that:

csp1 = -b2/5 - b3/5,           csp2 = -3b1/11 + 2b2/11 + 3b3/11,     csp3 = -2b1/11 + 3b2/55 + 31b3/110.

So, the basis are equivalent and they span the same space.

          The Column Space Of N By Finding the row Space of the transpose of N:

 First, define the transpose of a matrix A:

1- If A is an m x n matrix, the result is an n x m matrix, where the [i,j]th element of the result is equal to the [j,i]th element of the element of A. The result inherits the indexing function (for example, diagonal or spare) of A, if it has one.

2- If A is a vector, then it is treated as if it was a column vector, "transpose(A)" would therefore be a row vector.

To do that:

Now: To continue finding the column space of N by finding the row space of the transpose of A. let's find the transpose of N:

We knew that the row space of Nt is the column space of N, hence, we find the row space of Nt using row reduction:

We see that rows 1, 2 and 3 are nonzero, so they form the basis of row space of Nt, which is the column space of N:

to compare our result with the column space of N obtained by "colspace" command:

Obviously we have the same result.

         The Row Space Of N:

  Finding The Row Space of N through Row Reduction:

First, call our old matrix N:

 Then,  do Gauss Reduction to Row-Echelon form:

The three nonzero rows 1, 2 and 3 form the basis for the row space. So the row space is given by the following three vectors:

Finding the row space of N using the "rowspace" command:

To compare the two results, one can be expressed in terms of linear combination of the other, to make this, make a matrix out of the vectors written as column vectors, and use the gauss-Jordan method:

Then we call the augmented matrix M1 as follows:

Finally we use "gaussjord" command:

We obtained that:

 rsp1 = -b1/4 - 3b2 + b3/11,         rsp2 = 4b2 - 7b3/55,       rsp3 = b3/110.

Which, obviously indicated that the basis are equivalent. To see whether we can obtain the basis form By "gaussjord" direct command with the matrix or not ?

So, the first three row vectors of the Reduced Row -Echelon form the basis of the row space which we obtained before.

  1.3.7  Finding the basis for a set of vectors:

  We knew in the previous sections, how to find a basis for the nullspace, the column space and the row space of a matrix. To know how to find a basis for any set of vectors, we put that set of vectors into a matrix, as column vectors, and do the same as we have done before for a matrix. To show that, make a set of the 6 following vectors:

Now:  find the basis using the column space of a matrix.

Call the augmented matrix of these vectors:

Finally use "gaussjord" command to find the basis:

Oh!! there are pivots in all the columns, so all the vectors form a basis for the space spanned by them. To check this by using "basis" command:

So, the two methods agreed. Also one can find the basis using the row space of the matrix similar to what we have known before.

  1.3.8  The Rank of a Matrix:

 Definition: The rank of a matrix is the dimension of the row space, which is also the dimension of the column space.( The rank of a matrix = the number of its columns - the dimension of its nullspace ). Now, one can use "rank" command:

In this case the nullspace of the matrix V ( the augmented matrix of the set of our six vectors) is an empty set. so that its dimension is zero( has a nullity of zero ).

Now ( remember our old matrix N ) we expect that its rank is 3 because it has a nullity of 2:

Also, we can make use of "gausselim" command to find the rank of the matrix and its determinant by adding some options to the command "gausselim(V,'r','d')", where r is the rank and d is the determinant of the submatrix (V,1..n,1..n), where n is the rank of the matrix V.

To do that for the matrix N:

Here we compared the result for the determinant of N by "det"command because N is a square matrix, for non-square matrices their ranks make them to be square matrices.