3 Eigenvalues and Eigenvectors
By : Ali Mohamed Abu Oam asam295@hotmail.com
The purpose of this chapter is to introduce the concepts of eigenvalues and eigenvectors from both algebraic and geometric point of view under the following conclusion:
IF A is an nxn matrix then the following are equivalent:
a) The scalar 
is an eigenvalue of A.
b) The system of equations ( 
.I-A)x = 0 has nontrivial solution.
c) There is a nonzero vector x in 
such that Ax = 
x .
d) 
is a real solution of the characteristic equation det( 
.I-A)x = 0
e) An n by n matrix has at most n distinct eigenvalues.
The eigenvectors x of A corresponding to an eigenvalue 
are the nonzero vectors that satisfy: Ax = 
x. Equivalenty the eigenvectors corresponding to 
are the nonzero vectors in the solution space of (
I-A)x = 0. This solution space is the eigenspace of A corresponding to 
.
3.1 Computing Eigenvalues and Eigenvectors
The problem of finding nonzero solutions x to the matrix equation Ax = 
x arises in the formulation of many problems in engineering and physical sciences.
The differential equation y'' = - 
y that models the spring oscillation with y being the displacement from the equilibrium position and 
the spring constant is an example of such a problem with A being a second order linear differential transformation. This is called eigenvalue problem . The nonzero vector x is called the eigenvector corresponding to the eigenvalue 
.
3.1.1 Algebraic Solution:
Example:
Find the eigenvalues and the bases for the eigenspaces of the matrix A defined below:
| > |
A:=<<3,-2,0>|<-2,3,0>|<0,0,5>>; |
 |
(1) |
First load the package:
Warning, the protected names norm and trace have been redefined and unprotected
|
|
Then define a matrix and the identity of the same dimension:
| > |
A:=array([[3,-2,0],[-2,3,0],[0,0,5]]);
I3:=evalm(array(identity,1..3,1..3));
|
 |
(2) |
 |
(2) |
Where I3 is the identity matrix for the square matrix 3x3 , and evalm evaluates an expression involving matrices.
The Characteristic Polynomial:
1/ The Long way:
Let x be a vector defined below:
| > |
x:=Vector([[x1],[x2],[x3]]); |
 |
(3) |
The nonzero vectors x satisfying the equation:
| > |
evalm(A)*evalm(x)=lambda*evalm(x); |
 |
(4) |
This is equivalent to solving the matrix equation:
| > |
(evalm(A)-lambda*evalm(I3)*evalm(x))=matrix([[0],[0],[0]]); |
 |
(5) |
Or :
| > |
L:=evalm((A-lambda*I3)*(x))=matrix([[0],[0],[0]]); |
 |
(6) |
This homogeneous system posses a nontrivial solution if and only if the determinant of the coefficient matrix is equal to zero. That is, the coefficient matrix:
 |
(7) |
must have a zero determinant:
 |
(8) |
The resulting equation is called the Characteristic Polynomial of A.
The solution set of the characteristic polynomial is:
| > |
solve(det(B)=0,lambda); |
 |
(9) |
The elements of this solution set are the eigenvalues of A
2/ The Short Way :
Here use the command ( eigenvals( ) ) like this:
 |
(10) |
To compute the first eigenvector(eigenspace):
 |
(11) |
One can also compute an eigenvalue if he knows its corresponding eigenvector:
Let B be a matrix defined as:
| > |
B:=<<4,-3,1>|<4,-1,0>|<1,7,-4>>; |
 |
(12) |
and the vector w defined below is the eigenvector of B:
 |
(13) |
then the eigenvalue of ( B ) corresponding to the eigenvector w is
 |
(14) |
simply one can see that -3 is an eigenvalue corresponding to the eigenvector w ,( repeat that with the others eigenvectors to compute the others eigenvalues).
To find each eigenvalue and the eigenvector for the following matrix K :
| > |
K:=matrix(3,3,[0,1,0,0,0,1,4,-17,8]); |
 |
(15) |
Then define the following:
 |
(16) |
 |
(17) |
To find the first eigenvalue of K :
 |
(18) |
to find its multiplicity:
 |
(19) |
to find its eigenvector:
 |
(20) |
the 2nd eigenvalue:
 |
(21) |
its multiplicity:
 |
(22) |
its eigenvector:
 |
(23) |
3.1.2 Eigenvalues and Eigenvectors of the power Matrix
When we compute the eigenvalues and the eigenvectors of a matrix T ,we can deduce the eigenvalues and eigenvectors of a great many other matrices that are derived from T ,and every eigenvector of T is also an eigenvector of the matrices 
, 
,..., 
.
Example: Let T be a 3x3 matrix defined below:
| > |
T:=array([[-1,0,1],[0,4,0],[5,1,-1]]); |
 |
(24) |
Now find the eigenvalues and eigenvectors of T :
| > |
evT:=[eigenvectors(T)]; |
 |
(25) |
In this set of triples,the first entry of each is an eigenvalue,the 2nd is its multiplicity and the 3rd entry is an eigenvector.
To find the 2nd power of T :
 |
(26) |
To find eigenvectors :
| > |
ev(T^2):=[eigenvectors(T^2)]; |
 |
(27) |
Simply you can see we have the same eigenvectors as for T .
To repeat for great power:
 |
(28) |
To check eigenvectors:
| > |
ev(T^30):=[eigenvectors(T^30)]; |
 |
(29) |
Also we have the same eigenvectors. Where the eigenvalues have been changed,but their multiplicity will not.
3.1.3 Using Eigenvalues and Eigenvectors to ease Computation :
Constructing diagonalizable matrix which has specified eigenvalues and eigenvectors:
We will see how to use the equation M = KN(1/K) for this purpose, where N is diagonal with entries that are eigenvalues and K the matrix whose columns are eigenvectors of M . We first randomly choose the eigenvalues, and then the eigenvectors corresponding to each of these eigenvalues.
| > |
restart:with(linalg):
randomize():
|
Warning, the protected names norm and trace have been redefined and unprotected
|
|
| > |
N:=matrix(3,3,[[rand(-3..3)(),0,0],[0,rand(-3..3)(),0],[0,0,rand(-3..3)()]]); |
 |
(30) |
Thus, the diagonal of N supposed to be the eigenvalues of M.
(Now choose the eigenvectors randomly).
| > |
K1:=randvector(3,entries=rand(-3..3));
K2:=randvector(3,entries=rand(-3..3));
K3:=randvector(3,entries=rand(-3..3));
|
 |
(31) |
 |
(31) |
 |
(31) |
construct the matrix K by assembling K1, K2, K3 as its columns:
 |
(32) |
Now check that {K1,K2,K3} is independent by the rank, if the rank of K is not 3, then the set {K1,K2,K3}is not independent, here is:
 |
(33) |
Then compute the matrix M :
| > |
M:=evalm(K&*N&*inverse(K)); |
 |
(34) |
One can check, using the following command:
 |
(35) |
3.1.4 Geometric Solution (Linear Algebra Visualization) :
| > |
restart:
with(Student[LinearAlgebra]):
|
Warning, the protected name . has been redefined and unprotected
|
|
Where infolevel provides the user informations like entries,functions,integers,plot,animation,..etc..
To illustrate the concepts of eigenvalues and eigenvectors, the EigenPlot command shows the action of a given square matrix on selection of unit vectors emanating from the origin. The image vectors are drawn with tails at the tips of the corresponding input unit vectors (to prevent the plot from being too cluttered). The eigenvectors of the matrix are unit vectors whose images under the action of the matrix are colinear with the original vectors. In the following plot, there are two eigenvectors which are drawn in red and blue colors. The EigenPlot command, also gives for each eigenvalue its multiplicity and its corresponding eigenvectors.
| > |
EigenPlot(<<1,2>|<1,-1>>); |
| Eigenvector: < 1.366, 1. > |
|
| Eigenvector: < -.366, 1. > |
|
To see the input unit vectors as well, use the showunitvectors option.
| > |
EigenPlot(<<1,2>|<1,-1>>,showunitvectors); |
| Eigenvector: < 1.366, 1. > |
|
| Eigenvector: < -.366, 1. > |
|
Because the input vectors are unit vectors, the lengths and directions of the eigenvectors are determined by the corresponding eigenvalues, In the case of a zero eigenvalue, the visual information about the corresponding eigenvector is therefore lost.
To recover an indication of this, the eigenspaces are indicated by thin lines drawn in the same colors as the corresponding eigenvectors. See the following example ( in blue ):
| > |
EigenPlot(<<-1,0>|<0,0>>); |
In the 3-D case the eigenvectors corresponding to negative eigenvalues are shifted so that they remain outside the unitsphere, this reduces the visual clutter and allows you to see the action of the matrix more clearly ( in blue ) :
| > |
EigenPlot(<<1,2,3>|<2,0,1>|<1,-1,0>>); |
| Eigenvector: < .8422, .2468, 1. > |
|
| Eigenvector: < -1.277, 1.577, 1. > |
|
| Eigenvector: < .3577, -.593, 1. > |
|
To see the picture of the eigenvectors only, see the following example:
In this example we have many subcommands to be explained:
showsphere= false : not to plot the sphere.
showimagevectors= false : not to display the images of the unit vectors under the action of L.
eigenoptions: to display the eigenvectors in the required shape.
shape = harpoon: indicates that arrow is drawn as a line with single line indicating the head.
Hence the plot below shows only the three eigenvectors of L .
| > |
L:=<<1,2,3>|<2,0,1>|<1,-1,0>>; |
 |
(36) |
| > |
EigenPlot(L,showsphere=false,showimagevectors=false,eigenoptions
=[shape=harpoon],title="The Images of Eigenvectors");
|
| Eigenvector: < .8422, .2468, 1. > |
|
| Eigenvector: < -1.277, 1.577, 1. > |
|
| Eigenvector: < .3577, -.593, 1. > |
|
Note: This command ( EigenPlot ) , plots only real eigenvectors. Also a matrix may have fewer eigenvectors than its dimension. For either of these reasons the plot may show fewer eigenvectors than might otherwise be expected. The two following examples illustrate this fact:
| > |
EigenPlot(RotationMatrix(1)); |
| Eigenvalue: .5403+.8415*I |
|
| Eigenvalue: .5403-.8415*I |
|
In the above plot because the eigenvalues are not real, the corresponding eigenvectors were not shown, and the images of unit vectors laid out side the unitsphere. Where RotationMatrix(1) : command returns the 2x2 rotation matrix corresponding to the angle 1 measured in radians.
| > |
EigenPlot(<<1,0>|<1,1>>); |
Only one eigenvector was shown in red color to indicate that eigenvectors may be fewer than the dimension of the matrix.
3.1.5 Linear Transformations :
First: The LinearTransformPlot(M) command, where M is an mxn matrix, plot the transformation of the interval [-1,1], the unit circle, or the unit sphere for m=1,2 or 3 respectively, to the line, plane or 3-space for m=1,2 or 3 respectively under the action of M and gives the rank and the norm of M only.
For M is an nxn matrix the LinearTransformPlot command gives the rank,norm,det.,and for each eigenvalue gives its multiplicity and its corresponding eigenvectors. See the following examples:
| > |
LinearTransformPlot(<<1,2>|<0,-1>|<3,0>>,normalize=false); |
Here I inserted the subcommand normalize = false because if it set to true or not been inserted, the image object (on the left) is scaled by the inverse of the norm of M, so that the domain object and image object will have similar properties.
The following example for the square :
| > |
LinearTransformPlot(<<1/2,-5/3,0>|<0,4/3,0>|<-3/2,5/3,-1>>); |
| Eigenvector: < 1/2, 1, 0 > |
|
For square matrix is not recommended to use the subcommand normalize = false or normalize = true.
FOR the Linear Transformations of the 2x2 matrix see the following example:
| > |
LinearTransformPlot(<<1,0>|<-1,3>>,labels=[["x","y"],["u","v"]],
labeloptions=[font=[TIMES,ROMAN,12],color=red],titlefont=[TIMES,
ROMAN,20]);
|
Where Label: specifies the names to use as axis labels in the domain and range images.
labels= [["x","y"],["u","v"]]: the first list used for the image axes and the second for the domain axes.
labeloptions: provides options for the display of the label, here is for font and color.
[TIMES,ROMAN,12]: to determine the shape of the letters in size 12 for the axes of domain and range.
titlefont = [TIMES,ROMAN,20]: for the title of the plot in size for letters = 20.
FOR: 2x2 matrix the title is "The image of the unit Circle in the Plane".
2x3 matrix the title is "The image of the unit Sphere in the Plane".
3x3 matrix the title is "The image of the unit Sphere in the 3-Space".
( all the above plots can be animated ).
Second: The ApplyLinearTransformPlot: command used to show the action of a given linear transformation as applied to different domain objects. These input objects can be any of a regular set of objects( circle, square, sphere, cube, or grid ) or an arbitrary plotobject. The default object is a circle in the 2-D case and sphere in the 3-D case. Also, by default, 4 successive applications of the Transformation are applied ( where the input at each step after the first is the output from the previous step).
| > |
ApplyLinearTransformPlot(<<0,-1>|<1/2,1>>); |
| Eigenvector: < 1/2-1/2*I, 1 > |
|
| Eigenvector: < 1/2+1/2*I, 1 > |
|
| > |
P:=plot(cos(x),x=-Pi..Pi):
P;
|
Here to see Transformation under the action of the function, cos(x), where x belongs to the interval [-Pi..Pi].
| > |
ApplyLinearTransformPlot(<<0,-1>|<1/2,1>>,P); |
| Eigenvector: < 1/2-1/2*I, 1 > |
|
| Eigenvector: < 1/2+1/2*I, 1 > |
|
To animate the plot, use the following command:
| > |
ApplyLinearTransformPlot(<<1,0,-3>|<2,-1,1>|<1,-1,2>>,output=animation); |
| Eigenvector: < 1/6, -1/2, 1 > |
|
| Eigenvector: < .375-.3307*I, -.375+.3307*I, 1. > |
|
| Eigenvector: < .375+.3307*I, -.375-.3307*I, 1. > |
|
To see the action of the Transformation more clearly, use the iterations and trace options.
( the plot is well clearly under animation ).
| > |
ApplyLinearTransformPlot(<<1,1/2>|<-1/3,3/5>>,output=animation,
iterations=20,trace=6);
|
| Eigenvector: < .4+.7118*I, 1. > |
|
| Eigenvector: < .4-.7118*I, 1. > |
|
WHERE : iteration to give the numbers of the applications of a linear transformation.
trace function returns an expression sequence of the names of the procedures for which tracing was turned on ( off ). Here to determine how many transformations are there in the domain.
