Eigenvalues and Eigenvectors

From Math Images
Revision as of 06:20, 26 July 2013 by Xtian1 (talk | contribs)
Jump to: navigation, search
This is a Helper Page for:
Social Networks
Systems of Linear Differential Equations


If you are familiar with matrix multiplication, you know that a matrix times a vector gives out some vector, and most vectors resulting from the product of a vector and a matrix turns out different than the original vector,

i.e. A\vec{v}=\vec{x} where \vec{v} and \vec{x} are different vectors.

What if I told you that there are certain vectors that when multiplied to a matrix, the final vector is the same vector multiplied by a scalar?

i.e. A\vec{v}=\lambda\vec{v}, where \lambda is a scalar.

Amazing right? These vectors are called eigenvectors. We can figure out how to compute these vectors from any square (rows and columns are the same) matrix!

Definition: For a square matrix A and a non-zero vector \vec{v}, \vec{v}is an eigenvector of A if and only if \lambda\vec{v}=A\vec{v}, where \lambda is an eigenvalue of A associated with \vec{v}.

Before we begin our analysis, here are some concepts you need to be familiar with:

  • Determinant
    • Is a value attributed to a square matrix.
  • Invertibility
    • Definition can be found in Matrix
    • A square matrix A is not invertible if and only if the determinant of A is zero. Or,  det(A)= 0 implies that A is not invertible.
    • If a square matrix A is invertible, then  A\vec{v} = \vec{k} has only one solution \vec{v} for all \vec{k}.
  • Identity Matrix
    • Definition can be found in Matrix


In this section, we will see how certain values will change the entries of an original matrix A so that it can have eigenvectors. Let's try to find the eigenvector given that we know the definition:

 A\vec{v}-\lambda I \vec{v}=\vec{0}, where I is the identity matrix.
(A-\lambda I)\vec{v}=\vec{0}

So we have B\vec{v}=\vec{0}, where B is the square matrix (A - λI).

If B is invertible, then the only solution \vec{v} to B\vec{v}=\vec{0} is the zero vector. But \vec{v} cannot be zero (non-trivial), so we know that the zero vector cannot be the only solution. Therefore, B is not invertible. Since B is not invertible, we know that its determinant will equal zero!

We have to ask ourselves: what values can make B have a determinant of zero? Well we can take the determinant of (A-\lambda I) (which B represented) and see for what \lambda that determinant will equal zero. Since the only unknown value is λ and the determinant of a n×n matrix is just a nth degree polynomial (we call this the characteristic polynomial), we know this method always works. The values of \lambda are called eigenvalues, and we will work through a simple example.


Suppose we have

 A = \begin{bmatrix} 4 & -5 \\ 2 & -3 \end{bmatrix} .

To find the eigenvalues of A, we find the determinant of (A - λI):

|A-\lambda I| &= \begin{vmatrix} 4-\lambda & -5 \\ 2 & -3-\lambda \end{vmatrix} \\
&= (4-\lambda)(-3-\lambda) + 10 \\
&= \lambda^2 - \lambda - 2 \\
&= (\lambda - 2)(\lambda + 1) = 0 \\

So our eigenvalues are 2 and -1. We write these as λ1 = 2 and λ2 = -1. In general, an n×n matrix will have n eigenvalues because an nth degree polynomial will typically have n solutions (given that there are no repeated solutions).

The General 2×2 Case

Consider the general 2×2 matrix

 A = \begin{bmatrix} a & b \\ c & d \end{bmatrix} .

We find the eigenvalues of A by finding the determinant of (A - λI):

|A-\lambda I| &= \begin{vmatrix} a-\lambda & b \\ c & d-\lambda \end{vmatrix} \\
&= (a-\lambda)(d-\lambda) - bc \\
&= \lambda^2 - (a+d)\lambda + (ad - bc) = 0 \end{align}

Using the quadratic formula, we see that our eigenvalues are

\lambda_1 = \frac{1}{2} \left( a+d + \sqrt{(a+d)^2 - 4(ad-ac)} \right)
\lambda_2 = \frac{1}{2} \left( a+d - \sqrt{(a+d)^2 - 4(ad-ac)} \right).

Note that we will typically have two eigenvalues for a 2×2 matrix (except in the case when we have repeating solutions), since we're solving for a second-degree polynomial. Likewise, an n×n matrix will probably have n eigenvalues because an nth degree polynomial will typically have n solutions (once again given that there are no repeated solutions).

Another way of writing the Characteristic Polynomial

We define the trace of a matrix as the sum of its diagonal entries. So the trace of A (or tr(A)) is a + d. Remember that the determinant of A is ad - bc. We can rewrite the characteristic polynomial of a general 2×2 matrix as

\begin{align}0 &= \lambda^2 - (a+d)\lambda + (ad - bc) \\
&= \lambda^2 - \text{tr} (A) \lambda + \text{det} (A) = 0 \end{align}


After finding the eigenvalues of A, we must find the corresponding eigenvectors. Since we already have the equation (A - \lambda I)\vec{v} = \vec{0} available, why don't we use it? Plug one of the value of λ for A into the equation and find vector(s) that satisfy the equation.

We continue our previous example to illustrate this idea.


Remember that for our matrix

 A = \begin{bmatrix} 4 & -5 \\ 2 & -3 \end{bmatrix} , we have λ1 = 2 and λ2 = -1.

In this case, we will have two eigenvectors, one for each eigenvalue. Note that this is not always true. If for a n×n matrix there are fewer than n eigenvalues (repeated solutions for our polynomial), then at least one eigenvalue will have more than one eigenvector that corresponds to it. However in our case, we have two eigenvalues for a 2×2 matrix, so we know we will definitely have one eigenvector per eigenvalue.

We plug our λ's into the equation (A - \lambda I)\vec{v} = \vec{0} and find our eigenvectors.

λ1 = 2:  \begin{bmatrix} 4-2 & -5 \\ 2 & -3-2 \end{bmatrix} = \begin{bmatrix} 2 & -5 \\ 2 & -5 \end{bmatrix} Note that the two rows of the matrix are identical. So what vector, when multiplied by this matrix, gives the zero vector? One that could work is \vec{v}_1 = \begin{bmatrix} 5 \\ 2 \end{bmatrix}. This is our first eigenvector.
λ2 = -1:  \begin{bmatrix} 4-(-1) & -5 \\ 2 & -3-(-1) \end{bmatrix} = \begin{bmatrix} 5 & -5 \\ 2 & -2 \end{bmatrix} While the two rows of the matrix aren't identical, they are scalar multiples of each other. A vector that works is \vec{v}_2 = \begin{bmatrix} 1 \\ 1 \end{bmatrix} . This is our second eigenvector.

Thus we have the eigenvalues and the corresponding eigenvectors for A.

A Property of Eigenvectors

In the previous example, note that there are actually infinitely many eigenvectors. For example, \vec{v}_2 = \begin{bmatrix} 2 \\ 2 \end{bmatrix} works as well. However, all the possible eigenvectors for an eigenvalue are scalar multiples of each other. We defined eigenvalues and eigenvectors by stating

 A \vec{v} = \lambda \vec{v} , where \vec{v} is the eigenvector and λ is the eigenvalue of A.

From here, we see that the vector  c\vec{v} for any scalar c is an eigenvector too.

 \begin{align} A (c\vec{v}) &= c(A \vec{v}) \\
&= c(\lambda \vec{v}) = \lambda (c \vec{v}) \end{align} .

Another way to visualize this is to derive an equation for the eigenvectors in terms of their components. Suppose our eigenvector is

 \vec{v} = \begin{bmatrix} v_1 \\ v_2 \end{bmatrix} .

Then the equation (A - \lambda I)\vec{v} = \vec{0} can be written as

 \begin{bmatrix} a - \lambda & b \\ c & d - \lambda \end{bmatrix} \begin{bmatrix} v_1 \\ v_2 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix} .

Multiplying this out, we get two equations

 (a - \lambda) v_1 + bv_2 = 0 and  cv_1 + (d - \lambda) v_2 = 0 .

Since they are both equal to 0, we set them equal to each other and solve for v1 and v2.

 (a - \lambda) v_1 + bv_2 = cv_1 + (d - \lambda) v_2
 (a - \lambda) v_1 - cv_1 = (d - \lambda) v_2 - bv_2
 (a - \lambda - c) v_1 = (d - \lambda - b) v_2
 \frac{v_1}{v_2} = \frac{d - \lambda - b}{a - \lambda - c}

We have shown that for an eigenvalue λ of A, the components of its eigenvectors maintain the same ratio. Thus multiplying the eigenvectors by scalar constants will still work, since this ratio is preserved.

We have shown that the eigenvectors for an eigenvalue are actually a set of vectors. The eigenvector we write down as THE eigenvector is just a basis for this set of vectors.