Eigenvalues and Eigenvectors

From Math Images
Revision as of 06:15, 26 July 2013 by Xtian1 (talk | contribs)
Jump to: navigation, search
This is a Helper Page for:
Social Networks
Systems of Linear Differential Equations

Introduction

If you are familiar with matrix multiplication, you know that a matrix times a vector gives out some vector, and most vectors resulting from the product of a vector and a matrix turns out different than the original vector,

i.e. A\vec{v}=\vec{x} where \vec{v} and \vec{x} are different vectors.

What if I told you that there are certain vectors that when multiplied to a matrix, the final vector is the same vector multiplied by a scalar?

i.e. A\vec{v}=\lambda\vec{v}, where \lambda is a scalar.

Amazing right? These vectors are called eigenvectors. We can figure out how to compute these vectors from any square (rows and columns are the same) matrix!

Definition: For a square matrix A and a non-zero vector \vec{v}, \vec{v}is an eigenvector of A if and only if \lambda\vec{v}=A\vec{v}, where \lambda is an eigenvalue of A associated with \vec{v}.

Before we begin our analysis, here are some concepts you need to be familiar with:

  • Determinant
    • Is a value attributed to a square matrix.
  • Invertibility
    • Definition can be found in Matrix
    • A square matrix A is not invertible if and only if the determinant of A is zero. Or,  det(A)= 0 implies that A is not invertible.
    • If a square matrix A is invertible, then  A\vec{v} = \vec{k} has only one solution \vec{v} for all \vec{k}.
  • Identity Matrix
    • Definition can be found in Matrix

Eigenvalues

In this section, we will see how certain values will change the entries of an original matrix A so that it can have eigenvectors. Let's try to find the eigenvector given that we know the definition:

 A\vec{v}=\lambda\vec{v}
 A\vec{v}-\lambda\vec{v}=\vec{0}
 A\vec{v}-\lambda I \vec{v}=\vec{0}, where I is the identity matrix.
(A-\lambda I)\vec{v}=\vec{0}

So we have B\vec{v}=\vec{0}, where B is the square matrix (A - λI).

If B is invertible, then the only solution \vec{v} to B\vec{v}=\vec{0} is the zero vector. But \vec{v} cannot be zero (non-trivial), so we know that the zero vector cannot be the only solution. Therefore, B is not invertible. Since B is not invertible, we know that its determinant will equal zero!

We have to ask ourselves: what values can make B have a determinant of zero? Well we can take the determinant of (A-\lambda I) (which B represented) and see for what \lambda that determinant will equal zero. Since the only unknown value is λ and the determinant of a n×n matrix is just a nth degree polynomial (we call this the characteristic polynomial), we know this method always works. The values of \lambda are called eigenvalues, and we will work through a simple example.

Example

Suppose we have

 A = \begin{bmatrix} 4 & -5 \\ 2 & -3 \end{bmatrix} .

To find the eigenvalues of A, we find the determinant of (A - λI):

 \begin{align}
|A-\lambda I| &= \begin{vmatrix} 4-\lambda & -5 \\ 2 & -3-\lambda \end{vmatrix} \\
&= (4-\lambda)(-3-\lambda) + 10 \\
&= \lambda^2 - \lambda - 2 \\
&= (\lambda - 2)(\lambda + 1) = 0 \\
\end{align}

So our eigenvalues are 2 and -1. We write these as λ1 = 2 and λ2 = -1. In general, an n×n matrix will have n eigenvalues because an nth degree polynomial will typically have n solutions (given that there are no repeated solutions).

The General 2×2 Case

Consider the general 2×2 matrix

 A = \begin{bmatrix} a & b \\ c & d \end{bmatrix} .

We find the eigenvalues of A by finding the determinant of (A - λI):

 \begin{align}
|A-\lambda I| &= \begin{vmatrix} a-\lambda & b \\ c & d-\lambda \end{vmatrix} \\
&= (a-\lambda)(d-\lambda) - bc \\
&= \lambda^2 - (a+d)\lambda + (ad - bc) = 0 \end{align}

Using the quadratic formula, we see that our eigenvalues are

\lambda_1 = \frac{1}{2} \left( a+d + \sqrt{(a+d)^2 - 4(ad-ac)} \right)
\lambda_2 = \frac{1}{2} \left( a+d - \sqrt{(a+d)^2 - 4(ad-ac)} \right).

Note that we will typically have two eigenvalues for a 2×2 matrix (except in the case when we have repeating solutions), since we're solving for a second-degree polynomial. Likewise, an n×n matrix will probably have n eigenvalues because an nth degree polynomial will typically have n solutions (once again given that there are no repeated solutions).

Another way of writing the Characteristic Polynomial

We define the trace of a matrix as the sum of its diagonal entries. So the trace of A (or tr(A)) is a + d. Remember that the determinant of A is ad - bc. We can rewrite the characteristic polynomial of a general 2×2 matrix as

\begin{align}0 &= \lambda^2 - (a+d)\lambda + (ad - bc) \\
&= \lambda^2 - \text{tr} (A) \lambda + \text{det} (A) = 0 \end{align}

Eigenvectors

After finding the eigenvalues of A, we must find the corresponding eigenvectors. Since we already have the equation (A - \lambda I)\vec{v} = \vec{0} available, why don't we use it? Plug one of the value of λ for A into the equation and find vector(s) that satisfy the equation.

We continue our previous example to illustrate this idea.

Example

Remember that for our matrix

 A = \begin{bmatrix} 4 & -5 \\ 2 & -3 \end{bmatrix} , we have λ1 = 2 and λ2 = -1.

In this case, we will have two eigenvectors, one for each eigenvalue. Note that this is not always true. If for a n×n matrix there are fewer than n eigenvalues (repeated solutions for our polynomial), then at least one eigenvalue will have more than one eigenvector that corresponds to it. However in our case, we have two eigenvalues for a 2×2 matrix, so we know we will definitely have one eigenvector per eigenvalue.

We plug our λ's into the equation (A - \lambda I)\vec{v} = \vec{0} and find our eigenvectors.

λ1 = 2:  \begin{bmatrix} 4-2 & -5 \\ 2 & -3-2 \end{bmatrix} = \begin{bmatrix} 2 & -5 \\ 2 & -5 \end{bmatrix} Note that the two rows of the matrix are identical. So what vector, when multiplied by this matrix, gives the zero vector? One that could work is \vec{v}_1 = \begin{bmatrix} 5 \\ 2 \end{bmatrix}. This is our first eigenvector.
λ2 = -1:  \begin{bmatrix} 4-(-1) & -5 \\ 2 & -3-(-1) \end{bmatrix} = \begin{bmatrix} 5 & -5 \\ 2 & -2 \end{bmatrix} While the two rows of the matrix aren't identical, they are scalar multiples of each other. A vector that works is \vec{v}_2 = \begin{bmatrix} 1 \\ 1 \end{bmatrix} . This is our second eigenvector.

Thus we have the eigenvalues and the corresponding eigenvectors for A.

A Property of Eigenvectors

In the previous example, note that there are actually infinitely many eigenvectors. For example, \vec{v}_2 = \begin{bmatrix} 2 \\ 2 \end{bmatrix} works as well. However, all the possible eigenvectors for an eigenvalue are scalar multiples of each other. We defined eigenvalues and eigenvectors by stating

 A \vec{v} = \lambda \vec{v} , where \vec{v} is the eigenvector and λ is the eigenvalue of A.

From here, we see that the vector  c\vec{v} for any scalar c is an eigenvector too.

 \begin{align} A (c\vec{v}) &= c(A \vec{v}) \\
&= c(\lambda \vec{v}) = \lambda (c \vec{v}) \end{align} .

Another way to visualize this is to derive an equation for the eigenvectors in terms of their components. Suppose our eigenvector is

 \vec{v} = \begin{bmatrix} v_1 \\ v_2 \end{bmatrix} .

Then the equation (A - \lambda I)\vec{v} = \vec{0} can be written as

 \begin{bmatrix} a - \lambda & b \\ c & d - \lambda \end{bmatrix} \begin{bmatrix} v_1 \\ v_2 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix} .

Multiplying this out, we get two equations

 (a - \lambda) v_1 + bv_2 = 0 and  cv_1 + (d - \lambda) v_2 = 0 .

Since they are both equal to 0, we set them equal to each other and solve for v1 and v2.

 (a - \lambda) v_1 + bv_2 = cv_1 + (d - \lambda) v_2
 (a - \lambda) v_1 - cv_1 = (d - \lambda) v_2 - bv_2
 (a - \lambda - c) v_1 = (d - \lambda - b) v_2
 \frac{v_1}{v_2} = \frac{d - \lambda - b}{a - \lambda - c}

Thus the eigenvectors for an eigenvalue are actually a set of vectors. The eigenvector we wrote down is just the basis for this set of vectors.