Systems of Linear Differential Equations
|U.S. and Soviet Union nuclear warheads|
U.S. and Soviet Union nuclear warheads
- Differential equations have always been a popular research topic due to their various applications. A system of linear differential equations is no exception; it can be used to model arms races, simple predator prey models, and more.
- 1 Basic Description
- 2 A More Mathematical Explanation
- 2.1 Solving the Two Dimensional System
- 2.2 Arms Race Example
- 2.3 The Complex Eigenvalue Case
- 2.4 Stability of the Two Dimensional System
- 2.5 The More Rigorous Proof
- 3 Why It's Interesting
- 4 Teaching Materials
- 5 References
In 1945, the United States dropped atomic bombs on the Japanese cities Hiroshima and Nagasaki, ending World War II and establishing itself as a new superpower. The Soviet Union, while an ally of the United States during WWII, feared the bomb and spent the next few years developing their own atomic bomb, finally detonating their first nuclear weapon in 1949. This marked the beginning of a long and expensive arms race between the two powers. The competition for nuclear might, along with the countries' different ideologies (communism vs. capitalism), caused a political and psychological war during the second half of the 20th century, now known as the Cold War. During this period, both powers invested tremendous resources into their technology and weaponry, worried that the other was pulling ahead.
We will attempt to build a simple model to see the effects of the nuclear arms race. What type of model would fit best? Consider these two images.
Note that, starting in 1947, the Soviet Union rapidly increased its defense spending (Figure 2). The United States responded in kind beginning in 1948 (Figure 1). From that point on, the spending of the two powers pushed and pulled, staying within a fairly narrow range (the two higher points of United States defense spending around 1953 and 1968 are related to the Korean and Vietnam Wars). These fluctuating changes are connected with each power's analysis of both its own defense spending and that of the other power. The more the United States is already spending on defense, the less willing it is to spend on defense the following year. However, the more the Soviet Union spends on defense, the more likely the United States will be to increase its defense spending to not be left behind. With this dynamic, we can analyze the defense spending of each country by analyzing the change in defense spending from year to year. Since derivatives are the mathematical tool to analyze change, we will build our model around derivatives.
Let x(t) and y(t) be the defense spending of the United States and the Soviet Union respectively at time t (in years). Then x′(t) is the derivative of x over t and it represents how much the U.S. spending changes over t years. Likewise, y′(t) is the derivative of y over t and it represents how much the Soviet spending changes over t years. As mentioned above, the rate of U.S. defense spending depends negatively on its current defense spending and positively on the Soviet Union's current defense spending. Correspondingly, the rate of Soviet defense spending depends negatively on its current defense spending and positively on the United States's current defense spending. The model's starting point is the year in which the arms race started, when t = 0. Let x0 and y0 be the initial(t = 0) defense spending of the United States and the Soviet Union respectively The following equations fit these conditions.
- , where a is negative and b is positive.
- , where c is positive and d is negative.
This is a linked system of first order linear differential equations. In other words, equation x′(t) states that the United States lowers its budget by a dollars for every dollar it spent the previous year, and raises it by b dollars for every dollar in the Soviet Union's budget. Equation y′(t) states that the Soviet Union lowers its budget by d for every dollar it spent the previous year, and raises it by c dollars for every dollar in the United States's budget.
Note: We must use our model with a degree of skepticism because there are far more factors affecting the arms race than just the two countries' defense spending, such as available money/resources and the spending requirements of other important programs. Nevertheless, our model is useful for analyzing the general outline of the Cold War arms race and predicting its outcomes.
Introduction to Systems of Linear Differential Equations
We clarify a few terms:
- A differential equation is an equation that contains both function(s) and their derivatives.
- A nth order differential equation only involves up to the nth derivative of any function.
- A linked system of differential equations has at least one equation that involves different functions or different derivatives of functions.
- The expressions of linear equations are linear combinations of functions.
Each equation in our system is a linear combination of functions x(t) and y(t) and both have one derivative of either x or y. Hence we have a linked system of first order linear differential equations.
We can use eigentheory to solve for any system of first order linear differential equations. Indeed, the outcome of our nuclear arms race model depends on the eigenvalues (read the More Mathematical Explanation for details).
A More Mathematical Explanation
- Note: understanding of this explanation requires: *Linear Algebra, High school algebra, Single Variable Calculus
Solving the Two Dimensional SystemIn order to understand how a system of linear differential equ [...]
Solving the Two Dimensional System
In order to understand how a system of linear differential equations is useful in making sense of the Cold War Arms Race, we first need to understand the general system.
The general system of linear differential equations in two dimensions is
Since we are solving the general system, the coefficients a, b, c, and d are elements of ALL numbers (including imaginary numbers).
We can write this system with matrices in the form
- , where , , and .
In order to gain an understanding on how to solve this equation, we consider the one dimensional case.
The One Dimensional Case
If the two dimensional case involved the equation : with A a 2x2 matrix, then the A in the equation of the one dimensional case must be a 1x1 matrix, or just a constant. Consequently, will actually be u instead; it is no longer a vector. Hence the one dimensional equation is
- , where k is a real number.
Solving this requires basic knowledge of calculus. We can write this as
- (remember that u is a function of t)
- (rearranged variables).
Integrate both sides:
- . (So C = ec.)
Now we can apply the initial condition. Note that
- , so u(0) = C.
Thus the solution to the one dimensional case is
Back to the Two Dimensional System
We can make some connections between the one and two dimensional system. We see that A in the two dimensional case is like the k in the one dimensional case. The initial condition is also a vector instead of a scalar. Thus the solution to the one dimensional case suggests that the solution to our system is . We will explore the concept of raising e to a matrix later. Since the solution to the one dimensional case is u = Cekt, a good guess of the solution to the two dimensional case is for some scalar λ (we don't know what this is yet) and . This seemingly random equation will start to make sense once we plug our guess into the original equations and find λ and that work. We write .
We verify our guess by plugging it into our original system:
Now we must account for the fact that a 2x2 matrix can have two eigenvalues λ1 and λ2. Since differentiation is a linear operator, we know that the solution to our system is linear. We write the general solution as a linear combination
We now apply our initial condition. When t = 0, our general solution becomes
It is rare that and add up to the initial condition, so how do we solve this? Ideally, we multiply the two vectors by real number constants c1 and c2, but we must check that this still works for our system.
By multiplying constants to each term of our solution expression, our new proposed solution is
We verify this by plugging into our matrix equation .
In fact, multiplying constants to the terms is equivalent to finding a linear combination of our basis vectors and . Remember that differentiation is a linear transformation, so the linear combinations of existing solutions are naturally solutions as well.
Now that we have a more refined general solution, we need to know how to solve for the constants. Our initial condition now becomes
We can rewrite this as
- , which can be solved using Gaussian elimination.
Thus we have our particular solution for any initial condition. With the constants solvable, our general solution is
Arms Race Example
We will solve a system that follows the rules of our arms race model. Since the real Cold War arms race is hard to model, we will make assumptions on the actions of the United States and the Soviet Union. Assume that the United States lowers its budget by 3 dollars for every dollar it spent the previous year, and raises it by 6 dollars for every dollar in the Soviet Union's budget. Further assume that the Soviet Union lowers its budget by d for every dollar it spent the previous year, and raises it by c dollars for every dollar in the United States's budget. Finally assume that the United States defense spending is 130 billion dollars and the Soviet Union defense spending is 150 billion dollars at t = 0.
All of the constants are made up. The values of x and y are in tens of billions of dollars.
We can rewrite this as
To find the eigenvalues of A, remember that λ is an eigenvalue if and only if the determinant of (A - λI) is 0:
So our eigenvalues are λ = -8, 3. Using the eigenvalues, find the eigenvectors:
- For λ1 = -8: , so .
- For λ2 = 3: , so .
Now we will find the constants for our initial condition. Remember that we can use Gaussian elimination to find them.
- Start with , after Gaussian elimination --> , so c1 = 2 ⁄ 11 and c2 = 155 ⁄ 11.
Finally, we just plug all of our values back into our general solution to get our particular solution.
Thus we have the equations
These equations satisfy both the system of differential equations and the initial condition (check if doubtful).
We can now use these equations to help analyze the results of our hypothetical arms race. The results will happen many years later, so we can get a good estimate by taking the limit of x(t) and y(t) as t goes to ∞.
The results of this hypothetical arms race are disastrous; they indicate that tensions between the United States and Soviet will cause the arms race to spiral out of control, with both countries investing more and more of their resources into defense. The outcome of that is the economic collapse of at least one of the two powers. Thus, both countries should lessen the military and nuclear competition to avoid national turmoil.
We can represent our arms race example visually:
This is a vector field of all the paths in our made up model. The horizontal axis represent U.S. defense spending, and the vertical axis represent Soviet defense spending. In order to find the solution path for our initial condition, find the point (13, 15) on the plot and follow the arrows. While the vector field of all four quadrants is displayed, we are mainly interested in the first quadrant; it makes no sense for either country to have negative defense spending. Note that regardless of where the initial condition is in the first quadrant, the arrows lead to positive infinity . This is an unstable system, or a system that will reach infinity. Mathematically, the only factor that determines the stability of a system is its eigenvalues (more about this later).
Figure 3 also shows the vital role of the eigenvectors. Using Geometer's Sketchpad, we calculated the slopes of the two predominant lines. Line AC corresponds to ; they both have a slope of 1. Similarly, line AB corresponds to ; they both have a slope of 5 ⁄ -6, or -0.833. These eigenvectors dictate the possible paths our model can go through given different initial conditions.
The Complex Eigenvalue Case
So far, we have explored the systems with real eigenvalues. Now we will investigate a system that will contain complex eigenvalues.
In physics, springs are modeled by an equation known as Hooke's law. In an ideal environment (no air resistance, no friction, no heat generated by movement, etc.), an object of mass m, when set into motion attached to a spring, should oscillate forever. Hooke's law states that the mass's displacement x from the equilibrium and the force F the spring is exerting on the mass at time t is linearly related by a constant k, the spring constant. This constant quantifies the stiffness of the spring. Furthermore, this linear relationship is negative; the force is acting in a direction opposite of the displacement. We can write Hooke's law as
We combine Hooke's law with Newton's second law of motion, F = ma, to get
- , so .
Remember that acceleration a is the second derivative of displacement x. Thus we observe that Hooke's law is actually the differential equation
For the sake of simplificity, we assume k = m = 1. The only function in our differential equation is the displacement x, and it's a function of time t. The goal is to find the equation for x in terms of t. Note that our equation is actually a second order differential equation. So how do we solve this? We transform the equation into two first order differential equations by introducing a dummy variable y. Set , so . In physics, y is the velocity of the mass at time t (derivative of displacement is velocity). At time t = 0, the displacement is x0 and the velocity is always 0 (the velocity of the mass the instant it is released is 0). Now we have the system of first order linear differential equations
From here, solving this is the same as solving any other system of first order linear differential equations. We write this as the matrix equation
Solve for the eigenvalues of A:
So our eigenvalues are λ = i, -i. Using these eigenvalues, we find the eigenvectors.
- For λ1 = i: , so .
- For λ2 = -i: , so .
Now we will find the constants for our initial condition. Since we do not have a specific starting value for x, we will solve the system of equations instead of use Gaussian Elimination. When t = 0, we have
- and .
Looking at the second equation, we see that since ic1 = ic2, c1 = c2. Since the constants are equal, we will just call both constants c. We plug this fact into the first equation and we get x0 = 2c, so c = x0 ⁄ 2. Our general solution is then
Remember that we are only interested in x, the displacement of the mass. The general equation for x is
Our solution still has complex numbers. We use Euler's formula to understand what raising e by complex numbers mean. Euler's formula states
- for any x.
We now use Euler's formula in equation x:
The form of our solution is a constant multiplied by some type of function, quite similar to the real eigenvalue case. The main difference is the function; the function for the complex eigenvalue case is trigonometric while the function for the real eigenvalue case is exponential. This provides insight to the nature of complex numbers. Complex numbers might not be so "imaginary" after all; they seem to reside in a realm parallel to real numbers.
Just like with real eigenvalues, we can use a vector field to visualize our solution:
This solution is very interesting. Unlike the arms race model, we are interested in all four quadrants, since velocity and displacement can be negative. Note that regardless of where we start (except the origin), we will just orbit the origin in a perfect circle. Does that make sense? In an ideal spring system, the mass moves back and forth forever. Furthermore, the displacement and the velocity are negatively related. The velocity is at a maximum when the displacement is 0, and the displacement is at a maximum when the velocity is 0. Note that the vector field portrays both properties. Analyzing the stability of the system, the system does not end up at infinity. We call this type of system neutrally stable, neutrally because it doesn't approach the origin and stable because it doesn't reach infinity. The different stabilities of the arms race model and Hooke's law gives more insight to how important eigenvalues are to the stability of a system.
Stability of the Two Dimensional System
We judge the stability of a system by what happens when time goes to infinity. Each system also has an equilibrium. In our arms race example, the equilibrium is just the origin (in most cases it's the origin). An unstable system will go to infinity as time goes to infinity, and a stable system will not. There are two types of stable systems: asymptotically stable systems and neutrally stable systems. Neutrally stable systems circles in a closed path around the equilibrium (like Hooke's law). Asymptotically stable systems approach the equilibrium as time goes to infinity. The arms race example that we worked through is a case of an unstable system with a saddle point equilibrium. As mentioned above, the eigenvalues of a system are the only things required to determine the system's stability. We use our arms race example to help explain this. Since we judge the stability of a system by what happens when time goes to infinity, we can take the limit of our solution as time goes to infinity. Note that the only factor that affected the outcome of the solution (whether it converges or diverges) is the exponents on the e′s, and the exponents of the e′s are based on the eigenvalues. Thus, we can explore the relationship between the eigenvalues of our system and its stability.
For a 2x2 matrix, another way of writing the characteristic polynomial is
Thus we know by the quadratic formula that
With these in mind, we will analyze each type of stability and derive inequalities for λ in terms of detA and trA.
When we take the limit of our solution as time goes to infinity, the only factor that determines if the solution will go to infinity is the exponents on e, or the eigenvalues. For asymptotically stable systems, we want the system to approach 0. That means we want the real component of our eigenvalues to be negative. Note that trA must be negative. For the real eigenvalue case, we look at λ1 because if the real component of λ1 is negative, then the real component of λ2 is negative as well. We observe
- (Squared both sides, note that since trA < 0, the inequality sign changes.)
This condition also works for the complex eigenvalue case since it requires detA to be greater than 0 in order for the eigenvalues to be complex. The only difference between real and complex eigenvalues otherwise is how it converges to the equilibrium. For complex eigenvalues, the vector field actually spirals into the equilibrium. For real eigenvalues, the vector approaches the origin in some other curve. Regardless, we conclude that a system is asymptotically stable if and only if detA > 0 and trA < 0.
Hooke's law is an example of a neutrally stable system. A property of a neutrally stable system is that it must have purely imaginary eigenvalues. The existence of a real number within the eigenvalues causes the system to either go to equilibrium or go to infinity. In order to have purely imaginary eigenvalues, we must have trA = 0 and the expression under the square root to be less than zero. Since trA = 0, we have -4detA < 0, so detA > 0. Thus a system is neutrally stable if and only if detA > 0 and trA = 0.
Unstable systems approaches infinity as time passes, so the real component of the eigenvalue must be positive. We look at λ2 this time since if the real component of λ2 is positive, the real component of λ1 is positive as well. For the real eigenvalue case, we have
- (Squared both sides. Since both are positive, no need to change inequality sign.)
We get the same result as the asymptotically stable case (except trA > 0, and we stated before that this condition is also required for the complex eigenvalue case. Thus a system is unstable if and only if detA > 0 and trA > 0.
A special case arises when we observe the region around the equilibrium and notice how half of the region is unstable and the other half is unstable. In this case, the equilibrium is called a saddle point. The eigenvalues for this case are of opposite sign, one is positive and the other negative. Hence the eigenvalues must also be real, since complex eigenvalues cannot have different real components (trA is fixed). The arms race example is a great example of a saddle point. Observe that at the origin, the vectors from quadrant II and IV seem to be stable while the vectors from quadrant I and III seem to be unstable. Regardless of how half the vector field seems to be stable, we still characterize this case as unstable. Remember how in finding the limits of our solution, the result reached infinity. While the part of the expression with the negative eigenvalue converged to a number, the other part went to infinity, forcing the entire system to infinity. Thus the only way for a saddle point system to be stable is if the initial condition is a scalar multiple of the negative eigenvalue's eigenvector. Unlike the previous few cases, trA has no obvious restriction. We split this into 3 cases: trA < 0, trA = 0, trA > 0.
Case 1 (trA < 0)
λ2 is always negative, so we focus on making λ1 positive.
- (Squared both sides. Since -trA is positive, we do not need to change sign.)
The only condition for case 1 is detA < 0.
Case 2 (trA = 0)
We rewrite λ1 and λ2 taking into account trA = 0.
We need real numbers, so detA = 0. With that condition, λ1 is automatically positive and λ2 is automatically negative. Thus the only condition for case 2 is once again detA < 0.
Case 3 (trA > 0)
λ1 is always positive, so we focus on making λ2 negative.
- (Squared both sides. Since trA is positive, we do not need to change sign.)
The only condition for case 3 is detA < 0.
Thus the only condition to create a saddle point is detA < 0.
We summarize our findings here. The conditions for each case is listed below.
- Asymptotically stable: detA > 0 and trA < 0.
- Neutrally stable: detA > 0 and trA = 0.
- No saddle point: detA > 0 and trA > 0.
- With saddle point: detA < 0.
We can use a graph to summarize all our results.
From the image, we see that all of the positive stability cases is covered. The image was actually more specific than us, breaking asymptotically stable and unstable with no saddle point into real and complex eigenvalues. This is not too challenging; just set the expression under the square root for λ1 and λ2 to less than zero for complex eigenvalues or greater than zero for real eigenvalues.
The More Rigorous Proof
Requires a competent understanding of infinite series, Taylor Series, and Matrix algebra.
We have mentioned above that since the one-dimensional case is for initial condition u0, the solution to our system is . Using Taylor Series, we define
- for a square kxk matrix A. Note: A0 = I.
Now we have to show that solves :
So we know that solves . But what does mean? We use diagonalization. If the sum of the dimensions of a square matrix A's eigenspaces is equal to dim(A), then we can write A as , where P has the eigenvectors of A as its columns and L is a diagonal matrix with the eigenvalues of A as its diagonal entries. The eigenvalues of A in L must match up with the eigenvectors in P, meaning that if we choose a random eigenvector/column (say this is the kth eigenvector/column) of P, then the eigenvalue in the kth column of L must correspond to that eigenvector. Let us first consider etA.
Before we continue, consider . Remember in solving the two dimensional case, we made up constants c1 and c2 to satisfy the initial condition. For this more rigorous proof, we define
We solve for the constants by knowing that , so . Thus
When k = 2, note that this final expression expands into our two dimensional general solution. Thus the solution to any system of differential equations that can be written in the form is a linear combination of exponential functions with the eigenvectors and eigenvalues of A.
Why It's Interesting
We have shown how linear differential equations are highly applicable. They can model basic arms races and Hooke's law with a high degree of accuracy. We can use linear differential equations for any system that has derivatives and functions written in linear combinations of one another. Some include heat flow, bank interest, basic predator-prey models, and population through time. Through the Hooke's law case, we saw that our method for solving linear differential equations also works for second order equations. In fact, we can write any k order differential equation as a system of k first-order differential equations.
Even if we ignore their applicability, linear differential equations are fascinating topics to study. They help show how differentiation is a linear transformation by writing the system as a matrix equation and deriving a linear combination as the solution. Furthermore, they emphasize the importance of eigentheory. Without eigentheory, we could have solved the equations, much less determine the stability of our system. Eigentheory is not just a matrix stretching or shrinking a vector, it contains one of the core ideas of linear algebra - scalar multiplication. Coupled with the other core idea of linear algebra, linear combinations, eigentheory is an essential tool for linear algebra and all of mathematics.
- There are currently no teaching materials for this page. Add teaching materials.
Leave a message on the discussion page by clicking the 'discussion' tab at the top of this image page.