Chapter 4 · Systems of DEs

Matrix Algebra, Eigenvalues & Eigenvectors

The Foundation for Systems of Differential Equations

Dr. Mohamed Mabrok · Qatar University

The Big Picture: Why Matrices Matter

Coupled ODEs
Coupled Dynamics

Need Matrices!

$$\mathbf{x}' = A\mathbf{x}$$

Eigenvalues
Determine Behavior

Growth / Decay

Oscillation

Key Framework

      Systems of DEs → Matrix form → Find eigenvalues → Solve explicitly. This lecture covers step 3, the heart of the method.

Applications: Where Systems Arise

🌀

Coupled Springs

Two masses connected by springs — forces on each mass depend on positions of both.

$$m_1x_1'' = -k_1x_1 + k(x_2 - x_1)$$

⚡

Electrical Circuits

Multiple loops and meshes — currents and voltages are coupled via inductances and resistances.

LI_1'' + RI_1' = \ldots

🦌

Predator-Prey

Population dynamics — growth of prey depends on predators, and vice versa.

\frac{dx}{dt} = ax - bxy

The Common Thread

      In all these systems, the rates of change depend on multiple variables. Eigenvalues of the coefficient matrix determine stability and long-term behavior.

Matrix Operations: Quick Review

Basic Operations

      Matrix addition: add corresponding entries. Scalar multiplication: multiply all entries by the scalar.
    
      $$A + B = \begin{pmatrix} a_{11}+b_{11} & a_{12}+b_{12} \\ a_{21}+b_{21} & a_{22}+b_{22} \end{pmatrix}, \quad cA = \begin{pmatrix} ca_{11} & ca_{12} \\ ca_{21} & ca_{22} \end{pmatrix}$$

Matrix Multiplication is NOT Commutative!

      In general, $AB \neq BA$. The order matters! This is crucial when working with linear transformations.
    
      $$AB \text{ is generally different from } BA$$

Matrix Multiplication: Step-by-Step

The Rule: (Row) × (Column)

      The $(i,j)$ entry of $AB$ is the dot product of row $i$ of $A$ with column $j$ of $B$.

EXAMPLE 2×2 matrices

A = \begin{pmatrix} 2 & 1 \\ 3 & 4 \end{pmatrix}, \quad B = \begin{pmatrix} 1 & 0 \\ 2 & 3 \end{pmatrix}

AB = \begin{pmatrix} 2(1)+1(2) & 2(0)+1(3) \\ 3(1)+4(2) & 3(0)+4(3) \end{pmatrix} = \begin{pmatrix} 4 & 3 \\ 11 & 12 \end{pmatrix}

Geometric Meaning

      The columns of $A$ show where the standard basis vectors get mapped by the linear transformation. Multiplying $A\mathbf{x}$ applies this transformation to vector $\mathbf{x}$.

Determinants: A Critical Scalar

2×2 Determinant

      $$\det\begin{pmatrix} a & b \\ c & d \end{pmatrix} = ad - bc$$

EXAMPLE

\det\begin{pmatrix} 2 & 3 \\ 1 & 4 \end{pmatrix} = 2(4) - 3(1) = 8 - 3 = 5

Key Properties

      $\det(AB) = \det(A)\det(B)$, $\det(A^T) = \det(A)$, swapping rows flips sign, $\det(A) = 0$ iff $A$ is singular (non-invertible).

3×3 Determinant: Cofactor Expansion

      $$\det(A) = a_{11}(a_{22}a_{33} - a_{23}a_{32}) - a_{12}(a_{21}a_{33} - a_{23}a_{31}) + \ldots$$

Matrix Inverse: When It Exists

Invertibility Condition

      A square matrix $A$ is invertible (has an inverse $A^{-1}$) if and only if $\det(A) \neq 0$.
    
      $$AA^{-1} = A^{-1}A = I$$

2×2 Inverse Formula

      $$A^{-1} = \frac{1}{\det(A)} \begin{pmatrix} d & -b \\ -c & a \end{pmatrix}$$
    
      where $A = \begin{pmatrix} a & b \\ c & d \end{pmatrix}$.

EXAMPLE

A = \begin{pmatrix} 2 & 1 \\ 1 & 3 \end{pmatrix}, \quad \det(A) = 6 - 1 = 5, \quad A^{-1} = \frac{1}{5}\begin{pmatrix} 3 & -1 \\ -1 & 2 \end{pmatrix}

Eigenvalues: The Key Equation

Eigenvalue & Eigenvector

      $$A\mathbf{v} = \lambda \mathbf{v}$$
    
      $\lambda$ is an eigenvalue and $\mathbf{v}$ is the corresponding eigenvector (nonzero). The matrix only scales $\mathbf{v}$, no rotation!

Geometric Meaning

      Eigenvectors are special directions. When $A$ acts on an eigenvector, it just stretches or compresses along that direction. No rotation, no change in direction.

INTERPRETATION

\lambda > 1: \text{stretch}, \quad 0 < \lambda < 1: \text{compress}, \quad \lambda < 0: \text{reverse}, \quad \text{complex: rotate}

For x' = Ax

      If $\mathbf{x}(0) = \mathbf{v}$ (an eigenvector), then $\mathbf{x}(t) = e^{\lambda t}\mathbf{v}$ is the solution!

Finding Eigenvalues: The Characteristic Equation

DERIVATION

A\mathbf{v} = \lambda \mathbf{v} \quad \Rightarrow \quad (A - \lambda I)\mathbf{v} = \mathbf{0}

For a nonzero solution $\mathbf{v}$ to exist, the matrix $(A - \lambda I)$ must be singular:

\det(A - \lambda I) = 0

The Characteristic Equation

      This equation defines the characteristic polynomial. Its roots are the eigenvalues!
    
      $$\det(A - \lambda I) = 0$$

2×2 CASE

\det\begin{pmatrix} a-\lambda & b \\ c & d-\lambda \end{pmatrix} = \lambda^2 - \text{tr}(A)\lambda + \det(A) = 0

Three Cases: Eigenvalue Types

Case 1: Distinct Real

Discriminant > 0

Two different real eigenvalues $\lambda_1 \neq \lambda_2$

\text{Exponential growth/decay}

Case 2: Repeated

Discriminant = 0

Double root $\lambda$ (may need generalized eigenvector)

\text{May involve polynomial factors}

Case 3: Complex

Discriminant < 0

$\lambda = \alpha \pm \beta i$ (conjugate pair)

\text{Oscillation with growth/decay}

Physical Significance

      Real parts determine stability (decay if negative, grow if positive), imaginary parts determine oscillation frequency.

Worked Example

Find Eigenvalues & Eigenvectors of $A = \begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix}$

STEP 1 Form $A - \lambda I$

A - \lambda I = \begin{pmatrix} 4-\lambda & 1 \\ 2 & 3-\lambda \end{pmatrix}

STEP 2 Compute characteristic polynomial

\det(A-\lambda I) = (4-\lambda)(3-\lambda) - 2 = \lambda^2 - 7\lambda + 10 = 0

STEP 3 Factor & solve

(\lambda - 5)(\lambda - 2) = 0 \quad \Rightarrow \quad \lambda_1 = 5, \quad \lambda_2 = 2

STEP 4a For $\lambda_1 = 5$: solve $(A - 5I)\mathbf{v} = \mathbf{0}$

\begin{pmatrix} -1 & 1 \\ 2 & -2 \end{pmatrix}\begin{pmatrix} v_1 \\ v_2 \end{pmatrix} = \mathbf{0} \quad \Rightarrow \quad v_1 = v_2 \quad \Rightarrow \quad \mathbf{v}_1 = \begin{pmatrix} 1 \\ 1 \end{pmatrix}

STEP 4b For $\lambda_2 = 2$: solve $(A - 2I)\mathbf{v} = \mathbf{0}$

\begin{pmatrix} 2 & 1 \\ 2 & 1 \end{pmatrix}\begin{pmatrix} v_1 \\ v_2 \end{pmatrix} = \mathbf{0} \quad \Rightarrow \quad 2v_1 + v_2 = 0 \quad \Rightarrow \quad \mathbf{v}_2 = \begin{pmatrix} 1 \\ -2 \end{pmatrix}

Worked Example

Find Eigenvalues of $A = \begin{pmatrix} 2 & -1 \\ 1 & 0 \end{pmatrix}$

STEP 1-2 Characteristic polynomial

\det(A-\lambda I) = (2-\lambda)(0-\lambda) + 1 = \lambda^2 - 2\lambda + 1 = (\lambda - 1)^2 = 0

STEP 3 Find eigenvalues

\lambda = 1 \text{ (double root)}

STEP 4 Find eigenvectors

\begin{pmatrix} 1 & -1 \\ 1 & -1 \end{pmatrix}\begin{pmatrix} v_1 \\ v_2 \end{pmatrix} = \mathbf{0} \quad \Rightarrow \quad v_1 = v_2 \quad \Rightarrow \quad \mathbf{v} = \begin{pmatrix} 1 \\ 1 \end{pmatrix}

Note: Single Eigenvector!

      Double eigenvalue but only one eigenvector. Need generalized eigenvector for complete solution to the system.

Worked Example

Find Eigenvalues of $A = \begin{pmatrix} 1 & -2 \\ 1 & 3 \end{pmatrix}$

STEP 1-2 Characteristic polynomial

\det(A-\lambda I) = (1-\lambda)(3-\lambda) + 2 = \lambda^2 - 4\lambda + 5 = 0

STEP 3 Quadratic formula

\lambda = \frac{4 \pm \sqrt{16-20}}{2} = \frac{4 \pm \sqrt{-4}}{2} = \frac{4 \pm 2i}{2} = 2 \pm i

RESULT

\lambda = 2 \pm i \quad \text{where} \quad \alpha = 2, \quad \beta = 1

Physical Meaning

      Solutions involve $e^{2t}(\cos(t) + i\sin(t))$ — exponential growth combined with oscillation at frequency 1 rad/s.

How Eigenvalues Control System Behavior

Eigenvalue Type

\lambda \text{ real, } \lambda > 0

Exponential growth. Solution grows without bound.

\lambda \text{ real, } \lambda < 0

Exponential decay. Solution → 0 (stable).

\lambda = \alpha + \beta i, \alpha < 0

Damped oscillation (stable spiral).

\lambda = \alpha + \beta i, \alpha > 0

Growing oscillation (unstable spiral).

\lambda = \beta i \text{ (purely imaginary)}

Sustained oscillation (center).

Differential Equation Link

If $A\mathbf{v} = \lambda \mathbf{v}$ and we start with $\mathbf{x}(0) = c\mathbf{v}$, then:

\mathbf{x}(t) = ce^{\lambda t}\mathbf{v}

For complex $\lambda = \alpha + \beta i$:

\mathbf{x}(t) = e^{\alpha t}(c_1\cos(\beta t) + c_2\sin(\beta t))\mathbf{v}

Real part controls growth/decay, imaginary part controls rotation/oscillation frequency.

4-Step Method to Find Eigenvalues & Eigenvectors

STEP 1 Compute $A - \lambda I$ (subtract $\lambda$ from diagonal)

This gives a matrix that depends on the parameter $\lambda$.

STEP 2 Compute $\det(A - \lambda I)$ and set it equal to zero

The characteristic polynomial. Solve this to find the eigenvalues.

STEP 3 Solve for $\lambda$ from the characteristic equation

Use factoring, quadratic formula, etc. Each eigenvalue is a root.

STEP 4 For each $\lambda$, solve $(A - \lambda I)\mathbf{v} = \mathbf{0}$ for eigenvectors

Nonzero solutions form the eigenspace. Any nonzero vector in it is an eigenvector.

Common Mistakes to Avoid

⚠️ Frequent Errors

Forgetting the identity matrix: Write $A - \lambda I$, not just $A - \lambda$. The $I$ is crucial!
Arithmetic errors in the determinant: Compute $\det(A - \lambda I) = 0$ carefully. One mistake cascades!
Confusing eigenvalues with eigenvectors: $\lambda$ is a scalar (the eigenvalue). $\mathbf{v}$ is a vector (the eigenvector).
Forgetting eigenvectors are not unique: Any scalar multiple of an eigenvector is also an eigenvector. Choose simple ones (integers).
For complex eigenvalues, forgetting the conjugate pair: If $\lambda = \alpha + \beta i$ is an eigenvalue of a real matrix, so is $\bar{\lambda} = \alpha - \beta i$.
Not verifying the answer: Always check: $A\mathbf{v} = \lambda \mathbf{v}$. This catches mistakes!

Pro Tip

      Verify your eigenvalues satisfy $\lambda_1 + \lambda_2 = \text{tr}(A)$ and $\lambda_1 \lambda_2 = \det(A)$ for 2×2 matrices. Quick sanity check!

Summary & What Comes Next

Distinct Real

$\lambda_1 \neq \lambda_2$

Pure exponential growth or decay

Eigenvectors determine natural directions

Repeated

$\lambda_1 = \lambda_2 = \lambda$

May need generalized eigenvectors

Solutions involve $t \cdot e^{\lambda t}$ terms

Complex Conjugates

$\lambda = \alpha \pm \beta i$

Oscillation with exponential envelope

Frequency = $\beta$, decay/growth = $\alpha$

The Bridge to Solving Systems

      Now that we can find eigenvalues and eigenvectors, we'll use them to solve systems of differential equations: $\mathbf{x}' = A\mathbf{x}$. The complete solution will be a linear combination of these special exponential modes, one for each eigenvalue!

Real-World Payoff

      Engineers use eigenvalues to predict stability of control systems, vibration modes of structures, and dominant long-term behaviors of complex coupled systems.