02 First-Order Linear ODEs, Higher Order Constant Coefficient ODEs (Homogeneous Solution and Connection to Linear Algebra)

Our Guiding Principle: The Main Theorem for Linear ODEs

Before we dive deeper, let’s recall the profound result that serves as our roadmap. The structure of solutions to any linear ODE is not a mystery; it is a direct consequence of linearity, just like in linear algebra.

Summary: The Main Theorem

For a $k$-th order linear ODE, $y^{(k)}+\cdots +a_0(x)y=b(x)$:

1. The Homogeneous Core ($S_0$): The set of solutions to the homogeneous equation (where $b(x)=0$) forms a $k$-dimensional vector space. This is our “null space.” We just need to find $k$ linearly independent “basis” solutions.
2. The General Solution ($S_b$): The complete set of solutions is an affine space, formed by finding one particular solution ($y_p$) and shifting the entire homogeneous solution space by it. $y_{\text{general}}=y_p+y_h\text{where }{y}_h\in S_0$
3. Uniqueness: An initial value problem provides enough constraints to specify a single, unique solution.

A crucial remark: The set of inhomogeneous solutions, $S_b$, is NOT a vector space (unless $b=0$). It doesn’t contain the zero function!

---

Mastering First-Order Linear ODEs

Let’s apply this powerful framework to the first-order case: $y^{\prime }+a(x)y=b(x)$ where $a(x)$ and $b(x)$ are continuous functions.

The Homogeneous Solution (A Quick Recap)

For $y^{\prime }+a(x)y=0$, we found the solution by separating variables. The result is a one-dimensional vector space spanned by a single function.

Homogeneous Solution

Any solution to $y^{\prime }+a(x)y=0$ is of the form: $y_h(x)=K\exp (-A(x))\text{where }A(x)=\int a(x)dx$ The unique solution to the IVP with $y(x_0)=y_0$ is given by $y(x)=y_0\exp (A(x_0)-A(x))$.

The Inhomogeneous Solution: A Tale of Two Methods

Finding the particular solution $y_p$ is where the real work lies. Last time, we saw two powerful techniques.

1. Variation of Parameters (Systematic Method): We assume the particular solution mimics the homogeneous one, but we allow the “constant” to become a function: $y_p(x)=z(x)\exp (-A(x))$. Plugging this into the ODE magically simplifies and reveals that $z^{\prime }(x)=b(x)e^{A(x)}$. We find $z(x)$ by integration.

2. Educated Guess (Ansatz Method): If $b(x)$ has a simple form (polynomial, exponential), we guess that $y_p$ has a similar form and solve for the unknown coefficients.

There is a third, closely related perspective that is incredibly elegant.

A Clever Trick: The “Integrating Factor”

Let’s look at our ODE again: $\frac{dy}{dx}+a(x)y=b(x)$. The left-hand side is almost the result of the product rule. What if we could multiply the entire equation by some magic function, let’s call it $\mu (x)$, so that the left side becomes a perfect product rule derivative?

Let’s try it. Multiply by $\mu (x)$: $\mu (x)\frac{dy}{dx}+\mu (x)a(x)y=\mu (x)b(x)$ We want the left side to look like $\frac{d}{dx}(\mu (x)y)$. Let’s see what that is: $\frac{d}{dx}(\mu (x)y)=\mu (x)\frac{dy}{dx}+{\mu }^{\prime }(x)y$ Comparing our two expressions, we see they match perfectly if we choose $\mu (x)$ such that: ${\mu }^{\prime }(x)=\mu (x)a(x)$ This is a simple, separable ODE for our magic function $\mu (x)$! $\frac{{\mu }^{\prime }}{\mu }=a(x)⟹\ln |\mu |=\int a(x)dx=A(x)⟹\mu (x)=e^{A(x)}$ This function $\mu (x)=e^{\int a(x)dx}$ is called the integrating factor.

The Integrating Factor Method

1. Calculate the integrating factor $\mu (x)=e^{\int a(x)dx}$.
2. Multiply the entire ODE by $\mu (x)$. The left side will automatically collapse into $\frac{d}{dx}(\mu (x)y)$. $\frac{d}{dx}\left(y\cdot e^{A(x)}\right)=b(x)e^{A(x)}$
3. Integrate both sides to solve for the term in the parenthesis: $y\cdot e^{A(x)}=\int b(x)e^{A(x)}dx+C$
4. Solve for $y$: $y(x)=e^{-A(x)}\left(\int b(x)e^{A(x)}dx+C\right)$ Notice that this single formula gives both the particular solution (from the integral) and the homogeneous solution (from the $+C$ term)!

A Detailed Example

Let’s solve $x\frac{dy}{dx}-2y=x^2$.

1. Standard Form: First, we must put it in the standard form $y^{\prime }+a(x)y=b(x)$. Assuming $x\ne 0$, we divide by $x$: $\frac{dy}{dx}-\frac{2}{x}y=x$ So, $a(x)=-2/x$ and $b(x)=x$.

2. Homogeneous Solution: We solve $y^{\prime }-\frac{2}{x}y=0$. $\frac{y^{\prime }}{y}=\frac{2}{x}⟹\ln |y|=\int \frac{2}{x}dx=2\ln |x|+C=\ln (x^2)+C$ Exponentiating gives $y_h(x)=e^{\ln (x^2)+C}=e^C\cdot x^2=K{x}^2$. Our homogeneous solution is $y_h=K{x}^2$.

3. Particular Solution (Variation of Parameters): We guess $y_p(x)=K(x)x^2$. Its derivative is $y_p^{\prime }=K^{\prime }(x)x^2+K(x)(2x)$. Substitute into the standard form ODE: $\underset{y_p^{\prime }}{\underbrace{(K^{\prime }(x)x^2+2xK(x))}}-\underset{a(x)y_p}{\underbrace{\frac{2}{x}(K(x)x^2)}}=x$ The terms with $K(x)$ cancel, as expected: $K^{\prime }(x)x^2+\cancel{2xK(x)}-\cancel{2xK(x)}=x$ $K^{\prime }(x)x^2=x⟹K^{\prime }(x)=\frac{1}{x}$ Integrating gives $K(x)=\ln |x|$. So, our particular solution is $y_p(x)=K(x)x^2=x^2\ln |x|$.

4. General Solution: The full solution is the sum: $y(x)=y_p+y_h=x^2\ln |x|+K{x}^2$

Exercise

Try solving this same problem using the integrating factor method. You should arrive at the same general solution.

The Leap to Higher Order: Constant Coefficients

Solving linear ODEs where the coefficients $a_i(x)$ are arbitrary functions is extremely difficult. We can make a huge leap forward by simplifying the problem: what if all the coefficients are just constants? $y^{(k)}+a_{k-1}{y}^{(k-1)}+\cdots +a_1{y}^{\prime }+a_{0}y=b(x)$ This simplification allows us to develop a complete, powerful, and beautifully simple method for finding the homogeneous solution.

The Homogeneous Case: The Big Idea

Let’s focus on solving $Df=0$, where $D$ is our differential operator with constant coefficients.

Recall the first-order case: $y^{\prime }+ay=0$. The solution was $y=K{e}^{-ax}$. The exponential function is the star player.

What if we guess that the solution to the higher-order equation is also an exponential?

This is a brilliant leap of faith, an Ansatz. Let’s try a solution of the form $y(x)=e^{\lambda x}$ for some unknown constant $\lambda$ and see what happens.

Let’s plug $y=e^{\lambda x}$ into the homogeneous equation: $(e^{\lambda x}{)}^{(k)}+a_{k-1}(e^{\lambda x}{)}^{(k-1)}+\cdots +a_0(e^{\lambda x})=0$ The magic of the exponential is that its derivative is simple: the $j$-th derivative is just ${\lambda }^j{e}^{\lambda x}$. ${\lambda }^k{e}^{\lambda x}+a_{k-1}{\lambda }^{k-1}{e}^{\lambda x}+\cdots +a_0{e}^{\lambda x}=0$ We can factor out the $e^{\lambda x}$ term, which is never zero: $e^{\lambda x}\left({\lambda }^k+a_{k-1}{\lambda }^{k-1}+\cdots +a_0\right)=0$ For this equation to hold, the polynomial in the parenthesis must be zero.

The Characteristic Polynomial

For the linear ODE with constant coefficients, $y^{(k)}+\cdots +a_{0}y=0$, the associated characteristic polynomial is: $P(\lambda )={\lambda }^k+a_{k-1}{\lambda }^{k-1}+\cdots +a_0$ Our guess $y=e^{\lambda x}$ is a solution to the ODE if and only if $\lambda$ is a root of the characteristic polynomial.

We have transformed a difficult calculus problem (solving an ODE) into a much simpler algebra problem (finding the roots of a polynomial)!

By the Fundamental Theorem of Algebra, a polynomial of degree $k$ has exactly $k$ roots in the complex numbers (counting multiplicity). Each root gives us a solution to the ODE.

The Three Cases for the Roots

Case 1: Distinct Real Roots

If the characteristic polynomial has $k$ distinct real roots ${\lambda }_1,{\lambda }_2,\dots ,{\lambda }_k$, we immediately get $k$ linearly independent solutions: $e^{{\lambda }_{1}x},e^{{\lambda }_{2}x},\dots ,e^{{\lambda }_{k}x}$ The general homogeneous solution is then: $y_h(x)=c_1{e}^{{\lambda }_{1}x}+c_2{e}^{{\lambda }_{2}x}+\cdots +c_k{e}^{{\lambda }_{k}x}$

$y^{\prime \prime }-y=0$

• Characteristic Polynomial: $P(\lambda )={\lambda }^2-1=0$.
• Roots: ${\lambda }_1=1,{\lambda }_2=-1$.
• General Solution: $y_h(x)=c_1{e}^x+c_2{e}^{-x}$.

Case 2: Complex Conjugate Roots

If the coefficients $a_i$ are real, any complex roots of $P(\lambda )$ must come in conjugate pairs: $\lambda =a\pm bi$. This gives us two solutions: $e^{(a+bi)x}$ and $e^{(a-bi)x}$. Using Euler’s formula, $e^{i\theta }=\cos \theta +i\sin \theta$, we can find two real-valued solutions instead. $e^{(a+bi)x}=e^{ax}{e}^{ibx}=e^{ax}(\cos (bx)+i\sin (bx))$ By taking linear combinations of the complex solutions, we can isolate the real and imaginary parts to form our two real basis solutions: $e^{ax}\cos (bx)$ and $e^{ax}\sin (bx)$.

$y^{\prime \prime }+y=0$

• Characteristic Polynomial: $P(\lambda )={\lambda }^2+1=0$.
• Roots: $\lambda =\pm i$. (Here $a=0,b=1$).
• Real Solutions: $e^{0x}\cos (1x)=\cos (x)$ and $e^{0x}\sin (1x)=\sin (x)$.
• General Solution: $y_h(x)=c_1\cos (x)+c_2\sin (x)$.

Case 3: Repeated Roots

What if a root ${\lambda }_1$ has multiplicity $m>1$? We get one solution, $e^{{\lambda }_{1}x}$, but we need $m$ linearly independent solutions for that root. Where do the others come from?

Solutions for Repeated Roots

If $\lambda$ is a root of the characteristic polynomial with multiplicity $m$, then the $m$ linearly independent solutions corresponding to this root are: $e^{\lambda x},x{e}^{\lambda x},x^2{e}^{\lambda x},\dots ,x^{m-1}{e}^{\lambda x}$

$y^{\prime \prime }=0$

• Characteristic Polynomial: $P(\lambda )={\lambda }^2=0$.
• Roots: $\lambda =0$ with multiplicity 2.
• Solutions: $e^{0x}=1$ and $x{e}^{0x}=x$.
• General Solution: $y_h(x)=c_1(1)+c_2(x)=c_1+c_{2}x$. This matches what we know from direct integration!

The Bridge to Linear Algebra

Why do we use linear algebra terms like "characteristic polynomial" and "eigenvalues"?

Because it’s literally the same thing. Any $k$-th order linear ODE can be converted into a system of $k$ first-order linear ODEs, which can be written in matrix form.

Let’s see this with an example: $y^{\prime \prime }+3y^{\prime }+y=0$.

1. Define a state vector: Let $y_1=y$ and $y_2=y^{\prime }$. Our state vector is $Y=\left(\begin{array}{c}{y}_1\\ y_2\end{array}\right)$.
2. Find its derivative: $Y^{\prime }=\left(\begin{array}{c}{y}_1^{\prime }\\ y_2^{\prime }\end{array}\right)=\left(\begin{array}{c}{y}^{\prime }\\ y^{\prime \prime }\end{array}\right)$.
3. Express the derivative in terms of the state vector:
   • $y_1^{\prime }=y^{\prime }=y_2$.
   • $y_2^{\prime }=y^{\prime \prime }=-3y^{\prime }-y=-3y_2-y_1$.
4. Write as a matrix equation: $Y^{\prime }=\left(\begin{array}{c}{y}_1^{\prime }\\ y_2^{\prime }\end{array}\right)=\left(\begin{array}{c}0\cdot y_1+1\cdot y_2\\ -1\cdot y_1-3\cdot y_2\end{array}\right)=\underset{A}{\underbrace{\left(\begin{array}{cc}0& 1\\ -1& -3\end{array}\right)}}\left(\begin{array}{c}{y}_1\\ y_2\end{array}\right)=AY$ The problem is now $Y^{\prime }=AY$. The characteristic polynomial of the matrix $A$ is: $\det (A-\lambda I)=\det \left(\begin{array}{cc}-\lambda & 1\\ -1& -3-\lambda \end{array}\right)=(-\lambda )(-3-\lambda )-(1)(-1)=3\lambda +{\lambda }^2+1$ This is exactly the same characteristic polynomial we found with our $e^{\lambda x}$ guess! The roots $\lambda$ are truly the eigenvalues of the underlying linear system.

Part 2: The Inhomogeneous Equation with Constant Coefficients

Our final task is to find a particular solution $y_p$ for $Dy=b(x)$. We return to our friend, the educated guess, now formalized as the Method of Undetermined Coefficients.

The Method of Undetermined Coefficients

The idea is to propose a particular solution $y_p$ that has the same general form as the right-hand side $b(x)$, but with unknown (undetermined) coefficients. We then plug this Ansatz into the ODE and solve for the coefficients.

Here is a table of common forms for $b(x)$ and the corresponding Ansatz for $y_p$. ($P_n(x)$ denotes a generic polynomial of degree $n$).

If $b(x)$ is…	The Ansatz for $y_p$ is…
$P_n(x)e^{\alpha x}$	$Q_n(x)e^{\alpha x}$
$P_n(x)e^{\alpha x}\sin (\beta x)$	$Q_n(x)e^{\alpha x}\sin (\beta x)+R_n(x)e^{\alpha x}\cos (\beta x)$
$P_n(x)e^{\alpha x}\cos (\beta x)$	$Q_n(x)e^{\alpha x}\sin (\beta x)+R_n(x)e^{\alpha x}\cos (\beta x)$

We will put this powerful method into practice in our next session.