Eigenvalues and Eigenvectors
Characteristic Polynomial
Let $A$ be an $n \times n$ square matrix. An eigenvalue $\lambda$ of $A$ is a scalar such that there exists a nonzero vector $v$ (called the eigenvector) satisfying \[ A v = \lambda v. \]
The characteristic polynomial of $A$ is defined as \[ p_A(\lambda) = \det(A - \lambda I_n), \] where $I_n$ is the $n \times n$ identity matrix. The roots of this polynomial are the eigenvalues of $A$.
Find the characteristic polynomial of \[ A = \begin{bmatrix}2 & 1 \\ 1 & 2\end{bmatrix}. \]
Solution: \[ p_A(\lambda) = \det\begin{bmatrix}2-\lambda & 1 \\ 1 & 2-\lambda\end{bmatrix} = (2-\lambda)^2 - 1 = \lambda^2 - 4\lambda + 3. \]
Find the eigenvalues of \[ B = \begin{bmatrix}0 & 1 \\ -2 & -3\end{bmatrix}. \]
Solution: Characteristic polynomial: \[ p_B(\lambda) = \det(B - \lambda I) = \det\begin{bmatrix}-\lambda & 1 \\ -2 & -3-\lambda\end{bmatrix} = (-\lambda)(-3-\lambda) - (-2)(1) = \lambda^2 + 3\lambda + 2. \]
Factorizing: $\lambda^2 + 3\lambda + 2 = (\lambda+1)(\lambda+2) = 0$ Hence eigenvalues: $\lambda = -1, -2$.
Compute the characteristic polynomial of \[ C = \begin{bmatrix}1 & 0 & 2 \\ -1 & 3 & 1 \\ 0 & 0 & 4\end{bmatrix}. \]
Solution: \[ p_C(\lambda) = \det\begin{bmatrix}1-\lambda & 0 & 2 \\ -1 & 3-\lambda & 1 \\ 0 & 0 & 4-\lambda\end{bmatrix}. \]
Expanding along third row (or use upper-triangular property): \[ p_C(\lambda) = (4-\lambda) \det\begin{bmatrix}1-\lambda & 0 \\ -1 & 3-\lambda\end{bmatrix} = (4-\lambda)((1-\lambda)(3-\lambda) - 0) = (4-\lambda)(\lambda^2 -4\lambda +3). \]
So characteristic polynomial: $p_C(\lambda) = (\lambda^2 -4\lambda +3)(4-\lambda)$.
Finding Eigenvalues and Eigenvectors
To find the eigenvalues of a square matrix $A$:
Compute the characteristic polynomial: $p_A(\lambda) = \det(A - \lambda I)$.
Solve $p_A(\lambda) = 0$ to find all eigenvalues $\lambda_1, \lambda_2, \dots, \lambda_n$.
To find an eigenvector corresponding to a given eigenvalue $\lambda$:
Solve the system $(A - \lambda I)v = 0$ for a nonzero vector $v$.
Any nonzero solution $v$ is an eigenvector corresponding to $\lambda$.
Find the eigenvalues and eigenvectors of \[ A = \begin{bmatrix}2 & 1 \\ 1 & 2\end{bmatrix}. \]
Solution: Characteristic polynomial: \[ \det(A - \lambda I) = \det\begin{bmatrix}2-\lambda & 1 \\ 1 & 2-\lambda\end{bmatrix} = (\lambda - 1)(\lambda - 3). \] Eigenvalues: $\lambda_1 = 1$, $\lambda_2 = 3$.
For $\lambda_1 = 1$: \[ (A - I)v = \begin{bmatrix}1 & 1 \\ 1 & 1\end{bmatrix} \begin{bmatrix}x \\ y\end{bmatrix} = 0 \implies x + y = 0 \implies v = k\begin{bmatrix}1 \\ -1\end{bmatrix}. \]
For $\lambda_2 = 3$: \[ (A - 3I)v = \begin{bmatrix}-1 & 1 \\ 1 & -1\end{bmatrix} \begin{bmatrix}x \\ y\end{bmatrix} = 0 \implies x - y = 0 \implies v = k\begin{bmatrix}1 \\ 1\end{bmatrix}. \]
Find eigenvalues and eigenvectors of \[ B = \begin{bmatrix}0 & 1 \\ -2 & -3\end{bmatrix}. \]
Solution: Characteristic polynomial: \[ \det(B - \lambda I) = \lambda^2 + 3\lambda + 2 = (\lambda + 1)(\lambda + 2) \implies \lambda_1 = -1, \lambda_2 = -2. \]
For $\lambda_1=-1$: \[ (B + I)v = \begin{bmatrix}1 & 1 \\ -2 & -2\end{bmatrix} \begin{bmatrix}x \\ y\end{bmatrix} = 0 \implies x + y = 0 \implies v = k\begin{bmatrix}1 \\ -1\end{bmatrix}. \]
For $\lambda_2=-2$: \[ (B + 2I)v = \begin{bmatrix}2 & 1 \\ -2 & -1\end{bmatrix} \begin{bmatrix}x \\ y\end{bmatrix} = 0 \implies 2x + y = 0 \implies v = k\begin{bmatrix}-1/2 \\ 1\end{bmatrix} \sim k\begin{bmatrix}-1 \\ 2\end{bmatrix}. \]
Find the eigenvalues and an eigenvector of \[ C = \begin{bmatrix}3 & 0 & 0 \\ 0 & 2 & 1 \\ 0 & 0 & 2\end{bmatrix}. \]
Solution: Characteristic polynomial: \[ \det(C - \lambda I) = (3-\lambda)\det\begin{bmatrix}2-\lambda & 1 \\ 0 & 2-\lambda\end{bmatrix} = (3-\lambda)(2-\lambda)^2. \] Eigenvalues: $\lambda = 3$ (simple), $\lambda = 2$ (double).
For $\lambda=3$: \[ (C-3I)v = \begin{bmatrix}0 & 0 & 0 \\ 0 & -1 & 1 \\ 0 & 0 & -1\end{bmatrix}v = 0 \implies v = k\begin{bmatrix}1\\0\\0\end{bmatrix}. \]
For $\lambda=2$: \[ (C-2I)v = \begin{bmatrix}1 & 0 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0\end{bmatrix}v = 0 \implies v = k\begin{bmatrix}0\\1\\0\end{bmatrix} + l\begin{bmatrix}0\\0\\1\end{bmatrix}. \]
Properties of Eigenvalues and Eigenvectors
Some key properties of eigenvalues and eigenvectors of a square matrix $A$ are:
1. Trace and determinant relation: The sum of the eigenvalues of $A$ equals $\text{tr}(A)$ (sum of diagonal elements), and the product of eigenvalues equals $\det(A)$.
2. Eigenvalues of triangular matrices: If $A$ is upper or lower triangular, the eigenvalues of $A$ are exactly the diagonal entries.
3. Eigenvalues of powers: If $\lambda$ is an eigenvalue of $A$ with eigenvector $v$, then $\lambda^k$ is an eigenvalue of $A^k$ with the same eigenvector $v$.
4. Eigenvalues of inverse: If $A$ is invertible, then $1/\lambda$ is an eigenvalue of $A^{-1}$ for each eigenvalue $\lambda$ of $A$.
5. Eigenvalues of transpose: $A$ and $A^T$ have the same eigenvalues.
6. Eigenvectors corresponding to distinct eigenvalues are linearly independent.
7. Similarity invariance: If $B = P^{-1}AP$, then $A$ and $B$ have the same eigenvalues.
Trace and determinant relation: Let $A = \begin{bmatrix}2 & 1 \\ 0 & 3\end{bmatrix}$. Eigenvalues: $\lambda_1=2$, $\lambda_2=3$. Check: $\text{tr}(A) = 2+3 = 5$, sum of eigenvalues = 2+3=5. $\det(A)=6$, product of eigenvalues = 2*3=6.
Eigenvalues of a triangular matrix: Let $B = \begin{bmatrix}1 & 4 & 2 \\ 0 & 3 & 1 \\ 0 & 0 & -2\end{bmatrix}$ (upper triangular). Eigenvalues are the diagonal entries: $\lambda = 1, 3, -2$.
Eigenvalues of a power: Let $A = \begin{bmatrix}2 & 0 \\ 0 & 3\end{bmatrix}$, eigenvalues $\lambda_1=2$, $\lambda_2=3$. Then $A^2 = \begin{bmatrix}4 & 0 \\ 0 & 9\end{bmatrix}$, eigenvalues = $2^2=4$, $3^2=9$.
Eigenvalues of the inverse: If $A = \begin{bmatrix}2 & 0 \\ 0 & 3\end{bmatrix}$, eigenvalues = 2, 3. $A^{-1} = \begin{bmatrix}1/2 & 0 \\ 0 & 1/3\end{bmatrix}$, eigenvalues = 1/2, 1/3.
Eigenvalues of transpose: $A = \begin{bmatrix}1 & 2 \\ 0 & 3\end{bmatrix}$, eigenvalues: 1, 3. $A^T = \begin{bmatrix}1 & 0 \\ 2 & 3\end{bmatrix}$, eigenvalues still 1, 3.
Linear independence of eigenvectors: $A = \begin{bmatrix}2 & 0 \\ 0 & 3\end{bmatrix}$, eigenvectors for $\lambda=2$: $[1,0]^T$, $\lambda=3$: $[0,1]^T$. They are clearly linearly independent.
Similarity invariance: Let $A = \begin{bmatrix}2 & 1 \\ 0 & 3\end{bmatrix}$, $P = \begin{bmatrix}1 & 1 \\ 0 & 1\end{bmatrix}$, then $B = P^{-1}AP = \begin{bmatrix}2 & -1 \\ 0 & 3\end{bmatrix}$. Eigenvalues of $B$ are still 2, 3 (same as $A$).
Diagonalization of Matrices
A square matrix $A$ of order $n$ is said to be diagonalizable if there exists an invertible matrix $P$ and a diagonal matrix $D$ such that \[ A = P D P^{-1}. \]
Here:
The diagonal entries of $D$ are the eigenvalues of $A$.
The columns of $P$ are the corresponding eigenvectors of $A$.
Steps to diagonalize a matrix:
Find all eigenvalues of $A$.
For each eigenvalue, find a basis for its eigenspace (eigenvectors).
Form $P$ with eigenvectors as columns.
Form $D$ as a diagonal matrix with eigenvalues along the diagonal (matching the order of eigenvectors in $P$).
Note: A matrix is diagonalizable if and only if it has $n$ linearly independent eigenvectors.
Diagonalize \[ A = \begin{bmatrix}2 & 0 \\ 0 & 3\end{bmatrix}. \]
Solution: Eigenvalues: $\lambda_1=2$, $\lambda_2=3$ Eigenvectors: $v_1=[1,0]^T$, $v_2=[0,1]^T$
Form $P = [v_1 \, v_2] = \begin{bmatrix}1 & 0 \\ 0 & 1\end{bmatrix}$, $D = \begin{bmatrix}2 & 0 \\ 0 & 3\end{bmatrix}$
Check: $A = P D P^{-1}$
Diagonalize \[ B = \begin{bmatrix}4 & 1 \\ 0 & 2\end{bmatrix}. \]
Solution: Eigenvalues: $\lambda_1=4$, $\lambda_2=2$
Eigenvectors: \[ \lambda_1=4: (B-4I)v=0 \implies \begin{bmatrix}0 & 1 \\ 0 & -2\end{bmatrix}v=0 \implies v_1=[1,0]^T \] \[ \lambda_2=2: (B-2I)v=0 \implies \begin{bmatrix}2 & 1 \\ 0 & 0\end{bmatrix}v=0 \implies 2x + y=0 \implies v_2=[1,-2]^T \]
Form $P = [v_1 \, v_2] = \begin{bmatrix}1 & 1 \\ 0 & -2\end{bmatrix}$, $D = \begin{bmatrix}4 & 0 \\ 0 & 2\end{bmatrix}$
Then $A = P D P^{-1}$
Diagonalize \[ C = \begin{bmatrix}1 & 1 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 2\end{bmatrix}. \]
Solution: Eigenvalues: $\lambda=1$ (multiplicity 2), $\lambda=2$
For $\lambda=1$: $(C-I)v=0 \implies \begin{bmatrix}0 & 1 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 1\end{bmatrix}v=0 \implies v=[x,0,0]^T$ (choose $v_1=[1,0,0]^T$), $v_2=[0,0,1]^T$
For $\lambda=2$: $v_3=[0,0,1]^T$ (already chosen, can pick linearly independent vector for multiplicity)
Form $P$ from independent eigenvectors and $D = \text{diag}(1,1,2)$
Notes on diagonalization:
A matrix with $n$ distinct eigenvalues is always diagonalizable.
If an eigenvalue has multiplicity $m$, the dimension of its eigenspace must also be $m$ for diagonalization.
Symmetric matrices are always diagonalizable and have orthogonal eigenvectors.
Applications of Eigenvalues and Eigenvectors
Eigenvalues and eigenvectors have numerous practical applications in mathematics, engineering, and data science. Some key applications are:
1. Powers of a matrix: If a matrix $A$ is diagonalizable as $A = P D P^{-1}$, computing $A^k$ for large $k$ is simplified as $A^k = P D^k P^{-1}$. The eigenvalues along the diagonal of $D$ are raised to the power $k$.
2. Solving systems of linear differential equations: For systems of the form $\frac{d\mathbf{x}}{dt} = A \mathbf{x}$, eigenvalues determine the growth/decay rates, and eigenvectors determine the directions of solutions.
3. Quadratic form diagonalization: Eigenvectors are used to rotate coordinate axes to diagonalize a quadratic form, facilitating analysis in optimization, physics, and statistics.
4. Stability analysis in dynamical systems: In discrete or continuous systems, the eigenvalues of the system matrix determine whether equilibrium points are stable or unstable.
5. Principal Component Analysis (PCA): Eigenvectors of the covariance matrix identify the directions of maximum variance in data, while eigenvalues indicate the magnitude of variance along those directions.
6. Markov chains and steady-state distributions: The eigenvector corresponding to the eigenvalue $\lambda = 1$ of a stochastic matrix represents the steady-state probabilities.
7. Vibration analysis and mechanical systems: Eigenvalues correspond to natural frequencies, and eigenvectors indicate mode shapes of vibrating systems.
8. Google PageRank algorithm: Eigenvectors of the link matrix are used to rank the importance of web pages.
9. Image compression and signal processing: Eigenvectors (or singular vectors in SVD) are used to reduce dimensionality while preserving significant information.
10. Control theory and system design: Eigenvalues of system matrices are used to design stable and controllable systems.