Rank and Nullity
Vectors from Matrices
A vector can be seen as a matrix with either a single row or a single column:
1. A column vector is an $n \times 1$ matrix: \[ \mathbf{v} = \begin{bmatrix}v_1 \\ v_2 \\ \vdots \\ v_n\end{bmatrix} \]
2. A row vector is a $1 \times n$ matrix: \[ \mathbf{u} = \begin{bmatrix}u_1 & u_2 & \dots & u_n\end{bmatrix} \]
Thus, vectors can be viewed as special cases of matrices with either one column or one row.
Column vector from matrix: \[ \mathbf{v} = \begin{bmatrix}2 \\ -1 \\ 3\end{bmatrix} \in \mathbb{R}^3 \]
Row vector from matrix: \[ \mathbf{u} = \begin{bmatrix}1 & 0 & 4 & -2\end{bmatrix} \in \mathbb{R}^4 \]
A $2 \times 1$ matrix can be treated as a vector in $\mathbb{R}^2$: \[ \mathbf{w} = \begin{bmatrix}5 \\ 7\end{bmatrix} \]
A $1 \times 5$ matrix as a row vector: \[ \mathbf{x} = \begin{bmatrix}1 & -1 & 2 & 0 & 3\end{bmatrix} \]
Row Space, Column Space, and Null Space
For a matrix $A \in \mathbb{R}^{m \times n}$:
The row space of $A$, denoted $\text{Row}(A)$, is the subspace of $\mathbb{R}^n$ spanned by the row vectors of $A$.
The column space of $A$, denoted $\text{Col}(A)$, is the subspace of $\mathbb{R}^m$ spanned by the column vectors of $A$.
The null space (or kernel) of $A$, denoted $\text{Null}(A)$, is the set of all $x \in \mathbb{R}^n$ such that $Ax = 0$.
These spaces are fundamental because they reveal the structure of the solutions of linear systems.
Prerequisites: Echelon Forms
Before finding these subspaces, we need to understand the concept of echelon forms.
Row Echelon Form (REF) of a matrix $A$ is a form that satisfies:
All non-zero rows are above rows of all zeros.
The leading (first non-zero) entry of each row, called a pivot, is to the right of the pivot in the row above it.
All entries below a pivot are zeros.
Column Echelon Form (CEF) is similar to REF but applied to columns:
All non-zero columns are to the left of all-zero columns.
The topmost non-zero entry in each pivot column is above the pivot entry in the column to its right.
All entries to the right of a pivot in its row are zeros.
Finding the Spaces
Row Space: Reduce $A$ to REF using row operations. The non-zero rows of the REF form a basis for $\text{Row}(A)$.
Column Space: The pivot columns of the original matrix $A$ form a basis for $\text{Col}(A)$.
Null Space: Solve the homogeneous system $Ax=0$ using the REF to find free variables. The solution vectors form a basis for $\text{Null}(A)$.
Example 1: Row, Column, and Null Spaces
Let \[ A = \begin{bmatrix} 1 & 2 & 3 \\ 2 & 4 & 6 \\ 1 & 1 & 1 \end{bmatrix}. \] Find the row space, column space, and null space.
Solution: Perform row reduction step-by-step: \[ \begin{bmatrix} 1 & 2 & 3 \\ 2 & 4 & 6 \\ 1 & 1 & 1 \end{bmatrix} \overset{R_2-2R_1,\ R_3-R_1}{\longrightarrow} \begin{bmatrix} 1 & 2 & 3 \\ 0 & 0 & 0 \\ 0 & -1 & -2 \end{bmatrix} \overset{-R_3}{\longrightarrow} \begin{bmatrix} 1 & 2 & 3 \\ 0 & 0 & 0 \\ 0 & 1 & 2 \end{bmatrix} \overset{\text{swap } R_2 \leftrightarrow R_3}{\longrightarrow} \begin{bmatrix} 1 & 2 & 3 \\ 0 & 1 & 2 \\ 0 & 0 & 0 \end{bmatrix}. \]
Row Space: Non-zero rows form a basis: \[ \text{Row}(A) = \text{span}\{ [1,2,3], [0,1,2] \}. \]
Column Space: The pivot columns (1st and 2nd) of the original matrix form a basis: \[ \text{Col}(A) = \text{span}\left\{ \begin{bmatrix} 1 \\ 2 \\ 1 \end{bmatrix}, \begin{bmatrix} 2 \\ 4 \\ 1 \end{bmatrix} \right\}. \]
Null Space: From REF, equations are: \[ x_1 + 2x_2 + 3x_3 = 0, \quad x_2 + 2x_3 = 0. \] From the second equation, $x_2 = -2x_3$. Substituting into the first: \[ x_1 + 2(-2x_3) + 3x_3 = 0 \quad \Rightarrow \quad x_1 = x_3. \] Let $x_3 = t$, then the solution vector is: \[ x = t \begin{bmatrix} 1 \\ -2 \\ 1 \end{bmatrix}. \]
Thus, \[ \text{Null}(A) = \text{span}\left\{ \begin{bmatrix} 1 \\ -2 \\ 1 \end{bmatrix} \right\}. \]
Example 2: Already in Row Echelon Form
Let \[ A = \begin{bmatrix} 1 & 0 & 2 \\ 0 & 1 & 3 \\ 0 & 0 & 0 \end{bmatrix}. \]
Solution: Since $A$ is already in REF, we directly identify:
Row Space: \[ \text{Row}(A) = \text{span}\{[1,0,2],[0,1,3]\}. \]
Column Space: Pivot columns are the first and second columns: \[ \text{Col}(A) = \text{span}\left\{ \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}, \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} \right\}. \]
Null Space: From equations: \[ x_1 + 2x_3 = 0, \quad x_2 + 3x_3 = 0. \] Let $x_3 = t$. Then $x_1 = -2t$, $x_2 = -3t$, so \[ x = t \begin{bmatrix} -2 \\ -3 \\ 1 \end{bmatrix}. \]
Thus, \[ \text{Null}(A) = \text{span}\left\{ \begin{bmatrix} -2 \\ -3 \\ 1 \end{bmatrix} \right\}. \]
Rank of a Matrix
The rank of a matrix $A$ is the dimension of its row space (or equivalently its column space). It equals the number of leading $1$s (pivots) in the row echelon form of $A$.
Find the rank of \[ A = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{bmatrix}. \] Solution: Reduce: \[ \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{bmatrix} \sim \begin{bmatrix} 1 & 2 & 3 \\ 0 & -3 & -6 \\ 0 & -6 & -12 \end{bmatrix} \sim \begin{bmatrix} 1 & 2 & 3 \\ 0 & 1 & 2 \\ 0 & 0 & 0 \end{bmatrix}. \] Two pivots $\implies \text{rank}(A)=2$.
Find the rank of \[ A = \begin{bmatrix} 1 & 2 & 3 \\ 2 & 4 & 6 \end{bmatrix}. \] Solution: Second row = $2 \times$ first row. Only one independent row. \[ \text{rank}(A)=1. \]
Properties of Rank
$\text{rank}(A) \leq \min(m,n)$ for an $m \times n$ matrix $A$.
$\text{rank}(A) = \text{rank}(A^T)$.
$\text{rank}(AB) \leq \min(\text{rank}(A), \text{rank}(B))$.
If $A$ is invertible ($n \times n$), then $\text{rank}(A)=n$.
If $A$ has a zero row (or column), removing it does not change rank.
$\text{rank}(A+B) \leq \text{rank}(A) + \text{rank}(B)$.
If $A$ is diagonal/triangular, then $\text{rank}(A)$ is the number of nonzero diagonal entries.
Elementary row/column operations do not change the rank of a matrix.
Property 1: For $A = \begin{bmatrix}1 & 2 & 3 \\ 4 & 5 & 6\end{bmatrix}$, $\min(m,n) = \min(2,3)=2$. Reducing, \[ \begin{bmatrix}1&2&3\\4&5&6\end{bmatrix} \sim \begin{bmatrix}1&2&3\\0&-3&-6\end{bmatrix} \sim \begin{bmatrix}1&2&3\\0&1&2\end{bmatrix}. \] Two pivots $\implies \text{rank}(A)=2 \leq 2$.
Property 2: For $A=\begin{bmatrix}1&2\\3&6\\4&8\end{bmatrix}$, row 2 = $3\times$row 1, row 3 = $4\times$row 1, so $\text{rank}(A)=1$. Now $A^T=\begin{bmatrix}1&3&4\\2&6&8\end{bmatrix}$, col 2 = $3\times$col 1, col 3 = $4\times$col 1, so $\text{rank}(A^T)=1$. Thus, $\text{rank}(A)=\text{rank}(A^T)=1$.
Property 3: Let \[ A=\begin{bmatrix}1&0\\0&0\end{bmatrix},\quad B=\begin{bmatrix}0&0\\0&1\end{bmatrix}. \] Here, $\text{rank}(A)=1$, $\text{rank}(B)=1$. But \[ AB=\begin{bmatrix}1&0\\0&0\end{bmatrix}\begin{bmatrix}0&0\\0&1\end{bmatrix} = \begin{bmatrix}0&0\\0&0\end{bmatrix}, \] so $\text{rank}(AB)=0 \leq \min(1,1)$.
Property 4: If $A=\begin{bmatrix}2&1\\5&3\end{bmatrix}$, \[ \det(A)=2\cdot3 - 5\cdot1 = 1 \neq 0. \] Hence $A$ invertible, $\text{rank}(A)=2$.
Property 5: $A=\begin{bmatrix}1&2&3\\4&5&6\\0&0&0\end{bmatrix}$. Removing the zero row gives $\begin{bmatrix}1&2&3\\4&5&6\end{bmatrix}$, which has the same rank ($=2$).
Property 6: Let \[ A=\begin{bmatrix}1&0\\0&0\end{bmatrix},\quad B=\begin{bmatrix}0&0\\0&1\end{bmatrix}. \] $\text{rank}(A)=1$, $\text{rank}(B)=1$. Then \[ A+B=\begin{bmatrix}1&0\\0&1\end{bmatrix}, \quad \text{rank}(A+B)=2. \] So $\text{rank}(A+B)=2 \leq 1+1=2$.
Property 7: For triangular $A=\begin{bmatrix}2&5&1\\0&3&7\\0&0&0\end{bmatrix}$, the nonzero diagonal entries are $2,3$. So $\text{rank}(A)=2$.
Property 8: $A=\begin{bmatrix}1&2\\3&6\end{bmatrix}$. Row 2 = $3\times$ row 1, so $\text{rank}(A)=1$. Apply row operation $R_2 \to R_2 - 3R_1$: \[ \begin{bmatrix}1&2\\0&0\end{bmatrix}, \quad \text{rank}=1. \] Rank is unchanged.
Nullity of a Matrix
The nullity of $A \in \mathbb{R}^{m \times n}$ is the dimension of its null space. It measures how many free variables there are in the system $Ax=0$.
Find nullity of \[ A = \begin{bmatrix} 1 & 2 & 3 \\ 2 & 4 & 6 \end{bmatrix}. \] Solution: Row rank = 1, matrix has $n=3$ columns. By rank-nullity: $\text{nullity}(A) = 3-1=2$. Indeed, $x_1 + 2x_2 + 3x_3=0$ gives two free variables, basis: \[ \text{Null}(A) = \text{span}\left\{ \begin{bmatrix}-2\\1\\0\end{bmatrix}, \begin{bmatrix}-3\\0\\1\end{bmatrix}\right\}. \]
Find nullity of \[ A = \begin{bmatrix} 1 & 0 & 2 \\ 0 & 1 & 3 \\ 0 & 0 & 0 \end{bmatrix}. \] Solution: Rank=2, $n=3$. Nullity = $3-2=1$. Null space = span$\{[-2,-3,1]\}$.
Rank-Nullity Theorem
Rank-Nullity Theorem: For any $A \in \mathbb{R}^{m \times n}$, \[ \text{rank}(A) + \text{nullity}(A) = n. \] This balances the number of pivot variables (rank) and free variables (nullity).
Verify rank-nullity for \[ A = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{bmatrix}. \] Solution: Rank = 2 (row-reduced form has 2 pivots). Nullity = $3-2=1$. Check: $2+1=3=n$.
Verify rank-nullity for \[ A = \begin{bmatrix} 1 & 2 & 1 \\ 2 & 4 & 2 \end{bmatrix}. \] Solution: Second row = 2×first row $=>$rank=1. $n=3$ $=>$nullity=$3-1=2$. Check: $1+2=3=n$.
Applications of Rank and Nullity
Applications:
Consistency of systems: $Ax=b$ is consistent iff $\text{rank}(A)=\text{rank}([A|b])$.
Invertibility: $A$ is invertible iff $\text{rank}(A)=n$.
Linear dependence: Rank tells maximum independent rows/columns.
Geometry: Null space gives directions squashed to zero; column space gives range of transformation.
Solve \[ x+y+z=6,\quad 2x+2y+2z=12,\quad x+y+2z=7. \] Solution: Coefficient matrix $A=\begin{bmatrix}1&1&1\\2&2&2\\1&1&2\end{bmatrix}$. Row reduce $=>$rank$(A)=2$, rank$([A|b])=2$ $=>$system consistent. $n=3$, nullity=$1$ $=>$infinitely many solutions, parameterized by one free variable.
Check if $A=\begin{bmatrix}1 & 2 \\ 3 & 4\end{bmatrix}$ is invertible. Solution: Rank$(A)=2$, $n=2$. Rank = number of columns $=>$invertible. Indeed, $\det(A)=-2\neq 0$.