Special Matrices
Projection Matrices
A projection matrix $P$ is a square matrix that satisfies \[ P^2 = P. \] It "projects" a vector onto a subspace. If $v$ is any vector, $Pv$ is the projection of $v$ onto the subspace. Properties:
Eigenvalues of $P$ are 0 or 1.
$I - P$ is also a projection matrix.
If $P$ is symmetric ($P^T = P$), it is an orthogonal projection.
Let \[ P = \begin{bmatrix}1 & 0 \\ 0 & 0\end{bmatrix}. \] Check if $P$ is a projection matrix: $P^2 = \begin{bmatrix}1 & 0 \\ 0 & 0\end{bmatrix} = P$.
For \[ P = \begin{bmatrix}0.5 & 0.5 \\ 0.5 & 0.5\end{bmatrix}, \] compute $P^2 = \begin{bmatrix}0.5 & 0.5 \\ 0.5 & 0.5\end{bmatrix} = P$. So $P$ is a projection matrix.
Check if $I-P$ is also a projection: \[ I-P = \begin{bmatrix}0.5 & -0.5 \\ -0.5 & 0.5\end{bmatrix}, \quad (I-P)^2 = I-P. \] Thus, $I-P$ is a projection onto orthogonal subspace.
—
Orthogonal Matrices
A square matrix $Q$ is orthogonal if \[ Q^T Q = QQ^T = I. \] Properties:
$Q^{-1} = Q^T$.
Determinant: $\det(Q) = \pm 1$.
Columns (and rows) of $Q$ form an orthonormal set.
Eigenvalues satisfy $|\lambda| = 1$.
\[ Q = \begin{bmatrix}0 & 1 \\ -1 & 0\end{bmatrix}, \quad Q^T Q = I. \] So $Q$ is orthogonal.
\[ Q = \frac{1}{\sqrt{2}}\begin{bmatrix}1 & 1 \\ -1 & 1\end{bmatrix}, \quad Q^T Q = I. \] Columns are orthonormal, determinant = 1.
Check inverse: $Q^{-1} = Q^T = \frac{1}{\sqrt{2}}\begin{bmatrix}1 & -1 \\ 1 & 1\end{bmatrix}$.
—
Orthogonal
A real square matrix $A$ is called orthogonal if \[ A^T = A^{-1}, \quad \text{or equivalently } AA^T = A^T A = I. \] Thus, $A$ must be invertible and its rows and columns form orthonormal sets of vectors.
\[ A = \frac{1}{9} \begin{bmatrix} 8 & 4 & 9 \\ 4 & 9 & -9 \\ 9 & -9 & -4 \end{bmatrix} \]
It can be verified that $AA^T = I = A^T A$. Hence, $A^T = A^{-1}$ and $A$ is orthogonal.
Properties of Orthogonal Matrices:
The rows of an orthogonal matrix form an orthonormal set.
The columns of an orthogonal matrix form an orthonormal set.
If $A$ is orthogonal, then $\det(A) = \pm 1$.
For $n=2$, every real orthogonal matrix has the form \[ A = \begin{bmatrix} \cos \theta & \sin \theta \\ -\sin \theta & \cos \theta \end{bmatrix} \quad \text{or} \quad A = \begin{bmatrix} \cos \theta & \sin \theta \\ \sin \theta & -\cos \theta \end{bmatrix}. \] Both are orthogonal since $AA^T = I$.
Normal
A real square matrix $A$ is called normal if it commutes with its transpose: \[ AA^T = A^T A. \] Every symmetric, skew-symmetric, and orthogonal matrix is normal.
\[ A = \begin{bmatrix} 3 & -3 \\ -3 & 3 \end{bmatrix} \]
Then \[ AA^T = \begin{bmatrix} 45 & 0 \\ 0 & 45 \end{bmatrix}, \quad A^T A = \begin{bmatrix} 45 & 0 \\ 0 & 45 \end{bmatrix}. \]
Since $AA^T = A^T A$, the matrix $A$ is normal.
Idempotent Matrices
A matrix $A$ is idempotent if \[ A^2 = A. \] Properties:
Eigenvalues are 0 or 1.
$I-A$ is also idempotent.
Rank$(A) = \text{tr}(A)$ if $A$ is symmetric.
\[ A = \begin{bmatrix}1 & 0 \\ 0 & 0\end{bmatrix}, \quad A^2 = A. \] So $A$ is idempotent.
\[ A = \begin{bmatrix}0.5 & 0.5 \\ 0.5 & 0.5\end{bmatrix}, \quad A^2 = A. \] Hence idempotent.
Check $I-A$: \[ I-A = \begin{bmatrix}0.5 & -0.5 \\ -0.5 & 0.5\end{bmatrix}, \quad (I-A)^2 = I-A. \]
—
Partition (Block) Matrices
Matrices can be partitioned into submatrices (blocks): \[ A = \begin{bmatrix}A_{11} & A_{12} \\ A_{21} & A_{22}\end{bmatrix}. \] Block properties:
Block multiplication: conformable sizes.
Determinant of block-diagonal: $\det(A) = \det(A_{11})\det(A_{22})$.
Useful for LU decomposition or large systems.
\[ A = \begin{bmatrix}\begin{bmatrix}1&0\\0&1\end{bmatrix} & \begin{bmatrix}1\\2\end{bmatrix} \\ \begin{bmatrix}3 & 4\end{bmatrix} & \begin{bmatrix}5\end{bmatrix} \end{bmatrix}. \] Matrix is partitioned into 4 blocks.
For block-diagonal \[ A = \begin{bmatrix}B & 0 \\ 0 & C\end{bmatrix}, \quad \det(A) = \det(B)\det(C). \]
Block multiplication: \[ \begin{bmatrix}A & B \\ C & D\end{bmatrix} \begin{bmatrix}X & Y \\ Z & W\end{bmatrix} = \begin{bmatrix}AX + BZ & AY + BW \\ CX + DZ & CY + DW\end{bmatrix}. \]
—
Properties of Special Matrices
Summary of key properties:
Projection and idempotent: $P^2 = P$, eigenvalues = 0 or 1.
Orthogonal: $Q^T Q = I$, inverse = transpose, determinant = ±1.
Block matrices: useful in partitioned operations, determinant of block-diagonal = product of determinants.
Symmetric idempotent: rank = trace.
Symmetric idempotent: \[ A = \begin{bmatrix}1 & 0 \\ 0 & 0\end{bmatrix}, \quad \text{rank}(A) = \text{tr}(A) = 1. \]
Orthogonal check: \[ Q = \begin{bmatrix}0 & 1 \\ -1 & 0\end{bmatrix}, \quad Q^T Q = I, \det(Q)=-1. \]
Projection matrix property: \[ P = \begin{bmatrix}0.5 & 0.5 \\ 0.5 & 0.5\end{bmatrix}, \quad I-P = \begin{bmatrix}0.5 & -0.5 \\ -0.5 & 0.5\end{bmatrix}, \quad (I-P)^2 = I-P. \]
—
Applications of Special Matrices
Applications:
Projection matrices: least squares, computer graphics.
Orthogonal matrices: rotations, reflections, numerical stability.
Idempotent matrices: statistics (hat matrices), regression analysis.
Block matrices: large-scale computation, parallel processing, solving partitioned systems.