Quadratic Forms

A quadratic form in $n$ variables $x_1, x_2, \dots, x_n$ is an expression of the type \[ Q(x) = \sum_{i=1}^{n}\sum_{j=1}^{n} a_{ij} x_i x_j \] where $a_{ij} = a_{ji}$, i.e., $A = [a_{ij}]$ is a symmetric matrix. Equivalently, in matrix notation: \[ Q(x) = x^T A x, \quad x = \begin{bmatrix}x_1 \\ \vdots \\ x_n\end{bmatrix}, \quad A = A^T. \]

Symmetric matrix representation: Every quadratic form can be written uniquely using a symmetric matrix: \[ Q(x) = x^T \frac{A + A^T}{2} x \] even if $A$ is not symmetric initially.

If $A$ is symmetric, there exists an orthogonal matrix $P$ such that \[ P^T A P = D \] where $D$ is a diagonal matrix. Then, by the change of variables $y = P^T x$, the quadratic form becomes \[ Q(x) = x^T A x = y^T D y = \sum_{i=1}^{n} \lambda_i y_i^2 \] where $\lambda_i$ are the eigenvalues of $A$.

Let $Q(x) = x^T A x$.

• Positive definite: $Q(x) > 0$ for all $x \neq 0$ ($\lambda_i > 0$)

• Positive semi-definite: $Q(x) \ge 0$ ($\lambda_i \ge 0$)

• Negative definite: $Q(x) < 0$ for all $x \neq 0$ ($\lambda_i < 0$)

• Negative semi-definite: $Q(x) \le 0$ ($\lambda_i \le 0$)

• Indefinite: $Q(x)$ takes both positive and negative values (some $\lambda_i >0$, some $<0$)

By applying an orthogonal transformation $y = P^T x$, a quadratic form can always be expressed in canonical form: \[ Q(x) = \lambda_1 y_1^2 + \lambda_2 y_2^2 + \cdots + \lambda_n y_n^2 \] where $y_i$ are new variables along orthogonal axes, and $\lambda_i$ are eigenvalues of $A$.

The canonical form aligns the axes of the quadratic form with its principal directions. The signs of $\lambda_i$ determine the shape of the surface:

• Ellipsoid if all $\lambda_i>0$

• Hyperboloid if eigenvalues have mixed signs

• Paraboloid if some $\lambda_i=0$

Quadratic forms are used to:

• Test positive/negative definiteness of matrices

• Solve constrained optimization problems

• Analyze conic sections and 3D surfaces

• Diagonalize symmetric matrices (linking to orthogonal transformations)