{
"cells": [
{
"cell_type": "markdown",
"id": "65da8fed",
"metadata": {},
"source": [
"# Week 4: The Spectral Theorem"
]
},
{
"cell_type": "markdown",
"id": "cdd48f26",
"metadata": {},
"source": [
"## Key Terms"
]
},
{
"cell_type": "markdown",
"id": "ac445814",
"metadata": {},
"source": [
"* Symmetric and [Hermitian](https://en.wikipedia.org/wiki/Hermitian_matrix) matrices\n",
"* [The spectral theorem](https://en.wikipedia.org/wiki/Spectral_theorem) \n",
"* [Diagonalization](https://en.wikipedia.org/wiki/Diagonalizable_matrix)\n",
"* Orthogonal diagonalization"
]
},
{
"cell_type": "markdown",
"id": "3266bdaa",
"metadata": {},
"source": [
"## Preparation and Syllabus"
]
},
{
"cell_type": "markdown",
"id": "8475dbf1",
"metadata": {},
"source": [
"* Long Day: The rest of Chapter 2 \n",
"* Short Day: [Theme Exercise 2](/tema_dir/theme2.md)\n",
"* Python [demo](/demos/demo04_the_spectral_theorem.ipynb)"
]
},
{
"cell_type": "markdown",
"id": "9658cd4f",
"metadata": {},
"source": [
"___"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8c7c9cfd",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"from sympy import*\n",
"from dtumathtools import*\n",
"init_printing()"
]
},
{
"cell_type": "markdown",
"id": "aecae564",
"metadata": {},
"source": [
"## Exercises -- Long Day"
]
},
{
"cell_type": "markdown",
"id": "9c9a28be",
"metadata": {},
"source": [
"### 1: Types of Matrices\n",
"\n",
"Consider the matrices:\n",
"\n",
"\\begin{equation*}\n",
" A=\\begin{bmatrix} 1 & 0 & 0 \\\\ 0 & 2 & 0 \\\\ 0 & 0 & 3 \\end{bmatrix}, \\quad\n",
" B=\\begin{bmatrix} 1 & 2 & 3 \\\\ 3 & 1 & 2 \\\\ 2 & 3 & 1 \\end{bmatrix}, \\quad\n",
" C=\\begin{bmatrix} 1 & 2+i & 3i \\\\ 2-i & 1 & 2 \\\\ -3i & 2 & 1 \\end{bmatrix}, \\quad\n",
" D=\\begin{bmatrix} i & 2 & 3 \\\\ 2 & i & 2 \\\\ 3 & 2 & i \\end{bmatrix}.\n",
"\\end{equation*}\n",
"\n",
"Determine for *each* matrix whether it is symmetric, Hermitian, and/or normal. You may use SymPy to determine normality of the matrices. For your convenience the matrices are written in code here:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4460bddd",
"metadata": {
"tags": [
"remove-output"
]
},
"outputs": [],
"source": [
"A = Matrix.diag(1, 2, 3)\n",
"B = Matrix([[1, 2, 3], [3, 1, 2], [2, 3, 1]])\n",
"C = Matrix([[1, 2 + I, 3*I], [2 - I, 1, 2], [-3*I, 2, 1]]) \n",
"D = Matrix([[I, 2, 3], [2, I, 2], [3, 2, I]])"
]
},
{
"cell_type": "markdown",
"id": "12e2ac97",
"metadata": {},
"source": [
"### 2: Hermitian 2-by-2 Matrix. By Hand"
]
},
{
"cell_type": "markdown",
"id": "14d4e8ce",
"metadata": {},
"source": [
"We consider the Hermitian matrix $A$ given by: \n",
"\n",
"\\begin{equation*}\n",
" A=\\begin{bmatrix} 0 & i \\\\ -i & 0 \\end{bmatrix}.\n",
"\\end{equation*}\n",
"\n",
"This exercise concerns the computation of a **spectral decomposition** of $A$, which we know exists according to the Spectral Theorem (the complex case). We will find this decomposition of $A$ in three steps."
]
},
{
"cell_type": "markdown",
"id": "9357b665",
"metadata": {},
"source": [
"#### Question a"
]
},
{
"cell_type": "markdown",
"id": "153656b4",
"metadata": {},
"source": [
"Find all eigenvalues and corresponding eigenvectors of $A$. Check your answer with SymPy's `A.eigenvects()`."
]
},
{
"cell_type": "markdown",
"id": "6c28cd07",
"metadata": {},
"source": [
"#### Question b"
]
},
{
"cell_type": "markdown",
"id": "2f104132",
"metadata": {},
"source": [
"Determine an orthonormal basis consisting of eigenvectors of $A$."
]
},
{
"cell_type": "markdown",
"id": "5b21ec67",
"metadata": {},
"source": [
"#### Question c"
]
},
{
"cell_type": "markdown",
"id": "2faeef28",
"metadata": {},
"source": [
"This result applies to general $n \\times n$ matrices. Show that $A = U \\Lambda U^*$ if and only if $\\Lambda = U^* A U$, when $U$ is unitary."
]
},
{
"cell_type": "markdown",
"id": "b290fcdb",
"metadata": {},
"source": [
"#### Question d"
]
},
{
"cell_type": "markdown",
"id": "6fb33669",
"metadata": {},
"source": [
"Write out a unitary matrix $U$ and a diagonal matrix $\\Lambda$ such that $A = U \\Lambda U^*$. This formula is called a spectral composition of $A$. Check your result using the SymPy command:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b505f4f8",
"metadata": {
"tags": [
"remove-output"
]
},
"outputs": [],
"source": [
"A = Matrix([[0, I], [-I, 0]])\n",
"A.diagonalize(normalize = True)"
]
},
{
"cell_type": "markdown",
"id": "d1004c80",
"metadata": {},
"source": [
"### 3: Symmetric 3-by-3 Matrix"
]
},
{
"cell_type": "markdown",
"id": "d365dac5",
"metadata": {},
"source": [
"We are given the *real and symmetric* matrix\n",
"\n",
"\\begin{equation*}\n",
" A=\\begin{bmatrix} -2 & 1 & 1 \\\\ 1 & -2 & -1 \\\\ 1 & -1 & -2 \\end{bmatrix}.\n",
"\\end{equation*}\n",
"\n",
"Find a **spectral decomposition** of $A = Q \\Lambda Q^T$. In other words, state a *real orthogonal* matrix $Q$ and a *diagonal matrix* $\\Lambda$ such that \n",
"\n",
"\\begin{equation*}\n",
" A = Q \\Lambda Q^T\n",
"\\end{equation*} \n",
"\n",
"or, equivalently, \n",
"\n",
"\\begin{equation*}\n",
" Q^T \\, A\\, Q=\\Lambda\n",
"\\end{equation*} \n",
"\n",
"applies. Like in the previous question we know that it exists due to the Spectral Theorem (the real case)."
]
},
{
"cell_type": "markdown",
"id": "fa3911f5",
"metadata": {},
"source": [
"### 4: Spectral Decomposition with SymPy"
]
},
{
"cell_type": "markdown",
"id": "d85dfb06",
"metadata": {},
"source": [
"We consider the following matrices given in SymPy:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d94101de",
"metadata": {},
"outputs": [],
"source": [
"A = Matrix([[1, -1, 0, 0], [0, 1, -1, 0], [0, 0, 1, -1], [-1, 0, 0, 1]])\n",
"B = Matrix([[1, 2, 3, 4], [4, 1, 2, 3], [3, 4, 1, 2], [2, 3, 4, 1]])\n",
"A, B"
]
},
{
"cell_type": "markdown",
"id": "e12ef9cc",
"metadata": {},
"source": [
"We are informed that both matrices are real, *normal* matrices. This can be checked using:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ba0d8825",
"metadata": {},
"outputs": [],
"source": [
"A.conjugate() == A, B.conjugate() == B, A*A.T == A.T*A, B*B.T == B.T*B"
]
},
{
"cell_type": "markdown",
"id": "f3e4dd50",
"metadata": {},
"source": [
"We are furthermore informed that their eigenvalues are, respectively:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "de7bc108",
"metadata": {},
"outputs": [],
"source": [
"A.eigenvals(multiple=True), B.eigenvals(multiple=True)"
]
},
{
"cell_type": "markdown",
"id": "0744ceff",
"metadata": {},
"source": [
"#### Question a"
]
},
{
"cell_type": "markdown",
"id": "de2adef6",
"metadata": {},
"source": [
"Will the below SymPy commands give us the matrices that are involved in the spectral decompositions of $A$ and $B$? The call `A.diagonalize(normalize = True)` returns $(V,\\Lambda)$ where $A = V \\Lambda V^{-1}$ with *normalized* eigenvectors in $V$ and eigenvalues of $A$ in the diagonal matrix $\\Lambda$ (according to the eigenvalue problem from Mathematics 1a)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "197bfe74",
"metadata": {
"tags": [
"remove-output"
]
},
"outputs": [],
"source": [
"A.diagonalize(normalize = True), B.diagonalize(normalize = True)"
]
},
{
"cell_type": "markdown",
"id": "5df5c075",
"metadata": {},
"source": [
"#### Question b"
]
},
{
"cell_type": "markdown",
"id": "0021b564",
"metadata": {},
"source": [
"Does a unitary matrix exist which diagonalizes *both* $A$ and $B$? Meaning, does **one** unitary matrix exists such that $A = U \\Lambda_1 U^*$ and $B = U \\Lambda_2 U^*$, where \n",
"$\\Lambda_1$ is the diagonal matrix with eigenvalues of $A$ and $\\Lambda_2$ the diagonal matrix with eigenvalues of $B$?"
]
},
{
"cell_type": "markdown",
"id": "ae234813",
"metadata": {},
"source": [
"#### Question c"
]
},
{
"cell_type": "markdown",
"id": "e4cf76c1",
"metadata": {},
"source": [
"You have seen the matrix $U^*$ before (possibly with another column order). What kind of matrix is this?"
]
},
{
"cell_type": "markdown",
"id": "f2bfe936",
"metadata": {},
"source": [
"### 5: Diagonalization and Reduction of Quadratic Form"
]
},
{
"cell_type": "markdown",
"id": "925bcd39",
"metadata": {},
"source": [
"We consider the function $q : \\mathbb{R}^3 \\to \\mathbb{R}$ given by\n",
"\\begin{equation*}\n",
" q(x,y,z)=-2x^2-2y^2-2z^2+2xy+2xz-2yz+2x+y+z+5.\n",
"\\end{equation*}\n",
"\n",
"Note that $q$ can be split into two parts: a \"clean\" part with all second-degree terms, $k(x,y,z)=-2x^2-2y^2-2z^2+2xy+2xz-2yz$, and a part with all terms of less than second degree, meaning the first-degree polynomial $2x+y+z+5$."
]
},
{
"cell_type": "markdown",
"id": "413dbce4",
"metadata": {},
"source": [
"We are given the symmetric matrix\n",
"\n",
"\\begin{equation*}\n",
"A=\\begin{bmatrix} -2 & 1 & 1 \\\\ 1 & -2 & -1 \\\\ 1 & -1 & -2 \\end{bmatrix}.\n",
"\\end{equation*}"
]
},
{
"cell_type": "markdown",
"id": "29277c00",
"metadata": {},
"source": [
"#### Question a"
]
},
{
"cell_type": "markdown",
"id": "71c54c2f",
"metadata": {},
"source": [
"State a real, orthogonal matrix $Q$ and a diagonal matrix $\\Lambda$ such that\n",
"\n",
"\\begin{equation*}\n",
" Q^T \\, A\\, Q=\\Lambda.\n",
"\\end{equation*}\n",
"\n",
"You must choose $Q$ so it has $\\mathrm{det}\\,Q=1$. You may use SymPy for this exercise.\n",
"\n",
"**Note:**\n",
"Real, orthogonal matrices always have $\\mathrm{det}\\,Q = \\pm 1$ (why though?), so if your choice $Q$ has $\\mathrm{det}\\,Q = - 1$ then you can just change the sign of an arbitrary column or row. Real, orthogonal matrices with $\\mathrm{det}\\,Q = 1$ are said to be given a usual orientation. In $\\mathbb{R}^3$ this just means that the orthonormal basis in $Q$ establishes a *right-handed* coordinate system. This does not play any big role for us in this exercise."
]
},
{
"cell_type": "markdown",
"id": "2dadbc9b",
"metadata": {},
"source": [
"#### Question b"
]
},
{
"cell_type": "markdown",
"id": "65f69317",
"metadata": {},
"source": [
"Determine the expression $k(x,y,z),$ rewrite it to a matrix form, and *reduce* it."
]
},
{
"cell_type": "markdown",
"id": "b01a735a",
"metadata": {},
"source": [
"#### Question c"
]
},
{
"cell_type": "markdown",
"id": "21c95ddc",
"metadata": {},
"source": [
"Find an orthonormal basis with usual orientation for $\\mathbb{R}^3$ in which the expression for $q$ does not have mixed terms. Determine its expression."
]
},
{
"cell_type": "markdown",
"id": "2d69a778",
"metadata": {},
"source": [
"### 6: Standard Equation for the three Typical Conic Sections"
]
},
{
"cell_type": "markdown",
"id": "49585048",
"metadata": {},
"source": [
"In the following examples we will look at quadratic forms without *mixed terms* (as learned in the previous exercise we are able to get rid of these via diagonalization). Here it is possible to go one step further and remove the first-degree terms. This technique is called [completing the square](https://en.wikipedia.org/wiki/Completing_the_square). In the following we will be using the technique on our journey towards identification of so-called conic sections."
]
},
{
"cell_type": "markdown",
"id": "397b31d0",
"metadata": {},
"source": [
"#### Question a"
]
},
{
"cell_type": "markdown",
"id": "21cc04f6",
"metadata": {},
"source": [
"An ellipsis in the $(x,y)$ plane centred at $(c_1,c_2)$ with semi-axes $a$ and $b$ and with symmetry axes $x=c_1$ and $y=c_2$ has the standard equation\n",
"\n",
"\\begin{equation*}\n",
" \\frac{(x-c_1)^2}{a^2}+\\frac{(y-c_2)^2}{b^2}=1.\n",
"\\end{equation*}\n",
" \n",
"An ellipsis is given by the equation\n",
"\n",
"\\begin{equation*}\n",
" 4x^2+y^2+8x-6y+9=0.\n",
"\\end{equation*}\n",
"\n",
"Use the technique for completing the square, bring the equations to the standard form, and state the centre, semi-axes, and symmetry axes of the ellipsis."
]
},
{
"cell_type": "markdown",
"id": "476c976a",
"metadata": {},
"source": [
"#### Question b"
]
},
{
"cell_type": "markdown",
"id": "18e5bc56",
"metadata": {},
"source": [
"A hyperbola in the $(x,y)$ plane centred at $(c_1,c_2)$ with semi-axes $a$ and $b$ and with symmetry axes $x=c_1$ and $y=c_2$ has the standard equation\n",
"\n",
"\\begin{equation*}\n",
" \\frac{(x-c_1)^2}{a^2}-\\frac{(y-c_2)^2}{b^2}=1.\n",
"\\end{equation*}\n",
"\n",
"Or alternatively (If it is not horizontal but vertical):\n",
"\n",
"\\begin{equation*}\n",
" \\frac{(y-c_2)^2}{a^2}-\\frac{(x-c_1)^2}{b^2}=1.\n",
"\\end{equation*}\n",
"\n",
"A hyperbola is given by the equation\n",
"\n",
"\\begin{equation*}\n",
" x^2-y^2-4x-4y = 4.\n",
"\\end{equation*}\n",
"\n",
"Complete the square, bring the equation to standard form, and state the centre, semi-axes, and symmetry axes of the hyperbola."
]
},
{
"cell_type": "markdown",
"id": "f9bb8f0a",
"metadata": {},
"source": [
"#### Question c"
]
},
{
"cell_type": "markdown",
"id": "d4ea6d09",
"metadata": {},
"source": [
"A parabola in the $(x,y)$ plane with vertex (stationary point) $(c_1,c_2)$ and symmetry axis $x=c_1$ has the standard equation\n",
"\n",
"\\begin{equation*}\n",
" y-c_2=a(x-c1)^2.\n",
"\\end{equation*}\n",
"\n",
"Or alternatively, if the parabola is not vertical but horionzontal, which will give the symmetry axis $y=c_2$:\n",
"\n",
"\\begin{equation*}\n",
" x-c_1=a(y-c2)^2.\n",
"\\end{equation*}\n",
"\n",
"A parabola is given by the equation\n",
"\n",
"\\begin{equation*}\n",
" 2x^2+12x-y+17=0.\n",
"\\end{equation*}\n",
"\n",
"Complete the square, bring the equation to standard form, and state the vertex and symmetry axis of the parabola."
]
},
{
"cell_type": "markdown",
"id": "17df57c8",
"metadata": {},
"source": [
"### 7: The Partial Derivative Increases/Decreases the most in the Gradient Direction"
]
},
{
"cell_type": "markdown",
"id": "3e12e55f",
"metadata": {},
"source": [
"This exercise is from the Notes, and its purpose is to argue for why one in the gradient method moves in the direction of the gradient. \n",
"\n",
"Let $f: \\mathbb{R}^{n} \\to \\mathbb{R}$ be a function, for which all directional derivatives exist in $\\pmb{x} \\in \\mathbb{R}^{n}$. Assume that $\\nabla f(\\pmb{x})$ is not the zero vector."
]
},
{
"cell_type": "markdown",
"id": "a7f1a17c",
"metadata": {},
"source": [
"#### Question a"
]
},
{
"cell_type": "markdown",
"id": "506f8464",
"metadata": {},
"source": [
"Show that $\\pmb{u} := \\nabla f(\\pmb{x}) / \\Vert \\nabla f(\\pmb{x}) \\Vert$ is a unit vector."
]
},
{
"cell_type": "markdown",
"id": "6569cb34",
"metadata": {},
"source": [
"#### Question b"
]
},
{
"cell_type": "markdown",
"id": "db81326c",
"metadata": {},
"source": [
"Show that the scalar $|\\nabla_{\\pmb{v}}f(\\pmb{x})|$ becomes largest possible, when $\\pmb{v} = \\pm \\pmb{u}$."
]
},
{
"cell_type": "markdown",
"id": "9b743e54",
"metadata": {},
"source": [
"### 8: General Symmetric 2-by-2 Matrix \n",
"\n",
"We consider an arbitrary $2 \\times 2$ matrix. Such a matrix can be written as:\n",
"\n",
"\\begin{equation*}\n",
" A=\\begin{bmatrix} a & c \\\\ c & b \\end{bmatrix}\n",
"\\end{equation*}\n",
"\n",
"where $a,b,$ and $c$ are real numbers. (Why is this the case?)\n",
"\n",
"#### Question a\n",
"\n",
"Show that the eigenvalues of $A$ are real.\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"#### Question b\n",
"\n",
"Show that if $A$ is not a diagonal matrix, then it has two different (real) eigenvalues."
]
},
{
"cell_type": "markdown",
"id": "218844ce",
"metadata": {},
"source": [
"___\n",
"\n",
"## Thematic Exercise -- Short Day"
]
},
{
"cell_type": "markdown",
"id": "5d662bbb",
"metadata": {},
"source": [
"Today we will carry out [Theme Exercise 2](/tema_dir/theme2.md)."
]
}
],
"metadata": {
"jupytext": {
"formats": "ipynb,md:myst"
},
"kernelspec": {
"display_name": "python3",
"language": "python",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 5
}