Week 3: Preparation

Week 3: Preparation#

Reading Material#

We recommend that you read the textbook. Watching YouTube videos on the week’s topics can be useful, but it should not replace proper preparation for the week’s program and is not recommended as a standalone approach.

Read and study the following:

Key Concepts#

After reading, you should be able to explain the following key concepts:

  • Vector spaces with inner product and norm

  • \(\mathbb{R}^n\) and \(\mathbb{C}^n\)

  • Projections on the line

  • Projections on subspaces

  • Orthonormal bases

  • The Gram-Schmidt procedure

  • Orthogonal and unitary matrices

This week, we will explore these key concepts in great detail. We expect you to have familiarized yourself with these topics before lectures.

Preparatory Exercises#

I: Inner Product in \(\mathbb{R}^4\)#

Let \(\pmb{u} = (1, -2, 3, 4)\) and \(\pmb{v} = (-1, 0, 2, -3)\) be vectors in \(\mathbb{R}^4\) with the usual inner product defined.

  1. Calculate \(\langle \pmb{u}, \pmb{v} \rangle\).

  2. Compute the length (norm) of each vector.

  3. Can an angle \(\theta\) between the two vectors be determined?

Hint

Use the definition of the inner product:

\[\begin{equation*} \langle \pmb{u}, \pmb{v} \rangle = \pmb{u}_1 v_1 + \pmb{u}_2 v_2 + \pmb{u}_3 v_3 + \pmb{u}_4 v_4, \end{equation*}\]

and of the norm:

\[\begin{equation*} \|\pmb{u}\| = \sqrt{\langle \pmb{u}, \pmb{u} \rangle}. \end{equation*}\]

Answer

  1. Calculate \(\langle \pmb{u}, \pmb{v} \rangle\):

\[\begin{align*} \langle \pmb{u}, \pmb{v} \rangle &= (1)(-1) + (-2)(0) + (3)(2) + (4)(-3) \\ &= -1 + 0 + 6 - 12 = -7. \end{align*}\]

  1. The length (norm) of each vector:

\[\begin{align*} \|\pmb{u}\| &= \sqrt{1^2 + (-2)^2 + 3^2 + 4^2} = \sqrt{1 + 4 + 9 + 16} = \sqrt{30}, \\ \|\pmb{v}\| &= \sqrt{(-1)^2 + 0^2 + 2^2 + (-3)^2} = \sqrt{1 + 0 + 4 + 9} = \sqrt{14}. \end{align*}\]

  1. The angle \(\theta\) can be determined using the formula:

\[\begin{equation*} \cos\theta = \frac{\langle \pmb{u}, \pmb{v} \rangle}{\|\pmb{u}\|\|\pmb{v}\|} = \frac{-7}{\sqrt{30}\sqrt{14}} \approx -0.341. \end{equation*}\]

So the angle is approximately

\[\begin{equation*} \theta \approx \arccos(-0.341) \approx 110^\circ. \end{equation*}\]

II: An Orthonormal Basis in \(\mathbb{R}^5\)?#

Consider these vectors in \(\mathbb{R}^5\):

\[\begin{equation*} \pmb{a}_1 = (1,1,0,0,0), \quad \pmb{a}_2 = (0,1,1,0,0). \end{equation*}\]

Question a#

Verify that the Gram-Schmidt procedure that orthonormalizes vectors produces the following result:

\[\begin{equation*} \pmb{u}_1 = \left(\frac{1}{\sqrt{2}}, \frac{1}{\sqrt{2}}, 0, 0, 0\right), \quad \pmb{u}_2 = \left(-\frac{1}{\sqrt{6}}, \frac{1}{\sqrt{6}}, \frac{2}{\sqrt{6}}, 0, 0\right). \end{equation*}\]

Answer

First we let

\[\begin{equation*} \pmb{w}_1 = \pmb{a}_1 = (1,1,0,0,0). \end{equation*}\]

Then we normalize \(\pmb{w}_1\) to obtain the first orthonormal vector:

\[\begin{equation*} \pmb{u}_1 = \frac{\pmb{w}_1}{\|\pmb{w}_1\|} = \frac{(1,1,0,0,0)}{\sqrt{1^2+1^2}} = \left(\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}},0,0,0\right). \end{equation*}\]

Now the next intermediate vector is determined by removing the component along \(\pmb{u}_1\) from \(\pmb{a}_2\):

\[\begin{equation*} \pmb{w}_2 = \pmb{a}_2 - \langle \pmb{a}_2, \pmb{u}_1 \rangle \, \pmb{u}_1. \end{equation*}\]

We calculate the dot product:

\[\begin{equation*} \langle \pmb{a}_2, \pmb{u}_1 \rangle = (0)\left(\frac{1}{\sqrt{2}}\right) + (1)\left(\frac{1}{\sqrt{2}}\right) + (1)(0) = \frac{1}{\sqrt{2}}. \end{equation*}\]

We now have

\[\begin{equation*} \pmb{w}_2 = (0,1,1,0,0) - \frac{1}{\sqrt{2}}\left(\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}},0,0,0\right) = \left(0-\frac{1}{2},\,1-\frac{1}{2},\,1-0,\,0,\,0\right) = \left(-\frac{1}{2},\,\frac{1}{2},\,1,\,0,\,0\right). \end{equation*}\]

Finally, we normalize \(\pmb{w}_2\) to obtain the next orthonormal vector:

\[\begin{equation*} \pmb{u}_2 = \frac{\pmb{w}_2}{\|\pmb{w}_2\|}. \end{equation*}\]

We compute the norm:

\[\begin{equation*} \|\pmb{w}_2\| = \sqrt{\left(-\frac{1}{2}\right)^2 + \left(\frac{1}{2}\right)^2 + 1^2} = \sqrt{\frac{1}{4}+\frac{1}{4}+1} = \sqrt{\frac{3}{2}} = \frac{\sqrt{6}}{2}. \end{equation*}\]

Hence,

\[\begin{equation*} \pmb{u}_2 = \frac{\left(-\frac{1}{2},\,\frac{1}{2},\,1,\,0,\,0\right)}{\frac{\sqrt{6}}{2}} = \left(-\frac{1}{\sqrt{6}},\,\frac{1}{\sqrt{6}},\,\frac{2}{\sqrt{6}},\,0,\,0\right). \end{equation*}\]

This is the wanted result.

Question b#

Does the resulting list of vectors \(\pmb{u}_1, \pmb{u}_2\) constitute an orthonormal basis for \(\mathbb{R}^5\)?

Answer

No. They are orthonormal but they do not constitute a basis for \(\mathbb{R}^5\) since there are only two vectors in the list. To be a basis for \(\mathbb{R}^5\), five linearly independent vectors are needed.

III: A Linear Map that is a Projection?#

Consider the linear map \(\mathrm{proj}_Y: \mathbb{R}^4 \to \mathbb{R}^4\) given by:

\[\begin{equation*} \mathrm{proj}_Y(\pmb{x}) = P\pmb{x}, \quad \text{where} \quad P = \begin{bmatrix} 1/2 & 0 & 1/2 & 0 \\ 0 & 1/2 & 0 & 1/2 \\ 1/2 & 0 & 1/2 & 0 \\ 0 & 1/2 & 0 & 1/2 \end{bmatrix}. \end{equation*}\]

Question a#

Show that \(\mathrm{proj}_Y\) is a linear map.

Answer

Since \(\mathrm{proj}_Y(\pmb{x}) = P\pmb{x}\), and since matrix multiplication is linear, we have

\[\begin{equation*} P(\pmb{x}+\pmb{y}) = P\pmb{x} + P\pmb{y},\quad P(c\pmb{x}) = cP\pmb{x}. \end{equation*}\]

Hence, \(\mathrm{proj}_Y\) is linear.

Question b#

Show that the vectors \((1,0,-1,0)\) and \((0,1,0,-1)\) are orthogonal to the rows in \(P\). What does this have to do with the null space/kernel of \(P\)?

Answer

The rows in \(P\) are:

\[\begin{equation*} r_1 = \left(\frac{1}{2}, 0, \frac{1}{2}, 0\right) \quad \text{and} \quad r_2 = \left(0, \frac{1}{2}, 0, \frac{1}{2}\right), \end{equation*}\]

(where \(r_3 = r_1\) and \(r_4 = r_2\)).

We see that

\[\begin{equation*} (1,0,-1,0)\cdot r_1 = 1\cdot\frac{1}{2} + 0\cdot 0 + (-1)\cdot\frac{1}{2} + 0\cdot 0 = \frac{1}{2} - \frac{1}{2} = 0 \end{equation*}\]

and

\[\begin{equation*} (0,1,0,-1)\cdot r_2 = 0\cdot0 + 1\cdot\frac{1}{2} + 0\cdot0 + (-1)\cdot\frac{1}{2} = \frac{1}{2} - \frac{1}{2} = 0. \end{equation*}\]

Since these vectors are orthogonal to all rows in \(P\), they are located within the null space of \(P\).

This exercise will be continued later in this exercise in Week 3: Closure.