Week 1: Continuity

Key Terms

• Scalar functions, in particular quadratic forms

• Vector functions

• Visualisation of functions: Graphs and level curves/sets

• Continuity

• The usual inner product (dot product) and norm in \(\mathbb{R}^n\)

• Partial derivatives and the gradient vector

Preparation and Syllabus

• Revision: Study Chapter 0

• Long Day: Chapter 1, Section 2.1, Section 3.1, and Section 3.2

• Short Day: Section 2.2, Section 3.2, and Section 3.3

• Python demos: SymPy Intro, Week 1: Functions

---

Exercises – Long Day

1: Function or not?

Consider the following correspondance between \(a\) and \(b\) values:

\(a\)	\(b\)
1	0
2	1
0	3
1	2

We consider functions whose domains are subsets of \(\{0,1,2,3\}\) and whose co-domains are \(\{0,1,2,3\}\). We shall determine whether \(f\) and \(g\) define functions if we let \(f\) follow the rule that the first column (the \(a\) values) is input and the second column (the \(b\) values) is output of the function \(f\) and the domain is \(\{0,1,2\}\); and we let \(g\) follow the rule that the second column is input and the first column is output of function \(g\) with domain \(\{0,1,2,3\}\).

Does \(f\) define a function? Does \(g\)? If so, determine the range/image of the function and determine whether the function is injective and/or surjective.

Hint

Read this section in the Note.

Answer

Only \(g\) is a function since \(f\) both tries to define \(f(1)=0\) as well as \(f(1)=2\). The function \(g\) has \(\mathrm{im}(g) = \{0,1,2,\} \subset \{0,1,2,3\}\) and is thus not surjektive. Since \(g(0)=1\) and \(g(2)=1\), it is also not injective.

2: Equal Functional Expressions?

We consider functions \(f_i : \mathbb{R} \to \mathbb{R}\) given by:

\[\begin{gather*} f_1(x) = |x| \\ f_2(x) = \begin{cases} x & x > 0 \\ -x & x \le 0 \end{cases} \\ f_3(x) = \max(x,0) \\ f_4(x) = ReLU(x) + ReLU(-x) \end{gather*}\]

where \(x \in \mathbb{R}\).

Some of the functions are the same function. Find all those!

Hint

ReLU is defined in the Notes.

Hint

First find \(f_i(x)\) for the different functions for a couple of different values of \(x\).

3: Possible Visualisations

Discuss whether one can visualise the below functions – if so, plot them using SymPy/dtumathtools:

1. A scalar function of two variables \(f: \mathbb{R}^2 \to \mathbb{R}, \, f(x_1,x_2) = \sqrt{\vert x_1 x_2 \vert}\)

2. A scalar function of four variables \(f: \mathbb{R}^4 \to \mathbb{R}, \, f(x_1,x_2,x_3,x_4) = \sqrt{\vert x_1 x_2 x_3 x_4 \vert}\)

3. A complex scalar function of two variables \(f: \mathbb{R}^2 \to \mathbb{C}, \, f(x_1,x_2) = \sqrt{\vert x_1 x_2 \vert} + i \cos(x_1 + x_2)\)

4. A vector field in 2D \(\pmb{f}: \mathbb{R}^2 \to \mathbb{R}^2, \, \pmb{f}(x_1,x_2) = (-x_2/3, x_1/3)\)

5. A vector field in 3D \(\pmb{f}: \mathbb{R}^3 \to \mathbb{R}^3, \, \pmb{f}(x,y,z)= (x^3+yz^2, y^3-xz^2, z^3)\)

6. A function of the form \(\pmb{r}: [0,10] \to \mathbb{R}^3, \, \pmb{r}(t) = (\cos(t),\sin(t),t)\)

Note

The following Python commands can be of use: dtuplot.plot3d, dtuplot.plot_vector, dtuplot.plot3d_parametric_line.

Hint

When plotting a vector field, it can be useful to scale the vectors (if they are too short or long). This can be done by multiplying the vector field by a scalar, such as 0.1 * f instead of using just f.

4: Different(?) Quadratic Forms

Let \(\pmb{x} = [x_1,x_2]^T\) be a column vector in \(\mathbb{R}^2\). Define:

\[\begin{equation*} A_1 =\left[\begin{matrix}11 & -12 \\ -12 & 4\end{matrix}\right], \, A_2 =\left[\begin{matrix}11 & 0 \\ -24 & 4\end{matrix}\right], \text{and} \, A_3 =\left[\begin{matrix} 73/5 & -36/5 \\ -36/5 & 52/5 \end{matrix}\right] \end{equation*}\]

as well as

\[\begin{equation*} \pmb{b}_1 = \left[\begin{matrix}-20\\ 40\end{matrix}\right], \, \pmb{b}_2=\pmb{b}_1,\, \pmb{b}_3=\left[\begin{matrix}-44\\ 8\end{matrix}\right], \text{and}\, c = -60\,. \end{equation*}\]

Let \(q_i: \mathbb{R}^2 \to \mathbb{R}\) be given by:

\[\begin{equation*} q_i(\pmb{x}) = \pmb{x}^T A_i \pmb{x} + \pmb{b}_i^T \pmb{x} + c \end{equation*}\]

for \(i=1,2,3\). Such functions are called quadratic forms, see this definition.

Question a

Expand the expression for \(q_1(x_1,x_2)\). First by hand, then using Python. Then expand the expressions for \(q_2(x_1,x_2)\) and \(q_3(x_1,x_2)\) (by hand or using Python).

Hint

It can be necessary to use c = Matrix([-60]) since SymPy will not add scalars to \(1\times1\) matrices. Reversely, a \(1\times1\) SymPy matrix q1 can be converted into a scalar with either q1[0,0] or q1[0].

Answer

\( q_1(\pmb{x}) = 11 x_{1}^{2} - 24 x_{1} x_{2} - 20 x_{1} + 4 x_{2}^{2} + 40 x_{2} - 60\)

Question b

Is the quadratic matrix \(A\) in a quadratic form (such as \(\pmb{x}^T A \pmb{x}\)) uniquely given?

Hint

Look at \(q_1(\pmb{x})\) and \(q_2(\pmb{x})\).

Answer

No. For instance, \(q_1(\pmb{x}) = q_2(\pmb{x})\), while \(A_1 \neq A_2\).

Question c

Plot the graph of the function \(q_1\). Then plot some level curves. Which geometric form does the level curves have? Do the same for \(q_3\).

Question d

One of the functions has a minimum. Which? Where is it located approximately? What is this same point called for functions that do not have a minimum?

Answer

• \(q_3\) has a minimum.

• It is located at \((2,1)\) (just read this from a plot for now).

• \(q_1\) has a so-called saddle point at this same point, see Wikipedia.

5: Linear Vector Function

Let \(A \in \mathsf{M}_{3 \times 5}(\mathbb{R})\) be given by

\[\begin{equation*} A = \begin{bmatrix} 1 & 0 & 2 & 3 & 4 \\ 0 & -1 & 5 & 6 & 7 \\ 0 & 0 & -3 & 8 & 9 \end{bmatrix}. \end{equation*}\]

Consider the vector function \(\pmb{f}: \mathbb{R}^5 \to \mathbb{R}^3\) given by \(\pmb{f} = \pmb{x} \mapsto A\pmb{x}\), where \(\pmb{x}\) is a column vector in \(\mathbb{R}\).

Question a

State the 3 coordinate functions for \(\pmb{f}\).

Hint

Remember that the coordinate functions \(f_i\) are \(\pmb{f} = (f_1,f_2,f_3)\).

Question b

State the image \(\mathrm{im}(\pmb{f})\) of \(\pmb{f}\).

Hint

What is the rank of \(A\)? What is the column space \(\mathrm{col}A\) af \(A\)?

Answer

\( \mathrm{im}(\pmb{f})=\mathrm{col}A=\mathbb R^3\)

Question c

Is the vector function \(\pmb{f}\) surjective and/or injective?

Answer

\(\pmb{f}\) is surjective, since \(\mathrm{im}(\pmb{f})=\mathbb R^3\). It is not injective, since \(\mathrm{dim}\,\mathrm{ker}(A)=5-3=2.\)

6: Size of Vectors

Consider the following three vectors in \(\mathbb{R}^3\):

\[\begin{equation*} \pmb{v}_1 = \left[\begin{matrix}-10\\ -10\\ -10\end{matrix}\right], \, \pmb{v}_2 = \left[\begin{matrix}-10\\ -4\\ 14\end{matrix}\right], \, \pmb{v}_3 = \left[\begin{matrix}-10\\ -8\\-12\end{matrix}\right]. \end{equation*}\]

Which vector is the longest? Which vectors are orthogonal to each other? Which two vectors are closest to each other?

Note

We can think of the vectors as geometric vectors with their starting points at \(\pmb{0}=[0,0,0]^T\) and end points \(\pmb{v}_i\) for, respectively, \(i=1,2,3\). Now and then one writes this as \(\overrightarrow{\pmb{0}\pmb{v}_i}\).

7: Chatbots and (Dis-)Continuity

Ask a chatbot, such as BingChat or ChatGPT, to provide an example of a function \(f: \mathbb{R} \to \mathbb{R}\), that is continuous at exactly 3 points and discontinuous at all other points. Evaluate whether the chatbot’s answer is right or wrong. Can this function be plotted?

8: Python Function as Mathematical Function

Consider the following function:

One can define the same function with the following single-line command: myfun = lambda x1,x2: Matrix([ln(x1), x1 * x2**2 + 1]). There are Python-technical differences between the two methods, but in practical use they are very similar. We recommend in general using def commands. Lambda commands are, though, useful in situations where it is only used once and in calls to other functions, for instance sorted(['a1', 'z0'], key = lambda x: int(x[1])).

from sympy import *
from dtumathtools import *
init_printing()

def f(x1,x2):
    return Matrix([ln(x1), x1 * x2**2 + 1])

Note

• It is not necessary to define x1,x2 = symbols("x1 x2").

• Even though we often work with functions in this course, it is often not necessary to define Python functions. For instance, the derivative of \(f(x)=x^2 \cos(x)\) is easily found simply by:

x = symbols("x")
f = x**2 * cos(x)
f.diff()

Question a

What is the function value at \(f(2,-3)\)?

Question b

Write down the Python function as a mathematical function. The co-domain is \(\mathbb{R}^2\). What is the domain?

Hint

In Python one does not state “domains”. Hence, here we mean that the domain must be the largest possible.

Question c

Find the image.

Hint

The question is for which \((y_1,y_2)\in\mathbb R^2\) the equations \(y_1=\mathrm{ln}(x_1)\) and \(y_2=x_1\,x_2^2+1\) have a solution. First, work on the equation \(y_1=\mathrm{ln}(x_1)\).

Hint

If \(x_1\) only can be an arbitrary positive real number, which values can \(x_1\,x_2^2+1\) then take?

Answer

\(\mathrm{im}(f)=\mathbb R \times [1,\infty[\)

9: Continuity of First-Degree Polynomials

Prove that the polynomial function \(f : \mathbb{R} \to \mathbb{R}\), \(f(x)=3x\) is continuous at all points \(x \in \mathbb{R}\).

Hint

Use the \(\epsilon-\delta\) definition. See this equation in the Note.

Hint

Let \(x_0\) be an arbitrary real number, and let \(\epsilon > 0\) be given. You must now state how one can choose \(\delta > 0\) (it may depend on \(x_0\) and \(\epsilon\)) such that \( |x-x_0| < \delta \Rightarrow |f(x)-f(x_0)| < \epsilon\).

---

Exercises – Short Day

1: Partial Derivatives of Simple Scalar Function

Find the partial derivatives \(\frac{\partial f}{\partial x_1}\) and \(\frac{\partial f}{\partial x_2}\) of \(f(x_1, x_2) = x_1^3 + 3x_1 x_2 + x_2^3\). Compute the values of the partial derivatives at the point \((x_1,x_2)=(1,2)\).

Hint

• Remember the formula for partial derivatives with respect to each variable.

• Treat other variables as constants during differentiation.

Answer

• \(\frac{\partial f}{\partial x_1} = 3x_1^2 + 3x_2\)

• \(\frac{\partial f}{\partial x_2} = 3x_2^2 + 3x_1\)

• At the point \((1,2)\) we have: \(\frac{\partial f}{\partial x_1}(1, 2) = 9\), \(\frac{\partial f}{\partial x_2}(1, 2) = 15.\)

2: Quadratic Forms with Symmetric Matrices

Let \(A\) be an arbitrary \(n \times n\) matrix, and let \(\pmb{x}\) be a column vector in \(\mathbb{R}^n\). Define \(B\) as \(B = (A + A^T)/2\).

Question a

Show that the matrix \(B\) is symmetric.

Hint

A matrix \(B\) is symmetric if and only if \(B = B^T\). Remember that \(B = \tfrac{1}{2} (A + A^T) \).

Answer

\( B^T = ((A + A^T)/2)^T = \tfrac{1}{2} (A^T + (A^T)^T) = \tfrac{1}{2} (A^T + A) = \tfrac{1}{2} (A + A^T) = B \)

Question b

Show that \(\pmb{x}^T A \pmb{x} = \pmb{x}^T B \pmb{x}\).

Question c

Conclude that one always can assume that quadratic forms of the form \(\pmb{q}(\pmb{x}) = \pmb{x}^T A \pmb{x} + \pmb{b}^T \pmb{x} + c\) are given by a symmetric matrix \(A\).

3: Level Curves and Gradients

We consider a function \(f:\mathbb{R}^2\rightarrow \mathbb{R}\) given by the expression

\[\begin{equation*} f(x,y)=x^2-2y \end{equation*}\]

as well as its level curves

\[\begin{equation*} f(x,y)=c,\, c\in\mathbb{R}. \end{equation*}\]

Question a

Show that the level curve of an arbitrary \(c\) can be described by an equation of the form \(y=g_c(x)\), where \(g_c\) is a real function of \(x\), and draw the level curves that correspond to \(c\in\{−2,−1,0,1,2\}\).

Answer

The level curves are 5 parabolas that have undergone a parallel displacement relative to each other in the direction of the \(y\) axis. What is the distance between them?

Question b

Show that the point \(P=(2,1)\) is located at the level curve corresponding to \(c=2\), and find a parametric representation of this level curve.

Answer

For instance, one can choose \(\pmb{r}(u)=\left(u,\frac{1}{2}u^2-1\right)\), \(u\in\mathbb{R}\).

Question c

Determine the tangent vector at \(P\) that corresponds to the parametric representation, and show that the tangent vector is orthogonal to the gradient of \(f\) at \(P\).

Hint

The tangent vector is achieved by first differentiating each of the two coordinate functions from the parametric representation, and then inserting the parameter value corresponding to the given point.

Question d

Create using Python a collective plot of the level curves and the gradient vector field of \(f\).

Hint

Use SymPy’s dtuplot.

4: Visualisations - Hiking on a Mountain

We are looking at a topographical map of a mountain, where the circles are level curves of the altitude function that represents the landscape. The arrows indicate the gradient vector field of the altitude function. On the mountain we find an hiking trail which when seen from above is elliptical, drawn in red on the map.

Question a

Imagine that you are hiking along the red path counter-clockwise. State those points on the path where the increase is 0 (where you are neither walking upwards nor downwards).

Hint

There are four such points.

Answer

At a level curve the altitude (height above the zero-level) of course does not change. Thus, we are looking for points on the path where the path shares a tangent with the a level curve.

Question b

At which parts of the path is the path increasing, and where is it decreasing?

Question c

Now follow one of the level curves on the drawing all the way around and consider the direction of those gradient vectors that are located nearby along the way. Conclusion?

Question d

This mountain is of course quite special. But put on your hiking boots once more and give an intuitive argument for why the gradient vectors on all mountains will always be perpendicular to the level curves.

Answer

Suggestion: If we consider the level curve as a path that we are walking along, then the trip is nice and easy. That is because we are all the time walking horizontally. It seems natural that the increase will be largest if we suddenly change our direction by 90 degrees upwards. Agreed?

5: Discontinuity of Heaviside’s Step Function

Show that Heaviside’s step function given here is discontinuous at \(x_0=0\).

Hint

A function \(f\) is discontinuous at \(x_0\) if we can find an \(\epsilon > 0\) such that no \(\delta > 0\) can result in \(|x-x_0| < \delta \Rightarrow |f(x)-f(x_0)| < \epsilon\) being true.
Written with quantifiers: \(f\) is discontinuous at \(x_0\) if and only if \(\exists \epsilon >0 \forall \delta >0 \exists x : |x-x_0| < \delta \text{ and } |f(x)-f(x_0)| \ge \epsilon\).

Hint

Choose \(\epsilon = 1/2\) (the exact value is not important as long as it is \(<1\)).