Week 2: Preparation

Reading Material

• Reading: Read the rest of Chapter 3 (so, Definition 3.1.2 through Example 3.1.4 and Sections 3.4 through 3.8).

• Python: Demo 2

Key Concepts

Long Day will cover:

• Directional derivatives

• Differentiability

• The Jacobian matrix

• The gradient vector

• The generalized chain rule

• The Hessian matrix

Short Day is dedicated to Theme 1: The Gradient Method.

---

Preparatory Exercises

I: A Composite Function

Let \(g : \mathbb{R}^2 \to \mathbb{R}\) be given by

\[\begin{equation*} g(x,y) = e^{2x+y}, \end{equation*}\]

and let \(\pmb{f} : \mathbb{R} \to \mathbb{R}^2\) be given by

\[\begin{equation*} \pmb{f}(t) = \bigl(t^2, \sin(t)\bigr). \end{equation*}\]

Question a

Find the composite function \(g \circ \pmb{f}\).

Hint

First, find the expression for the composite function \((g \circ \pmb{f})(t)\)

Hint

Let \((x,y) := \pmb{f}(t) = (t^2, \sin(t))\) and substitute the expression for \(x\) and \(y\) into \(g(x,y)\).

Hint

Then, determine the domain and codomain of the composite function.

Answer

\(g \circ \pmb{f}: \mathbb{R} \to \mathbb{R}\) is given by \((g \circ \pmb{f})(t) = g(t^2, \sin(t)) = e^{2t^2 + \sin(t)}\) for \(t \in \mathbb{R}\).

Question b

Calculate the derivative

\[\begin{equation*} \frac{d}{dt}(g \circ \pmb{f})(t). \end{equation*}\]

Hint

The easiest approach is to differentiate the expression \(e^{2t^2 + \sin(t)}\) with respect to \(t\), but you can also use the chain rule.

Answer

\[\begin{equation*} \frac{d}{dt}(g \circ \pmb{f})(t) = e^{2t^2 + \sin(t)} \; \bigl(4t + \cos(t)\bigr). \end{equation*}\]

Question c

Is the composite function \(\pmb{f} \circ g\) well-defined?

Answer

Yes, in this case, it is. It is a function of the form \(\pmb{f} \circ g: \mathbb{R}^2 \to \mathbb{R}^2\). Try to find the functional expression yourself.

II: Partial Derivatives and Directional Derivatives

Let \(f : \mathbb{R}^2 \to \mathbb{R}\) be given by

\[\begin{equation*} f(x,y) = x^2 y + y^3. \end{equation*}\]

Question a

Calculate the partial derivatives \(\frac{\partial f}{\partial x}\) and \(\frac{\partial f}{\partial y}\) at the point \((1,2)\).

Hint

Differentiate \(f\) with respect to \(x\) and \(y\) separately, where you treat \(y\) and \(x\) as a constant, respectively.

Answer

\[\begin{equation*} \frac{\partial f}{\partial x}(x,y) = 2xy, \quad \frac{\partial f}{\partial y}(x,y) = x^2 + 3y^2. \end{equation*}\]

At the point \((1,2)\):

\[\begin{equation*} \frac{\partial f}{\partial x}(1,2) = 2 \cdot 1 \cdot 2 = 4, \quad \frac{\partial f}{\partial y}(1,2) = 1^2 + 3 \cdot 4 = 1 + 12 = 13. \end{equation*}\]

Question b

Calculate the gradient vector \(\nabla f(x,y)\), and state \(\nabla f(1,2)\).

Hint

The gradient of a function \(f\) is a vector consisting of its partial derivatives.

Hint

For a function of two variables the gradient is \(\nabla f(x,y) = \left( \frac{\partial f}{\partial x}(x,y), \frac{\partial f}{\partial y}(x,y) \right)\).

Answer

We have from the previous question that \(\nabla f(x,y) = (2xy, x^2 + 3y^2)\) and \(\nabla f(1,2) = (4, 13)\).

Question c

Calculate the directional derivative in the direction given by \(\pmb{e}_1=[1,0]^T\) at the point \((1,2)\). Also, calculate the directional derivative in the direction given by \(\pmb{e}_2=[0,1]^T\) at the point \((1,2)\).

Hint

The directional derivative in a “standard direction” corresponds to the partial derivatives.

Answer

At the point \((1,2)\), the directional derivative in the direction given by \(\pmb{e}_1\) is \(4\), and in the direction given by \(\pmb{e}_2\) it is \(13\).

Question d

What do the answers in questions b and c have to do with each other?

Answer

The partial derivatives are the directional derivatives in the “standard directions” \(\pmb{e}_1\) and \(\pmb{e}_2\).

III: A Vector Function in Three Variables

Let \(\pmb{f} : \mathbb{R}^3 \to \mathbb{R}^2\) be given by

\[\begin{equation*} \pmb{f}(x,y,z) = \bigl(xy + z, x - yz\bigr). \end{equation*}\]

Question a

Calculate \(\pmb{f}(1,2,3)\).

Hint

Substitute \((x,y,z) = (1,2,3)\) into the expression for \(\pmb{f}\).

Answer

\[\begin{equation*} \pmb{f}(1,2,3) = \bigl(1 \cdot 2 + 3, 1 - 2 \cdot 3\bigr) = (5, -5). \end{equation*}\]

Question b

Find the Jacobian matrix \(\pmb{J}_{\pmb{f}}(x,y,z)\). Calculate \(\pmb{J}_{\pmb{f}}(1,2,3)\).

Hint

The Jacobian matrix is a matrix that contains all the partial derivatives of all the coordinate functions. The vector function \(\pmb{f}=(f_1,f_2)\) has two coordinate functions, so what is the size of the Jacobian matrix?

Hint

The Jacobian matrix has the size: \(\pmb{J}_{\pmb{f}}(x,y,z) \in \mathbb{R}^{2 \times 3}\); remember that \(n=3\) is the number of input variables, and \(k=2\) is the number of “output” coordinate functions.

Answer

The Jacobian matrix \(\pmb{J}_{\pmb{f}}(x,y,z)\) is:

\[\begin{equation*} \pmb{J}_{\pmb{f}}(x,y,z) = \begin{bmatrix} \frac{\partial (xy + z)}{\partial x} & \frac{\partial (xy + z)}{\partial y} & \frac{\partial (xy + z)}{\partial z} \\ \frac{\partial (x - yz)}{\partial x} & \frac{\partial (x - yz)}{\partial y} & \frac{\partial (x - yz)}{\partial z} \end{bmatrix} = \begin{bmatrix} y & x & 1 \\ 1 & -z & -y \end{bmatrix}. \end{equation*}\]

At the point \((x,y,z)=(1,2,3)\):

\[\begin{equation*} \pmb{J}_{\pmb{f}}(1,2,3) = \begin{bmatrix} 2 & 1 & 1 \\ 1 & -3 & -2 \end{bmatrix}. \end{equation*}\]