Week 5: Taylor Approximations

Week 5: Taylor Approximations#

Key Terms#

  • Tangent lines and tangent planes

  • Taylor polynomials of one variable

  • Taylor polynomials of \(n\) variables

  • Taylor polynomials for vector functions

  • Taylor’s theorem

  • Taylor’s limit formula \(f(x) = P_K(x) + \phantom{x}\) term with the \(\varepsilon\) function

  • Remainder term and evaluation of remainder

Preparation and Syllabus#

Exercises – Long Day#

1: Approximating Polynomials. By Hand#

Question a#

Find for each of the following functions their first- and second-degree Taylor polynomials with expansion point \(x_0=0\).

  1. \(f(x)=\text{e}^x,\quad x\in \mathbb{R}\)

  2. \(f(x)=\cos (x), \quad x\in \mathbb{R}\)

  3. \(f(x)=\text{e}^{\sin( x)},\quad x\in \mathbb{R}\)

Answer

  1. \(P_1(x)=1+x\) and \(P_2(x)=1+x+\frac 12x^2\)

  2. \(P_1(x)=1\) and \(P_2(x)=1-\frac{1}{2}x^2\)

  3. \(P_1(x)=1+x\) and \(P_2(x)=1+x+\frac{1}{2}x^2\)

Question b#

The function

\[\begin{equation*} f(x)=\frac{1}{x},\quad x>0 \end{equation*}\]

can of course not be Taylor-developed from the expansion point \(x_0=0\). So, instead, determine the approximating polynomial of the first and second degree of \(f\) with expansion point \(x_0=1\).

Answer

\(P_1(x)=-x+2\) and \(P_2(x)=x^2-3x+3\).

Question c#

Plot the four functions together with their respective approximating polynomials of the first and second degree in SymPy.

2: Investigation of Taylor Polynomials#

Use SymPy to determine the approximating polynomial of degree 9, \(P_9(x)\), with expansion point \(x_0=0\) for the function \(\sin(x)\). Draw in the same coordinate system \(\sin( x)\) and \(P_9(x)\). How far to the side can one make the approximating polynomial follow the function? (Experiment with the degree of the polynomial.)

Hint

See how you determine approximating polynomials using the command series i the SymPy demo of the week.

Answer

The higher the degree one chooses for the approximating polynomium, the greater will the interval around \(x_0 = 0\) be over which \(\sin( x)\) and the approximating polynomial \(P_K(x)\) follow each other. One can get the approximating polynomial \(P_K(x)\) to follow the function \(\sin(x)\) arbitrarily far to the side. For \(n = 17\) it indeed matches quite well with \(\sin(x)\) in the interval \(\left[ -2\pi , 2\pi\right]\) and for \(n = 25\) it is even better.

3: Evaluation of Error from Approximation#

A function \(f:\mathrm{dom}(f) \to \mathbb{R}\), \(\mathrm{dom}(f) \subseteq \mathbb{R}\), is given by

\[\begin{equation*} f(x)=\sqrt{2x-1}. \end{equation*}\]

Question a#

Determine the domain \(\text{dom}(f)\) of \(f\).

Question b#

Determine the approximating polynomial \(P_3(x)\) of degree \(3\) for \(f\) with expansion point \(x_0=1\).

Question c#

Justify that the remainder function \(R_3(x)\) corresponding to \(P_3(x)\) can be expressed by

\[\begin{equation*} R_3(x)=-\frac{5}{8} \frac 1{(2\xi-1)^{7/2}} (x-1)^4 \end{equation*}\]

for an \(\xi\) between \(1\) and \(x\). Show by evaluation of the remainder function that the numerical value of the error one incurs from using \(P_3(3/2)\) instead of \(f(3/2)\) is less than or equal to \(\displaystyle{\frac 5{2^7}}\).

Note

The point of exercises like this one is that one can find the approximation to a function value of a difficult function solely by using a polynomial for which the value is easy to compute. And that one at the same time, purely by hand, can determine the upper limit of the error one incurs by using the approximation. We are not aiming for the precise error value since that is just as difficult to compute as the function value!

4: Approximation of Complex Function#

Approximating polynomials of complex functions of a real variable are formed using the same formula as for real functions of a real variable. In the following we consider the function \(f:\mathbb{R} \to \mathbb{C}\) given by:

\[\begin{equation*} f(x)=2\cos(x)+i\sin(2x), \quad x\in \mathbb{R}. \end{equation*}\]

Question a#

Determine the approximating polynomial \(P_3\) of degree three for \(f\) with expansion point \(x_0=0\).

Answer

\(P_3(x)=2+2\cdot i \cdot x-x^2-\frac 43 \cdot i\cdot x^3\).

Question b#

Determine, possibly using SymPy’s series method, the approximating polynomial \(Q_3\) of degree no higher than three for \(f\) with expansion point \(x_1=\frac{\pi}{2}\).

Question c#

The number \(1\) is closer to \(x_1=\frac{\pi}{2}\) than to \(x_0=0\). Why is it even so a better idea to use \(P_3\) rather than \(Q_3\) if one needs an approximated value of \(f(1)?\)

Question d#

What is the smallest degree an approximating polynomial for \(f(x)\) with expansion point \(x_0=0\) must have for the distance between \(f(1)\) and the polynomial value at \(1\) to be less than one hundredth?

Answer

\(P_7\) will do the job.

5: Approximation of Function with 3 Variables#

We are given the function

\[\begin{equation*} f(x,y,z)=\text{e}^{(x+1)yz}. \end{equation*}\]

Question a#

Determine the approximating polynomial of second degree \(P_2(x,y,z)\) for the function \(f(x,y,z)\) at \((x_0,y_0,z_0)=(0,0,0)\).

Answer

\[\begin{equation*} P_2(x,y,z) = yz+1 \end{equation*}\]

Question b#

Determine using \(P_2(x,y,z)\) an approximation of \(f(0.1,0.2,0.3)\).

Answer

\[\begin{equation*} f(0.1, 0.2, 0.3) \approx P_2(0.1, 0.2, 0.3) = 1.06 \end{equation*}\]

Question c#

Determine the absolute error of your approximation in Question b.

Hint

The absolute error is given by \(|P_2(x,y,z)-f(x,y,z)|\).

Answer

\[\begin{equation*} \text{absolute error}= |P_2(0.1,0.2,0.3)-f(0.1,0.2,0.3)|\approx0.0082 \end{equation*}\]

6: An Implementation of Taylor Approximation#

We will now have a look at an implementation of Taylor polynomials in Sympy/Python.

Question a#

Implement Taylor polynomials for arbitrary \(K> 0\) - see this equation in the book - based on the following setup.

def taylor(f,K,x0):
    Pk = 0
    for k in range(K+1):
        # Compute the k'th derivative of f at x0
        fk = # ???
        # Add the k'th term to the approximation
        Pk += # ???
    return Pk

Answer

def taylor(f,K,x0):
    Pk = 0
    for k in range(K+1):
        # Compute the k'th derivative of f at x0
        fk = diff(f,x,k).subs({x:x0})
        # Add the k'th term to the approximation
        Pk += fk/factorial(k)*(x-x0)**k
    return Pk

Question b#

Use your Python function from the previous question to compute the approximating polynomial of degree four, \(P_4(x)\), for the function \(f(x)=\sin(x)\) at \(x_0=2\pi/3\).

Answer

\[\begin{equation*} P_4(x)=- \frac{x}{2} + \frac{\sqrt{3} \left(x - \frac{2 \pi}{3}\right)^{4}}{48} + \frac{\left(x - \frac{2 \pi}{3}\right)^{3}}{12} - \frac{\sqrt{3} \left(x - \frac{2 \pi}{3}\right)^{2}}{4} + \frac{\sqrt{3}}{2} + \frac{\pi}{3} \end{equation*}\]

Question c#

Plot \(f(x)\) and \(P_4(x)\) from Question b together in Sympy/Python.

Question d#

Now we wish to implement the Taylor approximation of second degree for functions of multiple variables, according to the definition in the book. For the evaluation of matrices and functions in Sympy/Python one can for example use the following simple implementation (or built-in methods). Consider why this works.

def evaluateFunction(f,x,x0):
    for i in range(len(x)):
        f = f.subs({x[i]:x0[i]})
    return f

Next, use the following setup to implement the approximation.

def taylorN(f,x,x0):
    N = len(x)
    # Constant term
    const = # Use evaluateFunction to achieve f evaluated at x0
    # First-degree term
    J = # Find the Jacobian matrix
    J0 = # Use evaluateFunction to achieve J evaluated at x0
    first = # Compute the entire second term
    # Second-degree term
    H = # Find the Hessian matrix
    H0 = # Use evaluateFunction to achieve H evaluated at x0
    second = # Compute the entire last term
    # Result
    Pk = simplify(Matrix([const]) + first + second)[0]
    return Pk

Answer

def taylorN(f,x,x0):
    N = len(x)
    # Constant term
    const = evaluateFunction(f,x,x0)
    # First-degree term
    J = Matrix([f]).jacobian(x)
    J0 = evaluateFunction(J,x,x0)
    first = J0*(x-x0)
    # Second-degree term
    H = hessian(f,x)
    H0 = evaluateFunction(H,x,x0)
    second = 1/2*(x-x0).T*H0*(x-x0)
    # Result
    Pk = simplify(Matrix([const]) + first + second)[0]
    return Pk

Question e#

We are given the function

\[\begin{equation*} f(\pmb{x})=\exp(x_1x_2)+x_2\sin(x_3). \end{equation*}\]

Determine the approximating polynomial of second degree using your implementation in Question f at the expansion point \(\pmb{x}_0=(3,0,-\pi/2)\).

Answer

\[\begin{equation*} P_2(\pmb{x})=x_1x_2+\frac{9}{2}x_2^2-x_2+1 \end{equation*}\]

Question f#

Now determine both the absolute and the relative error by using the approximation from Question e at the points

\[\begin{equation*} \pmb{y}_1 = (3.1,0.2,-\pi/2),\quad \pmb{y}_2 = (2.9,-0.2,-\pi/2). \end{equation*}\]

Hint

The absolute error is given by \(|P_2(x_1,x_2,x_3)-f(x_1,x_2,x_3)|\).

Hint

The relative error is given by \(|P_2(x_1,x_2,x_3)-f(x_1,x_2,x_3)|/|f(x_1,x_2,x_3)|\).

Answer

For \(\pmb{y}_1\) the absolute error is \(0.0589\) and the relative error is 3.55 %, and for \(\pmb{y}_2\) the absolute error is \(0.0401\) and the relative error is 5.28 %.

7: A Second-Degree Taylor Polynomial is a Quadratic Form#

We consider a function \(f: \mathbb{R}^n \to \mathbb{R}\) whose first- and second-order partial derivatives exist at the point \(\pmb{x}_{0}\). Show that the second-degree Taylor polynomial \(P_{2,f,\pmb{x}_{0}}(\pmb{x})\) defined in the book is a quadratic form \(q\).

In the quadratic form you must express the matrix \(A\), the column vector \(\pmb{b}\), and the constant \(c\) in terms of the vector \(\pmb{x}_{0}\) and \(f\) and its (partial) derivatives of the first and second order (in matrix form as gradient vector and Hessian matrix) evaluated at the point \(\pmb{x}_{0}\).

Hint

Begin by expanding all inner products in the definition of \(P_{2,f,\pmb{x}_{0}}(\pmb{x})\) using the linearity of the inner product. Collect all constant terms (those that are not dependent on \(\pmb{x}\)) and set them equal to \(c\). Continue from here with the linear terms in \(\pmb{x}\) and finally the quadratic terms.

Exercises – Short Day#

The term Taylor’s limit formula, also refered to as simply “Taylor’s formula” (see the book), implies that \(f(x)\) is isolated on the left-hand side and the approximating polynomial and the remainder term expression with an epsilon function are gathered on the right-hand side.

1: Function of one Variable#

Question a#

Determine with the expansion point \(x_0=0\) Taylor’s limit formula of second degree for the function

\[\begin{equation*} f(x)=2\cos(x)-2\sin(2x), \quad x\in \mathbb{R}. \end{equation*}\]

Answer

\(f(x)=2-4x-x^2+x^2\cdot \varepsilon(x)\).

Question b#

A smooth function \(f\) of one variable fulfills that \(f(2)=1\), \(f'(2)=1\), and \(P_2(1)=1\). Determine the approximating polynomial of second degree, \(P_2(x)\), for \(f\) with expansion point \(x_0=2\).

Hint

Use the two first pieces of information to write out \(P_2(x)\). There is actually only one unknown. Use the last piece of information to find this unknown.

Answer

\(P_2(x)=-1+x+(x-2)^2\)

2: Taylor’s Limit Formula. By Hand#

This exercise provides a method for computing a limit value of a fraction, in which both numerator and denominator go towards zero.

Question a#

Write out Taylor’s limit formula for the function \(\ln(1+x)\) with expansion point \(x_{0}=0\) of degree 1, as well as 2 and 3.

Question b#

Which of the three results in Question a can not be used to find the limit value:

\[\begin{equation*} \lim_{x\to 0}\frac{\ln(1+x)-x}{x^2}? \end{equation*}\]

Hint

Insert the expression from Taylor’s limit formula for \(\ln(1+x)\) of degree 1, and then investigate whether the limit value can be determined. In this exercise we do not need to use Taylor’s limit formula in place of the denominator. Why?

Hint

Now see, after reducing with \(x\), what the limit value becomes.

Hint

Then try with degree 2 and degree 3.

Answer

It is sufficient to expand to second degree.

Answer

\[\begin{equation*} \lim_{x\to 0}\frac{\ln(1+x)-x}{x^2}=\lim_{x\to 0}\frac{-\frac{x^2}{2}+x^2\varepsilon (x)}{x^2}= \lim_{x\to 0}(-\frac{1}{2}+\varepsilon (x))= -\frac{1}{2}. \end{equation*}\]

Question c#

Now compute using Taylor’s limit formula the following limit value:

\[\begin{equation*} \lim_{x\to 0}\frac{x(\text{e}^x+1)-2(\text{e}^x-1)}{x^3}. \end{equation*}\]

Hint

The denominator is of degree 3, so we should expand to the third degree.

Answer

\[\begin{equation*} \lim_{x\to 0}\frac{x(\text{e}^x+1)-2(\text{e}^x-1)}{x^3}=\frac{1}{6} \end{equation*}\]

3: Trick Exercise#

A function \(f\in C^{\infty}(\mathbb{R}^2)\) fulfills the equations

\[\begin{equation*} f(x,0)=\mathrm{e}^x\quad\text{and}\quad f'_y(x,y)=2y\cdot f(x,y). \end{equation*}\]

Question a#

Find the approximating polynomial of second degree for function \(f\) with \((x_0,y_0)=(0,0)\) as expansion point.

Hint

Yes, that’s right, you seem to be missing som information… Try writing out the expansion formula with the information that you have.

Hint

\[\begin{equation*} P_2(x,y)=f(0,0)+f'_x(0,0)\cdot x+f'_y(0,0)\cdot y+\frac{1}{2}(f''_{xx}(0,0)\cdot x^2+2f''_{xy}(0,0)\cdot xy+f''_{yy}(0,0)\cdot y^2). \end{equation*}\]

Hint

Find the missing pieces of information!

Answer

\(P_2(x,y)=1+x+\frac{1}{2}x^2+y^2\).

4: Taylor’s Formulas and Approximation#

We are given the function \(f: \mathbb{R}^2 \to \mathbb{R}\):

\[\begin{equation*} f(x,y)=\text{e}^{x+xy-2y} \quad \text{for} \quad (x,y)\in\mathbb{R}^2. \end{equation*}\]

Question a#

Write out the second-degree Taylor-polynomial for \(f\) with the expansion point \((x_0,y_0)=(0,0)\) in the usual form (without vectors and matrices).

Hint

See the following lemma in Section 4.5 of the book..

Question b#

Compute the gradient \(\nabla f(0,0)\) and Hessian matrix \(\pmb{H}_f(0,0)\), and write out in matrix form the second-degree Taylor polynomial for \(f\) with expansion point \((x_0,y_0)=(0,0)\).

Hint

See definition in the book.

Question c#

We now want an approximated value for \(f(\frac 34, \frac12)\) using an approximating second-degree polynomial for \(f\). It is of course convenient just to use the approximating second-degree polynomial with the expansion point \((0,0)\) that we have from the first question. On the other hand, \((\frac 34, \frac 12)\) is located a little bit closer to \((1,1)\) from where it is also relatively easy to expand from. So, maybe we should rather use \((1,1)\) as expansion point? What difference would that make?

Determine the approximating polynomials of second degree \(P_2(x,y)\) and \(Q_2(x,y)\) for \(f\) with the expansion points, respectively, \((0,0)\) and \((1,1)\). Compute the values of them at the point \((\frac 34, \frac 12)\) and compare with a computer value of \(f(\frac 34, \frac 12)\).

Answer

\(P_2(x,y)=1+x-2y-xy+\frac{1}{2}x^2+2y^2\) \(Q_2(x,y)=2x-y-(x-1)(y-1)+2(x-1)^2+\frac{1}{2}(y-1)^2\)

Answer

The deviation by using \(P_2\) is \(2.04\%\), while it by using \(Q_2\) is \(0.719\%\). So, in this case it might be worth the trouble to develop \(Q_2\ldots\)

5: Application of Approximating Polynomium#

A function \(f:\mathbb{R}^2 \to \mathbb{R}\) is given by

\[\begin{equation*} f(x,y)=\sqrt{x^2+y^2} \quad \text{for} \quad (x,y)\in\mathbb{R}^2. \end{equation*}\]

Question a#

Determine the approximating polynomial \(P_2(x,y)\) of second degree for \(f\) with the expansion point \((x_0,y_0)=(3,4)\).

Hint

See the definition in the book with the formula for the approximating second-degree polynomial.

Answer

\(\displaystyle{P_2(x,y)=5+\frac{3}{5}(x-3)+\frac{4}{5}(y-4)+\frac{8}{125}(x-3)^2-\frac{12}{125}(x-3)(y-4)+\frac{9}{250}(y-4)^2.}\)

Question b#

In this question we will illustrate the error that we incur by using the approximating second-degree polynomial instead of the exact value.

Determine, using the result from Question a, the length of the diagonal of a rectangle with side lengths 2.9 and 4.2 (you may use SymPy for the calculations).

Hint

The distance from the origin to an arbitrary point \((x,y)\) is a function of 2 variables. And yes, this is of course just \(f(x,y)=\sqrt{x^2+y^2}\).

Hint

Imagine the rectangle positioned in the \((x,y)\) coordinate system such that the diagonal links the points \((0,0)\) and \((2.9,4.2)\).

Answer

The length can be approximated by using \(P_2\) as we found above with the expansion point \((3,4)\). In short, \(P_{2}(2.9,4.2)=5.10400\).

Question c#

Compare with a SymPy value of the diagonal length.

Answer

The computer value is around 5.10392, so the approximation is not that bad after all.

Question d#

Is the different significant?

Answer

The difference is about \(8\cdot10^{-5}\), which is not much compared to the diagonal length. The difference \(8\cdot10^{-5}\) is called the absolute error, while the absolute error relative to (that is, divided by) the diagonal length is called the relative error.

Contents