Lecture 1.2:
Errors of numerical approximations

There are several potential sources of errors in a numerical calculation. Two sources are universal in the sense that they occur in any numerical computation. They are round-off and truncation errors. Inaccuracies of numerical computations due to the errors result in a deviation of a numerical solution from the exact solution, no matter whether the latter is known explicitly or not. To examine the effects of finite precision of a numerical solution, we introduce a relative error:

ERROR = | approximate value - exact value | / |exact value |

Round-off errors

Numbers are represented in a computer by a finite number of digits of precision. The simplest variant for hardware implementation is to keep the first n digits and to chop off all remaining digits. A more accurate scheme is to examine the (n+1)-st digit and to round the n-th digit to the nearest integer. This procedure leads to round-off errors.

Consider the number pi. It is irrational, i.e. it has infinitely many digits after the period:

pi = 3.14159265358979...

The round-off error of computer representation of the number pi depends on how many digits are left out. Make sure that you understand each line of the following rounding off the number pi:

of digits
for pi
1 3.100 0.141593 1.3239%
2 3.140 0.001593 0.0507%
3 3.142 0.000407 0.0130%

Round-off errors may accumulate, propagate and even lead to catastrophic cancellations leading to loss of accuracy of numerical calculations.

Truncation errors

Truncation of an infinite series to a finite number of terms leads to the truncation error. An infinite power series (Taylor series) represents a local behaviour of a function near a given point. If one replaces the series by the n-th order polynomial, the truncation error is said to be order of n, or order of O(hn), where h is the distance to the given point. Consider another irrational number e:

e = 2.71828182845905...

and compare it with the Taylor series of the function exp(x) near the given point x = 0:

exp(x) = 1 + x + x2/2 + x3/6 + ...

Check a few Taylor series approximations of the number e = exp(1):

order of n approximation
for e
3 2.500000 0.218282 8.030140%
4 2.666667 0.051615 1.898816%
5 2.708333 0.009948 0.365984%

Two MATLAB examples illustrate errors in numerical approximations:

Computation of pi as the area of a unit circle
      Discusses accuracy of elementary numerical integration
Computation of e by solving the differential equation for exponential growth
      Discusses accuracy of an elementary numerical ODE solver