Convergence and stability of iterative methods

To illustrate the main issues of iterative numerical methods, let us consider
the problem of root finding, i.e. finding of possible roots *x = x _{*}*
of a nonlinear equation

I = I(V) = V^{3} - 1.5*V^{2} + 0.6*V

The tunnel diode is connected with a resistor *R*
and a voltage source *E*. Applying the Kirchhoff's voltage law,
one can find that the steady current through the diode must satisfy the equation:

I(V) = ( E - V ) / R

For given *E* and *R*, this equation is of the form *f(x) = 0*,
where the voltage *x = V* is to be found. Three versus one solutions are possible
(click to enlarge the figure)

In order to actually compute these roots, we discuss here three main iterative methods.
More methods including globally convergent but slow *bracketing* methods are
also available for root finding (**optional** chapters 2.2, 2.3, 2.5 of
the main textbook).

Construct the following mapping from *x _{k}* to

x_{k+1} = g(x_{k}) = x_{k} + f(x_{k})

Starting with an initial approximation *x _{0}*, use the iterative
scheme above to find

The main issue of the iterative method is to check or to prove
if the sequence really converges to
a fixed point *x _{*}*. If not, we say that the method

It will be found during the class that the contraction mapping method is linearly stable provided

| g'(x_{*}) | = | 1 + f'(x_{*}) | < 1,

where the prime denotes the derivative of the functions *g(x)* and *f(x)*.
If the method is stable, the absolute error *err _{k}* reduces
with iterations as

Construct another mapping from *x _{k}* to

x_{k+1} = x_{k} - f(x_{k}) / f'(x_{k})

This is the Newton-Raphson method based on the approximation of a function
*f(x)* by the straight line tangent to the curve *f(x)*
at *x = x _{k}*:

y = f(x_{k}) + f'(x_{k}) (x - x_{k})

The estimate on the root is obtained by setting *y = 0* at *x = x _{k+1}*.
The Newton method is ALWAYS linearly stable (but it may diverge if

The slope of the curve *f(x)* at the point *x = x _{k}*
can be evaluated even if the derivative

x_{k+1} = x_{k} - f(x_{k}) (x_{k} - x_{k-1})
/ (f(x_{k}) - f(x_{k-1}))

This approximation when the derivative is replaced by the quotient is called
*the backward finite difference*. We shall study them later in much details.

It is easier to use the secant method compared to the Newton's method. However,
the downsides are (i) two starting points *x _{0}* and

Two MATLAB examples illustrate different algorithms for root finding in the problem above. Watch for convergence of the algorithms depending on the initial approximation of a root!

- Contraction mapping method
- Discusses stability of the method for each of three roots (for a steady current in a tunnel diode)
- Newton's method
- Discusses the fast (quadratic) rate of convergence for a single root (for the steady current)