CHAPTER 2. Numerical Approximation

The importance of error was introduced in our discussion of the falling parachutist in Chapter 1. Recall that we determined the velocity of a falling parachutist by both analytical and numerical methods. Although the numerical technique yielded estimates that were close to exact analytical solution, there was a discrepancy, or error, because the numerical method involved an approximation. Actually, we were fortunate in that case because the availability of an analytical solution allowed us to compute the error exactly. For many applied engineering problems, we cannot obtain analytical solutions. Therefore, we cannot compute exactly the errors associated with our numerical methods. In these cases, we must settle for approximations or estimates of the errors.

Although perfection is a laudable goal, it is rarely, if ever, attained. For example, despite the fact that the model developed from Newton`s second law is an excellent approximation, it would never in practice exactly predict the parachutist`s fall. A variety of factors such us winds and slight variations in air resistance would result in deviations from the prediction. If these deviations are systematically high or low, then we might need to develop a new model. However, if they are randomly distributed and tightly grouped around the prediction, then the deviations might be considered negligible and the model deemed adequate. Numerical approximations also introduce similar discrepancies into the analysis. Again, the question is: How much error is present in our calculations and is it tolerable?


SIGNIFICANT FIGURE

The concept of a significant figure, or digit, has been developed to formally designate the reliability of a numerical value. The significant digits of a number are those that can be used with confidence. They correspond to the number of certain digits plus one estimated digit. It is conventional to set the estimated digit at one-half of the smallest scale division on the measurement device.

Although it is usually a straightforward procedure to ascertain the significant figures of a number, some cases can lead to confusion. For example zeros are not always significant figures because they may be necessary just to locate a decimal point. The numbers 0.00001845, 0.0001845, and 0.001845 all have four significant figures. Similarly, when trailing zeros are used in large numbers, it is not clear how many, if any, of the zeros are significant. For example, at face value the number 45,300 may have three, four or five significant digits, depending on whether the zeros are known with confidence. Such uncertainty can be resolved by using scientific notation, where 4.53 x10^4, 4.530 x10^4, 4.5300 x10^4 designate that the number is known to three, four and five significant figures, respectively.


ACCURACY AND PRECISION

The errors associated with both calculations and measurements can be characterized with regard to their accuracy and precision. Accuracy refers to how closely a computed or measured value agrees with the true value. Precision refers to how closely individual computed or measured values agree with each other. These concepts can be illustrated graphically with the following pictures:





Numerical methods should be sufficiently accurate or unbiased to meet the requirements of a particular engineering problem. They also should be precise enough for adequate engineering design. In the next terms that I am going to introduce in this blog, you should understand the term error as both inaccuracy and the imprecision of predictions.


ERROR DEFINITIONS

Both documents bellow have some important information about the main kind of errors in numerical methods. Just take some time to watch them.

Errors Ppt

Numerical Methods Odes
http://www.sys-bio.org

Stability and Condition

The condition of a mathematical problem relates to its sensitivity to changes in its input values. We say that a computation is numerically unstable if the uncertainty of the input values is grossly magnified by the numerical method.

These ideas can be studied using a first order Taylor series



This relationship can be employed to estimate the relative error of f(x) as in



The relative error of x is given by



A condition number can be defined as the ratio of these relative errors



The condition number provides a measure of the extent to which an uncertainty in x is magnified by f(x). A value of 1 tells us that the function`s relative error is identical to the relative error in x. A value greater than 1 tells us that the relative error is amplified, whereas a value less than 1 tells us that it is attenuated. Functions with very large values are said to be ill-conditioned. Any combination of factors in the last equation, that increases the numerical value of the condition number will tend to magnify uncertainties in the computation of f(x).

Total Numerical Error

The total numerical error is the summation of the truncation and round-off errors. In general, the only way to minimize round-off errors is to increase the number of significant figures of the computer. Further, we have noted that round-off error will increase due to subtractive cancellation or due to an increase in the number of computations in an analysis. In contrast, the truncation error can be reduced by decreasing the step size. Because a decrease in step size can lead to subtractive cancellation or to increase in computations, the truncation errors are decreased as the round-off errors are increased. Therefore, we are faced by the following dilemma: The strategy for decreasing one component of the total error leads to an increase of the other component. In a computation, we could conceivably decrease the step size to minimize truncation errors only to discover that in doing so, the round-off error begins to dominate the solution and the total error grows! Thus, our remedy becomes our problem (the figure bellow shows this behavior). One challenge that we face is to determine an appropriate step size for a particular computation. We would like to choose a large step size in order to decrease the amount of calculations and round-off error begins to negate the benefits of step-size reduction.



In actual cases, however, such situations are relatively un common because most computers carry enough significant figures that round-off errors do not predominate. Nevertheless, they sometimes do occur and suggest a sort of “numerical uncertainty principle” that places an absolute limit on the accuracy that may be obtained using certain computerized numerical methods.


BIBLIOGRAPHY
Numerical Methods for Engineering. Steven C. Chapra-Raymond P. Canale. McGrawHill fifth edition.