Examining the stability question for the general problem (3) is too complicated. Instead, we examine the stability of numerical methods for the model problem

d y / d x = λ y ( x ) ,         y ( x 0 ) = y 0 (4)

whose analytical solution is

y ( x ) = y 0 exp ( λ ( x − x 0 ) ) (5)

where λ is a constant. The equation

d y / d x = λ y ( x ) (6)

is called the “Dahlquist test equation” [26] . The simple test problem (4) has traditionally been used for stability analysis since we can easily obtain analytical expressions describing the solution produced by the numerical method. Studying the behavior of a numerical method in solving this problem is also useful in predicting its behavior in solving the general problem (3). Indeed, if we expand d y / d x = f ( x , y ( x ) ) about ( x 0 , y 0 ) , we obtain

d y d x ≈ f ( x 0 , y 0 ) + ∂ f ∂ x | ( x = x 0 , y = y 0 ) ( x − x 0 ) + ∂ f ∂ y | ( x = x 0 , y = y 0 ) ( y ( x ) − y 0 ) (7)

= λ ( y ( x ) − y 0 ) + g ( x ) (8)

with λ = ∂ f / ∂ x | ( x = x 0 , y = y 0 ) and g ( x ) = f ( x 0 , y 0 ) + ∂ f / ∂ x | ( x = x 0 , y = y 0 ) ( x − x 0 ) .

This is a valid approximation if | x − x 0 | is sufficiently small, i.e., x ∈ [ x 0 − h , x 0 + h ] where h is a small positive real number. Introducing V ( x ) = y ( x ) − y 0 , we obtain

d V / d x ≈ λ V ( x ) + g ( x ) . (9)

The inhomogeneous term g ( x ) will drop out of all derivations concerning numerical stability, because we are concerned with differences of solutions of the equation. Dropping g ( x ) in Equation (9), we obtain the model Equation (6).

If we are dealing with a set of m equations, i.e.

d y / d x = f ( x , y ) , (10)

where y ( x ) = ( y 1 ( x ) , ⋯ , y m ( x ) ) T is a vector of m components, and f ( x , y ) = ( f 1 ( x , y 1 , ⋯ , y m ) , ⋯ , f m ( x , y 1 , ⋯ , y m ) ) T is a vector-valued function of m + 1 variables, the partial derivative ∂ f / ∂ y intervening in Equation (7) becomes a matrix called the Jacobian of f . Thus the model equation becomes

d y / d x = J y (11)

where J is the Jacobian of f and can be regarded as a constant matrix. For a linear system of differential equations, d y / d x = A ( x ) y , determined by the matrix A ( x ) , the Jacobian J is precisely the matrix A ( x ) . The differential system (11) reduces to a set of m equations like (6), i.e.

d Z i d x = λ i Z i ,         1 ≤ i ≤ m (12)

with λ 1 , ⋯ , λ m the eigenvalues of J and Z i , 1 ≤ i ≤ m , the components of the vector Z defined by

Z = R y ( x ) (13)

where R = ( R 1 , ⋯ , R m ) is a matrix of right eigenvectors of the Jacobian, which means that J R i = λ i R i , i = 1 , 2 , ⋯ , m .

We now study the behavior of the solution computed by the explicit Euler method (EEM) [1] [27] for the test problem (4) using a fixed step size h. We assume that we have computed solution values y 1 , y 2 , ⋯ , y n at points x 1 , x 2 , ⋯ , x n , respectively, where x j = x 0 + j h for j = 1 , 2 , ⋯ , n . The above mentioned numerical method, when applied to the model problem (4), leads to the following equation:

y n + 1 = y n + λ h y n . (14)

The ratio of the computed solutions at x n + 1 and x n is given by

y n + 1 y n = 1 + λ h . (15)

We compare this with the ratio of the true solutions at the same points, which is

y ( x n + 1 ) y ( x n ) = exp ( λ x n + 1 ) / exp ( λ x n ) = exp ( λ h ) . (16)

It is clear that when λ is real, 1 + λ h is a reasonable approximation to exp ( λ h ) except if λ h < − 2 . For large negative λ h , exp ( λ h ) is much smaller than 1, while 1 + λ h is (in magnitude) greater than 1. This means that the numerical solution is growing while the true solution is decaying. We say that the numerical solution produced by the EEM for λ h < − 2 is unstable.

Another way of looking at stability is to look at the propagation of local errors. For the EEM we have [1] [13] [27] :

e n + 1 = ( 1 + h ∂ f ∂ y ) e n = ( 1 + h λ ) e n , (17)

where e n = y ( x n ) − y n . If ∂ f / ∂ y = λ , we say that the errors are growing if | 1 + λ h | > 1 . In other words, the errors grow if λ h < − 2 and, for such step sizes, the method is called unstable.

To sum up, the requirement | y n | → 0 as n → ∞ , which, for λ < 0 , mirrors the asymptotic behavior of the analytical solution (5), implies that the inequality

0 < h < − 2 λ (18)

must be satisfied. If λ ≪ 0 , then the step size of integration is severely restricted by Equation (18) even though the true solution y 0 exp ( λ ( x − x 0 ) ) becomes negligible very quickly.

Before finishing this subsection, let us define some stability concepts for one-step numerical methods. For this purpose, we consider the model problem (4) where λ is a constant (possibly complex) and x 0 = 0 . If we assume that y ( 0 ) = η , the exact solution of this problem is

y ( x ) = η exp ( λ x ) . (19)

It is clear that lim x → ∞ y ( x ) = 0 if and only if Re ( λ ) < 0 . We have therefore to look for the conditions that have to be imposed on the above mentioned numerical methods in order that the numerical solution y n = y ( n h ) → 0 as n → ∞ where h is the step size of integration. By applying any one-step numerical method to the model problem in question, we obtain [20] [28]

y n + 1 = R ( z ) y n (20)

where z = λ h and R ( z ) is the so-called stability function. In order that y n tends to zero as n → ∞ , we must impose Re ( λ h ) < 1 thereby implying some constraints on the step size h. The set is called the stability domain of the numerical method. This latter one is said to be A-stable (absolutely stable) [1] [29] if its stability domain is included in ℂ − 1 = { z ; Re ( z ) ≤ 1 } . It is L-stable [1] if, apart from being A-stable, the stability function has the property lim Re ( λ h ) → − ∞ | R ( λ h ) | = 0 . A numerical scheme is called A ( α ) -stable [30] [31] for α ∈ ] 0 , π / 2 [ if its region of absolute stability includes the infinite wedge

S α = { z = λ h ∈ ℂ ; | π − arg ( z ) | < α } . (21)

This definition calls for a numerical method to have a stability region which is unbounded but which does not include the whole complex left hand half-plane. One of the reasons why this is such a useful definition is that many problems have eigenvalues of the Jacobian that lie in a sector S α .

L-stable methods are the most stable ones [32] and the backward Euler method [1] [27] is an example of an A-stable method.

We now briefly discuss stiffness and the difficulties involved in solving stiff equations. As discussed earlier, a system of ODEs of the form d y / d x = f ( x , y ) may be approximated by d y / d x = J y over a small interval of the independent variable x. If J is diagonalizable, this new system may be transformed into

d Z / d x = D Z , (22)

where D is a diagonal matrix with eigenvalues λ i of J (possibly complex) on its diagonal, and Z is related to y by Equation (13).

Many differential equation systems of practical importance in scientific modeling exhibit a distressing behavior when solved by classical numerical methods. This behavior is distressing because these systems, classified as stiff, are characterized by very high instability when approximated by standard numerical methods.

An intuitive idea of what stiff equations are is that they are problems with some smooth and some transient solutions, where all solutions reach the smooth one after a short time (after the transient phase has finished) [31] .

Since stiffness is closely related to the behavior of perturbations to a given solution, it is important to study the effect of a small perturbation ε Y ( x ) to a solution y ( x ) . The parameter ε is small, in the sense that we are interested only in asymptotic behavior of the perturbed solution as this quantity approaches zero. If y ( x ) is replaced by y ( x ) + ε Y ( x ) in the differential equation and the solution expanded in a series in power of ε , with ε 2 and higher powers replaced by zero, we obtain the system

y ′ ( x ) + ε Y ′ ( x ) = f ( x , y ( x ) ) + ε J Y ( x ) (23)

where J ( x ) is the Jacobian of f , y ′ ( x ) = d y / d x and Y ′ ( x ) = d Y ′ / d x . Subtracting Equation (10) from Equation (23), we obtain the equation

Y ′ = J ( x ) Y ′ ( x ) (24)

which controls the behavior of the perturbation. The Jacobian matrix J ( x ) has a crucial role in the understanding of stiff problems. In fact, its spectrum is sometimes used to characterize stiffness [7] [33] [34] . In an interval Δ x , chosen so that there is a moderate change in the value of the solution to y ′ = f ( x , y ( x ) ) , and very little change in J ( x ) , the eigenvalues of J ( x ) determine the growth rate of components of the perturbation. The existence of one or more large and negative values of λ Δ x , for λ ∈ σ ( J ( x ) ) , the spectrum of J ( x ) , indicates that stiffness is almost certainly present. If J ( x ) possesses complex eigenvalues, then we interpret this test for stiffness as the existence of a λ = Re ( λ ) + i Im ( λ ) ∈ σ ( J ( x ) ) such that Re ( λ Δ x ) is negative with large magnitude. A definition of stiffness in terms of the spectrum of the Jacobian matrix has been proposed by Gaffney [35] [36] who, in 1984 discussed the concept of stiffness oscillatory systems. This definition states that the IVP

y ′ ( x ) = f ( x , y ) ,     y ( x 0 ) = y 0 (25)

is considered to be stiff oscillatory over the interval S if for every x ∈ S the eigenvalues { λ j = u j + i v j , j = 1 , 2 , ⋯ , m } of the Jacobian J = ( ∂ f / ∂ y ) possess the following properties:

u j < 0 ,       j = 1 , 2 , ⋯ , m , (26)

max 1 ≤ j ≤ m | u j | ≫ min 1 ≤ j ≤ m | u j | (27)

or if the stiffness ratio S satisfies

S = max j , k | u j u k | ≫ 1 (28)

and | u j | ≪ | v j | for at least one pair of j in 1 ≤ j ≤ m .

If an explicit method like the Euler’s one is used to solve the differential system y ′ ( x ) = f ( x , y ( x ) ) when J has a real eigenvalue λ ≪ 0 , the step size of integration is controlled by a transient solution (which quickly becomes negligible), whereas outside the transient phase we wish the step size to be controlled by accuracy alone. This suggests a rather more pragmatic definition of stiffness, where the definition is not based on an analysis of the actual differential equation to be solved but is instead based on the relative performance of implicit and explicit integration methods. Perhaps the best and the oldest definition of this type is due to Curtiss and Hirschfelder [6] who, in 1952, said: “Stiff equations are equations where certain implicit methods, and in particular backward-differentiation formulae (BDFs), perform better, usually tremendously better, than explicit methods”.

The same situation can occur for complex eigenvalues with negative real parts. The problem is that the EEM is not A-stable or even A ( α ) -stable for any α < π / 2 . The same is true for all explicit methods like Euler’s one: no such explicit method can be A ( α ) -stable. We are therefore forced to use implicit methods, like the backward Euler method, to solve stiff systems. These methods require more work per step than explicit methods, however, since a system of nonlinear algebraic equations must be solved at each step. For further details of stiff problems, the reader is referred to Shampine and Gear [11] , Lambert [37] and Söderling et al. [38] .

Before finishing this section, let us introduce a definition of stiffness which involves a certain norm that depends on the differential system to be solved. If J ( x , y ) , the Jacobian matrix of the system, is diagonalizable, this definition is as follows. A system of ODEs is stiff if its stiff indicator, defined by

σ ( J ( x , y ) ) = m [ J ( x , y ) ] + M [ J ( x , y ) ] 2 (29)

is “large” and negative. Here, m [ J ] and M [ J ] are the greatest lower bounds (glb) and the least upper bound (lub) logarithmic Lipschitz constants, respectively [38] [39] . Note that M [ J ] corresponds to the usual logarithmic norm [39] .

The stiffness indicator is easily computed. Let λ [ A ] denote the eigenvalues of a matrix A . Then, for the Euclidian norm, it holds that

M 2 [ A ] = max λ [ H e ( A ) ] ;     m 2 [ A ] = min λ [ H e ( A ) ] (30)

where H e ( A ) = ( A + A T ) / 2 denotes the hermitian part of the matrix A . Other norms can also be used, provided that m [ J ] and M [ J ] are computed using well-known expressions.

By demanding that the interpolating function (32) coincides with the theoretical solution at the endpoints of the interval [ x n , x n + 1 ] , the following recursion formula is readily obtainable [15] [20] [40] :

y n + 1 = y n + R f n + S f n ( 1 ) (34)

where we use the notations f n = f ( x n , y n ) , f n ( 1 ) = d f ( x , y ) / d x | x = x n . R and S represent diagonal matrices defined by

R = Ω 1 Φ − Ω 1 Ξ , S = Φ + Ξ (35)

where Φ and Ξ are diagonal matrices with non zero entries in the main diagonal given by

Φ j = exp ( Ω j 1 h ) − 1 Ω j 1 ( Ω j 1 + Ω j 2 ) , Ξ j = exp ( − Ω j 2 h ) − 1 Ω j 2 ( Ω j 1 + Ω j 2 ) (36)

The components of the stiffness matrices are given by [15] [20] :

Ω j 1 = 1 2 ( − D j + D j 2 + 4 E j ) ,         Ω j 2 = Ω j 1 + D j , (37)

where D j and E j , ( j = 1 , ⋯ , m ) are expressed in terms of the components of the derivatives f n ( k ) , k = 0 , 1 , 2 , 3 :

D j = f j n ( 0 ) f j n ( 3 ) − f j n ( 1 ) f j n ( 2 ) f j n ( 1 ) f j n ( 1 ) − f j n ( 0 ) f j n ( 2 ) , E j = f j n ( 1 ) f j n ( 3 ) − f j n ( 2 ) f j n ( 2 ) f j n ( 1 ) f j n ( 1 ) − f j n ( 0 ) f j n ( 2 ) (38)

provided that the denominator in Equation (38) is not zero.

The complex interpolation formula (see Equation (33)) is adopted if in Equation (37), the following relationship holds:

( D j ) 2 < − 4 E j . (39)

The components of the stiffness matrices, which, in this case, are Ω 1 and Ω 1 * , can be written as

Ω j 1 = λ j + i u j ,           Ω j 2 = Ω j 1 * = λ j − i u j , (40)

for some real numbers λ j and u j . By imposing the same restrictions on (33) as (32), we still obtain the interpolation formula (34) but now with components R j and S j of R and S , respectively, given by [15]

R j ( λ j , u j ) = − e λ j h [ ( λ j 2 − u j 2 ) sin ( h u j ) − 2 λ j u j cos ( h u j ) ] − 2 λ j u j u j ( λ j 2 + u j 2 ) , (41)

S j ( λ j , u j ) = e λ j h [ λ j sin ( h u j ) − u j cos ( h u j ) ] + u j u j ( λ j 2 + u j 2 ) . (42)

Fatunla [15] has shown that his method, when applied to the scalar test problem y ′ = λ y , is L-stable and exponentially fitted for any complex value λ with negative real part. This means that the corresponding stability function R ( λ h ) = exp ( λ h ) gives the optimum stability properties. Furthermore, it can be shown that the jth component of the local truncation error at t = t n + 1 is given by [20] [40] :

T j n + 1 = h 5 5 ! [ f j n ( 4 ) + ( Ω j 2 3 − Ω j 2 2 Ω j 1 + Ω j 2 Ω j 1 2 − Ω j 1 3 ) f j n ( 1 )     − Ω j 1 Ω j 2 ( Ω j 1 2 − Ω j 1 Ω j 2 + Ω j 2 2 ) f n ( 0 ) ] + O ( h 6 ) (43)

for the real interpolation formula and by [15]

T j n + 1 = h 5 5 [ f j n ( 4 ) + ( λ j 2 + u j 2 ) ( 3 λ j 2 − u j 2 ) f j n + 4 λ j ( u j 2 λ j 2 ) f j n ( 1 ) ] + O ( h 6 ) (44)

for the complex interpolation formula.