This paper addresses the problem for solving a Continuous-time Riccati equation with an indefinite sign of the quadratic term. Such an equation is closely related to the so called full information H
∞ control of linear time-invariant system with external disturbance. Recently, a simultaneous policy update algorithm (SPUA) for solving H
∞ control problems is proposed by Wu and Luo (
Simultaneous policy update algorithms for learning the solution of linear continuous-time H∞ state feedback control, Information Sciences, 222, 472-485, 2013). However, the crucial point of their method is to find an initial point, which ensuring the convergence of the method. We will show one example where Wu and Luo’s method is not effective and it converges to an indefinite solution. Three effective methods for computing the stabilizing solution to the considered equation are investigated. Computer realizations of the presented methods are numerically compared on the computational platforms MATLAB and SCILAB.
The continuous-time algebraic Riccati equations and their extensions have been investigated extensively in the literature. Recently, the H¥ control problem was solved for linear time-invariant system [1] -[3] and for stoch- astic systems [4] -[7] .
Wu and Luo [8] have commented the iterative solution of the following continuous-time algebraic Riccati equation
Note that this equation has indefinite quadratic part. Assume there exists a positive semidefinite solution to (1) with property that real parts of eigenvalues of are negative. Such type solution is called a stabilizing solution.
The H¥ linear quadratic problems have been introduces by Basar and Bernhard [9] as a two-player zero sum gane. We consider a model for a a two-player zero-sum game, where the control function is a minimizing player (or a controller player) of the functional and the disturbance function is a maximizing player (or a disturbance player), where
The controller player aims to minimize the and the disturbance player aims to maximize the under a constrain of the system:
Knowing the stabilizing solution to (1) we define the following functions:
The functions have the property
And thus they form the equilibrium point of the two-player zero-sum game described by (2) and the functional. This fact is well known in the literature and it can be derived using the Pontryagin’s Maximum Principle for example. Moreover, the stabilizing solution is very important solution to Equation (1).
So, why we need to study the iterative equations for computing the stabilizing equation to (1)? Many re- searchers have investigated Riccati Equation (1) and more specially how to compute his stabilizing solution. Lanzon et al. [1] have proposed two effective methods. The first method constructs two matrix sequences where the first sequence converges to the stabilizing solution. The second method avoids the second matrix sequence and defines one matrix sequence which directly approximates the stabilizing solution. Later, Wu and Luo [8] have studied the same equation and the proposed method in [1] . They have commented that the second Lanzon’s method (it is Algorithm 2 [8] ) is not fully effective and by this reason they have introduced the new method described as Algorithm 4 in their paper [8] . Here, we consider an example where these two algorithms will be compared.
Example 1. Let us we take the following matrix coefficients to (1) (using the MATLAB notations):
;
;;;;.
We execute Algorithm 2 [8] and Algorithm 4 [8] with the initial point. Starting Algorithm 2 in MATLAB we obtain the following stabilizing solution to (1) after 4 main iterations and (the average number of iterations for the inner loop is 7):
And the solution computed by Algorithm 4 is
Note that the matrix is positive definite and while the matrix is
indefinite and. In addition. Thus these two solutions are different! Which of them is sought? We have to check whether the corresponding matrices stabilize system (2).
We compute eigenvalues of and. The eigenvalues of have negative real parts
and the matrix has one positive eigenvalue. Thus the matrix is the stabilizing solution to Equation (1) while the matrix is not the stabilizing solution to (1). In addition we have execute the same example with the open software SCILAB (http://www.scilab.org/scilab/about). We apply the SCILAB’s function “lyap” for Algorithm 2 and Algorithm 4. After 4 main iterations Algorithm 2 in SCILAB computes the stabilizing
solution with. After 18 iterations with Algorithm 4 in SCILAB we obtain the same
solution with. The solution is indefinite and it is not the stabilizing solution.
So, this example gives us the conclusion that the Algorithm 4 described in [8] compute only a solution to (1) and this solution is not always positive definite and this solution is not always stabilizing.
In this reason we confirm that the Lanzon’s method [1] is an effective method for computing the stabilizing solution. His main essential feature is that the iterative process includes two iterative loops-the out loop and the inner loop. We extend the ideas described by Lanzon et al. [1] and Feng and Anderson [6] to propose iterative methods where one matrix sequence is constructed. Here we introduce additional two iterative methods which lead directly to the stabilizing solution. Our contribution is to apply two computational schemes for realization the first iterative equation. Moreover, the second iterative equation is a new method for computing the stabi- lizing solution to (1). We present a few examples for testing the introduced recurrence equations on the MATLAB and SCILAB computational platforms.
We write or if is positive definite or is positive semidefinite for any two symmetric matrices X and Y. We use some properties of positive definite and positive semidefinite matrices. A matrix A is said to be asymptotically stable if all the eigenvalues of A lie in the open left half plane.
2. Iterative Methods for Stabilizing Solution to (1)
The first method is the Lanzon’s method [1] and Algorithm 2 from [8] . We present the main theorem with pro-
perties for constructing two matrix sequences of positive semidefinite matrices. The matrix
sequences are constructed as follows. We take
We find as the stabilizing solution of the algebraic Riccati equation with definite quadratic part:
where
The matrices are stabilizing solutions for the sequence of algebraic Riccati Equations (4). We will prove that the second sequence is monotonically non-decreasing and converges to the unique stabilizing solution to set of Equation (1). We reformulate the convergence theorem introduced in [1] (Theorem 3) and we present it as sufficient conditions to existence the stabilizing solution to (1).
Theorem 1 Assume there exist symmetric matrices and such that and and
, and the pair is a stabilizable one. Then for the matrix sequences
defined by (3), (4) are satisfied for
1) is stabilizable;
2);
3) the matrix is asymptotically stable for;
4).
Proof. The proof follows the proof of Theorem 3 from [1] .
Theorem 1 presents sufficient conditions for the equation has a solution. Such type conditions are introduced here for the considered equation for the first time. Theorem 1 confirms the convergence properties of iterative method (3), (4).
Further on, we consider an alternative iteration process where one matrix sequence is constructed. Consider the behaviour of the controller player (u(t)). Assume the controller player knows the matrix. He wants to
find. Then he takes. The system (2) becomes
And the functional is
The corresponding Riccati equation regarding to is
where
Based on recurrence Equation (5) we derive the following new iteration:
with. We perform iteration (6) using two recurrence approaches. The first one is to solve Equation (6) as a Riccati equation. We call this approach “(6) + care”. The second one is to solve Equation (6) applying the Lyapunov iteration:
with. The matrix sequence converges to. Iteration (7) defines the inner loop for iteration (6). We call the second approach “(6) + lyap”. In fact, that is an extension of the Algorithm 2 [8] .
The notation “(6) + care” means that the iteration (6) is solved as a Ricacti equation with unknown matrix. Each solution) of (6) is computed as a solution to Riccati Equation (6). The notation “(6) + lyap” stands for the fact that the solution to Equation (6) is computed as a limit of the sequence and each matrix is a solution to iteration (7). Iteration (7) describes the inner loop for finding the matrix sequence
defined by iteration (6) and it is a Lyapunov iteration for computing.
Thus, Equation (6) can be considered as a new iteration formula. This equation constructs a new matrix
sequence which converges to the stabilizing solution to (1). It is easy to see that the recurrence
Equation (6) is obtained from (4) when we substitute. This fact is observed by Praveen and Bhasin in Lemma 2 [2] . Thus is the stabilizing solution to (7) with, we conclude
the matrix pair is a stabilizable pair and is positive de-
finite. This is enough to start iterative process (7) with. Following Theorem 9.1.1 derived by Lancaster and Rodman [10] it is sufficient to claim iterative process (7) converges.
Further on, we extend the idea for constructing the matrix sequence. When we put
in (4) we obtain:
Next, we extricate the term and continue with some matrix
manipulations. We derive
We apply the following implementation for the latest recurrence equation:
Our thoughts and algebraic manipulations for deriving recurrence Equation (8) show that it is equivalent to the main iterative process (3)-(4). Thus iteration (9) constructs a new matrix sequence which converges to the stabilizing solution of (1). In order to execute iteration (9) we apply the following algorithm:
1) We take, and as a small positive number.
2) We compute as a solution to the equation.
3) For we carry out.
a) Compute and
b) Find as a solution to the Lyapunov equation.
c) Algorithm stops when the inequality holds.
4) The stabilizing solution is.
3. Numerical Experiments
We carry out experiments for solving a continuous-time algebraic Riccati equation with an indefinite quadratic
term (1). We construct two matrix sequences and for each example. The first matrix
sequence is computed using the iterative process (3)-(4). Iteration (4) is a Riccati equation and is its stabilizing solution. In addition, we apply two iterations (6) and (9) for computing the stabilizing solution to (1), where one matrix sequence is established. We perform iteration (6) in two ways “(6) + care” and “(6) + lyap”. We are solving Riccati recurrence Equations (4) and (6) with the MATLAB procedure care where the flops are
per one iteration. The MATLAB procedure lyab is applied for solving (7) and (9) and the flops are
per one iteration.
Moreover, we have carried out experiments in the open source software SCILAB
http://www.scilab.org/scilab/about. It provides a computing environment for scientific applications. There are functions for solving linear and nonlinear matrix equations. We apply the “ricc” function for solving a con- tinuous Riccati equation and “lyap” function for a linear Lyapunov equation.
Our experiments are executed in MATLAB on a 2.20 GHz Intel (R) Core (TM) i7-4702MQ CPU computer. We use two variables tolR and tol for small positive numbers to control the accuracy of computations. We
denote and. All iterations stop when the inequality
is satisfied for some. That is a practical stopping criterion. However, iteration “(6) + lyap” defines two loops- external and inner. The inner loop stops when the inequality is satisfied for some integer.
For our purpose we have executed hundred runs of each value of n for two family of examples. The tables report the maximal number It of iterations for which the inequality holds and the average number of iterations for all hundred runs of each size. In addition, the variable stands for the average number of iterations executed by (7) in order to obtain the stabilizing solution through (6) for all hund- red runs of each size. For instance, the iteration “(6) + lyap” executes main iterations and for Example 1. The column “CPU” presents the CPU time for execution the corresponding iterations. In our de- finitions the functions randn (p, k) and sprand (q, m, 0.3) return a p-by-k matrix of pseudorandom scalar values and a q-by-m sparse matrix respectively (for more information see the MATLAB description).
Example 2. We consider a family of examples in case, where the coefficient real matrices are given as follows: and C were constructed using the MATLAB notations:
Results from experiments in Example 2 are given in Table 1 with for all values of n.
Example 3. We consider a family of examples in case, where the coefficient real matrices are given as follows: and C were constructed using the MATLAB notations:
Results from experiments for Example 3 are given in Table 2 with for all values of n.
The application of all iterative methods shows that they achieve the same accuracy for different number of iterations. Our conclusions based on experiments are:
1) The execution the iterations (3), (4) and “(6) + care” takes almost the same CPU time (see the corresponding
Example 2. Results from 100 runs for each value of n
(3)-(4)
(6) + lyap (Alg 2)
(6) + care
(9)
n
It
avIt
CPU
It
avIt
CPU
It
avIt
CPU
It
avIt
CPU
The MATLAB Execution
10
3
2.6
0.5 s
4
6.9
0.32 s
4
3.15
0.54 s
5
4.1
0.16 s
11
3
2.7
0.5 s
4
7.2
0.37 s
4
3.26
0.57 s
6
4.22
0.25 s
12
3
2.8
0.56 s
4
7.24
0.39 s
4
3.28
0.62 s
6
4.31
0.22 s
13
3
2.8
0.57 s
4
7.46
0.45 s
4
3.43
0.64 s
7
4.56
0.26 s
14
3
2.9
0.65 s
4
7.7
0.40 s
4
3.62
0.65 s
8
4.86
0.25 s
15
3
2.9
0.76 s
4
7.9
0.49 s
4
3.69
0.75 s
9
5.02
0.37 s
The SCILAB Execution
10
3
2.6
0.4 s
4
6.9
0.36 s
4
3.10
0.45 s
5
4.0
0.29 s
11
3
2.7
0.5 s
4
7.1
0.40 s
4
3.22
0.58 s
5
4.1
0.31 s
12
3
2.7
0.56 s
4
7.2
0.45 s
4
3.27
0.64 s
6
4.2
0.35 s
13
3
2.8
0.72 s
4
7.5
0.52 s
4
3.38
0.80 s
7
4.4
0.4 s
14
3
2.9
0.77 s
4
7.7
0.55 s
4
3.64
0.94 s
7
4.7
0.45 s
15
3
2.9
0.92 s
4
8.0
0.64 s
4
3.76
1.10 s
8
5.0
0.52 s
Example 3. Results from 100 runs for each value of n
(3)-(4)
(6) + lyap (Alg 2)
(6) + care
(9)
n
It
avIt
CPU
It
avIt
CPU
It
avIt
CPU
It
avIt
CPU
The MATLAB Execution
15
4
3
0.42 s
11
8.3
0.5 s
4
4.0
0.57 s
12
6.6
0.28 s
16
4
3
0.45 s
11
8.3
0.4 s
5
4.0
0.57 s
14
6.7
0.28 s
17
4
3
0.54 s
10
8.5
0.39 s
5
4.0
0.57 s
11
7.0
0.32 s
18
4
3
0.57 s
11
8.8
0.57 s
5
4.0
0.60 s
12
7.5
0.32 s
19
4
3.1
0.61 s
17
8.8
0.56 s
6
4.1
0.68 s
21
7.7
0.38 s
20
4
3.1
0.64 s
12
9.3
0.54 s
5
4.1
0.65 s
26
8.3
0.42 s
The SCILAB Execution
15
4
3
0.68 s
9
8.1
0.44 s
5
4.0
0.82 s
10
6.2
0.37 s
16
4
3
0.72 s
12
8.5
0.47 s
5
4.0
0.87 s
10
6.5
0.38 s
17
4
3
0.96 s
11
8.5
0.54 s
4
4.0
1.12 s
12
6.7
0.44 s
18
4
3
1.03 s
11
8.7
0.58 s
4
4.0
1.20 s
11
6.9
0.49 s
19
4
3.1
1.21 s
11
8.7
0.63 s
5
4.1
1.40 s
15
7.3
0.57 s
20
4
3.1
1.29 s
12
9.0
0.67 s
7
4.2
1.53 s
25
8.0
0.70 s
columns of the tables). Note that the procedure care in these iterations have to be applied;
2) Iterations based on the solution of Lyapunov equations faster than the iterations based on the solution of Riccati equations;
3) The new iteration (9) is fastest than other iterative methods;
4) Comparing the MATLAB Execution and the SCILAB Execution we note the MATLAB implementations of the considered iterative methods are faster than the same executed in the SCILAB environment. However, the SCILAB implementations achieve the same accuracy and based on the fact it is an open source software we deduce the SCILAB is an useful tool for education to master and PhD students.
The conclusions are indicated by implemented numerical simulations.
4. Conclusion
We have studied two iterative processes for finding the stabilizing solution to generalized Riccati Equations (2). We have made numerical experiments for computing this solution and we have compared the considered methods numerically. We have compared the results from the experiments in regard of number of iterations and CPU time for executing. Our numerical experiments confirm the effectiveness of proposed new method (9). It is introduced here and moreover numerical experiments show its efficiency.
Acknowledgements
The present research paper was supported in a part by the EEA Scholarship Programme BG09 Project Grant D03-91 under the European Economic Area Financial Mechanism.
Cite this paper
Ivan G.Ivanov,Ivelin G.Ivanov,Nikolay C.Netov, (2015) On the Iterative Solution to H∞ Control Problems. Applied Mathematics,06,1263-1270. doi: 10.4236/am.2015.68119
ReferencesLanzon, A., Feng, Y., Anderson, B. and Rotkowitz, M. (2008) Computing the Positive Stabilizing Solution to Algebraic Riccati Equations with an Indefinite Quadratic Term via a Recursive Method. IEEE Transactions on Automatic Control, 53, 2280-2291. http://dx.doi.org/10.1109/TAC.2008.2006108Praveen, P. and Bhasin, S. (2013) Online Partially Model-Free Solution of Two-Player Zero Sum Differential Games, Preprints of the 10th IFAC International Symposium on Dynamics and Control of Process Systems. The International Federation of Automatic Control.Vrabie, D. and Lewis, F. (2011) Adaptive Dynamic Programming for Online Solution of a Zero-Sum Differential Game. Journal of Control Theory and Applications, 9, 353-360. http://dx.doi.org/10.1007/s11768-011-0166-4Dragan, V., Freiling, G., Morozan, T. and Stoica, A.-M. (2008) Iterative Algorithms for Stabilizing Solutions of Game Theoretic Riccati Equations of Stochastic Control. Proceedings of the 18th International Symposium on Mathematical Theory of Networks & Systems. http://scholar.lib.vt.edu/MTNS/Papers/078.pdfDragan, V. and Ivanov, I. (2011) Computation of the Stabilizing Solution of Game Theoretic Riccati Equation Arising in Stochastic H∞ Control Problems. Numerical Algorithms, 57, 357-375. http://dx.doi.org/10.1007/s11075-010-9432-7Feng, Y.T. and Anderson, B.D.O. (2010) An Iterative Algorithm to Solve State-Perturbed Stochastic Algebraic Riccati Equations in LQ Zero-Sum Games. Systems & Control Letters, 59, 50-56.
http://dx.doi.org/10.1016/j.sysconle.2009.11.006Zhu, H.-N., Zhang, C.-K. and Bin, N. (2012) Infinite Horizon LQ Zero-Sum Stochastic Differential Games with Markovian Jumps. Applied Mathematics, 3, 1321-1326. http://dx.doi.org/10.4236/am.2012.330188Wu, H.-N. and Luo, B. (2013) Simultaneous Policy Update Algorithms for Learning the Solution of Linear Continuous-Time H∞ State Feedback Control. Information Sciences, 222, 472-485.
http://dx.doi.org/10.1016/j.ins.2012.08.012Basar, T. and Bernhard, P. (1995) H∞ Optimal Control and Related Minimax Design Problems: A Dynamic Game Approach. Systems & Control: Foundations and Applications Series, Birkhauser, Boston, 1995.Lancaster, P. and Rodman, L. (1995) Algebraic Riccati Equations. Clarendon Press, Oxford.