This paper presents a design method of
H2 and
H∞-feedback control loop for nonlinear smooth gene networks that are in control affine form. Formulaic solution methodology for solving the nonlinear partial differential equations, namely the
Hamilton-Jacobi-Bellman and
Hamilton-Jacobi-Isaacs equations through successive Galerkin’s approximation is implemented and the results are compared. Throughout the implementation, there were several caveats that need to be further resolved for practical applications in general cases. Such issues and the clarification of causes are mathematically established and reviewed.
Gene Regulatory Network GMA System Galerkin’s Approximation Feedback Design of Biomolecular Systems Hamilton-Jacobi Equation Nonlinear Control1. Introduction
Over the recent decades, there had been increasing interest in control theoretic approach in synthetic biology [1]. Especially when dealing biomolecular circuits, external disturbances caused from molecular events are present which impose difficulties in observation of the state variables and design of the feedback controllers [2]. To attain certain amount of robustness and performance of the feedback control, quantitative inspection of the synthetic gene circuits was attempted. Regarding the traditional proportional-integral-differential (PID) control design, [3] demonstrated for gene circuits under stochastic disturbance. The biggest merit of PID control design is that the priori knowledge about the system is not necessary for the design of feedback control. In variational approaches, [4] used bang-bang control, which is typically obtained by solving the maximum principle. As there were animated researches held with the model predictive control (MPC) recently, [5] compared the three approaches.
Meanwhile at the other side of the academia, there had been breakthroughs on numerical solution methodology of nonlinear partial differential equations (PDEs), namely the Galerkin’s Approximation. The approximation is done by finding an appropriate linear combination of generating functions, or basis functions, to reduce the size of the residual of PDE to zero. Such method is meaningful since the typical optimal control problems are involved with the nonlinear PDEs, the Hamilton-Jacobi-Bellman (HJB) and Hamilton-Jacobi-Isaacs (HJI) equations. By solving these PDEs, the optimal control input required for minimization of the performance measure is determined. The PDE that needs to be solved depends on the performance measure. For typical choices so-called the H 2 and H ∞ feedback control, HJB and HJI need to be solved respectively [6]. But when the system is nonlinear, these PDEs are well-known to be extremely hard to solve. By exploiting Galerkin’s approximation, [7] had established a rigorous mathematical approach by using Galerkin’s approximation in iterative manner to solve HJB. Such methodology was named Successive Galerkin Approximation (SGA). Then it was proposed in [8] a similar approach to solve HJI and the point-wise convergence properties were also proved. Performance of such method was also verified. Moreover, it was confirmed in [9] about the superior performance of the controller designed through SGA over controller designed by linearization of the nonlinear dynamics. Although such methodology also requires precise modeling of the system we aim to control, thanks to chemists and biologists, a large portion of biochemical systems have reliable state dynamics and stochastic modeling of the intrinsic disturbances [10]. Unlike the PID control, H ∞ -feedback control provides quantitative amount of margin of the control system that can withstand against the external disturbance which makes it possible to maximize the disturbance rejection performance. Hence such optimal control strategy is preferred over traditional control strategies if an accurate modeling of the system is available.
To this end, we present an implementation of H 2 and H ∞ feedback control design methodology with SGA. Along the simulation, several limitations of the SGA algorithms are indicated and mathematical reasonings are provided.
2. Feedback Control Design Problem
Let us consider the smooth nonlinear control system represented in the below form.
x ˙ ( t ) = f ( x ) + g ( x ) u ( x ) + k ( x ) w , x ( 0 ) = x 0 (1)
where x ∈ ℝ n , u ∈ ℝ m , w ∈ ℝ q for each t, and f : ℝ n ↦ ℝ n , g : ℝ n ↦ ℝ n × m , k : ℝ n ↦ ℝ n × q . u ( x ) denotes the state feedback. The system is observable through output y ( t ) = h ( x ( t ) ) where h : ℝ n ↦ ℝ p is a smooth function with h ( 0 ) = 0 . When the unknown external disturbance v is entering the system, it is desired to find an appropriate state feedback u ( x ) that achieves robust stability and performance. Throughout this paper, we impose the below assumption about the system. With this assumption, we introduce the two design purposes, namely H 2 and H ∞ -feedback throughout this section.
Assumption 1 f ( 0 ) = 0 and h ( 0 ) = 0 .
2.1. H 2 -Feedback Control Design
For H 2 -feedback control design problem, the performance measure is set as J = ∫ 0 ∞ l ( x ) + ‖ u ( x ) ‖ R 2 d t where l = x T Q x , ‖ u ‖ R 2 = u T R u for some Q , R ≻ 0 . It was shown from the previous studies including [11] that for the solution V ( x ) of the Hamilton-Jacobi-Bellman Equation (HJB) below, feedback control u ( x ) = − 1 2 R − 1 g ( x ) T ∂ V ∂ x attains minimum of the above performance measure.
∂ V ∂ x T f ( x ) + l ( x ) − 1 4 ∂ V ∂ x T g ( x ) R − 1 g ( x ) T ∂ V ∂ x = 0, V ( 0 ) = 0 (2)
Although the PDE (2) can be easily solved by casting into form of algebraic Riccati equation in linear cases, gene networks typically include high nonlinearities and hence PDE (2) is extremely hard to solve in general.
2.2. H ∞ -Feedback Control Design
Unlike the H 2 -feedback control, the objective of H ∞ -feedback control is to achieve asymptotic stability and disturbance-to-output L 2 -gain under certain level γ > 0 . Such condition can be expressed as below.
∫ 0 T ‖ y ( t ) ‖ 2 + ‖ u ( t ) ‖ R 2 d t ≤ γ 2 ∫ 0 T ‖ w ( t ) ‖ P 2 d t , ∀ T ≥ 0 (3)
where w ∈ L 2 ( 0, T ) and ‖ w ‖ P 2 = w T P w for some P ≻ 0 . It was proved in [6] that for solution V ( x ) of the Hamilton-Jacobi-Isaacs Equation (HJI) below, feedback control u ( x ) = − 1 2 R − 1 g ( x ) T ∂ V ∂ x achieves the condition of Equation (3). Moreover, it was also proven that the worst case disturbance if given by w ( t ) = 1 2 γ 2 P − 1 k T ∂ V ∂ x . The inequality of Equation (3) is tight for such choice of u ( x ) and w ( t ) and when V ( x 0 ) = 0 .
∂ V ∂ x T f ( x ) + h ( x ) T h ( x ) − 1 4 ∂ V ∂ x T ( 1 γ 2 k ( x ) P − 1 k ( x ) T − g ( x ) R − 1 g ( x ) T ) ∂ V ∂ x = 0, V ( 0 ) = 0 (4)
Like the HJB, HJI is not tractable in general cases.
3. Successive Galerkin Approximation
A typical strategy to solve PDEs in numerical manner is Galerkin method [12]. Galerkin method approximates the solution with linear combination of finite number of trial functions.
[8] applied Galerkin method in sequential manner to obtain convergent solutions of HJB and HJI. Point-wise convergence of such method was also proved. Such approach was named Successive Galerkin Approximation (SGA). Although convergence and consistency under finite number of trial functions was not provided, with reasonable number and choice of trial (or basis) functions, it was verified that the feedback design through SGA performs decently. By taking the notations from [8], we summarize the SGA algorithm for HJB and HJI respectively in this section.
Let us begin by introducing the notations. For the basis functions { ϕ 1 , ϕ 2 , ⋯ , ϕ N } , let Φ = [ ϕ 1 ϕ 2 ⋯ ϕ N ] T . Now when the solution is approximated by V ( x ) ≈ c 1 ϕ 1 + ⋯ + c N ϕ N , let c = [ c 1 c 2 ⋯ c N ] T . We further denote the integration region of Galerkin method by Ω . Now we begin by introducing the iteration matrices for solution of the HJB.
A 1 = ∫ Ω Φ ( x ) f ( x ) T ⋅ ∇ Φ ( x ) T d x A 2 ( u 0 ) = ∫ Ω Φ ( x ) u 0 ( x ) T g ( x ) T ⋅ ∇ Φ ( x ) T d x b 1 = − ∫ Ω l ( x ) d x b 2 ( u 0 ) = − ∫ Ω ‖ u 0 ( x ) ‖ R 2 ⋅ Φ ( x ) d x G i = ∫ Ω Φ ∂ ϕ i ∂ x T g ( x ) R − 1 g ( x ) T ⋅ ∇ Φ ( x ) d x , i = 1, ⋯ , N (5)
The derivation of the above formulas (5) can be found at [8].
In Algorithm 1, c ( i − 1 ) is the result from the ( i − 1 ) -th iteration and ε is the tolerance for termination criterion.
For solution of HJI, construction of the iteration matrices is as below.
A 1 = ∫ Ω Φ ( x ) f ( x ) T ⋅ ∇ Φ ( x ) T d x A 2 ( u 0 ) = ∫ Ω Φ ( x ) u 0 ( x ) T g ( x ) T ⋅ ∇ Φ ( x ) T d x b 1 = − ∫ Ω h ( x ) T h ( x ) ⋅ Φ d x b 2 ( u 0 ) = − ∫ Ω ‖ u 0 ( x ) ‖ R 2 ⋅ Φ ( x ) d x G i = ∫ Ω Φ ∂ ϕ i ∂ x T g ( x ) R − 1 g ( x ) T ⋅ ∇ Φ ( x ) d x , i = 1, ⋯ , N K i = ∫ Ω Φ ∂ ϕ i ∂ x T k ( x ) P − 1 k ( x ) T ⋅ ∇ Φ ( x ) d x , i = 1, ⋯ , N (6)
In Algorithm 2, c ( i , j ) is the result from the ( i , j ) -th iteration, or the i-th inner loop and j-th outer loop iteration. ε is the tolerance for termination criterion. It was proved in the previous researches the convergence of both Algorithm 3 and Algorithm 3 for complete set of basis functions of continuous L 2 space. But under finite number of basis functions, convergence of the solution procedure and convergence of the residual of the nonlinear PDE to zero was not proved. Due to finite computation resources, this is the case in practical applications. Our consequent results in this paper will review the results obtained from the SGA algorithms and point out such issues especially for the application in HJI case.
4. Problem Description
For illustrative purpose, we consider the gene regulatory network in cascaded form brought from [10]. The system dynamics f ( x ) is represented by the below first order ODE. Though we specify the system dynamics as below, large amount of gene network systems are represented in the similar form and hence such an approach can be employed at various situations [10]. For the H ∞ -feedback control design of this system, linearization and application of fuzzy interpolation was attempted in [2].
x ˙ 1 = 4 x 1 0.5 − 4 x 1 0.5 x 2 − 1 + 1.5 w x ˙ 2 = 4 x 1 0.5 x 2 − 1 − 2 x 2 0.5 − 2 x 2 x ˙ 3 = 2 x 2 0.5 − 2 x 3 0.5 x ˙ 4 = 2 x 2 − 2 x 4 0.5 (7)
where w ( t ) is the external disturbance and the initial conditions are given by x 1 ( 0 ) = 2.5 , x 2 ( 0 ) = 2 , x 3 ( 0 ) = 2 , x 4 ( 0 ) = 2 . Such representation is also called the generalized-mass-action(GMA) form. It is evident that one equilibrium point is x e = [ 1 1 1 1 ] T . We sketch a formulaic design method of H 2 and H ∞ -feedback controls for this system. Rewriting the Equation (2), it follows
∂ V ∂ x T ( f ( x ) + g ( x ) u ( x ) ) + l ( x ) + ‖ u ‖ R 2 = d d t V ( x ( t ) ) + x T Q x + ‖ u ‖ R 2 = 0 (8)
Because boundness of V ( x ) has to be guaranteed for Galerkin’s approximation to be used, l i m t → ∞ d d t V ( x ( t ) ) = l i m t → ∞ − x T Q x − ‖ u ‖ R 2 = 0 . Hence it is evident that it is required to have l i m t → ∞ x ( t ) = l i m t → ∞ u ( t ) = 0 for application of the SGA algorithm. Due to such reason, we consider the perturbed system as below.
x ˜ ˙ 1 = 4 ( x ˜ 1 + 1 ) 0.5 − 4 ( x ˜ 1 + 1 ) 0.5 ( x ˜ 2 + 1 ) − 1 + 1.5 w x ˜ ˙ 2 = 4 ( x ˜ 1 + 1 ) 0.5 ( x ˜ 2 + 1 ) − 1 − 2 ( x ˜ 2 + 1 ) 0.5 − 2 ( x ˜ 2 + 1 ) x ˜ ˙ 3 = 2 ( x ˜ 2 + 1 ) 0.5 − 2 ( x ˜ 3 + 1 ) 0.5 x ˜ ˙ 4 = 2 ( x ˜ 2 + 1 ) − ( 2 x ˜ 4 + 1 ) 0.5 (9)
By such perturbation, the system now has x ˜ e = [ 0 0 0 0 ] T as an equilibrium point.
5. Numerical Demonstration & Discussion
In this section, we provide the numerical demonstration results of the previously described nonlinear gene network system. For both H 2 and H ∞ -feedback designs, total six basis functions were used and are given by { ϕ 1 , ϕ 2 , ϕ 3 , ϕ 4 , ϕ 5 , ϕ 6 } = { x ˜ 1 2 , x ˜ 2 2 , x ˜ 3 2 , x ˜ 1 x ˜ 2 , x ˜ 2 x ˜ 3 , x ˜ 1 x ˜ 3 } . Such choice was to obtain linear approximation of the true solution u * ( x ) = − 1 2 R − 1 g T ∂ V * ∂ x of HJB or HJI and satisfy l i m t → ∞ u ( x ( t ) ) = 0 simultaneously. The integration region was set as ( x ˜ 1 , x ˜ 2 , x ˜ 3 , x ˜ 4 ) ∈ [ − 0.3,3 ] × [ − 0.3,2 ] × [ − 0.3,2 ] × [ − 0.3,2 ] . For the feedback input, g ( x ) = [ I 2 O 2 ] T was considered where I 2 and O 2 are the 2 × 2 identity and zero matrices respectively. Numerical simulation was done by Simulink software on Matlab 2019b with Macbook Pro, 2.4GHz Intel Core i5.
5.1. H 2 -Feedback Design
For the design parameters, Q = d i a g ( [ 10 1 1 1 ] ) and R = I 2 was chosen. With the given basis, it took 7 iterations to converge within the tolerance ε = 10 − 3 . Converged solution is shown in Table 1. The state and control variables from the designed H 2 controller are depicted in Figure 1. Simulation results with external disturbance entering through the state dynamics of x ˜ 1 are depicted in Figure 2 together with the random disturbance. The designed control system withstands such disturbance decently.
Obtained solution of the HJB
Basis Functions
Coefficients c i
ϕ 1
3.7839
ϕ 2
0.1924
ϕ 3
1.3827
ϕ 4
2.2592
ϕ 5
0.4725
ϕ 6
−0.2893
5.2. H ∞ -Feedback Design
For the H ∞ -feedback control design, as from [2], it is desirable to find the smallest γ that the solution of HJI exists. Hence for the design parameter, γ = 3.5 was used for the initial choice and we designed the feedback control throughout the solution. It took total 8 iterations for the outer loop to converge while inner loops typically took 4 iterations to converge within the same tolerance from previous subsection. Converged solution is shown in Table 2. Though the H 2 feedback controller had shown slightly rapid convergence to the desired equilibrium point, both controllers show similar performances. Comparing with the results from [2] obtained from fuzzy interpolation after linearization, Figure 3 and Figure 4 show more rapid convergence. Such superiority of the feedback design without linearization was also confirmed at [9]. Then by trial and error, it was confirmed that the SGA algorithm also converges for smaller values of γ ≥ 0.4 . But for the solution obtained for γ less than 2, the SGA algorithm did converge but the designed feedback controller exhibited unstable properties. Converged solution is shown in Table 3. Simulation results for γ = 1.0 is depicted in Figure 5 and Figure 6 which shows unstable response regardless the presence of disturbance. We speculate that this is due to non-zero residual of the HJI (4). Because the choice of our basis was obviously not a complete set that can span the whole continuous L 2 space, although the projection of the residual onto the subspace generated by our basis functions may be zero, it does not imply that the residual itself is zero. Regarding this, we would like to point out that the solution obtained from the SGA algorithm is indeed the utmost solution with the given basis, but it does not necessarily imply that it is the exact solution of HJI. This could be resolved through adding the number of basis functions or choosing appropriate basis functions depending on the system dynamics through heuristic. Though it was proposed in [8] a clever methodology exploiting tensor products to mitigate the computational burden of massive number of numerical integrations, in practice we are only allowed to use finite number of basis functions due to finite computing power. Also it should be pointed out that in a lot of cases, full-state observability of the system is not present and hence there are limitations in choice of basis functions too.
Obtained solution of the HJI with γ = 3.5
Basis Functions
Coefficients c i
ϕ 1
1.9798
ϕ 2
0.2858
ϕ 3
1.787
ϕ 4
1.7534
ϕ 5
0.2893
ϕ 6
−0.2927
Obtained solution of the HJI with γ = 1.0
Basis Functions
Coefficients c i
ϕ 1
0.8388
ϕ 2
0.6259
ϕ 3
23.7091
ϕ 4
−1.9829
ϕ 5
3.9033
ϕ 6
−10.6959
6. Conclusions & Future Scope
In this paper, nonlinear H 2 and H ∞ -feedback controls have been developed for nonlinear gene regulatory network in GMA form. With the choice of basis functions from Section 5, Successive Galerkin Approximation did converge rapidly for both the Hamilton-Jacobi-Bellman and Hamilton-Jacobi-Isaacs equations. Because the majority of gene network systems are of the form of Equation (7), such feedback design methodology from this paper could be applied in a wide range of systems in GMA form. Comparison between the designed H 2 and H ∞ -feedback control systems was made and for the hyper parameters used, it was confirmed that the two controllers exhibited similar performances.
It was also confirmed from our simulation that even when the SGA algorithm converged, there were cases where the corresponding feedback control system was unstable. This is due to nonzero residual of the HJI, and it needs to be further resolved by improvements in the algorithm for solving HJB and HJI. This issue could indeed be overcome by different choice of basis functions, but still there is no guarantee that the solution obtained with new basis functions will not exhibit such instability. Additionally in practice, there are limitations in observability of the state variables, which again impose restriction on the choice of the basis functions. Hence it requires further research to analyze the feedback control systems’ properties under finite number of basis functions to overcome these issues. Our consequent work will include breakthroughs in such issues discussed and consideration of stochastic disturbance entering through the system dynamics due to molecular noises.
Conflicts of Interest
The author declares no conflicts of interest regarding the publication of this paper.
Cite this paper
Bae, A.W. (2022) -Feedback Control Design for Nonlinear Gene Networks via Successive Galerkin’s Approximation. Computational Molecular Bioscience, 12, 95-108. https://doi.org/10.4236/cmb.2022.122006
ReferencesZhang, W.H. and Chen, B. (2006) State Feedback H<sub>∞</sub> Control for a Class of Nonlinear Stochastic Systems. SIAM Journal on Control and Optimization, 44, 1973-1991.Shannon, B., Zamora-Chimal, C.G., Postiglione, L., Salzano, D., Grierson, C.S., Marucci, L., Savery, N.J. and di Bernardo, M. (2020) In Vivo Feedback Control of an Antithetic Molecular-Titration Motif in Escherichia coli Using Microfluidics. ACS Synthetic Biology, 9, 2617-2624. https://doi.org/10.1021/acssynbio.0c00105van der Schaft, A.J. (1992) L<sub>2</sub>-gain Analysis of Nonlinear Systems and Nonlinear State-Feedback H<sub>∞</sub> Infinity Control. IEEE Transactions on Automatic Control, 37, 770-784. https://doi.org/110.1109/9.256331Saridis, G.N. and Lee, C.-S.G. (1979) An Approximation Theory of Optimal Control for Trainable Manipulators. IEEE Transactions on Systems, Man, and Cybernetics, 9, 1522-159. https://doi.org/10.1109/TSMC.1979.4310171Bea, R.W. (1998) Successive Galerkin Approximation Algorithms for Nonlinear Optimal and Robust Control. International Journal of Control, 71, 717-743.
https://doi.org/10.1080/002071798221542Volt, E.O. (2000) Computational Analysis of Biochemical Systems: A Practical Guide for Biochemists and Molecular Biologists. Cambridge Univ. Press, Cambridge.Modi, S., Dey, S. and Singh, A. (2021) Noise Suppression in Stochastic Genetic Circuits Using PID Controllers. PLOS Computational Biology, 17, e1009249.
https://doi.org/10.1371/journal.pcbi.1009249Fletcher, C.A.J. (1984) Computational Galerkin Methods. Springer, Berlin.Fiore, G., Perrino, G., di Bernardo, M. and di Bernardo, D. (2016) In Vivo Real-Time Control of Gene Expression: A Comparative Analysis of Feedback Control Strategies in Yeast. ACS Synthetic Biology, 5, 154-162.
https://doi.org/10.1021/acssynbio.5b00135Oudjama, F., Boumédiène, A., Messirdi, M. and Boubekeur, D. (2021) Comparison of Linear and Nonlinear H-Infinity Controllers for an Electric Vehicle Driven by the Permanent Magnet Synchronous Motor. International Journal on Emerging Technologies, 12, 247-257.Del Vecchio, D., Dy, A.J. and Qian, Y.L. (2016) Control Theory Meets Synthetic Biology. Journal of the Royal Society Interface, 13, Issue 120.
https://doi.org/10.1098/rsif.2016.0380Chen, B. and Chang, Y.-T. and Wang, Y.-C. (2008) Robust H<sub>∞</sub>-Stabilization Design in Gene Networks under Stochastic Molecular Noises: Fuzzy-Interpolation Approach. Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on, 38, 25-42.