Over the recent decades, along with the development of control engineering and computational methods, there had been numerous attempts to adopt a control theoretic approach to problems of synthetic biology [1] . In particular, designing feedback controllers of biomolecular circuits is a typical problem in the synthetic biology field that requires control theory engaged into. The essential goal of these attempts is to attain quantitative inspection of the system response and exploit them to find an appropriate feedback control of the overall system.

In particular, the Heparin-controlled blood clotting network is a typical objective system of feedback control. Heparin, or unfractionated heparin (UFH), is a widely used anticoagulant, discovered by Jay McLean and William Henry Howell in 1916. Heparin is one of the World Health Organization’s Lists of Essential Medicines since it is an essential medication for various cardiac surgeries such as implant of ECMO circuit or Coronary Artery Bypass Graft.

Heparin binds to anti-Thrombin and brings conformational change to enhance its biological activity. Enhanced anti-Thrombin is able to more effectively reduce the action of Thrombin, thereby shutting down the blood clotting mechanism. Heparin by itself cannot break down the clot but it can induce the breakdown by supporting the body’s natural clot lysis.

Four major proteins (Prothrombin, Thrombin, Fibrinogen, and Fibrin) are related in a cascaded positive feedback pathway in blood clotting. Intrinsic or extrinsic pathway activates Factor Xa. Activated Factor Xa catalyzes the conversion of Prothrombin to Thrombin. Then Thrombin catalyzes the conversion of Fibrinogen to Fibrin, which then combines to form long fibrin threads that entangle platelets, building up a spongy mass that gradually hardens and contracts to become the blood clot.

The effect of Heparin in the gene network of blood clotting can be expressed in a Generalized-Mass-Action (GMA) form with stochastic noises [2] . Regarding the feedback control of gene regulatory networks, there had been numerous researches attempting to exploit the advantages of each control method.

As the well-known Proportional-Integral-Differential (PID) control is widely adopted in many control situations in electrical, mechanical, aerospace, and many other fields, it was demonstrated in [3] feedback control design of gene circuits under stochastic noise. The biggest benefit of PID control is that it can be implemented without priori knowledge regarding the system dynamics. In the situations where system dynamics are available, [4] employed a variational approach, the Bang-Bang Control, which can be obtained by solving the Pontryagin maximum principle. Along with the recent concerns on Model-Predictive-Control (MPC) in autonomous driving problems, [5] implemented the aforementioned three methods and compared them. While numerous previous researches concentrated on the design methodology of feedback control, it is desired to secure the robustness of the designed controller under various disturbance situations.

Including the aforementioned Heparin networks, typical biomolecular circuits suffer from the presence of external noise entering through the system dynamics and molecular noises that are modeled stochastically [6] . Such an issue makes it difficult to have a clear inspection of state variables and generate the feedback signal. To overcome such an issue, a control strategy to minimize the noise-to-output gain and attain a certain amount of robustness in stability, so-called the H ∞ controller was used in aerospace control field [7] . The biggest merit of H ∞ controller is that it provides a quantitative amount of margin in both the performance and stability against external disturbance. For H ∞ controller design under finite energy disturbance, it is required to solve the nonlinear partial differential equation (PDE), the first-kind Hamilton-Jacobi-Isaacs (HJI) equation. When there is a stochastic noise that could have unbounded energy, a solution of the second-kind HJI equation is required. As the analytic solutions of nonlinear PDEs are not available in general cases, without any approximations, there was no general methodology to solve H ∞ control design problems until recent breakthroughs.

In the studies of PDE theories, there had been recent breakthroughs in the numerical solution method of two nonlinear PDEs, the aforementioned first-kind HJI equation and the Hamilton-Jacobi-Bellman equation, namely Galerkin’s Method. Basic idea is to express in linear combination of complete set of basis functions the solution and find the appropriate coefficients to make residual of the PDE zero. [8] presented a solution procedure to obtain pointwise convergent solution by applying Galerkin’s method iteratively. Convergence was also proved alongside. Such a method is attractive since it even confirmed the superior performance of the controller over the one designed with linearization. Although it requires precise knowledge of the system dynamics, a vast majority of biomolecular circuits that are aimed to be controlled have reliable modeling of system dynamics. But to retain robustness under stochastic noise, which every biomolecular circuit may suffer due to molecular events, solution procedure for second-kind HJI is required.

To this end, we propose a solution methodology of second-kind HJI through SGA. The validity of such a method is checked on Simulink software on MATLAB through Monte-Carlo simulation. With such methodology, we provide accurate protocols of Heparin use in cardiac surgery procedures. From hereon, let us denote the second-kind HJI equation by HJI equation for simplicity.

Let us consider a nonlinear control system as below.

x ˙ ( t ) = f ( x ) + g ( x ) u ( x ) + k ( x ) v ( t ) + h ( x ) W ˙ ,   x ( 0 ) = x 0 (1)

where x ∈ ℝ n , u ∈ ℝ m , v ∈ ℝ q , W ∈ ℝ for each t, and f : ℝ n ↦ ℝ n , g : ℝ n ↦ ℝ n × m , k : ℝ n ↦ ℝ n × q , h : ℝ n ↦ ℝ which are Lipschitz. u ( x ) denotes the state feedback, v ( t ) denotes exogenous disturbance, and W is the 1-dimensional standard Wiener process. The system is observable through output y ( t ) = C x ( t ) where C ∈ ℝ n × p . Under existence of the unknown external disturbance v and stochastic noise W ˙ entering the system dynamics, it is desired to find design a state feedback u ( x ) that can maintain robust stability and performance. Throughout this paper, we impose the below assumption about the system. With this assumption, we introduce the H ∞ -feedback design problem throughout this section.

Assumption 1. f ( 0 ) = 0 and h ( 0 ) = 0 .

If the system dynamics does not satisfy the above assumption, we can always perturb the system by equilibrium point as x ˜ = x − x e and enforce it.

H ∞ -Feedback Design Problem

The main objective of the H ∞ control design is to achieve asymptotic stability and disturbance-to-output L 2 -gain under certain level γ > 0 . Due to presence of stochastic noise,

E [ ∫ 0 ∞ ‖ y ( t ) ‖ 2 + ‖ u ( x ( t ) ) ‖ R 2 d t ] ≤ γ 2 E [ ∫ 0 ∞ ‖ v ( t ) ‖ P 2 d t ] (2)

where v ∈ L 2 ( ℝ + ) , ‖ v ( t ) ‖ P 2 = v ( t ) T P − 1 v ( t ) , and ‖ u ( x ( t ) ) ‖ R 2 = u ( x ( t ) ) T R − 1 u ( x ( t ) ) for P , R ≻ 0 . Then from the studies of stochastic noise systems [9] , the below theorem was proved.

Theorem 1. For the Hamilton-Jacobi-Isaacs equation below,

∂ V ∂ x T f ( x ) + x T C T C x + 1 4 ∂ V ∂ x T ( 1 γ 2 k ( x ) P − 1 k ( x ) T     − g ( x ) R − 1 g ( x ) T ) ∂ V ∂ x + 1 2 h ( x ) T ∂ 2 V ∂ x 2 h ( x ) = 0,   V ( 0 ) = 0 (3)

if there exists a smooth solution V ( x ) ≥ 0 with boundary condition V ( 0 ) = 0 , then the state feedback

u ( x ) = − 1 2 R − 1 g ( x ) T ∂ V ∂ x (4)

satisfies Equation (2) and the worst case disturbance is given by

v ( t ) = 1 2 γ 2 P − 1 k ( x ) T ∂ V ∂ x (5)

The proof of the theorem above without considering the presence of stochastic noise term was proposed and proved in [6] , while it was considered here the general case with stochastic noise entering through the system dynamics.

Remark 1. By substituting Equations (4) and (5) to Equation (3), HJI can be modified as

∂ V ∂ x T ( f ( x ) + g ( x ) u ( x ) + k ( x ) v ) + x T C T C x + ‖ u ( x ) ‖ R 2 − γ 2 ‖ v ‖ P 2 + 1 2 h ( x ) T ∂ 2 V ∂ x 2 h ( x ) = 0,   V ( 0 ) = 0 (6)

where dependency on time is dropped for simplicity.

It is noteworthy that without any linearization, HJI above is not tractable in general cases.

In this section, we depict an iterative solution procedure of Equation (3). Up to our knowledge, it was first proposed in [8] to solve HJI in iterative manner. But due to presence of stochastic terms, it is desired to modify the iterative algorithm. Before stating the algorithm, let us begin by defining the below notation to represent the partial differential equation.

Definition 1. A ( u , v , V ) ≡ ∂ V ∂ x T ( f ( x ) + g ( x ) u ( x ) + k ( x ) v ) + x T C T C x   + ‖ u ( x ) ‖ R 2 − γ 2 ‖ v ‖ P 2 + 1 2 h ( x ) T ∂ 2 V ∂ x 2 h ( x ) .

For u and v in Equations (4) and (5), the PDE A ( u , v , V ) = 0 with boundary condition V ( 0 ) = 0 is equivalent to HJI (3) (Algorithm 1).

Algorithm 1. Iterative solution procedure for HJI.

Galerkin’s method is employed in various situations where numerical solutions of PDEs are needed [10] . With a finite number of basis functions, the solution is approximated by a linear combination in a least square manner. It was proposed in [8] to apply Galerkin’s method sequentially to obtain pointwise convergent solutions of HJI, namely Successive Galerkin’s Approximation (SGA). Convergence of such method under complete set of basis was also confirmed. From the conclusions from the previous section, by adopting the notations from [8] we further expand the idea of SGA to solve HJI under presence of stochastic noise in this section.

To begin by introducing the notations, given finite set of basis functions { ϕ 1 , ϕ 2 , ⋯ , ϕ N } , let Φ = [ ϕ 1 ϕ 2 ⋯ ϕ N ] T . Now if the solution is approximated by V ( x ) ≈ c 1 ϕ 1 + ⋯ + c N ϕ N , let c = [ c 1 c 2 ⋯ c N ] T . Let us further denote the integration region of Galerkin method by Ω . The below Equation (7) are the iteration matrices for implementation of SGA where the detailed derivation procedure can be found at [8] .

A 1 = ∫ Ω     Φ ( x ) f ( x ) T ⋅ ∇ Φ ( x ) T d x A 2 ( u 0 ) = ∫ Ω     Φ ( x ) u 0 ( x ) T g ( x ) T ⋅ ∇ Φ ( x ) T d x b 1 = − ∫ Ω     x T C T C x ⋅ Φ ( x ) d x b 2 ( u 0 ) = − ∫ Ω ‖ u 0 ( x ) ‖ R 2 ⋅ Φ ( x ) d x

G i = ∫ Ω     Φ ( x ) ∂ ϕ i ∂ x T g ( x ) R − 1 g ( x ) T ⋅ ∇ Φ ( x ) d x ,   i = 1, ⋯ , N K i = ∫ Ω     Φ ( x ) ∂ ϕ i ∂ x T k ( x ) P − 1 k ( x ) T ⋅ ∇ Φ ( x ) d x ,   i = 1, ⋯ , N H = 1 2 ∫ Ω     Φ ( x ) h ( x ) T ∇ 2 Φ ( x ) h ( x ) d x (7)

In Algorithm 2, the result from the ( i , j ) -th iteration, or the i-th inner loop and j-th outer loop iteration is denoted by c ( i , j ) . Criterion for termination was set to be ε . For complete set of functions of L 2 space, it was proved in previous works the convergence of Algorithm 2.

Algorithm 2. Successive galerkin approximation.

Performance of Algorithm 2 is confirmed in Heparin-controlled blood clotting network. Although this paper aims for design in Heparin network, vast majority of gene networks share analogous formulations in GMA form. The system dynamics is given as below.

x ˙ 1 = 4 x 1 0.5 − 4 x 1 0.5 x 2 − 1 + 1.5 w x ˙ 2 = 4 x 1 0.5 x 2 − 1 − 2 x 2 0.5 − 2 x 2 x ˙ 3 = 2 x 2 0.5 − 2 x 3 0.5 x ˙ 4 = 2 x 2 − 2 x 4 0.5 (8)

where w ( t ) is the external disturbance and stochastic noise entering the dynamics. The initial conditions are set as x 1 ( 0 ) = 2.5 , x 2 ( 0 ) = 2 , x 3 ( 0 ) = 2 , x 4 ( 0 ) = 2 . Above representation of the gene regulatory network is also called the generalized-mass-action (GMA) form. For the design of H ∞ -feedback control, fuzzy interpolation approach together with linearization of nonlinear dynamics was proposed in [6] . Now to guarantee the boundedness of V ( x ) , l i m t → ∞ x ( t ) = l i m t → ∞ u ( t ) = 0 [8] . Hence the perturbed system as below is considered.

x ˜ ˙ 1 = 4 ( x ˜ 1 + 1 ) 0.5 − 4 ( x ˜ 1 + 1 ) 0.5 ( x ˜ 2 + 1 ) − 1 + 1.5 w x ˜ ˙ 2 = 4 ( x ˜ 1 + 1 ) 0.5 ( x ˜ 2 + 1 ) − 1 − 2 ( x ˜ 2 + 1 ) 0.5 − 2 ( x ˜ 2 + 1 ) x ˜ ˙ 3 = 2 ( x ˜ 2 + 1 ) 0.5 − 2 ( x ˜ 3 + 1 ) 0.5 x ˜ ˙ 4 = 2 ( x ˜ 2 + 1 ) − 2 ( x ˜ 4 + 1 ) 0.5 (9)

By such perturbation, x ˜ e = [ 0 0 0 0 ] T is an equilibrium point of the system.

Throughout this section, we provide a numerical demonstration result of Algorithm 2 applied to the previously described nonlinear gene network system. Total six basis functions { ϕ 1 , ϕ 2 , ϕ 3 , ϕ 4 , ϕ 5 , ϕ 6 } = { x ˜ 1 2 , x ˜ 2 2 , x ˜ 3 2 , x ˜ 1 x ˜ 2 , x ˜ 2 x ˜ 3 , x ˜ 1 x ˜ 3 } were used. Choice of such basis functions was intended to obtain a linear approximation of V ( x ) and to satisfy ϕ ( 0 ) = 0, i = 1, ⋯ ,6 . For the integration region, { ( x ˜ 1 , x ˜ 2 , x ˜ 3 , x ˜ 4 ) ∈ [ − 0.3,3 ] × [ − 0.3,2 ] × [ − 0.3,2 ] × [ − 0.3,2 ] } was used. For the

feedback input function, g ( x ) = [ 1 0 0 0 0 1 0 0 ] T x was used. Numerical simulations were implemented on Simulink and Matlab 2019b with Apple Macbook Pro, 2.4 GHz Intel Core i5.

When it comes to design of H ∞ -feedback control, find the smallest possible value of γ such that the smooth solution of HJI (3) is required. By trial and error, we chose γ = 3 and implemented the proposed SGA algorithm. While it took 9 iterations for the outer loop to converge, it took minimum 4 to maximum 7 iterations for the inner loops to converge. Tolerance for convergence criterion was set by ε = 10 − 3 for both inner and outer loops. The converged solution is shown in Table 1. Compared with the results obtained by fuzzy interpolation and linearization from [6] , results in Figure 1 and Figure 2 exhibit more rapid convergence characteristics. Such superiority of performance is due to the design procedure without linearization and such improvement was also confirmed in

[11] but without the existence of stochastic noise. To test the robustness of the designed controller, or the gene regulatory circuit, we implemented Monte-Carlo simulation with random seed fed into band-limited white noise as external disturbance. Total of 100 simulations were done and the robustness of the controller was confirmed. Although the presence of a jittering attitude due to noise, is shown in Figure 3 and Figure 4 the robust performance and stability.

Throughout this paper, H ∞ -feedback control design problem for nonlinear dynamics under the presence of stochastic noise was developed. To solve the well-known Hamilton-Jacobi-Isaacs equation, Successive Galerkin Approximation was employed. The robustness of the designed feedback control was confirmed through the Monte-Carlo simulation. Aiming to provide accurate protocols of Heparin use in cardiac surgery procedures, the proposed algorithm was validated in a Heparin-controlled blood clotting network. However since the vast majority of gene network systems are expressed in GMA form, we further expect the methodology presented in this paper to be applied in a wider range of nonlinear systems.

The author declares no conflicts of interest regarding the publication of this paper.

Bae, A.W. (2022) -Feedback Control of Heparin-Controlled Blood Clotting Network for Cardiac Surgeries. Journal of Biosciences and Medicines, 10, 57-67. https://doi.org/10.4236/jbm.2022.108006