This subsection introduces the used subsampling and the random subsampling. For system subsampling, we first determine the sampling interval m 1 and the total sample M 1 , and select position “1” as the sampling starting point to generate the sampling row index. For random subsampling, the total sample M 2 is determined, and the row index of unduplicated samples is randomly generated. The method of subsampling is to combine the row index generated by system subsampling and random subsampling and apply it to the vector X i and the matrix Φ at the same time [8]. We mark the process of subsampling as S , then the linear system obtained after subsampling is

where S X i ∈ ℝ M 1 + M 2 , S Φ ∈ ℝ ( M 1 + M 2 ) × N and the subsampling process is shown in Figure 1.

Figure1. Take row-identical subsamples of vector X i and matrix Φ in the linear system (9).

In the process of integrating the linear system (4), there will be a large difference of orders of magnitude, which will lead to poor identification effect. In order to solve this problem, we need to normalize the data to maintain data stability and improve the generalization ability of the model. For the vector S X i in the linear system (10), the maximum absolute is used to scale to it. Then, we have

For matrix S Φ , applying the L 2 norm normalization to S Φ column-wise:

then all normalized column vectors D j are represented as columns of the new matrix D by:

Therefore, the linear system after subsampling and normalization is as follows:

where Y i ∈ ℝ M 1 + M 2 , D ∈ ℝ ( M 1 + M 2 ) × N , c i ∈ ℝ N . Using this method to preprocess data can help us to identify the governing equations of nonlinear dynamical systems more effectively in terms of reducing calculation cost, processing unbalanced data and improving data stability.

However, it is also worth noting that in practical applications, due to the fact that the data contains noise, Equation (14) does not strictly hold so that

This approximation makes it very difficult for us to solve the inverse problem. In order to deal with this challenge, we need to find an optimal c ^ i , which makes the error between Y i and D c i extremely small, and promotes the sparsity of the coefficient vector c i . For this, we use the Lasso Regression [39]-[41] to solve this problem:

where λ is the regularization parameter.

In this subsection, we propose to use the Frank-Wolfe algorithm [34][35] to solve the Lasso problem corresponding to Equation (16). Frank-Wolfe algorithm is an iterative method for solving convex optimization problems, especially when the objective function is a differentiable convex function and the constraint is convex set. Therefore, it is especially effective when dealing with convex optimization problems with L 1 constraints.

According to [39], Equation (16) can be equitably expressed as:

where α is the regularization parameter. Problem (17) is equivalent to (16) via the KKT conditions: the Lagrange multiplier for the ℓ 1 -ball constraint acts as the regularization parameter λ . Thus, a larger λ (stronger sparsity) corresponds to a smaller α . In practice, we treat α as the main hyperparameter and select it empirically as described in Section IV. Then, we solve the convex optimization problem (17) to find the optimal sparse coefficient vector c i by using the Frank-Wolfe algorithm.

We define the objective function as

Since the Frank-Wolfe algorithm is also called the conditional gradient algorithm, which determines the optimization direction by calculating the gradient, we need to compute the gradient of the objective function g ( c i ( p ) ) at the p -th iteration:

where p = 1 , 2 , ⋯ , p max ， p max is the maximum number of iterations. During each iteration, for a given gradient ∇ g ( c i ( p ) ) , we need to find a direction s ( p ) that minimizes the following linear subproblem:

This problem can be simplified by using the dual norm. For the L 1 norm constraint problem, the dual norm is the L ∞ norm [42], i.e.

To find the optimal direction s ( p ) , we need to minimize the inner product 〈 ∇ g ( c i ( p ) ) , s 〉 . This is equivalent to moving in the direction of the maximum absolute component of the gradient under the constraints of the L 1 norm, where the maximum absolute component is given by [38]

Since we need the inner product 〈 ∇ g ( c i ( p ) ) , s 〉 to be minimum and ‖ s ‖ 1 ≤ α , we need to choose the direction of s to be opposite to the direction of the gradient and take the largest possible step size α in the direction of the gradient

component with the largest absolute value [38], i.e.

where e j ( p ) is a unit vector that takes the value “1” in the direction of the maximum absolute component of the gradient.

Next, we perform an exact linear search between the current solution c i ( p ) and the direction s ( p ) to find the optimal step size γ ^ to minimize the objective function g ( c i ) :

where d ( p ) = s ( p ) − c i ( p ) indicates the search direction, and

To find the optimal step size γ ^ , we take the derivative of Equation (25) and make the derivative zero, then we have

Once the optimal step size γ ^ ( p ) in each iteration is calculated, we can update the current solution c i ( p ) by the optimal step size to get the next solution c i ( p + 1 ) , i.e.

We define the dual gap h ( p ) :

If the dual gap h ( p ) is less than the preset threshold, the algorithm is considered to have converged and the iteration is stopped. Additionally, the used algorithm also will stop the iterations when the maximum number of iterations is reached.

Finally, the normalized solution vector c i is restored to the original scale:

where we define

for nonlinear dynamical system identification based on the Frank-Wolfe algorithm is summarized in Algorithm1.

Consider the following Logistic equation

This is a common model of population dynamics. We use the ode45 function in MATLAB to generate the simulation data x ( t ) of the Logistic equation under the initial condition x ( 0 ) = 8 × 10 − 8 , the time interval t ∈ [ 0 , 10 ] , and the time-step Δ t = 0.001 . The generated data trajectory is shown by the solid red line in Figure 2. However, to model the practical case, we add the noise to the data via x ˜ ( t ) = x ( t ) + η ( t ) , where η ( t ) ~ N ( 0 , σ 2 ) , here we choose σ = 10 − 5 .

After obtaining the noisy measurement data x ˜ ( t ) containing the noise, we can combine the initial condition x ( 0 ) = 8 × 10 − 8 and Equation (8) to get the displacement vector X , and then define the matrix Θ as follows

The high maximum degree (9) is intentionally chosen to test the method’s ability to suppress irrelevant higher-order terms under noise. By combining Equation (6) and Equation (7), we can construct the feature matrix Φ.

Next, we use the subsampling method proposed in this paper, which combines systematic subsampling and random subsampling, to simultaneously conduct subsampling on the displacement vector X and the feature matrix Φ. When we perform systematic subsampling, the sampling interval m 1 = 100 and the total sample M 1 = 12 are selected, and the position “1” is selected as the sampling starting point to generate the sampling row index. When conducting random subsampling, the total sample M 2 = 30 is used, and the row index of unduplicated samples was randomly generated. The row index generated during the subsampling process is then applied to the displacement vector X and the feature matrix Φ to obtain S X and S Φ . The subsampling process is shown in Figure 1.

Then, we apply the maximum absolute value scaling in Equation (11) and the L 2 norm normalization in Equation (12) to normalize the vector S X and the matrix S Φ , respectively, and then get the linear system: Y = D c . Finally, we can use the Frank-Wolfe algorithm proposed in this paper to solve the sparse vector c . In the process of applying the Frank-Wolfe algorithm, we set the maximum number of iterations p max = 40000 , the regularization parameter α = 3 , the threshold value t o l = 10 − 8 , and the parameter τ = 10 − 4 . The value α = 3 was chosen empirically after testing a small set of candidate values to achieve a desirable trade‑off between sparse coefficient support and data fitting accuracy.

With the noise intensity σ = 10 − 5 , the identification results for the Logistic equation are shown in Table1. In addition, we also plotted the data trajectories of the learned governing equation, as shown by the dashed black line in Figure 2.

By comparing the true coefficients with the learned coefficients in Table1, and analyzing the data trajectory of the exact system without noise and the identified system with noise intensity σ = 10 − 5 in Figure 2, it can be demonstrated that the Frank-Wolfe algorithm proposed in this paper, which combines subsampling and data normalization, accurately identifies the correct activity items. This also demonstrates the effectiveness of our approach. In addition, we can see from Fig. that the trajectory of the system identified by the method proposed in this paper almost coincides with the real trajectory, which also shows that the method is robust to the measurement noise. The experimental MSE is 1.4545 × 10⁻⁷ and the RMSE is 3.8137 × 10⁻⁴.

Figure 2.Data trajectories of logistic equation under given conditions. The solid red line represents the true trajectory of the exact system. The dashed black line represents the data trajectory of the sparse identified system under the condition of noise intensity σ = 10 − 5 .

Table 1. Identification results of Logistic Equation (31). The “Basis” represents the used basis functions. The “True” indicates the true coefficients. The “Learning” represents the parameters identified using the proposed method.

Consider the Lokta-Volterra system

The Lotka-Volterra system, also known as the predatorprey model, is a mathematical model used to describe the dynamics of biological populations. Under the initial condition [ x 1 ( 0 ) , x 2 ( 0 ) ] = [ 10 − 5 , 5 × 10 − 4 ] , the time interval t ∈ [ 0 , 200 ] , and the time-step Δ t = 0.01 , the simulation data of Lotka-Volterra system is generated. The generated data trajectory is shown by the solid red line in Figure 3(a). Similarly, in the presence of noise, we set the noise intensity σ = 10 − 5 .

We calculate the displacement vector X i ( i = 1 , 2 ) according to Equation (8). We construct a polynomial candidate library of total degree 4, including all monomials of the form x 1 p 1 , x 2 p 2 with p 1 , p 2 ≥ 0 and 0 ≤ p 1 + p 2 ≤ 4 . We define the matrix Θ as follows:

Although the true system (32) contains only linear and bilinear terms { x 1 , x 2 , x 1 x 2 } , the higher‑degree monomials are included to test the ability of the Frank‑Wolfe algorithm to suppress irrelevant terms and recover the correct sparse support under noisy measurements. Then, according to Equation (6) and Equation (7), the feature matrix Φ is obtained.

After obtaining the displacement vector X i ( i = 1 , 2 ) and the feature matrix Φ, we need to subsample for X i ( i = 1 , 2 ) and Φ. In this case, we need to subsample the displacement vectors X 1 and X 2 and their corresponding feature matrices Φ₁ and Φ₂ respectively. The feature matrices Φ₁ and Φ₂ are the same here, and we have added subscripts for the convenience of the subsampling process.

For the displacement vector X 1 and the feature matrix Φ₁, we select the sampling interval m 11 = 30 , the total sample M 11 = 28 , and the position “1” as the sampling starting point to generate the sampling row index. When conducting random subsampling, we select the total sample M 12 = 6 and randomly generate the row index with no duplicate samples. Then the row index generated by the subsampling is applied to both the displacement vector X 1 and the feature matrix Φ₁ to obtain S X 1 and S Φ 1 . For the displacement vector X 2 and the feature matrix Φ₂, we select the sampling interval m 21 = 150 , the total sample M 21 = 110 , and the position “1” as the sampling starting point to generate the sampling row index. When conducting random subsampling, we select the total sample M 22 = 8 and randomly generate the row index with no duplicate samples. Then the row index generated by the subsampling is applied to both the displacement vector X 2 and the feature matrix Φ₂ to obtain S X 2 and S Φ 2 .

Next, Equation (11) is used to perform maximum absolute value scaling on the vector S X i ( i = 1 , 2 ) , and Equation (12) is used to perform L 2 norm normalization on the matrix Y i = D i c i ( i = 1 , 2 ) . Thus, the linear system: Y i = D i c i ( i = 1 , 2 ) is obtained. Finally, we use the Frank-Wolfe algorithm proposed in this paper to solve the convex optimization problem with L 1 constraints in Equation (16), and then obtain the sparse vectors c 1 and c 2 .

In the process of applying the Frank-Wolfe algorithm to solve the sparse vector c i ( i = 1 , 2 ) , for the sparse vector c 1 , we set the maximum number of iterations p 1 max = 100000 and the regularization parameter α 1 = 500 . For the sparse vector c 2 , we set the maximum number of iterations p 2 max = 10200 and the regularization parameter α 2 = 500 . They share a threshold value t o l = 10 − 8 and a parameter τ = 10 − 6 . Both α 1 = 500 and α 2 = 500 were selected empirically based on several trials, where larger values of α (weaker sparsity) led to overfitting and smaller values (stronger sparsity) eliminated relevant terms. The value 500 gave the best compromise for both equations under the same subsampling setting. The identification results are shown in Table2.

Additionally, we also plotted the true trajectories of each component of the Lotka-Volterra system and the identified trajectories of each component of the learned governing equations, as shown in Figure 3(b) and Figure 3(c).

By comparing and analyzing the true and estimated coefficients in Table2, as well as the true and the identified trajectories in Figure 3, we verify the effectiveness of the proposed method for the identification of the Lokta-Volterra system. However, it can be seen from Figure 3(a) that there are still some errors in the identification of the entire system. In addition, we also calculated the MSE and RMSE between the true parameters and the estimated parameters of the Lotka-Volterra system, as shown in Table 3.

Table 2. Identification results of Lokta-Volterra system (32).

Figure3. (a) Phase diagram of Lotka-Volterra system. The solid red line represents the true trajectory of the exact system. The dashed black line represents the data trajectory of the sparse identified system under the condition of noise intensity σ = 10 − 5 . (b) The true trajectory of the state component x 1 ( t ) is represented by the solid red line. The identified trajectory is represented by dashed black line. (c) The true trajectory of the state component x 2 ( t ) is represented by the solid blue line. The identified trajectory is represented by dashed black line.

Table 3.MSE and RMSE between the true parameters and the estimated parameters of the Lotka-Volterra system.

Consider the two-dimensional damped harmonic oscillator with cubic dynamics

The two-dimensional damped harmonic oscillator generates the simulation data under the initial condition [ x 1 ( 0 ) , x 2 ( 0 ) ] = [ 1 , 1.8 ] , the time interval t ∈ [ 0 , 10 ] , and the time-step Δ t = 0.001 . The data trajectory is shown in Figure 4(a). In the presence of noise, we set the noise intensity σ = 10 − 5 .

After obtaining the noisy measurement data x ˜ , we can obtain the displacement vector X i ( i = 1 , 2 ) and the matrix Θ, where we construct a polynomial candidate library of total degree 4, including all monomials in x 1 and x 2 with 0 ≤ p 1 + p 2 ≤ 4 . We define the matrix Θ as follows:

Although the true dynamics (33) contain only the cubic terms x 1 3 and x 2 3 , we include all lower- and higher-degree monomials to evaluate whether the Frank-Wolfe algorithm can correctly identify the sparse support and reject the irrelevant terms in the presence of noise. Then we can obtain the feature matrix Φ by Equation (6) and Equation (7).

When subsampling the displacement vector X i ( i = 1 , 2 ) and the feature matrix Φ, we use the same method as in the two examples above. We choose m 11 = 100 , M 11 = 30 , M 12 = 20 and m 21 = 100 , M 21 = 30 , M 22 = 20 . And then we have the vector S X i ( i = 1 , 2 ) and the matrix S Φ i ( i = 1 , 2 ) . Then we use the method proposed in this paper to normalize the vector S X i ( i = 1 , 2 ) and the matrix S Φ i ( i = 1 , 2 ) respectively, and we can get the linear system: Y i = D i c i ( i = 1 , 2 ) .

Finally, we can use the Frank-Wolfe algorithm to solve the sparse coefficient vector c i ( i = 1 , 2 ) . When solving the sparse coefficient vector c 1 , we set the maximum number of iterations p 1 max = 3000 and the regularization parameter α 1 = 3 . When solving the sparse coefficient vector c 2 , we set the maximum number of iterations p 2 max = 26000 and the regularization parameter α 2 = 10 . They share a threshold value t o l = 10 − 8 and a parameter τ = 2 × 10 − 5 .

These values were determined by empirical tuning: α 1 = 3 and α 2 = 10 each produced a sparse coefficient vector whose nonzero entries matched the true support while keeping the residual ‖ D c i − Y i ‖ 2 low. Different α values for the two equations reflect their different sensitivities to the ℓ₁ constraint. The identification results are shown in Table 4.

Table 4. Identification results of the two-dimensional damped oscillator (32).

We also plotted the true trajectories of each component of the two-dimensional damped harmonic oscillator and the identified trajectories of each component of the learned governing equation, as shown in Figure 4(b).

Figure4. Phase diagram of the two-dimensional damped oscillator and the identified trajectories of state components x and y.

By comparing the true coefficients and learned coefficients in Table3, and analyzing the true trajectories and identified trajectories in Figure 4(a) and Figure 4(b), it can be effectively demonstrated that the Frank-Wolfe algorithm proposed in this paper, which combines subsampling and data normalization, can correctly identify active items and approximate coefficients. The MSE and RMSE between the true parameters and the estimated parameters are shown in Table 5.

Table 5.MSE and RMSE between the true parameters and the estimated parameters of the two-dimensional damped oscillator.

Consider the following Lorenz system

The Lorenz system is a three-dimensional nonlinear dynamical system that is one of the classic examples of chaos theory, demonstrating complex dynamical behavior. The Lorenz system generates the simulation data under the initial condition [ x 1 ( 0 ) , x 2 ( 0 ) , x 3 ( 0 ) ] = [ − 152 , 150 , 300 ] , the time interval t ∈ [ 0 , 10 ] , and the time-step Δ t = 0.001 . The data trajectory is shown in Figure 5. In the presence of noise, we set the noise intensity σ = 10 − 5 .

From the noisy measurement data x ˜ , we can get the displacement vector X i ( i = 1 , 2 , 3 ) and the matrix Θ, where we construct a polynomial candidate library of total degree 3, including all monomials in { x 1 , x 2 , x 3 } with 0 ≤ p 1 + p 2 + p 3 ≤ 3 . The matrix Θ is defined as follows:

Although the true Lorenz system (34) contains only linear and bilinear terms { x 1 , x 2 , x 3 , x 1 x 2 , x 1 x 3 } , the higher-degree monomials are included to rigorously test the ability of the Frank-Wolfe algorithm to suppress irrelevant terms and recover the exact sparse support in a chaotic system under noise. Then the feature matrix Φ can be obtained by Equation (6) and Equation (7).

Next, the displacement vector X i ( i = 1 , 2 , 3 ) and the feature matrix Φ are subsampled. In the subsampling process, m 11 = 190 , M 11 = 20 , M 12 = 5 are selected for the displacement vector X 1 and the feature matrix Φ 1 . For the displacement vector X 2 and the feature matrix Φ 2 , we choose m 21 = 149 , M 21 = 52 , M 22 = 8 . For the displacement vector X 3 and the feature matrix Φ 3 , we choose m 31 = 33 , M 31 = 6 , M 32 = 36 .Then we normalize the vector S X i ( i = 1 , 2 , 3 ) and the matrix S Φ i ( i = 1 , 2 , 3 ) according to Equation (11) and Equation (12), and obtain the linear system: Y = D c . Finally, the sparse coefficient vector c i ( i = 1 , 2 , 3 ) is solved by the Frank-Wolfe algorithm proposed in this paper. When solving the sparse coefficient vector c 1 , we set the maximum number of iterations p 1 max = 70000 and the regularization parameter α 1 = 3.8 . When solving the sparse coefficient vector c 1 , we set the maximum number of iterations p 2 max = 49600 and the regularization parameter α 2 = 14 . When solving the sparse coefficient vector c 3 , we set the maximum number of iterations p 3 max = 68000 and the regularization parameter α 3 = 9.59 . They share a threshold value t o l = 10 − 8 and a parameter τ = 10 − 6 . Each α i was chosen empirically by trying a set of candidate values and selecting the one that produced the correct sparsity pattern with the smallest fitting error. The distinct values (3.8, 14, 9.59) are necessary because the three equations have different magnitudes and sensitivities to the ℓ 1 constraint. The identification results are shown in Table6. In addition, we also plotted the true trajectories of each state component of the Lorenz system, as shown in Figure 6. The MSE and RMSE between the true parameters and the estimated parameters are shown in Table7.

Table6. Identification results of Lorenz system (34).

Figure5. Identified trajectories of the Lorenz system.

Figure6. Trajectory of state component x, y, z.

Table 7.MSE and RMSE between the true parameters and the estimated parameters of the Lorenz system.

By comparing the true coefficients with the learned coefficients in Table5, and analyzing the true trajectories and identified trajectories in Figure 5 and Figure 6, we can see that the method proposed in this paper is very effective for the identification of complex Lorenz system. Additionally, it can be seen from Figure 5(a) and Figure 5(b) that the considered Lorentz system is sensitive to the system parameters. When the identified parameters contain a very small error, the true trajectory of the Lorenz system also exhibits slight deviations.