The equation of motion of the rotor system can be written as:

$M_R\ddot{u}+\left(C_B+\Omega G_R\right)\stackrel{\dot{}}{u}+\left[K_R+K_B\right]u=Q$(1)

here, the subscripts $R$ and $B$ denote contributions from the rotor and support, respectively. $M$, $K$, $C$, and $G$ represent the mass matrix, stiffness matrix, damping matrix, and gyroscopic matrix, respectively, while $Q$ denotes the load vector.

The critical speeds of the rotor system can be determined using Equation (1) through methods such as the Campbell diagram method and the characteristic equation method. The mathematical principles of the characteristic equation method are described as follows:

For rotor systems, the frequencies of excitation forces acting upon the system are invariably expressible in the following form:

${\omega }_f=n\Omega$(2)

where $\Omega$ denotes the rotational speed of the rotor. According to Equation (2), the excitation force acting on the rotor system can be expressed as:

$Q\left(t\right)=Q_0{\text{e}}^{j{\omega }_{f}t}$(3)

The rotor system response induced by this excitation force can be written as:

$u\left(t\right)=u_0{\text{e}}^{j{\omega }_{f}t}$(4)

Substituting Equations (2)-(4) into Equation (1) gives:

$D{u}_0=Q_0$(5)

where:

$D=-{\Omega }^2\left[n^{2}M-jnG\right]+j\Omega nC+K$(6)

The characteristic equation method solves the eigenvalue problem for $n=1$.

To calculate critical speeds, set $n=1$ and construct matrix $A$:

$A=\left[\begin{array}{cc}0& I\\ -M_v^{-1}{K}_v& -M_v^{-1}{C}_v\end{array}\right]$(7)

where:

$\{\begin{array}{cc}\begin{array}{l}{M}_v=-M+jC\\ K_v=K=K_R+K_B\\ C_v=jC\end{array}& \begin{array}{l}M=M_R\\ C=C_B\\ C=C\end{array}\end{array}$(8)

The critical speeds of the rotor system can be obtained by solving the eigenvalues of the system matrix. In Equation (8), $j=\sqrt{-1}$ denotes the imaginary unit. The characteristic equation method requires only a single eigenvalue solution to determine all critical speeds of the system, demonstrating superior computational efficiency and enhanced suitability for rotor critical speed optimization.

Critical speed is the core index reflecting the dynamic characteristics of the rotor system, this paper takes the first three orders of critical speed of this engine as the optimization objective function, as shown in Equation (9):

$obj=\min {\displaystyle {\sum }_{i=1}^3\frac{\left|\frac{{\omega }_i}{\lambda }-{\Omega }_i\right|}{{\Omega }_i}}$(9)

where $\lambda =1.0$ is the specified similarity ratio; ${\omega }_i,i=1,2,3$ is the first three orders of critical speed of the model rotor system; ${\Omega }_i,i=1,2,3$ is the first three orders of critical speed of the aero-engine rotor system.

Optimization algorithms can be broadly classified into exact algorithms and heuristic algorithms. Gradient-based optimization algorithms utilize gradient information of the objective function to iteratively approach optimal solutions through local derivative analysis. These methods are characterized by high computational efficiency and rapid convergence rates, making them particularly suitable for low-dimensional, continuous, smooth convex optimization problems. However, their effectiveness depends on the differentiability of the objective function and they are prone to local optima trapping.

In contrast, heuristic algorithms employ stochastic search mechanisms inspired by natural phenomena or swarm behaviors to achieve global exploration. These gradient-free methods demonstrate strong capability in handling high-dimensional, non-convex, multimodal, or discrete-variable optimization problems, exhibiting enhanced robustness and potential for global convergence.

As a heuristic algorithm, Genetic Algorithm (GA) translates biological evolutionary principles—specifically the “survival of the fittest”—into mathematical optimization processes via fitness function mapping mechanisms. Its unique encoding-decoding strategies and swarm intelligence characteristics endow it with exceptional global exploration capabilities in solving complex nonlinear optimization problems.

In the context of rotor system model design based on genetic optimization algorithms, similarity-related challenges are transformed into computable optimization problems. This approach effectively resolves model construction difficulties under incomplete similarity conditions, offering a methodological framework for developing testbench systems for highly complex dynamic systems.

The objectives of rotor system dynamic optimization design include:

1) Adjusting critical speeds to ensure the optimized rotor’s operational speed maintains sufficient safety margins, effectively avoiding its critical speed ranges;

2) Optimizing the structural parameters of the model rotor to minimize vibration amplitude under operational conditions.

Design variables, as the core adjustable parameters influencing the optimization scheme, directly determine the validity of the optimization results. The total number of variables corresponds to the dimensionality of the optimization problem. To ensure engineering applicability, all design variables must be assigned reasonable numerical constraint ranges.

A rotor system typically consists of three fundamental components: rigid discs, rotating shafts, and support bearings. Each component can be treated as an independent similarity unit, with similarity characteristics categorized into geometric configuration similarity and kinetic performance similarity. Among these, geometric configuration similarity (structural morphology) plays a dominant role in the similarity analysis of rotor systems, as illustrated in Figure 3 .

For the studied aero-engine rotor system, its characteristic parameters include:

1) Diameter and thickness of 3 axial compressor discs,

2) Outer diameter and thickness of 1 centrifugal compressor disc,

3) Outer diameter and thickness of 2 turbine discs.

To achieve the optimization design objectives, the thickness similarity constants and diameter similarity constants of each disc in the rotor system are selected as optimization design variables. Based on the rotor system’s overall performance and structural characteristics, the chosen optimization variables are illustrated in Figure 4 , with their respective value ranges summarized in Table 1 .

The lower limit of the inner diameter and outer diameter of the disk is set to 1 mm, and the lower limit value of some outer diameters is larger because the structure requires. The minimum thickness of the disc is set at 5 mm, because too thin a disc is not conducive to the design of subsequent rotor testers. The upper limit of disk parameters is also preliminarily selected according to the shape of the prototype rotor, and should not be too large. The overall guaranteed mass and moment of inertia distribution is similar to that of the prototype engine.

To reduce manufacturing complexity, the hollow tie-rod shaft was redesigned as a solid configuration, increasing total rotor mass by 12.5%. Theoretical analysis confirmed that this modification would elevate the first critical speed by 9.2% if uncompensated (from 7913 rpm to 8645 rpm). However, through multi-variable co-optimization targeting mass redistribution (e.g., −15% axial compressor hub inner diameters) and stiffness balancing (e.g., +8% turbine disk thickness).

An GA optimization program was developed using MATLAB. At the initial stage of program execution, the population is randomly initialized, and MATLAB automatically triggers a background process to call the program. Subsequently, ANSYS is utilized to compute the first three critical speeds of all individuals in the population. Through batch processing scripts, ANSYS writes key parameters such as forward/backward whirling frequencies and resonant frequencies for each order into a designated text file upon termination of each iterative calculation.

MATLAB then extracts these characteristic parameters from the file and calculates the fitness value of each individual based on the optimization objective function. Individuals are ranked according to their fitness values. Finally, selection, crossover, mutation, and reinsertion operations are performed based on fitness to generate a new population. The algorithm automatically terminates and outputs the globally optimal parameter combination when the number of iterations reaches the predefined maximum limit. The complete logical framework of this optimization process is illustrated in Figure 5 .

For the optimization design objectives, the GA-based optimization program was configured with 15 design variables, a population size of 20, and a maximum iteration limit of 200 generations. To address the challenge of high-dimensional optimization design variables and enhance convergence efficiency while avoiding local optima traps, the generation gap was set to 0.8 and the mutation rate to 0.07.

The population size (N = 20) was determined through preliminary sensitivity analysis to balance computational efficiency and solution diversity, while the mutation rate (0.07) was selected based on Goldberg’s recommendation for medium-dimensional problems [9] .

The initial geometry of the prototype rotor before optimization is displayed in Figure 6 , and the iterative optimization process is illustrated in Figure 7 . After each optimization iteration, the allowable ranges of optimization variables were

dynamically adjusted through an adaptive mechanism: if a parameter value exceeded 95% of its current range for two consecutive iterations, the corresponding bound was expanded by 10% to avoid search space confinement. For example, the outer diameter of the centrifugal compressor disc (H4) initially ranged from 12 - 30 mm. During iteration 1, W4 reached 29 mm (94.4% of its upper limit), prompting an expansion to 31.8 mm. This strategy ensured continuous exploration while adhering to manufacturing constraints.

The objective function value of the model rotor was reduced from 0.021 (initial optimization) to 0.007 (final iteration) through the GA-driven process. During optimization, the axial compressor exhibited a rearward mass shift alongside inner diameter reduction at disk hubs to clearance limits, while the centrifugal compressor retained stable mass distribution. Concurrently, the turbine experienced mass redistribution toward the rear section.

As shown in Table 2 , the optimized rotor achieves first three critical speeds of 7838 rpm, 15,487 rpm, and 55,877 rpm, with relative deviations of 0.95%, 0.98%, and 0.28% from the prototype, respectively. Abaqus Explicit dynamics simulations under identical boundary conditions yielded critical speeds of 7852 rpm, 15,410 rpm, and 55,790 rpm, deviating by ≤0.5% from ANSYS results, confirming solver independence.

Figure 8 demonstrates identical critical mode shapes between the optimized rotor and the prototype:

1st & 2nd modes: Predominantly exhibit pitching mode characteristics.

3rd mode: Dominated by bending deformation, aligned with the prototype’s dynamic behavior.