Support Vector Machine (SVM) methods were applied to the spray-drying process for both regression and classification tasks, aiming to optimize process parameter prediction and operational state identification. SVM Regression for Process Parameter Prediction using a Radical Basis Function (RBF) kernel: The SVM regression (SVR) model was optimized with hyperparameters (C = 100, epsilon = 0.2, gamma = scale) identified via grid search and cross-validation. The SVR showed strong predictive performance with a test R² of 0.96, indicating it explains 96% of the variance in the target variable. Error metrics on test data were low (MAE = 3.82, RMSE = 5.25, MAPE = 0.0077), suggesting robust generalization and minimal overfitting.

Figure 5(a) . SVR training performance visualization showing predicted vs. actual values, residual distribution, and feature importance rankings with time and outside temperature as leading predictors. Figure 5(b) . SVR test performance metrics visualization showing close alignment between predicted and actual values, with most predictions falling within a narrow error band, demonstrating strong model performance on unseen data. Figure 5(c) . Correlation plot between actual and predicted values from the SVR model showing a strong linear relationship with points tightly clustered along the diagonal, indicating high prediction accuracy. Figure 5(d) . Residual plot showing error distribution in SVR predictions with residuals symmetrically distributed around zero, confirming the model’s ability to make unbiased predictions across the range of values.

The SVM classifier, also using an RBF kernel with hyperparameters C = 100 and gamma = 1, exhibited limited performance in classifying operational states, with test balanced accuracy dropping sharply to 0.12 from 0.65 in training—indicative of severe overfitting. Precision, recall, and F1-scores on the test set were equally low. Figure 6 highlights feature importance for classification, with time and tower variables being most influential.

The study reveals that while SVM regression effectively predicts continuous process parameters and supports manufacturing control, SVM classification struggles with identifying operational states due to high-class complexity and dimensionality challenges. The time feature’s dominance in classification suggests temporal patterns significantly influence state transitions, highlighting a need for time-series-specific models. Table 4 compares regression and classification performance, emphasizing regression’s practical utility and classification’s limitations in this context.

The Naïve Bayes classifier was applied for operational state classification due to its ability to handle uncertainty and incorporate prior knowledge, despite the unrealistic feature independence assumption. Figure 7 visualizes the probability distributions modeled by the classifier, showing clear class separations for some features but overlaps for others, indicating varying discriminative power.

Figure 8(a) class probability distribution for feature 1 showing the probability density across different values, with multiple peaks indicating several distinct operational modes. Figure 8(b) class probability distribution for feature 2 showing how this parameter influences classification decisions, with varying probability densities across the parameter range. Figure 8(c) class probability distribution for feature 3 demonstrating more tightly clustered probability distributions, indicating this parameter may have more behavior that is consistent across operational states.

Performance metrics for Naïve Bayes classification showed moderate accuracy (0.49 overall) but poor balanced accuracy (0.01) and low precision, recall, and F1-score (all at 0.01), indicating limited effectiveness across minority classes.

To overcome Naïve Bayes limitations, Bayesian Networks were implemented, capturing probabilistic dependencies between process variables and modeling causal relationships. These networks enable inference of unobserved variables, prediction of downstream effects, and diagnosis of root causes in spray-drying processes.

While Bayesian Networks had slightly better quantitative performance than Naïve Bayes, their key strength lies in providing qualitative insights into parameter interdependencies that support operator decision-making.

As summarized in Table 5 , the Bayesian methods provide valuable probabilistic insights and uncertainty quantification vital for risk-aware manufacturing decisions. However, the Naïve Bayes assumption of feature independence limits classification performance, while Bayesian Networks improve modeling at the cost of increased complexity.

Interpretability is a key benefit of Bayesian approaches, aligning well with operator knowledge and enabling integration of ML results into practical workflows. While these models may not yield the highest raw predictive accuracy, their strength lies in uncertainty management, causal inference, and handling missing data—important for real-world industrial optimization.

Decision tree regression was implemented to build interpretable models that predict process parameters, with an emphasis on both accuracy and transparency to support operator understanding. Hyperparameter optimization through grid search identified the best settings, including the use of the absolute error criterion, a maximum tree depth of 10, and minimum samples per leaf and split set to prevent overfitting while capturing important relationships. The regression model demonstrated strong predictive performance, achieving a high-test R² value of 0.95, indicating that 95% of the variance in the target variable was explained, alongside low errors measured by MAE and MAPE. Figure 9 visually depicts the hierarchical structure of the decision tree, where the most influential parameters are positioned near the root, reflecting their cascading impact on the spray-drying process. Feature importance analysis ( Figure 10 ) identified Gas, Time, and Outside Temperature as the primary drivers of model predictions, consistent with domain knowledge about thermal dynamics. The model generated 173 explicit decision rules, allowing operators to follow a clear decision path, exemplified by a sample sequence involving thresholds on Gas, Time, Output Temperature, Main Blade, and Humidity that culminated in a specific predicted value. This level of transparency enhances user trust and interpretability.

For classification, decision trees were employed to categorize operational states using optimized hyperparameters that included entropy as the splitting criterion, a max depth of 10, and feature subset selection via log2. While the model showed moderate performance on the training set (balanced accuracy of 0.76), there was a severe drop on the test set, with balanced accuracy falling to 0.09, indicating strong overfitting likely caused by class imbalance and limited discriminative information. Figure 11 highlights that the most important features for classification differ somewhat from regression, with Main Blade and Outside Temperature dominating, followed by Time and Output Temperature.

The results underscore several important points: the key advantage of decision trees lies in their interpretability, offering explicit decision rules that align with how operators think about the process, facilitating acceptance and integration into operational workflows. Consistent with prior findings for support vector machines, regression tasks achieve stronger predictive results than classification, which appears more challenging due to dataset limitations. Feature importance insights confirm that parameters related to Gas, Time, and temperature are critical for both prediction and classification tasks, guiding future focus areas for sensor deployment and process control. However, the classification results reveal overfitting despite hyperparameter tuning, suggesting that ensemble methods might be better suited for robust operational deployment.

Table 6 summarizes the performance comparison between decision tree regression and classification. Regression attains high-test accuracy (R² = 0.95) with a moderate gap between training and testing, while classification suffers from a severe performance drop (balanced accuracy drops from 0.76 to 0.09). The top features differ by task: Gas leads regression predictions, whereas Main Blade is most influential in classification. Both models use trees of depth 10 and maintain high interpretability, but classification has limited practical utility in its current form, whereas regression supports effective parameter prediction and control.

Following the evaluation of individual models, hybrid and ensemble approaches were explored to harness the complementary strengths of different algorithms for spray drying optimization. Figure 12 compares the performance of several models—Random Forest, XGBoost, Gradient Boosting, and Support Vector Regression (SVR)—across multiple metrics. Among these, Random Forest exhibited the best overall performance with an R² of 0.962, followed by XGBoost (0.947), Gradient Boosting (0.929), and SVR (0.929). This comparison highlights the advantage of ensemble methods in combining multiple learners to enhance prediction accuracy and robustness. Table 7 details key performance metrics, showing Random Forest leading with the lowest MAE (3.24) and RMSE (5.15), confirming its efficacy for the task.

Visualizations of the ensemble models ( Figures 13-16 ) provide deeper insights into their behavior and operational relevance. Figure 13 illustrates prediction error distributions, learning curves, and feature contribution plots that demonstrate how these models integrate multiple weak learners. Learning curves in Figure 14 reveal that ensemble approaches reduce overfitting by maintaining close training and validation performances. Figure 15 quantifies feature interaction strengths, with darker cells indicating strong interactions, particularly between Gas, Output Temperature, and Time, confirming the complex, nonlinear dependencies inherent in spray drying. Figure 16 presents parameter sensitivity analyses, showing how varying inputs influence predictions, thus offering valuable operational guidance for parameter adjustments.

The discussion of these results emphasizes several key points. Ensemble methods consistently improve performance by combining the diverse strengths of different algorithms, supporting the theoretical premise that model diversity mitigates individual limitations. These approaches blend the boundary-defining power of SVMs, the probabilistic reasoning of Bayesian methods, and the rule-based logic of decision trees into comprehensive predictive frameworks. However, these gains come with trade-offs: ensemble models require higher computational resources and typically sacrifice interpretability compared to simpler models like decision trees, a critical factor when considering deployment contexts. Nevertheless, the rich visualizations and sensitivity analyses generated by ensemble models provide actionable insights that can enhance operational decision-making.

Table 7 presents a comprehensive comparison of ensemble methods tailored for manufacturing optimization. Random Forest offers the best balance of accuracy, computational speed, and ease of implementation, making it suitable for general parameter prediction. XGBoost excels with high-dimensional data but is more sensitive to hyperparameters. Gradient Boosting Supports incremental learning but is slightly less accurate, while Stacked Ensembles deliver the highest accuracy (R² = 0.968) at the cost of increased training time, memory usage, and reduced interpretability. Each method aligns with different operational needs, balancing accuracy, complexity, and real-time capability.

To bridge the gap between accuracy and interpretability in real-world deployment, a tiered machine learning model deployment strategy is proposed. In this approach, a high-accuracy ensemble model (e.g., Random Forest or XGBoost) is deployed to generate predictive outputs in the background, while a simpler, interpretable model such as a decision tree operates in parallel to provide real-time, understandable justifications to human operators. For example, the ensemble model may detect that a specific combination of temperature and flow rate leads to a 7% reduction in product moisture content, while the decision tree displays an actionable rule such as “If inlet temperature > 180˚C and atomizer speed < 12 k RPM, then moisture risk = high.” This architecture enables operators to trust and act on model outputs, combining the precision of advanced analytics with the transparency necessary for effective human-machine collaboration. It also aligns with current industry needs for explainable AI in manufacturing environments [3] [15] .

This study comprehensively evaluates various machine learning algorithms for spray drying optimization, highlighting their relative strengths and limitations ( Table 8 ). Support Vector Machines (SVM) exhibit high regression accuracy (R² = 0.96) and robustness but face classification challenges and limited interpretability. Naïve Bayes and Bayesian Networks provide probabilistic insights and uncertainty quantification, useful for rapid classification and fault diagnosis, though with some accuracy and complexity trade-offs. Decision Trees offer high interpretability and operator-aligned decision rules but suffer from overfitting and limited classification accuracy. Ensemble methods like Random Forest and XGBoost achieve superior regression performance (R² of 0.962 and 0.947, respectively), balancing accuracy and robustness, but at the cost of reduced interpretability and higher computational demands.

Machine learning insights generally align with operator expertise, especially the critical role of Gas flow and temperature parameters. However, models also reveal novel temporal dependencies and interaction effects (e.g., Gas-Humidity interactions) not previously emphasized by operators ( Table 9 ). Difficulties in classifying operational states reflect operator observations of continuous process transitions rather than discrete states. Decision tree-extracted rules complement operator heuristics by providing more precise numerical thresholds, enhancing decision support. Furthermore, complex anomaly patterns and quality predictors identified by machine learning extend qualitative operator assessments, offering quantitative guidance for process optimization and multi-objective trade-off management.

The research results comprehensively address key questions on the application of machine learning to optimize spray-drying operations. Machine learning techniques enable highly accurate parameter prediction, with regression models achieving R² values exceeding 0.95. Decision trees provide interpretable decision support rules that align with operator mental models, enhancing real-world usability. Ensemble methods, particularly Random Forest, demonstrate robust performance across various operational conditions, ensuring reliable optimization even as manufacturing environments change. Bayesian approaches contribute uncertainty quantification that supports risk-aware decision-making, while visualization tools translate complex relationships into intuitive guides for operators. Overall, these machine-learning approaches offer a powerful toolkit for improving efficiency, quality, and decision-making in spray-drying manufacturing.

RQ-1: How can ML approaches improve prediction and control of manufacturing processes?

Machine learning enhances manufacturing prediction and control by replacing traditional first-principles-based models with data-driven approaches grounded in Statistical Learning Theory [21] . In this research, a spray-drying study, techniques such as Support Vector Machines (SVMs), Random Forests (RF), and Decision Trees (DTs) demonstrated high predictive power. The SVM, using squared loss, achieved an impressive R² = 0.96 and MAPE = 0.0077, validating the utility of Reproducing Kernel Hilbert Space (RKHS) theory for function approximation [22] . Meanwhile, the Random Forest model reached R² = 0.962, leveraging ensemble learning through bootstrap aggregation and random feature selection [6] .

Decision Trees provided interpretable rule-based outputs with R² = 0.95, aiding operator trust and process transparency [23] . These results align with the bias-variance decomposition framework, where Random Forests reduce variance and SVMs reduce bias via regularization.

Key variables—gas flow, time, and temperature—were consistently identified as most influential using mutual information (I > 0.75) and Spearman correlation (>0.85), supporting their control-critical roles [24] . The Represented Theorem [25] underpins kernel-based models like SVMs, ensuring sparse, generalizable solutions, while VC theory provides formal generalization bounds based on model complexity [21] .

These results collectively show that machine learning provides accurate, theoretically grounded, and interpretable tools for improving process control in manufacturing systems.

RQ-2: How can effective methods of conducting high-dimensional manufacturing dataset reasoning be achieved?

Effective strategies for managing high-dimensional datasets in manufacturing include ensemble learning, kernel-based methods, and information-theoretic approaches. Even though our dataset included only nine primary features, the applied framework is scalable and generalizable.

Random Forests, through feature bagging and bootstrap aggregation, implicitly regularize high-dimensional models and preserve robustness without overfitting [6] . SVMs utilize RKHS regularization norms to control complexity, yielding sparse models that often use just a subset (~20%) of the training data [22] . This sparsity is advantageous in high-dimensional settings by enhancing interpretability and reducing computation.

To ensure informative feature selection, mutual information-based techniques were applied, minimizing redundancy while maximizing predictive content [24] . Furthermore, Principal Component Analysis (PCA) revealed that five components retained over 85% of the variance, consistent with rate-distortion theory in information science [26] .

From a computational standpoint, algorithmic complexity analysis showed that Random Forests benefit from parallelizability, whereas SVMs—though accurate—demand higher resources. In scenarios with larger datasets, scalable methods such as online stochastic gradient descent (SGD) [27] , MapReduce, and sketching techniques [28] are practical solutions for memory-efficient and distributed learning.

These findings indicate that with proper algorithmic and theoretical tools, high-dimensional manufacturing data can be managed effectively without sacrificing performance or interpretability.

This research demonstrates that machine learning substantially improves spray-drying manufacturing through accurate parameter prediction, multi-objective optimization, and interpretable decision support. Among the methods studied, ensemble models—especially Random Forest—achieved the highest predictive accuracy (R² = 0.962), while decision trees provided the most interpretable insights despite slightly lower accuracy. Models consistently identified Gas Flow, Time, and Temperature as the most influential process parameters. Regression models outperformed classification approaches, uncovering complex nonlinear interactions within the spray-drying system and highlighting the rich mathematical structure underlying manufacturing dynamics.

The synergy of machine learning results with domain knowledge confirmed many operator insights while revealing novel temporal and interaction effects, indicating that ML complements operator expertise by refining parameter thresholds and quantifying relationships aligned with human mental models. Random Forest’s ensemble design offers robust generalizability and computational efficiency, making it suitable for real-time optimization, while Support Vector Machines, despite strong theoretical foundations, are computationally intensive and less interpretable. Bayesian methods, though offering modest predictive accuracy, provided valuable probabilistic reasoning for risk-aware process management. Decision Trees, with over 170 explicit rules, remain essential for human-in-the-loop decision support due to their transparency.

The mixed-methods approach—integrating data-driven modeling with qualitative expert input—enabled the connection between algorithmic intelligence and lived experience, fostering a symbiotic decision-support ecology. Key methodological contributions include forward-chaining time-series splits for realistic validation, rule extraction to quantify qualitative knowledge, and advanced feature engineering that embeds domain knowledge via temporal abstractions and interaction terms (e.g., temperature-humidity ratios). A tiered deployment strategy is proposed, combining high-accuracy predictors (Random Forest, XGBoost) with interpretable models (Decision Trees) to balance accuracy and usability. This architecture supports scalable integration within industrial roles, underpinning applications such as real-time quality prediction dashboards and operator-guided corrective actions. The strong predictive performance (e.g., MAPE < 1% on key quality metrics) offers immediate potential for waste reduction and enhanced repeatability.

This study confirms the power of statistical learning in enhancing manufacturing precision. While ensemble methods maximize predictive accuracy, decision trees ensure interpretability critical for operator acceptance. Coupled with domain knowledge, the hybrid intelligence framework enables proactive quality control and informed decision-making. Machine learning tools empowered operators with predictive dashboards and actionable insights, shifting manufacturing from reactive to predictive control, laying a foundation for ongoing quality improvement, energy efficiency, and enhanced operator engagement.

Future research should focus on integrating these models into live control systems and expanding their applicability across diverse manufacturing domains, with emphasis on scalability, generalizability, and ethical transparency. Recommended directions include:

By combining machine learning, numerical optimization, and financial engineering, future work can transition from predictive analytics toward prescriptive, economically optimized decision support systems for smart, sustainable manufacturing.