As the quest to extract the richest possible insights from data proliferates, a vast arsenal of statistical tools has been deployed, ranging from the conventional to the profoundly innovative. Traditional forms of regression and time series analysis, bounded by their inherent parametric or non-parametric assumptions, have been fundamental in our data modeling approaches [1] . However, the advent of Artificial Neural Networks (ANN), inspired by the intricate workings of human neurons, provides a radical departure from statistical dogma, fostering a broad spectrum of applications including natural language processing, medical diagnoses, agriculture, criminology, statistics, among others ( [2] - [4] ). Other areas of application include sediment load prediction [5] and emission prediction [6] At the heart of ANN lies the paramount process of learning, governed by two main types of parameters - parameters and hyperparameters. While the former is updated by the network, the latter is under the researcher’s control, creating a crucial gateway for modulating ANN performance [7] . Consequently, the meticulous selection of hyperparameters has resulted an array of studies, each endeavoring to optimize prediction or classification outcomes [8] [9] . Among these hyperparameters, Activation Functions (AFs) merit particular attention. Defined as the transfer function enabling signal transmission from one neuron to another, AFs have a profound impact on network training [1] [10] - [13] . While numerous AFs have been developed, each carrying unique strengths and weaknesses, their efficacy is sometimes compromised by the complexity of the data being modeled or their inherent limitations, as in the case of the constant AF [1] . This has stimulated the concept of Fusion of Activation Functions (FAF), which amalgamates different AFs with an aim to create more robust functions for improved signal transfer [14] . However, these attempts have sometimes been met with suboptimal results, as discretionary choices lead to less-than-optimal training and prediction outcomes. Therefore, this research presents an exploration into the depths of FAFs, with a specific focus on the prediction of Container Dwell Time—The length of time freight containers occupie a space from reception to delivery or transfer to the next transport mode. By delving into a more comprehensive suite of FAFs, this study strives to pioneer a path towards improved performance in such predictive tasks, ultimately enriching the potential of ANNs in data modeling and prediction.

The scholarly research dissected underscores the indispensable role and multifaceted nature of Activation Functions (AFs) within Artificial Neural Networks (ANNs). These functions serve as the communication mechanism between neurons, aiding the propagation of signals through the network [1] . Over time, several distinctive types of AFs have been introduced, each with unique strengths and weaknesses. For instance, the step function, sigmoid function, hyperbolic tangent, Rectified Linear Unit (ReLU), Leaky ReLU (LReLU), Parametric ReLU (PReLU), Exponential Linear Unit (ELU), Gaussian Error Linear Unit (GELU), and Swish are among the well-documented Afs [15] - [21] . However, the application of singular AFs across diverse problems and datasets has presented challenges, primarily due to their limitations in different scenarios. The phenomenon of the vanishing gradient syndrome is one such challenge observed in complex datasets when using sigmoid AFs [13] . This predicament led to the exploration of a more versatile approach: The Fusion of Activation Functions (FAFs) [14] . FAFs, essentially the combination of multiple AFs, have been found to improve performance, particularly with larger and more complex datasets, and promote faster network convergence [21] - [25] . However, a significant portion of the extant literature focuses mainly on classification tasks, neglecting regression problems. Furthermore, many studies have prioritized network training speed, overlooking the paramount importance of prediction accuracy in various applications. Whilst training speed focuses on the amount of time required for convergence (completion of training), prediction or classification accuracy focuses on the size of the deviation between the desired outcome and the predicted outcome (usually measured in percentage) [26] [27] . Obviously. This significant gap in the literature serves as the impetus for the current investigation, aiming to enhance the application of FAFs for improving the prediction accuracy of ANNs in modeling Container Dwell Time (CDT). The focus on prediction accuracy is due to two things, first, due to the limited literature and two, its importance in terms of value. CDT is a crucial parameter in the administration and management of container terminals. In addition to the research on AFs and FAFs, other studies indirectly support the broader potential of ANNs in dealing with intricate tasks. These works, including those by [28] , underscore the potential of leveraging strategies like FAFs to enhance ANN performance and accuracy. Although these studies do not directly delve into FAFs, their findings nevertheless offer corroborative evidence of the potential applicability and value of enhanced predictive models in various contexts. While the literature has seen extensive research into the discovery and fusion of new AFs, it has notably fallen short in their application to regression problems and assessing their impact on prediction accuracy. This critical review of the literature presents an opportunity for future research to bridge this gap. It provides a promising prospect for the exploration of FAFs in these areas, potentially paving a new path towards enhancing prediction accuracy in ANNs.

The study considers 307,594 container records from 2014 to 2022 at the Port of Tema made up of import and transit containers delivered within the period under review. The main dataset is main of 307,594 container records extracted from the TOS system of Ghana Ports and Harbours Authority. The data is made up of the dwell days of containers as the dependent variable as well as their characteristics as the main independent variables. From [29] [30] , it is believed that these features influence the dwell time of the container.

Below are the characteristics of the container under consideration ( Table 1 ).

The method used for this study is adopted from [31] who, after reviewing about 97 Machine Learning (ML) research articles in 10 different application areas recommended for the adoption of the following life cycle in conducting ML projects. Data Collection, Data Pre-processing, Model Training, Model Testing and Model Evaluation. This involved the extraction of Container Dwell Time (CDT) data from the TOS system of Ghana Ports and Harbours Authority. This extraction was done to include all the required features of the container after which the dwell days were computed for each record. To be able to make the data machine learning-ready, the following pre-processing steps were taken:

1) Outliers were detected and replaced with the mean CDT.

2) The data was grouped under categorical and numeric features.

3) To achieve standardization, one-hot encoding was applied to the categorical features.

4) Standard Scaling was applied to the numeric features.

5) Principal Component Analysis was used to drop non-informative numeric features.

6) The dataset was then reorganized and divided into input and output feature sets.

Model Training and Testing

The pre-processed data was split into Training, Test and Validation sets.

The main library used was the Random Search CV from the scikit-learn library of Python.

Random Search CV takes the default hyperparameter arguments, (estimator, param_distributions, *, n_iter = 10, scoring = None, n_jobs = None, refit = True, cv = None, verbose = 0, pre_dispatch = ‘2*n_jobs’, random_state = None, error_score = nan, return_train_score = False) which are variable.

The data was then trained with sets of the set hyperparameters in Table 2 till convergence.

The Mean Squared Error (MSE) is the loss function used to evaluate the performance of the model during training and testing. It measures the average of the squared deviations between the predicted and the expected. Represented by:

$\text{MSE}=\frac{1}{N}\underset{i=1}{\overset{N}{{\displaystyle \sum }}}{\left(y-\hat{y}\right)}^2$

N = Sample Size

y = Actual output

$\hat{y}$ = Predicted output

The Principal Component Analysis (PCA), a statistical procedure used to identify the principal components that best capture the variability in a dataset, conducted on three numerical features (BL Type, BL Version, and BOE Version), showed variance levels of 30%, 33%, and 37% respectively (see Figure 1 ). This significant variation suggests that these features hold valuable information and should not be eliminated during data preprocessing. Instead, they should be included in the feature set for training the ANN. This underscores the importance of feature selection and the value of PCA in identifying influential variables in a dataset.

The application of the RandomizedSearchCV function from the scikit-learn library in Python was employed to identify the optimal hyperparameters for the ANN model. This function uses a randomized search on hyperparameters (see Table 3 ), which is both time-efficient and effective, particularly when the dimensionality of the parameter space is high. The search converged on three hidden layers, a constant learning rate, the ajaLT activation function, and stochastic gradient descent (SGD) momentum. This optimal configuration suggests a potential direction for the construction of ANNs for similar problems.

In Table 4 , the comparison between the actual CDT and predicted CDT shows a high degree of accuracy, reinforcing the effectiveness of the chosen hyperparameters and the ANN model itself. The activation function, ajaLT, which is a fusion of different activation functions, appears to contribute significantly to the model’s performance, suggesting that this approach could be more widely applied in future ANNs ( Figure 2 ). The high degree of similarity between actual and predicted CDT values confirms the validity of the approach used, both in terms of the ANN structure and the choice of hyperparameters.

A comparison of the fusion of activation functions (FAFs) with standard mono activation functions was also carried out (see Figure 2 ). This comparison showed that the ajaLT FAF (a fusion of the logistic function and the hyperbolic tangent function) outperformed other FAFs and mono activation functions, with an accuracy level of 82%. This provides strong evidence for the superiority of FAFs in this context, reaffirming previous findings [32] and advocating for their broader adoption in ANNs.

The results suggest that the use of FAFs in ANNs can lead to improved modeling outcomes when predicting CDT. In particular, the ajaLT FAF, which is a fusion of the logistic function and the hyperbolic tangent function, demonstrated superior performance in terms of prediction accuracy. Furthermore, the use of PCA helped to identify important features to include in the model, while the RandomizedSearchCV function provided an efficient means to determine the optimal hyperparameters for the ANN.

Building upon a wealth of empirical literature that emphasizes the crucial role of Activation Functions (AFs) within Artificial Neural Networks (ANNs), the findings of our study demonstrate the application and performance of AFs within a real-world context [1] [15] - [21] . Our research reaffirms the mounting agreement within the scientific community that the fusion of various AFs, referred to as Fusion of Activation Functions (FAFs), can significantly improve network performance and promote quicker convergence, especially when working with larger or more complex datasets [14] [21] - [25] . The existing literature reveals a significant challenge when applying sigmoid AFs to complex datasets: the issue of vanishing gradients [13] . To address this prevalent problem, our study proposes a novel solution—applying FAFs, specifically the ajaLT function, which represents a fusion of the logistic function and the hyperbolic tangent function. This combination demonstrated a superior performance when compared to other singular AFs, contributing substantially to the high prediction accuracy observed in our results. Such an approach not only offers a fresh perspective on dealing with the vanishing gradient problem, but it also introduces a potentially game-changing methodology for future ANN applications. Another important contribution of our study lies in addressing the knowledge gap in the existing literature. The majority of previous research has primarily focused on the application of AFs and FAFs to classification tasks, largely overlooking regression problems. We tackled this underrepresented area by applying FAFs to model the Container Dwell Time (CDT), a key regression problem in the realm of container terminal administration and management [33] - [36] . Through this application, our study adds valuable knowledge to the yet under-explored area of FAFs application in regression tasks, thereby broadening the scope and usability of FAFs in various domains. Furthermore, our study takes a distinctive approach in comparison to the bulk of existing research. Many studies have prioritized network training speed, often neglecting the equally important aspect of prediction accuracy [37] . In contrast, we emphasized the fundamental aspect of prediction accuracy in our research, showcasing the impressive accuracy level achieved (82%) through the use of FAFs. This observation underscores the potential of FAFs in significantly enhancing prediction accuracy, thereby extending the applications of ANNs across a multitude of fields. Our findings not only echo but also significantly contribute to the ongoing discourse in the empirical literature regarding the role and advantages of FAFs in enhancing the performance of ANNs. We advance the discussion by demonstrating that FAFs, specifically the ajaLT function, can be effectively used in regression problems, such as predicting CDT, with high prediction accuracy. These insights add substantial value to the existing body of knowledge by emphasizing the potential of FAFs in ANNs, which in turn illuminates new research avenues.

The extensive exploration into the use of FAFs in ANNs reaffirms the potential of these functions in addressing complex problems within diverse domains. As a pivotal addition to the scientific literature, this study highlights the versatility of FAFs, especially in improving prediction accuracy, setting a promising trajectory for further research and applications in this dynamic and rapidly evolving field.

The conclusions of this study are multi-faceted and significantly contribute to the evolving body of knowledge on the application and performance of Activation Functions (AFs) in Artificial Neural Networks (ANNs). Our study aligns with and extends upon the extensive empirical literature that underscores the indispensable role of AFs in ANNs. First, our research confirmed the effectiveness of Fusion of Activation Functions (FAFs), a burgeoning concept in the field. We demonstrated that the ajaLT function, a fusion of the logistic function and the hyperbolic tangent function, showed superior performance compared to singular AFs. This fusion strategy effectively countered the common issue of vanishing gradients, typically associated with the use of sigmoid AFs in complex datasets. Therefore, our study provides a promising approach to tackle the vanishing gradient problem, offering a significant contribution to the ANNs field. Second, we addressed a gap in the current literature by demonstrating the application of FAFs in regression problems. Specifically, we applied FAFs to model the Container Dwell Time (CDT), an essential parameter in the administration and management of container terminals. By extending the application of FAFs to regression problems, our research diversifies the utility of ANNs and FAFs, broadening their scope and potential impact in various contexts. Third, our study highlighted the importance of prediction accuracy over network training speed, a priority often overlooked in the current literature. We achieved a high prediction accuracy (82%) through the use of FAFs, emphasizing their potential in enhancing the prediction accuracy of ANNs. Our research has reinforced the significance of FAFs in improving the performance of ANNs, particularly in complex datasets and regression problems. We demonstrated the versatility and efficacy of the ajaLT function and emphasized the importance of prediction accuracy. Our findings offer valuable insights and provide a robust foundation for future research, paving the way for more effective and accurate prediction models within the domain of ANNs. The potential applications of these findings extend across a myriad of fields, opening up new avenues for research and real-world applications.

Theory Implications

The results of this study provide valuable insights into the theoretical understanding of Artificial Neural Networks (ANNs) and the use of Activation Functions (AFs). The effectiveness of the Fusion of Activation Functions (FAFs), particularly the ajaLT function, contributes to the understanding of complex data modeling in ANNs. The research further extends the applicability of FAFs beyond classification tasks to regression problems. This broadens the scope of existing ANN theory, providing a richer understanding of how to tackle the common issue of vanishing gradients and improve prediction accuracy.

Policy Relevance

The successful application of FAFs to model Container Dwell Time (CDT) has policy implications, particularly for container terminal management and administration. Given that CDT is a crucial factor in the efficient functioning of these facilities, the ability to accurately predict it can inform strategic decision-making and policy development. Policies that improve logistical efficiency, reduce bottlenecks, and enhance overall terminal operations can be better informed by the predictive capabilities of ANNs leveraging FAFs.

Practice Implications

In practical terms, this study demonstrates the potential of FAFs to enhance the accuracy of ANNs in various applications. For professionals working with ANNs, particularly in data-heavy environments, the use of FAFs may lead to improved performance, more accurate predictions, and faster network convergence. Additionally, this research paves the way for further exploration and utilization of FAFs in different contexts, expanding the toolbox for data scientists, machine learning engineers, and other practitioners in the field.

Social Impact

From a social perspective, the study’s findings could have wide-ranging impacts. Improved prediction accuracy in ANNs could enhance various sectors, including healthcare, logistics, and finance, among others. For instance, more accurate predictions of CDT could lead to more efficient container terminal operations, potentially reducing costs, minimizing delays, and enhancing overall productivity. These improvements could contribute to economic growth, job creation, and overall societal well-being. Further, the study could inspire future researchers and practitioners to innovate and expand upon the potential applications of FAFs, driving advancements in technology that benefit society at large.

Building on the successful application of Fusion of Activation Functions (FAFs) in the prediction of Container Dwell Time (CDT) observed in this study, it is crucial for future research to consider exploring the applicability of FAFs in a wider spectrum of regression problems. This could serve to provide a more comprehensive understanding of the potential use cases for FAFs, thus, illuminating novel avenues for predictive modeling. In addition, although the ajaLT function, a fusion of the logistic function and the hyperbolic tangent function, has demonstrated superior performance in this study, numerous other combinations of activation functions remain unexplored. Consequently, future studies are encouraged to delve into these combinations to assess their performance and potentially discover new effective FAFs that could contribute significantly to the field of predictive modeling. Given the priority of prediction accuracy over network training speed in this study, further research would benefit from a detailed investigation into the trade-offs between these two critical factors. Such understanding could provide valuable insights that would inform the practical decision-making process when it comes to choosing an appropriate AF or FAF for a specific problem or dataset. On a policy level, it is recommended that policymakers, especially those in the domain of container terminal management and administration, take into account the potential benefits of utilizing predictive models like the one presented in this study for strategic decision-making. The ability to accurately predict Container Dwell Time (CDT) could considerably enhance operational efficiency and, as such, policies should be established to encourage the utilization of such advanced predictive tools in practice. To maximize the potential of FAFs in improving prediction accuracy and network performance, it is necessary to offer training to practitioners in the field of data science and machine learning. Educational institutions, organizations, and learning platforms should strive to incorporate modules on FAFs in their training programs. This would foster capacity building and ensure the broad adoption of FAFs in the field. Beyond the scope of FAFs, there are numerous hyperparameters in neural networks that could influence the performance of the model. It is recommended for future research to explore these hyperparameters in the context of FAFs to further enhance the predictive power of these networks. Lastly, there is an urgent need to raise societal awareness about the potential benefits that can be derived from accurate predictive models, such as those empowered by FAFs. By informing the wider public of these benefits, support for further research and application of this technology could be enhanced, which in turn, would contribute to the advancement of predictive modeling.

While this study offers significant insights into the application of Fusion of Activation Functions (FAFs) for predicting Container Dwell Time (CDT), it is not without its limitations, which serve to highlight future research directions. Firstly, the study was largely centered on the ajaLT activation function, a fusion of the logistic function and the hyperbolic tangent function, which outperformed the standard mono activation functions in the prediction of CDT. However, it is crucial to note that the performance of activation functions, whether singular or fused, can vary depending on the nature of the dataset and the problem at hand. Therefore, the superior performance of ajaLT in this study does not necessarily guarantee similar results across all use cases. Secondly, although the study achieved a relatively high prediction accuracy of 82%, there is still room for improvement. The use of other machine learning techniques, beyond the scope of this study, could potentially result in higher prediction accuracy. Additionally, this study’s focus was primarily on prediction accuracy, rather than network training speed. While accuracy is indeed paramount in many applications, the trade-off between accuracy and training speed is an important factor to consider, particularly in use cases where real-time predictions are required. This study, however, did not delve into these trade-offs, which leaves room for future research. Lastly, the study was conducted using a specific dataset related to container dwell time. Although it demonstrated the potential of FAFs in the field of container terminal management, the application of these findings to other datasets and fields may not yield the same results. In light of these limitations, future research should explore the applicability of various FAFs across a broad range of datasets and problem types. There is also a need for comprehensive studies to understand the trade-offs between prediction accuracy and network training speed when applying FAFs. Additionally, further investigations into the influence of other hyperparameters on the performance of ANNs when using FAFs would be beneficial. Overall, this study paves the way for more extensive research on the application and optimization of FAFs in predictive modeling.