The following section provides a detailed examination of significant studies in the field of human activity recognition. In computer vision, one of the primary fields of study is action recognition, which is a subset of gesture recognition [18] . Researchers have employed a variety of methods to develop action-recognition systems, such as artificial intelligence (AI), hand-crafted features combined with traditional machine learning algorithms, and diverse deep learning approaches [19] . Traditional machine learning algorithms and a range of manual feature extraction methods were mostly used to construct the present human activity recognition (HAR) [20] . Conventional machine learning techniques for action recognition typically follow a three-step procedure. At the first phase, features are extracted using manually created descriptors. Then, a particular algorithm is used to encode these features. In the final stage, the encoded features are classified using a suitable machine-learning method [21] . Two distinct techniques are employed in different tasks: local feature-based approaches and global feature-based approaches. The main goal of local feature-based techniques is to characterize features as separate patches, interest points, and gesture information. The learned cues relevant to the current task are aligned with these features. Global features, on the other hand, encompass the entire region of interest.

As noted in the references [22] , background removal and tracking techniques are frequently used to accomplish this. Yasin et al. [23] , who have presented a fundamental method for identifying actions in video sequences by utilizing keyframe selection and applying it to human activity recognition (HAR). Zhao et al. [24] presented the HAR system that leverages keyframes and employs conventional machine learning techniques for multi-feature fusion. Yala et al. [25] introduce a novel activity recognition system that relies on streaming data. The strategy shows a remarkable level of accuracy in recognizing important human activities. Nunes et al. [26] introduced a framework aimed at daily life human activity recognition. Many features are first extracted by the suggested method. Following this, two successive automatically recognized key positions are encircled around each human activity frame, from which the maximal static and dynamic features are retrieved. Kantorov and Laptev [27] identified the application of Fisher vectors for feature encoding and effectively used linear classifiers to obtain accurate action identification. In [28] , Lan et al. presented a process for improving action recognition systems’ operational strategies by switching from data-driven to data-independent approaches. While standard machine learning algorithms have made significant progress in the last 10 years, they still have limits because of human cognition. These limitations include labor intensiveness, time consumption, and the difficulty of feature engineering [29] . Deep learning is a move toward more automated and adaptive techniques that can get over the limitations of handcrafted features. Deep learning departs from the conventional three-step machine learning architecture by introducing a contemporary end-to-end framework. With this framework, classification tasks can be completed simultaneously with the learning and representation of highly discriminative visual features. This following presents the previous related publication for human activity recognition (HAR) using deep leaning techniques. Table 1 presents the list of papers and its corresponding dataset with accuracy.

From this literature reviews, we can understand the deep learning model that uses feature extraction and classification techniques to improve the accuracy of human action recognition.

During the preprocessing step, a region-wise sliding window approach is implemented to account for variation in each consecutive frame. This approach helps in ignoring unnecessary regions, such as the background. The result of this process is a binary image is extracted through background subtraction, which is then subjected to a noise removal technique. To enhance the image, the binary image is initially converted to RGB color. Then, the RGB image is transformed into the Hue Saturation Intensity (HSI) format. The next stage is drawing a box around a person to identify them. This step aims to improve the quality of the video-extracted frames by enhancing the foreground features. The following are detailed steps involved in the preprocessing stage are presented in Figure 2 .

In this stage, three types of feature extractors named Histogram of Oriented Gradients (HOG), Gabor, and chromatic features that are used to analyze each frame. These three types of feature extractors are considered in our study because the goal is to accurately identify and classify various activities, optimal feature selection becomes pivotal for creating models that are both accurate and practical for real-world deployment [34] . The resulting feature vectors have dimensions of 1 × 3780 for HOG, 1 × 60 for Gabor, and 1 × 9 for co-occurrence matrices and chromatic features. These features capture aspects like gradient distributions, textures, spatial relationships, and color characteristics in the frames, contributing to a comprehensive set for further analysis or classification.

Feature extraction using HOG Method, the image is initially divided into smaller segments, which are then processed individually. Afterwards, these segments are combined back together. To compute directional gradients $G_x$ and $G_y$ the Sobel kernel function is utilized on the processed images according to the following mathematical equations [35] .

$F_{se{g}_{\left|G\left(i,j\right)\right|}}=\sqrt{G_x{\left(i,j\right)}^2+G_y{\left(i,j\right)}^2}$ (1)

$F_{se{g}_{\varnothing G}}\left(i,j\right)=ta{n}^{-1}\left(\frac{G_x\left(i,j\right)}{G_y\left(i,j\right)}\right)$ (2)

where, $\left|G\right|$ denotes magnitude, ${\varnothing }_G$ supplies the gradient angle, i and j stand for rows and columns concurrently. Based on the gradient, the angle divides the cell votes into bins. Later, each block of the histogram is used to create the standardized vector.

The following equation represents the eight bin cells that are used to implement the HOG feature descriptor on the segmented image.

$F_{se{g}^{V_i^N}}=\frac{V_i}{\sqrt{V_2^2+{\in }^2}}$ (3)

where, the vector V is the non-normalized vector that contains all of the histograms in a block, and $\in$ is a minor constant that does not split by zero. A single block containing all of these vectors is the HOG feature vector. Each feature’s range and mean variance are also measured. Figure 3 shows the HOG features and tis graphical representation.

The following equation illustrates how a complex sinusoidal wave is used to use the “Gaussian Kernel” feature of a modified 2D Gabor filter in the geographic area.

$F_{seg}=\frac{f{s}^2}{\pi Y\eta }\exp \left(-\frac{p^{\prime }+Y^2{q}^{\prime }}{2{\sigma }^2}\right)\exp \left(2\pi fs{x}^{\prime }+\varnothing \right)$ (4)

here, Y represents the spatial characteristics where the elliptical support of the Gabor function is defined; p' and q' are detailed in the following equations. fs indicates sinusoidal frequency, $\varnothing$ represents band similarity direction of an activity described by Gabor, $\varnothing$ indicates the phase offset, and σ indicates the Standard Deviation (SD) of the Gaussian wrapper [36] . The gabor feature has five scales and six directions of implementation. The chosen measurement for the gabor feature is 1 × 30. By using the Gabor feature, one may measure the variance and mean. Figure 3(c) describes the graphical representation of HOG features.

Because of this method’s extensive use, it has become a standard. In contrast, other studies depended on a limited set of functions, including entropy (H), correlation (COR), energy (E), and local homogeneity (LH). To calculate this Chromatic Features, we used these equations (5)-(9) [37] .

$E={\displaystyle {\sum }_i{\displaystyle {\sum }_j{\left[h\left(i,j|d,\theta \right)\right]}^2}}$ (5)

$H=-{\displaystyle {\sum }_i{\displaystyle {\sum }_j\left[h\left(d,\theta \right)\log h\left(d,\theta \right)\right]}}$ (6)

$I={\displaystyle {\sum }_i{\displaystyle {\sum }_j\left[{\left(i-j\right)}^{2}h\left(d,\theta \right)\right]}}$ (7)

$LH={\displaystyle {\sum }_i{\displaystyle {\sum }_j\frac{\left(i,j|d,\theta \right)}{1+{\left(i+j\right)}^2}}}$ (8)

$COR={\displaystyle {\sum }_i{\displaystyle {\sum }_j\frac{\left(i-{\mu }_x\right)\left(j-{\mu }_y\right)h\left(i,j|d,\theta \right)}{{\sigma }_x{\sigma }_y}}}$ (9)

where, ${\sigma }_x$ and ${\sigma }_y$ are the vertical statistics, ${\mu }_x$ is the horizontal mean, and ${\sigma }_x$ is the variance. Quantifying color information in images in demonstrate in Figure 3(d) .

In the presented technique, Principal Component Analysis (PCA) is employed for the purpose of feature selection. This allows the identification and selection of the most significant features from the outcomes of various methods, namely Histogram of Oriented Gradients (HOG), Gabor filters, and chromatic feature vectors. In general, the PCA method transforms a set of n vectors from a d-dimensional space to another space with d' dimensions [39] . This is represented by the equation, which gives the resulting in vectors as ( $x_1^{\text{'}},x_2^{\text{'}},\cdots ,x_i^{\text{'}},\cdots ,x_n^{\text{'}}$).

$x_r^{\text{'}}={\displaystyle {\sum }_{n=1}^{d^{\prime }}{a}_{n,r}}{e}_n,d^{\prime }\le d$ (10)

where, $e_n$ displays the greatest eigenvalues of the distribution and the eigenvectors corresponding to the d'-dimensional space. Conversely, an, r represents predictions of the vectors $x_r$ across the eigenvectors.

The action recognition method will be efficient and effective because of feature fusion. Additionally, in complex settings, this improves the human action classification rate. Compared to the original Gabor and HOG features, feature fusion in this method yields significantly better results in both the high brightness environment and the dark background. The feature vectors have sizes of 1 × 60, 1 × 3780, and 1 × 9 for HOG, Gabor, and chromatic features, respectively. Let, C1, C2, C3, ⋯, Cn be the human activity classes that require classification in order to perform feature fusion. Consider, $\Delta =\left\{\varnothing \text{v}\varnothing \in RN\right\}$ presents the total number of model training samples $\left\{{Ύ}_{HOG},{Ύ}_{Gab},{Ύ}_{Chrom}\right\}\in R_{HOG+Gab+Chrom}^N$ are the three feature vectors that have been extracted. The size is defined as:

$F{V}_1=\left\{j_1,\cdots ,j_k\right\},F{V}_2=\left\{y_1,\cdots ,y_k\right\},F{V}_3=\left\{d_1,\cdots ,d_k\right\}$ (11)

The sizes of the feature vectors are denoted by FV₁, FV₂, and FV₃, representing HOG, Gabor, and cooccurrence matrices with chromatic features, respectively. These feature vector sizes can be further described using the set k, where k ∈ {60, 3780, 9}. The sizes of extracted feature sets are: (Ύ_HOG→1 × 3780, Ύ_Gab→1 × 60, Ύ_Chrom→1 × 9). The final extracted vector is indicated as:

$F\left(Ø\right)={{\displaystyle {\sum }_j^{F{V}_1}Ύ}}_{HOG}+{{\displaystyle {\sum }_t^{F{V}_2}Ύ}}_{Gab}+{{\displaystyle {\sum }_d^{F{V}_3}Ύ}}_{Chrom}$ (12)

$F\left(\text{Ø}\right)=\left\{\left(1\times 3780\right)+\left(1\times 60\right)+\left(1\times 9\right)\right\}$ (13)

$Final\left(\text{Ø}\right)=\left\{\left(1\times 3849\right)\right\}$ (14)

We compared five different ways of making predictions: linear SVM (LS), cubic-SVM (CS), Complex tree (CT), fine-KNN (FK), and subspace KNN (SK). Figure 4 shows how we selected and combined features to get the best results.

Subspace-KNN did the best on the KTH dataset, and cubic-SVM did better than the other classifiers model on the Weizmann dataset.

The total success of the machine learning model, which includes the suggested network, is heavily influenced by the caliber and applicability of the dataset. In this study, we consider Weizmann Dataset and KTH dataset.

Weizmann dataset comprises 2513 images depicting various human activities, performed by nine different actors and covering five types of human behavior. After selecting and combining features, classification techniques are functional to assess the results. Figure 5 provides a sample of images from the Weizmann dataset. This dataset consists of five classes: Hand Waving, Running, Jumping, Walking, and Bending [39] .

The KTH dataset comprises 1628 images showcasing six distinct types of human activities. Figure 6 displays sample images from the Weizmann datasets, which encompass activities such as boxing, clapping, hand waving, running, and walking [40] . Table 2 presents the combined summary of both datasets (Cn for Class label, n = 1, 2, 3…).

In Section 4.2 of the research, they checked how well their new algorithm performed. They used a few different measurements to do this:

These measurements help us to calculate overall the classification performance.

To quantify the results, three distinct experiments are conducted, each involving a different number of features. Table 3 provides a detailed description of all experiments, specifying the number of classes, folds, and features. When assessing the performance of a machine learning model, a technique called “k-fold cross-validation” is commonly used. This involves dividing the dataset into subsets (folds), training the model on some folds, and evaluating it on others. The process is repeated multiple times. Using different values for k helps in obtaining a more reliable estimate of how well the model performs. After all the runs, the results are averaged to get a comprehensive assessment of the model’s effectiveness. This approach provides a more robust evaluation compared to a single train-test split. Each experiment uses five classification methods and calculates sensitivity, specificity, precision, and AUC (Area Under Curve) for the Weizmann and KTH datasets. This helps us see which classification method is the best fit for these specific datasets.

In Experiment 1, we observed that cubic-SVM achieved the highest specificity of 99.80% for the Weizmann dataset among all the algorithms. Meanwhile, Fine-KNN attained an accuracy of 99.93%, specifically in terms of precision, for the KTH dataset, as indicated in Table 4 .

In Experiment 2, we observed that cubic-SVM achieved the highest sensitivity of 99.85% for the Weizmann dataset among all the algorithms. Meanwhile, Subspace-KNN attained an accuracy of 99.94%, specifically in terms of specificity, for the KTH dataset, as indicated in Table 4 .

In Experiment 3, we observed that cubic-SVM achieved the highest sensitivity of 99.84% for the Weizmann dataset among all the algorithms. Meanwhile, Fine-KNN attained an accuracy of 99.93%, specifically in terms of specificity, for the KTH dataset, as indicated in Table 4 .

LS = Linear-SVM, CS = Cubic-SVM, CT = Complex Tree, FK = Fine-KNN, SK = Subspace-KNN.

Table 5 compares the previously implemented algorithms and the proposed algorithm. The table provides a clearer understanding of the performance metrics and highlights the proposed algorithm’s superiority. The basis for this conclusion is derived from the discussion that follows.

The proposed algorithm distinguishes itself by incorporating three distinct feature extractors. This strategic combination yields a notable improvement in accuracy compared to the algorithms that were previously implemented. The enhanced accuracy is a result of the synergistic effect created by the integration of these feature extractors, which collectively contribute to a more robust and effective algorithm. In summary, the proposed algorithm surpasses its predecessors in terms of accuracy, making it a promising advancement in the field. The utilization of multiple feature extractors enhances the algorithm’s ability to capture and leverage diverse information, leading to improved performance in comparison to existing methods. This reinforces the significance of the proposed approach and its potential for applications requiring high-precision and reliable results.