Attribute reduction is a hot topic in rough set research. As an extension of rough sets, neighborhood rough sets can effectively solve the problem of information loss after data discretization. However, traditional greedy-based neighborhood rough set attribute reduction algorithms have a high computational complexity and long processing time. In this paper, a novel attribute reduction algorithm based on attribute importance is proposed. By using conditional information, the attribute reduction problem in neighborhood rough sets is discussed, and the importance of attributes is measured by conditional information gain. The algorithm iteratively removes the attribute with the lowest importance, thus achieving the goal of attribute reduction. Six groups of UCI datasets are selected, and the proposed algorithm SAR is compared with L2-ELM, LapTELM, CTSVM, and TBSVM classifiers. The results demonstrate that SAR can effectively improve the time consumption and accuracy issues in attribute reduction.
Rough set theory [
Attribute reduction [
The attribute reduction algorithm proposed in literature [
To facilitate the research work, this section introduces the basic knowledge of neighborhood rough sets, attribute reduction methods, and other related concepts.
The classical rough set must be discretized before processing the actual sample data, in which the integrity of the data and the authenticity of the discretization results cannot be guaranteed. Neighborhood rough sets, as an extension of rough sets, can handle continuous or mixed-type data in real-life situations and effectively address the problem of information loss after data discretization, which is a common issue in classical rough set models.
Let S = { U , A T = A ∪ D , δ } be a neighborhood rough set decision system, where U is the set of all objects, A is the set of conditional attributes, and D is the set of decision attributes, with A ∩ D = ∅ .
Definition 1 [
δ ( x i ) = { x ∈ U | d ( x , x i ) ≤ δ } (1)
Here, δ ≥ 0 , d ( x , x i ) represents the distance between x and xi, and common distance calculation methods include Manhattan distance, Euclidean distance, and norm distance functions. In this paper, the Euclidean distance function is used to calculate d ( x , x i ) . The δ-neighborhood information granule, denoted as δ ( x i ) , δ-neighborhood granule, is denoted as xi neighborhood granule.
Property 1: Given a decision system S and a neighborhood radius δ, for any subset B ⊆ A , of conditional attributes, if δ 1 < δ 2 ,then δ 1 ( x ) ⊆ δ 2 ( x ) .
Proof: For any ∀ x i ∈ δ 1 ( x ) , we have d ( x , x i ) < δ 1 < δ 2 . If d ( x , x i ) < δ 1 , then δ 1 ( x ) ⊆ δ 2 ( x ) .
Definition 2 [
R _ B ( D ) = ∪ i = 1 n R _ B ( X i ) R ¯ B ( D ) = ∪ i = 1 n R ¯ B ( X i ) (2)
where:
R _ B ( X ) = { x i ∈ U | δ ( x i ) ⊆ X i } R ¯ B ( X ) = { x i ∈ U | δ ( x i ) ∩ X i ≠ ∅ } (3)
The boundary of the decision system is defined as:
B N ( D ) = R ¯ B ( D ) − R _ B ( D ) (4)
The positive region and negative region of the neighborhood decision system are defined as:
P o s B ( D ) = R _ B ( D ) , N e g B ( D ) = U − R ¯ B ( X ) (5)
The dependency degree of decision attribute D on conditional attribute B is defined as:
γ B ( D ) = | P o s B ( D ) | | U | (6)
Property 2: Given a decision system S and a neighborhood radius δ, for any subset B ⊆ A of conditional attributes, the following properties hold:
1) For any ∀ B 1 ⊆ B 2 , R δ _ B 1 ( X ) ⊇ R δ _ B 2 ( X ) ;
2) If δ 1 < δ 2 , then P o s B δ 2 ⊂ P o s B δ 1 ;
3) For any B 1 ⊆ B 2 ⊆ ⋯ ⊆ A , have P o s B 1 ( D ) ⊆ P o s B 2 ( D ) ⊆ ⋯ ⊆ P o s B A ( D ) , then γ B 1 ≤ γ B 2 ≤ ⋯ ≤ γ B A .
Attribute reduction is a core problem in rough set theory research. Its core idea is to remove redundant attributes while maintaining the classification ability of the information system, thereby extracting key attributes, and simplifying the information system. Depending on the measurement method of attribute importance, commonly used measurement criteria for attribute reduction include attribute dependency-based and conditional information entropy-based attribute reduction.
Definition 3 [
I N D ( B ) = { ( x , y ) ∈ U × U | ∀ a ∈ B , f ( x , a ) = f ( y , a ) } (7)
Here, x and y are any two samples on the domain U, and the indistinguishable relationship determined by the attribute subset can be seen as dividing the domain U into multiple partitions, denoted as U / I N D ( B ) .
Definition 4 [
1) G is independent, meaning that every element in G is indispensable.
2) I N D ( D ) = I N D ( P ) , meaning that it does not affect the partition of the knowledge base.
Then G is called a reduction of P, denoted as G ∈ R e d ( P ) . Here, R e d ( P ) represents the set of all reductions of P. If G satisfies the following condition:
I N D ( P − { G } ) ≠ I N D ( P ) (8)
Then G is called indispensable in P. The set of all indispensable knowledge sets in P is called the core of P, denoted as C o r e ( P ) .
Definition 5 [
1) γ B ( D ) = γ A ( D ) , meaning P o s B ( D ) − P o s A ( D ) ;
2) For any ∀ a ∈ B , γ B ( D ) > γ B − { a } ( D ) .
Then, attribute subset B is called a relative reduction of the conditional attribute set A. To represent attribute reduction based on dependency functions using information measures, Hu defined the notion of information droplets in decision systems.
Definition 6 [
H ( D | B ) = − 1 n ∑ i = 1 n log | [ x i ] B ∩ [ x i ] D | | [ x i ] B | (9)
Here, [ x i ] B represents the fuzzy equivalence class, [ x i ] D represents the decision class. For ∀ a ∈ B , B is a dependency-based reduction if and only if H ( D | B ) = H ( D | A ) and H ( D | B ) > H ( D | B − { a } ) .
As an effective measure of uncertainty, information entropy can be improved and optimized on the basis of combining with classical rough sets, which can deal with mixed data effectively. During the attribute reduction process based on attribute importance, the conditional information entropy is considered as a measure of attribute importance, which can effectively enhance the discriminative power of attributes. The larger the conditional information entropy, the higher the attribute importance in the neighborhood rough set, and vice versa.
Definition 7: Given a neighborhood decision system S, for ∀ B ⊆ A , if H ( D | B ) = H ( D | A ) and H ( D | B ) > H ( D | B − { a } ) , then the conditional attribute subset B is a reduction of the conditional attribute set A. B 1 , B 2 , ⋯ , B n are all the possible reductions of A.
Definition 8: In a neighborhood rough set decision system S = { U , A T = A ∪ D , δ } , for ∀ B ∈ A , the conditional information entropy with respect to the conditional attribute B is defined as:
H ( B ) = − 1 n ∑ i = 1 n log 1 δ B ( x i ) (10)
Definition 9: In the neighborhood rough set decision system S = { U , A T = A ∪ D , δ } , for any decision attribute set B, the process of selecting a subset B that has the same discernibility effect on objects as the attribute set AT is called attribute reduction. The conditional information entropy of D with respect to B is defined as follows:
H ( D | B ) = − 1 n ∑ i = 1 n log δ B ( x i ) ∩ [ x i ] D δ B ( x i ) (11)
Definition 10: Given a neighborhood decision system S, for ∀ a ⊆ B , the attribute importance of a with respect to B is defined as follows:
S i g ( a , B , D ) = H ( D | B ) − H ( D | B ∪ { a } ) (12)
Conclusion 1: In a neighborhood rough set decision system S = { U , A T = A ∪ D , δ } , for any decision attribute sets B, if B 1 ⊆ B 2 , then H ( D | B 1 ) ≤ H ( D | B 2 ) .
Proof: Since B 1 ⊆ B 2 , then δ B 1 ( x i ) ⊇ δ B 2 ( x i ) . It follows from Equation (11) that H ( D | B 1 ) − H ( D | B 2 ) ≤ 0 , which implies H ( D | B 1 ) ≤ H ( D | B 2 ) .
Conclusion 2: In a neighborhood rough set decision system S = { U , A T = A ∪ D , δ } , for ∀ a ⊆ B , the attribute importance of a with respect to B, denoted as S i g ( a , B , D ) , is higher when the conditional information entropy H ( D | B ) is larger. Conversely, when H ( D | B ) is smaller, the attribute a is considered less important.
Example 1: Given a neighborhood decision information system S = { U , A T = A ∪ D , δ } , you can see this table (
According to the decision attribute D, the domain U can be divided into D 1 = { x 1 , x 5 } , D 2 = { x 2 , x 4 } , and D 3 = { x 3 , x 6 } . The conditional information entropy is calculated as follows:
H ( D | B ) = − 1 n ∑ i = 1 n log δ B ( x i ) ∩ [ x i ] D δ B ( x i ) = 3.2749 (13)
Then H ( D | B ∪ { a 1 } ) = 2.5874 , H ( D | B ∪ { a 2 } ) = 2.3854 , H ( D | B ∪ { a 3 } ) = 3.1028 . The formula S i g ( a , B , D ) = H ( D | B ) − H ( D | B ∪ { a } ) has been used to determine the attribute importance S i g ( a 1 , B , D ) = 0.6875 , S i g ( a 2 , B , D ) = 0.8895 , S i g ( a 3 , B , D ) = 0.1721 of conditional attributes a 1 , a 2 , a 3 . Therefore, based on the attribute importance values, the order of importance is a 2 > a 1 > a 3 , and attribute a 3 can be considered as a redundant attribute in the reduction process.
The traditional neighborhood rough set attribute reduction algorithm uses greedy search strategy to select attributes, and adds new candidate condition attributes with cycles to produce new attribute subsets. However, this method requires many cycles and has high time complexity. To address neighborhood rough set attribute sets, the attribute reduction algorithm based on attribute importance is proposed, utilizing conditional information entropy as a measure of importance. The algorithm uses forward search and heuristic information, and the detailed description is as follows:
| U | a1 | a2 | a3 | d |
|---|---|---|---|---|
| x1 | 0.20 | 0.25 | 0.25 | 1 |
| x2 | 0.40 | 0.40 | 0.50 | 2 |
| x3 | 0.30 | 0.25 | 0.45 | 3 |
| x4 | 0.45 | 0.30 | 0.50 | 2 |
| x5 | 0.45 | 0.50 | 0.55 | 1 |
| x6 | 0.60 | 0.45 | 0.55 | 3 |
Algorithm 1: Attribute Reduction Method for Neighborhood Rough Sets Based on Attribute Importance (SAR).
Input: Decision information system S = { U , A T = A ∪ D , δ } , redius δ.
Output: Reduction set B
Step 1: Set B = ∅ .
Step 2: Calculate the conditional information entropy H ( D ) | B for the entire neighborhood decision system.
Step 3: DO
1) Calculate the conditional information entropy H ( D | B ∪ { a i } ) for each conditional attribute a i ( i = 1 , 2 , ⋯ , n ) ;
2) Calculate the attribute importance of a i , S i g ( a i , B , D ) = H ( D | B ) − H ( D | B ∪ { a i } ) ;
3) Select the attribute with the highest attribute importance S i g ( a i , B , D ) = max ( a i ) ;
4) If S i g ( a i , B , D ) > S i g ( B , D ) , add a i to the reduction set B and go to Step 3. Otherwise, go to Step 4.
Step 4: Output the reduction result.
This paper utilizes conditional information entropy to calculate the attribute importance. In the reduction process, attribute importance is calculated only once. The algorithm improves efficiency and reduces computational complexity when dealing with datasets with a large number of attributes, while also enhancing the accuracy of reduction.
To validate the effectiveness of the proposed SAR algorithm, 8 UCI datasets were selected for experimentation. You can see this table (
A 10-fold cross-validation method was employed in the experiments. Ten different radii were selected: 0.02,0.04, ⋯ ,0.2 . The data was divided into 10 subsets, with each subset used as the test set while the remaining 9 subsets were used as the training set in a rotating manner. The average classification accuracy of the 10 experiments was calculated as the performance metric for the attributes. The SAR algorithm proposed in this paper was compared with L2-ELM, LapTELM, CTSVM, and TBSVM classifiers in terms of classifying the test set samples.
Initially, 100 artificial datasets were divided into positive and negative classes, represented by + and *, respectively. To examine the impact of outliers on the classification performance, 6 outliers were introduced into the artificial datasets, with an equal distribution of 3 positive and 3 negative outliers; you can see this figure (
| No. | Dataset Name | Samples | Conditional Attributes |
|---|---|---|---|
| 1 | WDBC | 569 | 30 |
| 2 | Vote | 432 | 16 |
| 3 | QSAR | 1055 | 41 |
| 4 | Cancer | 699 | 9 |
| 5 | spectf | 267 | 44 |
| 6 | Pima | 768 | 8 |
During the experiment, the time consumption for attribute reduction was recorded for SAR, L2-ELM, LapTELM, CTSVM, and TBSVM classifiers. The comparison results are shown in the following table, you can see this table (
To demonstrate the effectiveness of SAR in terms of classification accuracy, SAR was compared with L2-ELM, LapTELM, CTSVM, and TBSVM classifiers on the test set samples. The experimental results are as follows, you can see this figure (
The experimental results show that SAR achieves higher classification accuracy than L2-ELM, LapTELM, CTSVM, and TBSVM classifiers for most radii. Therefore, SAR can effectively improve the classification accuracy of attribute reduction. To further validate the advantages of SAR, experiments were conducted on artificial datasets using the five algorithms. You can see this figure (
| L2-ELM | LapTELM | CTSVM | TBSVM | S3AR | |
|---|---|---|---|---|---|
| WDBC | 1.545 | 2.555 | 3.114 | 1.864 | 0.451 |
| Vote | 1.478 | 5.607 | 2.501 | 6.867 | 0.455 |
| QSAR | 3.101 | 4.523 | 7.060 | 8.302 | 3.457 |
| Cancer | 4.274 | 5.881 | 3.151 | 7.359 | 1.274 |
| spectf | 0.991 | 4.388 | 1.582 | 5.539 | 1.522 |
| Pima | 1.967 | 4.857 | 3.810 | 2.689 | 0.699 |
From the above figure, it can be observed that SAR still exhibits good classification performance on artificial datasets with outliers. To better illustrate the stability of SAR in the classification process, experiments were conducted to analyze the variation rate of SAR, L2-ELM, LapTELM, CTSVM, and TBSVM classifiers at radii of δ = 0.1 and δ = 0.2 . You can see this figure (
The above experiments provide a comparison of the classification accuracy and precision of the five algorithms on six datasets. Additionally, the variation rates at radii of δ = 0.1 and δ = 0.2 were compared across different datasets. The experimental results indicate that, under the same conditions of algorithm, radius, and metric, SAR performs as well as L2-ELM, LapTELM, CTSVM, and TBSVM classifiers in terms of classification performance.
This paper proposes a neighborhood rough set attribute reduction method based on attribute importance. It utilizes conditional information entropy to calculate attribute importance and obtain the final attribute reduction results. By comparing the time consumption and classification accuracy of SAR, L2-ELM, LapTELM, CTSVM, and TBSVM classifiers, the proposed SAR method shows good classification performance with lower time consumption. Compared with other algorithms, S3AR algorithm has a faster running speed, but the time cost is still very high. In the future, we will focus on how to optimize the algorithm to reduce the time cost.
This work was supported by the National Natural Science Foundation of China under grant numbers 11971210 and 12371459.
The authors declare no conflicts of interest regarding the publication of this paper.
Su, P.Y., Qin, F. and Li, F. (2023) A Neighborhood Rough Set Attribute Reduction Method Based on Attribute Importance. American Journal of Computational Mathematics, 13, 578-593. https://doi.org/10.4236/ajcm.2023.134031