Algorithm 2 illustrates the steps to create and update the hash table which is used in RAN-LSH learning algorithm (Lines 4, 8 and 17 in Algorithm 1). Each subregion is allocated to an entry in a hash table, where each entry is composed of five items: hash value, prototype, margin flag, outlier flag and the occurrence frequency (Figure 1: top right). The index or a hash value is used as a key to find a matched entry of a similar item which has been registered previously in hash table (see the first condition in

Line 6). Hash values are a set of hash functions, which are given as follow:

where and are the upper and lower values of projections on the th eigenvector, respectively. Here, is the number of partitions which determines the granularity for a projection.

The next item is prototype. A prototype is the mean vector of all data allocated to each entry (Lines 7 and 11) which is calculated as follows:

where and are the previous prototype which is registered in the entry and training data that belong to th subset of unique hash values, respectively. Meanwhile, is the previous occurrence frequency of the entry and is the number of training data that has a similar hash value. is regarded as the representative point of the subregion.

The third item is the margin flag and it is calculated as follows (Line 12):

where the output margin is given by subtracting the second largest network output from the largest network output as follows:

The fourth item is the outlier flag which is determined as follows (Line 12):

The last item is the occurrence frequency of similar data in an entry. Whenever a new training data is assigned to an entry, the occurrence frequency is increased by one. The details of margin flag and outlier

flag are discussed in the following Section 2.2 and 3.4, respectively.

When a large number of data are given simultaneously under an incremental learning environment, it is impor- tant to learn only essential data in a classifier model. Obviously, this is because the learning must be completed as quickly as possible; otherwise, the next data may be given before the learning is completed. In RAN-LSH, the data selection is conducted by using LSH. First, all training data in a given training set are projected to eigenvectors. Then, for each training data, the projection value is encoded into a hash code whose granularity is determined by the number of partitions, and the obtained hash codes are transformed into a hash value.

If a matched entry with the same hash value is found and the margin flag is “1”, it means the classifier is well trained (Line 6 in Algorithm 2). Then, the mean vector is calculated and the occurrence frequency is incremented by. Otherwise (Line 10 in Algorithm 2), the output margin in Equation (5) is calculated

and the margin flag and outlier flag are updated by Equations (4) and (6), respectively. Note that the training data associated with the margin flag are eliminated from the training set (see Algorithm 1 Lines 12 - 14). On the contrary, if the margin flag is “0”, it means a given data should be trained. After the learning phase, the margin flag would be updated. Nevertheless, updating the margin flag of every prototype in the hash table would increase the learning time. As mentioned before, the prototype with

means the classifier is “well-trained” around the prototype. Thus, this prototype does not need to be updated. Meanwhile, prototype with should be updated because there would probably be regions that have become “well-trained” after the learning phase (Line 17 in Algorithm 1 and Line 12 in Algorithm 2).

LSH is also used to find RBF bases near to the training data (Lines 8 - 13 in Algorithm 3). In RAN-LSH, only the connection weights connected to the selected RBF bases are updated in the following procedures. Firstly, the hash values of RBF bases are retrieved from a hash table. Next, the LSH distance for each th RBFs is calculated as follows:

Then, only RBF bases whose LSH distance is less than a threshold are selected for a learning purpose. This is because it is considered that if the LSH distance is large, the RBF output would become very small and the weight update could be negligible. Finally, the selected RBF bases are used to solve the linear equation in [18] .

Let the number of inputs, RBF units, and outputs be, , and, respectively. RBF outputs and the network outputs of inputs are calculated as follows:

where, , and are the center of th RBF unit, the variance of the th RBF unit, the connection weight from the th RBF unit to the th RBF unit and the bias, respectively.

Algorithm 3 shows the learning algorithm of RAN classifier. In RAN-LSH, RBF centers are not trained but selected based on the output error. If the output error is large, it indicates that a new RBF unit should be added (Lines 5 and 22). As mentioned above, only connection weights for active RBF units are updated (Lines 15 - 17).

Figure 1(a) illustrates the process of obtaining double bounce emails. Let us consider a case that a spammer sends a large number of emails. In many cases, almost all emails will reach existing users. However, it is very likely that some email addresses are no longer in use for some reason. Therefore, the email server would return such emails with unknown addresses to the sender. If the spammer intends to send a malicious email, it is also likely that the spammer has faked the originating address and such emails would be re-sent to the receiver. This type of unreachable error email is called “double-bounce email” [7] and they are usually disposed of by the email server on the receiver’s side. We utilize this mechanism of generating double-bounce emails to collect malicious spam emails automatically.

To use double-bounce emails as training data under the supervised learning, we would need their class labels. Needless to say, spammers try to conceal their malicious intention; therefore, it is not easy to determine the maliciousness from the collected double-bounce emails. The only way to identify the maliciousness is by click- ing the URLs. Evidently, this is very dangerous for general users; therefore, we use a crawling-type web mali- ciousness analyzer called SPIKE, which was developed by the National Institute of Information and Communi- cations Technology (NICT) in Japan.

Figure 2 illustrates how the maliciousness of URLs in a spam email is analyzed in SPIKE [21] . The URL links in the email are first extracted from a double-bounce email and SPIKE downloads the html file and attached materials (e.g., java scripts, pdf, doc files) in the entrance page. It then continues to find other URLs in the downloaded pages again. This process is conducted recursively by crawling the linked websites, and all the downloaded materials are analyzed. Emails that are only link to a normal webpage with non-malicious contents are considered as non-malicious spam emails (i.e., all contents of Webpage 1 - 6 in Figure 2 are normal), whereas the emails with at least one suspicious content (i.e., one of Webpage 1 - 6 in Figure 2 is malicious) are identified as malicious spam emails.

In order for the classifier to carry out the classification task effectively, the classifier requires instances as the input instead of the raw spam emails for the learning purpose. The instances consist of informative features with a fixed-length which are extracted from the emails. Thus, appropriate pre-processing steps are required so that the arbitrary data of text messages are transformed into features with numerical features. Figure 1(b) demons- trates the pre-processing module of the spam email detection system. Feature extraction of spam emails involves tokenizing and lemmatizing the documents into bag-of-words (BoW). Tokenization breaks the sentences in the emails into pieces of words and removes frequent words called stop words such as “the”, “which”, “are”, etc. Besides filtering out stop words, lemmatization also reduces the number of words in BoW by transforming redundant words that end with “ing”, “ed” and “s” into their root word (e.g., “learned” to “learn”).

The BoW features usually consist thousands or millions of feature vectors. In general, only some features are informative and are able to differentiate different classes. Therefore, feature selection is carried out to select the most informative features in order to reduce the number of dimensions and avoid the computational complexity. Firstly, the initial training data are transformed into feature vectors with term frequency-inverse document fre-

quency (TF-IDF) feature representation. Next, linear Support Vector Machines (SVM) is used as a feature selection strategy that requires two steps; training using linear SVM [22] and eliminating features with low weights [23] . SVM is able to find a decision boundary by optimizing the following objective function:

which maximizes the margin of hyperplane between two classes and contains

only a minimum training error (i.e., training data located above the support vectors which belong to the class of the training data). The parameter controls the trade off between margin maximization and errors of the SVM on training data, where a larger corresponds to a higher penalty to errors. The weights obtained is used to select number of features by choosing the highest -rank weights.

To represent the selected features of initial training data and the remaining training data, the normalized TF-IDF [24] is used to measure the importance of a word to a document (i.e., document refers to the spam email) in the collection of documents given by the following equation:

where, , and are the frequency of term in a document, the total

frequencies of all terms in document, the total number of document in corpus and number of documents which have term, respectively. The normalized term frequency (TF) is used to provide a balanced value to all documents that have a different number of words. If term appears frequently in a document and seldom occurs in other documents in, the value of TF-IDF would be high where both TF and inverse document frequency (IDF) obtain high values. This indicates that term is important to document. Otherwise, if either the occurrence of term is low in document or term is always appears in other documents in, this would indicate that term is not important to document where the value of TF-IDF is low or “0”.

After going through the entire procedure above, these data are used as the input to the classifier model. The details of the classifier model are discussed in the previous Section 2.2.

Although SPIKE can judge the maliciousness of spam emails, the analysis takes time, from a few minutes to even longer than ten minutes. Therefore, it is difficult to check all the collected double-bounce emails by SPIKE in real time. We introduce the outlier detection mechanism into RAN-LSH in order to reduce the number of spam emails to be checked by SPIKE. That is, only a new type of unknown spam emails (i.e., outlier) should be selected and sent to SPIKE for labeling. For this purpose, we propose a spam email detection system by combining RAN-LSH classifier [15] and SPIKE, so that the learning time is accelerated compared to when using SPIKE alone. In this study, we detect an outlier based on the output margin, outlier threshold and the occurrence frequency threshold. The data with low output margins are considered as unknown emails for the current classifier and thus should be categorized as outlier. In addition, the number of similar data in each entry is also important to decide whether the data is outlier or not. We assume that the data that do not frequently occurred (i.e., data allocated to an entry with small) can also be categorized as outlier although the output margins are slightly higher than. The outlier flag is calculated using Equation (6). The algorithm of the outlier detection is summarized in Algorithm 4.

The detection performance is evaluated under incremental learning settings to study the following effects: 1) the effect of threshold parameters and 2) the effectiveness of daily updates.

In the former experiment, we investigate the effects of the following three threshold parameters to the perfor- mance: threshold of accumulation ratio, that of output margin, and that of tolerant distance. In the latter experiment, we study the effect of incremental learning through comparison with the batch learning scheme. Figure 3 illustrates how labeled spam emails are trained in (a) batch learning scheme and (b) incremen- tal learning scheme. In the batch learning scheme, we adopt the conventional RBF network (RBFN) (i.e., RBFN usually used as batch learning [25] ) as a classifier and a sliding window is introduced to define a data set to be trained every day. In this experiment, the time-window size is preliminarily determined as 12 days via the cross- validation using the spam emails collected during a different period. Therefore, as seen in Figure 3(a), the first learning stage is carried out on Day 12 using a set of spam emails collected from Day 1 to Day 12, and the data set from Day 13 is used to test the performance. Then, the time-window is shifted by one day at the second learning stage; that is, a set of spam emails collected from Day 2 to Day 13 is used for training, and the data set from Day 14 is used to test. Note that RBFN is retrained with a set of 12-day spam emails at every learning stage in a batch mode. On the other hand, in the incremental learning scheme, batch learning is first applied to an initial data set, which is composed of spam emails collected in the first 12 days. After that, this initial detection model is updated daily using one day training data and the system is tested using the next day data. A set of spam emails collected is forwarded to SPIKE to get the labels on maliciousness, and the pairs of a spam email and its class label are used as a training set on the next day. The number of collected spam emails is different every day. Their maximum, minimum, and average numbers are 756, 26, and 207, respectively.

In this detection system, three parameters are determined empirically. The parameters are accumulation ratio, output margin and tolerant distance. In the following experiment, the parameters are set to:, , and. Meanwhile, the error threshold is set to 0.5. The other parameters, RBF width, the number of partitions, errors penalty of SVM, outlier threshold, occurrence frequency threshold and time-window size are determined in the initial learning phase through the cross-validation. There are 20,448 double bounce emails with 8334 malicious spam emails and 12,114 non-malicious spam emails used in this study that were collected from 1st March 2013 to 10th May 2013.

It is important for the proposed system to correctly classify not only positive examples (malicious spam emails) but also negative ones (non-malicious spam emails). Therefore, in this study, three evaluations are used which are: recall rate, precision rate and F1 measure rate. The actual class labels are the class labels given by SPIKE, whilst the prediction labels are obtained by the detection system. The recall rate and the precision rate measure the ability of the detection system to classify the malicious spam emails (positive samples) correctly which takes into account different types of error. On the other hand, the F1 measure is the harmonic mean of recall and precision rate. The recall rate considers type II error (i.e., a malicious spam email is wrongly classi- fied as non-malicious spam email), whereas the precision rate considers type I error (i.e., a non-malicious spam email is wrongly classified as malicious spam email). If the malicious spam email detection system obtained a low recall rate, the users are exposed to the danger of malware infection because some users may click an URL that leads to malicious websites. Such a misclassification must be avoided in any cases. In the second situation, there is low risk of the malware attack. The system only gives strict conditions where most of the non-malicious spam emails are categorized as malicious spam emails. Therefore, to design a good malicious spam email detec- tion system, it is crucial to have at least high percentage of recall rate to reduce the risk of malware attack.

However, if the malicious spam email detection system is able to obtain high rate of both recall and precision, we can say that the developed system is nearly a perfect detection system.

First, let us examine the threshold parameters and their effect to the detection system. Here, we study the influence of, and so that the parameters are optimized to ensure fast learning property of the detection system while having low misclassification rate. The first parameter is the threshold of accumulation ratio. If is set to a large value, an eigenspace to define hash functions has high-dimensions. Thus, the length of a hash code becomes long, resulting in the enlargement of a hash table. Therefore the searching of similar data would require a longer time since there are many hash values registered in hash table. The next parameter is the output margin threshold. This parameter controls the amount of selected data to be learned by RAN-LSH. As the value of is set to a higher value, the representation of the “not well-learned” region would become wider and the number of selected data is increased in the incremental learning phase, resulting in slower learning. The third important parameter is the tolerant distance which determines the distance of near RBF bases. By updating the weights of only near RBF bases (i.e., using small value of), the time needed to solve the linear equation using Singular Value Decomposition (SVD) is shorten, thus the learning time would be accelerated.

To determine an appropriate value of each parameter, the cross-validation is performed for the initial training set, and the obtained parameter values are fixed over the incremental phase. Table 1(a) and Table 1(b) show the F1 measure and the learning time, respectively, using several combination values of the accumulation ratio and output margin. As seen in Table 1(a), the highest F1 measure is obtained when and. For output margin, F1 measure does not differ much from the F1 measure with output margin. This result is not surprising because high value of output margins represents that the data are “well-learned” or correctly classified. By adding these data, it is expected that the classification rate would not improve although the number of selected data and RBF bases created are increased. As a result for learning using more training data shown by a higher, the learning time would also increase. We assume that the data which are located on the border of different classes should have network outputs in the range of “0.4” to “0.6”.

Therefore, the output margins can be estimated to be between “0” to “0.2” where the data with output margins in these range are important to be learned to reduce the misclassification rate.

On the other hand, Table 2 demonstrates the suitable value of the tolerant distance using appropriate value of and obtained previously which are “0.9” and “0.2”, respectively. Tolerant distance also give influence to the classification rate and the speed of the detection system by controlling the distance which defines the area of near RBF bases. As we can see, the suitable value for the tolerant distance is 2. It means that only the RBF centers that differ from the given training data at two projection vector are used to update the weight. If the is too small, it indicates that the area of selected RBF bases is not enough to approximate the weights correctly. Whereas for that is too large, it would be similar to the approach of updating weights using all RBF bases. Thus, the size of RBF outputs in would be bigger and therefore, the decomposition steps using SVD would require a longer time. Even though the results show the evaluation performance during initial learning, we expect a similar result from the incremental learning phase. It is because parameter, and are also required during the incremental learning phase. For the next experiment, we set the value of, and to be “0.2”, “0.9” and “2”, respectively.

All learning parts in the detection system including pre-processing and classifier module are very crucial which give effect to the performance result. In this experiment, we compare the performance of the proposed online detection system with different learning scheme and classifier model to see the competency of the proposed method. Figure 4 and Figure 5 show the recall rate and precision rate for the detection system with the follow- ing three combinations of classifiers and learning schemes: RBFN (batch learning), RAN (incremental learning), and RAN-LSH (incremental learning) (see Figure 3). The batch learning is carried out using 12-days of training data and it is retrained incrementally. While for the incremental learning, the classifier is updated incrementally using 1-day of training data. As seen in Figure 4 and Figure 5, the proposed one-pass learning of the detection system is capable to learn and carry out the classification task effectively since our proposed system obtained almost the same classification rate as the memory-based learning approach (i.e., batch learning). In fact, our proposed method does not need large memory size to store the training data compared to the memory-based learning. In this study, 12-days length of window size is used for the batch learning to learn incrementally, whereas for incremental learning, only 1-day data set is used as training data. Besides that, we also compare the performance of conventional classifier RAN using the same incremental learning scheme. Our previous study in [15] shows that RAN-LSH can learn fast. As seen in Figure 4 and Figure 5, our detection system obtains a comparable result against the conventional classifier model.

The overall performance is evaluated by averaging over the whole incremental learning phase. The recall rate, precision rate and F1-measure for the three learning models are summarized in Table 3. From Table 3, we can see that the proposed detection system can learn 482 times faster than the conventional RBFN model, and can learn 46 times faster than RAN. This is because our detection system can find a set of similar data in a given training set very quickly using LSH; thus, only a data set falling in an untrained region are selected to learn and the others are discarded. In addition, only near RBF centers are used to update the connection weights. The recall rate and F1 measure of RBFN are higher than those of the other incremental models because this model keeps a large number of data for training; where this requires high computational costs and large memory. Since

the classification rate of recall rate, precision rate and F1 measure do not differ much from the other model, we can conclude that the proposed system is able to update efficiently and able to give class label of the incoming emails within a short time.