The Federal Office for Economic Affairs and Export Control (BAFA) of Germany promotes digital concepts for increasing energy efficiency as part of the “Pilotprogramm Einsparzähler”. Within this program, Limón GmbH is developing software solutions in cooperation with the University of Kassel to identify efficiency potentials in load profiles by means of automated anomaly detection. Therefore, in this study two strategies for anomaly detection in load profiles are evaluated. To estimate the monthly load profile, strategy 1 uses the artificial neural network LSTM (Long Short-Term Memory), with a data period of one month (1 M) or three months (3 M), and strategy 2 uses the smoothing method PEWMA (Probalistic Exponential Weighted Moving Average). By comparing with original load profile data, residuals or summed residuals of the sequence lengths of two, four, six and eight hours are identified as an anomaly by exceeding a predefined threshold. The thresholds are defined by the Z-Score test, i.e., residuals greater than 2, 2.5 or 3 standard deviations are considered anomalous. Furthermore, the ESD (Extreme Studentized Deviate) test is used to set thresholds by means of three significance level values of 0.05, 0.10 and 0.15, with a maximum of k = 40 iterations. Five load profiles are examined, which were obtained by the cluster method k-Means as a representative sample from all available data sets of the Limón GmbH. The evaluation shows that for strategy 1 a maximum F1-value of 0.4 (1 M) and for all examined companies an average F1-value of maximum 0.24 and standard deviation of 0.09 (1 M) could be achieved for the investigation on single residuals. In variant 3 M the highest F1-value could be achieved with an average F1-value of 0.21 and standard deviation of 0.06 (3 M) for summed residuals of the partial sequence length of four hours. The PEWMA-based strategy 2 did not show a higher anomaly detection efficacy compared to strategy 1 in any of the investigated companies.
Establishing an environmentally compatible and sustainable energy supply is one of the central challenges of our time. For the implementation of the energy transition by 2030 [
Limón GmbH is therefore developing algorithms for the automated analysis of energy data and load profile data in cooperation with the University of Kassel as part of the BAFA [
Anomaly detection has a wide range of applications, such as economics, network traffic, industrial process control, and statistics. Therefore, a number of books and survey papers exist that deal with anomaly detection [
Anomalies can be defined as follows: “Anomalies are patterns in data that do not conform to a well-defined notion of normal behavior” [
Unusual data points or sequences in load profiles can be categorized according to different types. Point anomalies and partial sequence anomalies are relevant to this study, but additional types or further possibilities of systematizing anomalies exist [
In this work, a residual-based anomaly detection method is used for anomaly detection. In a first step, the normal behavior of a load profile is estimated using a mathematical model. In a second step, deviations (residuals) between estimated and observed values are formed and a decision is made whether an observation is anomalous by means of residual analysis.
Different regression models can be created from the known load profile data. Through these models, expected values can be formed and these represent a kind of normal behavior of a load profile, which is anomaly-free in the best case. In order to determine an expected value, a distinction is made between estimation and prediction models. The conceptual distinction between estimation and prediction is based on Blázquez-García et al. (2018) [
The estimation model by LSTM algorithm uses two variants. The first variant uses a period of one month, abbreviated as 1 M in the following, to build the model. This corresponds to the period in which anomalies are to be identified. In the second variant, the model is created with data from three months (3 M), i.e., two previous months and the month to be examined for anomalies are used. The additional analysis using a 3 M estimation model avoids the danger compared to the 1 M estimation model of estimating possible anomalies in the estimation model and thus hiding them.
The estimate based on the PEWMA method uses three smoothing parameters αp = (0.3, 0.6 and 0.9) and are tested with regard to the possibility of successfully identifying anomalies. The larger αp, the more the resulting curve is smoothed. Following Renshaw [
The estimated load profiles are compared with the observed load profiles of a month and the deviations are analyzed (residuals analysis). This analysis is carried out either on the level of individual residuals or based on the summation of successive residuals as partial sequence anomalies. The partial sequences of length two, four, six, eight hours are examined, each being sliding sequence windows. Residuals and residual sequences whose expression exceeded a defined threshold were scored as anomalies. Thresholds are systematically varied by using a generalized Extreme Studentized Deviate (ESD) [
The Precision is defined as the number of true positives (tp) divided by the sum of tp and the number of false positives (fp):
P = t p t p + f p (1)
The Recall R describes the ratio of tp to the sum of tp and false negatives (fn):
R = t p t p + f n (2)
The F1 measure considers both P and R and forms the harmonic mean of both previous metrics:
F 1 = 2 ⋅ P ⋅ R P + R (3)
All metrics output numerical values between 0 and 1. In order to apply them, it is necessary to know the number of data points that have been classified as anomalous or normal by the anomaly detection method. Furthermore, the population of anomalies present in the data set must be known.
Limón GmbH has been able to evaluate data in the form of load profiles from several hundred companies over the last 10 years. This total material was grouped by using the k-Means clustering method to five prototypical load profiles. In order to predefine anomalies, the companies to be investigated were examined for anomalous profiles in expert interviews prior to the evaluation. The anomalies defined by the experts are used as references to determine the efficacy of the anomaly detection strategies.
The predefined anomalies of companies 1 to 4 can be described as temporally contextual anomalies, since the load profiles follow a temporal rhythm, i.e., day-night and weekday-weekend dependencies exist, and unusual profiles exist in the context of the respective point in time. Company 5 (see
It is found that the majority of the companies have more predefined anomalous
sequences than point anomalies. Point anomalies are identified by evaluating the residual values of individual time points. However, it is possible that single residual studies also identify anomalous subsequences. This is because the estimation in the range of anomalous sequences also differs from the given load profile data in part by only single high residual values. Therefore, the investigation of partial sequence anomalies using summed residual values can also identify single point anomalies. To address this difficulty, the following counting method for anomalies is established: 1) Anomalous sequence regions are counted as detected only once if identified more than once. 2) If point anomalies and anomalous sequences occur together in a load profile, then each anomalous sequence counts as one anomaly each.
Estimation using the two different strategies is followed by examination of the single and summed residuals to identify anomalies using threshold testing by Z-Score and ESD. In the following, two exemplary individual results and subsequent overarching results are discussed.
For the analysis of conspicuous single residuals for company 1, the LSTM model (1 M) is estimated and deviations between observed and estimated values are evaluated using the Z-Score and ESD test (see
| Company 1 (1 M) | Precision | Recall | F1 measure | tp | fp | fn |
|---|---|---|---|---|---|---|
| ESD α = 0.05 | 0.17 | 0.33 | 0.22 | 1 | 5 | 2 |
| ESD α = 0.10 | 0.14 | 0.33 | 0.20 | 1 | 6 | 2 |
| ESD α = 0.15 | 0.14 | 0.33 | 0.20 | 1 | 6 | 2 |
| Z-Score > 3 | 0.08 | 0.33 | 0.13 | 1 | 11 | 2 |
| Z-Score > 2.5 | 0.10 | 0.67 | 0.17 | 2 | 19 | 1 |
| Z-Score > 2 | 0.06 | 0.67 | 0.10 | 2 | 34 | 1 |
F1 = 0.22. For less stringent significance levels (α = 0.10 and 0.15), the values deteriorate. For the analysis by Z-Score thresholds, although two of the three reference anomalies can be identified as true positives under the two thresholds of Z = 2.0 and Z = 2.5, at the same time the number of false positive assignments increases considerably to 19 and 34, respectively.
The analysis of the data in Strategy 2 showed that the application of the PEWMA-based approach only seems to make more sense for company 5. The PEWMA method cannot utilize temporal contextual rhythms, such as weekday-weekend dependencies, and these are significantly responsible for the load profiles in companies 1 through 4.
For company 5, the highest anomaly detection quality results in the case of αP = 0.9, i.e., with a very strongly smoothed curve. For two of the three reference anomalies with unusual maximum values, very large residuals arise. Their correct positive identification is thus facilitated (see
Residuals and residual sequences whose expression exceeds a defined threshold are evaluated as anomalies. The thresholds are systematically varied in an ESD test procedure or as a Z-Score test with three test thresholds each. An overarching evaluation of the threshold procedures allows the suitability for the residual-based anomaly detection approach to be assessed.
| αP = 0.3 | αP = 0.6 | αP = 0.9 | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Company 5 | P | R | F1 | P | R | F1 | P | R | F1 |
| Z-Score > 3 | 0.09 | 0.33 | 0.14 | 0.09 | 0.33 | 0.14 | 0.18 | 0.67 | 0.29 |
| Z-Score > 2.5 | 0.05 | 0.33 | 0.09 | 0.11 | 0.67 | 0.19 | 0.12 | 0.67 | 0.20 |
| Z-Score > 2 | 0.03 | 0.33 | 0.05 | 0.06 | 0.67 | 0.10 | 0.08 | 1.00 | 0.14 |
smaller than 0.05. Furthermore, the ESD test identifies anomalies depending on the amount of incoming data points. The high number of incoming data of n = 720, which corresponds to a period of about one month, seems to lead to the ESD test being too insensitive, since the λi value remains very stable with continuous iteration compared to Ri values of the test statistic [
By taking a cross-strategy view, the question of whether the efficacy of anomaly detection depends on the specific load profile of a company can be answered. Companies 1 to 4 have relatively similar load profile characteristics. Comprehensible temporal rhythms exist. Among other things, the experts identified deviations as reference anomalies that did not follow these rhythms. The LSTM network underlying strategy 1 can, in principle, recognize many of these input variables or temporal rhythms thanks to the information from the time stamp and take them into account in the regression model. Consequently, it could be assumed that the LSTM algorithm should be able to estimate the cyclic changes of the power values. In the evaluation, however, it is difficult to estimate individual load peaks in the maximum power range of the day or the slopes of suddenly rising or falling power values at the beginning or end of a core working time. In these areas, high residuals arise when comparing the expected values estimated by the LSTM network with the observed data. These are detected as anomalies in the residual analysis, but do not correspond to the reference anomalies resulting in low F1 measures. However, the strategy based on the LSTM network can detect anomalies much more efficiently than strategy 2 for the first four companies. In contrast, the load profile of company 5 does not show temporal dependencies. Using both strategy 1 and strategy 2, similar anomaly detection qualities have resulted for this company. For the LSTM network, estimation based on the information from the timestamp is challenging, since these hardly influence the load profile. Other influencing factors not known for this work seem to play a role and cannot be used as input variables for the LSTM network. For the PEWMA based strategy 2 it can be observed that the efficacy of the anomaly detection is influenced by the context in which the anomalies are located. The higher the proportion of unusual power peaks or drops in the context of the overall course, the higher the anomaly detection efficacy.
Furthermore, by comparing the cross-company results of strategy 1, it is possible to evaluate the effect of the length of the time period of the data used to estimate an LSTM model (1 M vs. 3 M) on the efficacy of anomaly detection.
A closer look at the data pool of the company from company 4 reveals that two of the previous months used for the 3 M estimate have very different load profiles. Consequently, the resulting estimate does not provide a good basis for identifying the reference anomalies. However, it also shows comparatively low F1 values overall, since there is a high number of false positive assignments
depending on the threshold. Presumably, the use of significantly longer periods for training the LSTM network could serve to improve the anomaly detection efficacy.
By summarizing the results of single residual or residual sequence analysis and comparing them across strategies, the effects on the efficacy of anomaly detection can be evaluated.
A more differentiated picture emerges for variant 3 M when the F1 values are considered. The combination of threshold values of Z > 2.5 or Z > 3 and residual sequences of four and six summed hours (sum 4 & sum 6) shows the highest F1 values in the overall comparison. Consequently, summation can lead to improved anomaly detection quality under certain conditions. In the analysis of the data, not explicitly presented here, using strategy 2, an F1 measure of 0.40 was shown for company 5 with the residual summation of six hours as well as Z values greater than 3. Comparing these values with the strategy 1 analysis of the data of the company from company 5, identically high F1 values are shown.
Considering all the data analyzed and using the Z-Score thresholds, it can be concluded that the use of strategy 1 based on the LSTM network is preferable to the use of strategy 2.
The F1 values obtained from the two strategies are rather low compared to examples from the literature. For example, Hundman et al. (2020) use an LSTM network to identify anomalies in multivariate time series of telemetry data from a spacecraft, where different types of thresholding procedures are tested [
authors report Precision values of up to P = 0.92 in combination with a Recall of R = 0.63. Therefore, this publication suggests that other thresholding techniques should be tested for their usefulness, such as adaptive thresholding techniques. The publication leaves open which characteristics the curve progression of the described telemetry data has. Accordingly, the results cannot be compared with those of this work without exception. The PEWMA method has been successfully applied, for example, to streaming data from Twitter with resulting F1 values of up to 0.80 [
This paper presents the comparison and adaption of two strategies for anomaly detection based on the LSTM network and the PEWMA method. The results show that anomaly can be partial to fully recognized by the chosen approaches previously defined, but with the maximum averaged F1 value of 0.24. In many cases, the presence of a large number of false positive assignments leads to a decrease of F1 values. In further efforts, approaches should be found to minimize the false positive results and thus improve the associated efficacy of the anomaly detection.
The chosen technique of being able to identify anomalies using information from timestamps and associated performance values alone is probably one reason why the F1 values are relatively low. The incorporation of additional factors affecting power values in the sense of a multivariate estimation model can possibly achieve higher anomaly detection efficacy. Furthermore, alternative methods for thresholding, such as adaptive methods, should be tested to determine their suitability. It must be considered that the anomalies identified by the expert interviews are not necessarily exhaustive. Thus, the false positive assignments may well have identified anomalies that were not noticed in the expert interviews. This will be considered further in future studies.
Overall, however, it can already be stated that the use of automated anomaly detection procedures can significantly reduce the search area for several hundred load profiles in a company and thus reduce the effort of manual analysis.
The authors declare no conflicts of interest regarding the publication of this paper.
Krawiec, P., Junge, M. and Hesselbach, J. (2021) Comparison and Adaptation of Two Strategies for Anomaly Detection in Load Profiles Based on Methods from the Fields of Machine Learning and Statistics. Open Journal of Energy Efficiency, 10, 37-49. https://doi.org/10.4236/ojee.2021.102003