MTAD RF: Multivariate Time-series Anomaly Detection based on Reconstruction and Forecast

Kenan Qin

¹ School of Compter Science, Shaanxi Normal Unversity, Xi’an, ShaanXi, 710062, China

² Bayero University, Kano, Kano 700241, Nigeria

Anomaly detection in multivariate time series is an important research direction, which helps to improve the security of industrial systems by detecting abnormally unreliable devices. Multivariate time series (MTS) anomalies not only need to pay attention to the time correlation between different time series but also need to consider the abnormal changes in the relationship between different variables. Once the influence relationship between two variables that influence each other is ignored, it will likely lead to false positives or false positives. At the same time, the degree of influence between different time series or different features is also inconsistent, just like what happened recently have radically different influences on the present. Furthermore, most of the existing models are weak in detecting no abnormality. To tackle these issues, in this paper, we propose a new model of multivariate time series anomaly detection based on reconstruction and forecast, named MTAD RF. First, we capture the temporal and feature correlations of MTS through two parallel GAT layers, and at the same time distinguish the influence degree between different time series or different features based on attention coefficients. Second, we leverage the generative power of VAE and the single-step forecast power of MLP to jointly detect known and unknown anomalies based on reconstructed and predicted models. Major practical implications of the proposed approach is missing. Finally, anomalies are detected and explained based on temporal and feature anomaly scores. Experiments demonstrate that our model outperforms current state-of-the-art methods on 4 real-world datasets, with an average F1 score of about 95% and excellent anomaly diagnostic ability.

Anomaly Detection, Multivariate Time-series, Graph Attention Network,Variational AutoEncoder

Kenan Qin, Mengfan Xu, Bello Ahmad Muhammad, and Jing Han (2023). MTAD RF: Multivariate Time-series Anomaly Detection based on Reconstruction and Forecast. Journal of Networking and Network Applications, Volume 3, Issue 1, pp. 45–57. https://doi.org/10.33969/J-NaNA.2023.030105.

[1] G. Sivapalan, K. K. Nundy, S. Dev, B. Cardiff, and D. John, “Annet: A lightweight neural network for ECG anomaly detection in IOT edge sensors,” IEEE Transactions on Biomedical Circuits and Systems, vol. 16, no. 1, pp. 24–35, 2022.

[2] M. Jain, G. Kaur, and V. Saxena, “A K-means clustering and SVM based hybrid concept drift detection technique for network anomaly detection,” Expert Systems with Applications, vol. 193, p. 116510, 2022.

[3] S. G S and R. Balakrishnan, “A statistical-based light-weight anomaly detection framework for Wireless Body Area Networks,” The Computer Journal, vol. 65, no. 7, pp. 1752–1759, 2021.

[4] R. Chalapathy and S. Chawla, “Deep Learning for Anomaly Detection: A Survey,” arXiv.org, 2019, [Online]. Available: https://arxiv.org/abs/1901.03407.

[5] S. Tuli, G. Casale, and N. R. Jennings, “Tranad,” Proceedings of the VLDB Endowment, vol. 15, no. 6, pp. 1201–1214, 2022.

[6] X. Wang, D. Pi, X. Zhang, H. Liu, and C. Guo, “Variational transformer-based anomaly detection approach for multivariate time series,” Mea-surement, vol. 191, p. 110791, 2022.

[7] Y. Su, Y. Zhao, C. Niu, R. Liu, W. Sun, and D. Pei, “Robust anomaly detection for multivariate time series through Stochastic Recurrent Neural Network,” Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019.

[8] D. Park, Y. Hoshi, and C. C. Kemp, “A multimodal anomaly detector for robot-assisted feeding using an LSTM-based variational autoencoder,” IEEE Robotics and Automation Letters, vol. 3, no. 3, pp. 1544–1551, 2018.

[9] J. Li, W. Pedrycz, and I. Jamal, “Multivariate Time Series Anomaly Detection: A framework of Hidden Markov models,” Applied Soft Computing, vol. 60, pp. 229–240, 2017.

[10] K. Hundman, V. Constantinou, C. Laporte, I. Colwell, and T. Soder-strom, “Detecting spacecraft anomalies using lstms and Nonparametric dynamic Thresholding,” Proceedings of the 24th ACM SIGKDD Inter-national Conference on Knowledge Discovery & Data Mining, 2018.

[11] Z. Chen, C. K. Yeo, B. S. Lee, and C. T. Lau, “Autoencoder-based network anomaly detection,” 2018 Wireless Telecommunications Sym-posium (WTS), 2018.

[12] T. Kieu, B. Yang, and C. S. Jensen, “Outlier detection for Multidi-mensional Time Series using Deep Neural Networks,” 2018 19th IEEE International Conference on Mobile Data Management (MDM), 2018.

[13] J. Audibert, P. Michiardi, F. Guyard, S. Marti, and M. A. Zuluaga, “USAD,” Proceedings of the 26th ACM SIGKDD International Con-ference on Knowledge Discovery & Data Mining, 2020.

[14] C. Zhang, D. Song, Y. Chen, X. Feng, C. Lumezanu, W. Cheng, J. Ni, B. Zong, H. Chen, and N. V. Chawla, “A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, pp. 1409–1416, 2019.

[15] M. Munir, S. A. Siddiqui, A. Dengel, and S. Ahmed, “Deepant: A deep learning approach for unsupervised anomaly detection in time series,” IEEE Access, vol. 7, pp. 1991–2005, 2019.

[16] H. Zhao, Y. Wang, J. Duan, C. Huang, D. Cao, Y. Tong, B. Xu, J. Bai,

J. Tong, and Q. Zhang, “Multivariate time-series anomaly detection via graph attention network,” 2020 IEEE International Conference on Data Mining (ICDM), 2020.

[17] T. Cover and P. Hart, “Nearest neighbor Pattern Classification,” IEEE Transactions on Information Theory, vol. 13, no. 1, pp. 21–27, 1967.

[18] I. T. Jolliffe and J. Cadima, “Principal component analysis: A review and recent developments,” Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, vol. 374, no. 2065,

p. 20150202, 2016.

[19] A. Ben-Hur, “Support vector clustering,” Scholarpedia, vol. 3, no. 6, p. 5187, 2008.

[20] R. J. Hyndman and Y. Khandakar, “Automatic time series forecasting: Theforecastpackage forr,” Journal of Statistical Software, vol. 27, no. 3, 2008.

[21] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, 2015.

[22] B. Zong, Q. Song, M. R. Min, W. Cheng, C. Lumezanu, D. Cho, and

H. Chen, “Deep autoencoding gaussian mixture model for unsupervised anomaly detection,” International conference on learning representations, 2018.

[23] P. Veliˇckovi´c, G. Cucurull, A. Casanova, A. Romero, P. Li`o, and Y. Bengio, “Graph attention networks,” arXiv preprint arXiv:1710.10903, 2018.

[24] T. N. Kipf and M. Welling, “Semi-supervised classification with graph Convolutional Networks,”arXiv preprint arXiv:1609.02907, 2017.

[25] D. P. Kingma and M. Welling, “Auto-encoding vari-ational Bayes,” arXiv.org, 2014, [Online]. Available: https://arxiv.org/abs/1312.6114v10..

[26] C. dos Santos and M. Gatti, “Deep convolutional neural networks for sentiment analysis of short texts,” Proceedings of the 25th International Conference on Computational Linguistics, pp. 69–78, 2014.

[27] H. Ren, B. Xu, Y. Wang, C. Yi, C. Huang, X. Kou, T. Xing, M. Yang, J. Tong, and Q. Zhang, “Time-series Anomaly detection service at Microsoft,” Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019.

[28] H. Touvron, P. Bojanowski, M. Caron, M. Cord, A. El-Nouby, E. Grave, G. Izacard, A. Joulin, G. Synnaeve, J. Verbeek, and H. Jegou, “RESMLP: Feedforward Networks for image classification with data-efficient training,” IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1–9, 2022.

[29] K. Kellogg, S. Thurman, W. Edelstein, M. Spencer, G.-S. Chen, M. Underwood, E. Njoku, S. Goodman, and B. Jai, “NASA’s Soil Moisture Active Passive (SMAP) observatory,” 2013 IEEE Aerospace Conference, 2013.

[30] A. P. Mathur and N. O. Tippenhauer, “Swat: A water treatment testbed for research and training on ICS Security,” 2016 International Workshop on Cyber-physical Systems for Smart Water Networks (CySWater), 2016.

[31] K. J¨arvelin and J. Kek¨al¨ainen, “Cumulated gain-based evaluation of IR techniques,” ACM Transactions on Information Systems, vol. 20, no. 4, pp. 422–446, 2002.

[32] J. Tang, M. Qu, M. Wang, M. Zhang, J. Yan, and Q. Mei, “Line,” Proceedings of the 24th International Conference on World Wide Web, 2015.

[33] D. Wang, P. Cui, and W. Zhu, “Structural Deep Network embedding,” Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016.

[34] J. Wang, P. Huang, H. Zhao, Z. Zhang, B. Zhao, and D. L. Lee, “Billion-scale commodity embedding for e-commerce recommendation in Alibaba,” Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2018.