Examining a generic streaming architecture for smart manufacturing's Big data processing in Anomaly detection: A review and a proposal
Abstract
The smart manufacturing industry has witnessed a rapid increase in data generation due to the integration of sensors, IoT devices, and other advanced technologies. With this huge amount of data, the need for efficient data processing methods becomes critical for identifying anomalies in real-time. With the rise of Industry 4.0 practices, digitally enabled manufacturing units are shifting their focus towards Smart Manufacturing paradigm for better productivity, throughput and increased business volume. Traditionally digital manufacturing units have considered different AI approaches like Neural Network, Statistical Methods, Deep Learning etc. to detect and predict anomalies in their production lines. But with the Smart Manufacturing ecosystem, a manufacturing unit must integrate manufacturing intelligence in real-time across entire production lines through sensor data of IOT devices. Hence the traditional anomaly detection systems fall short to respond well, under the changed scenario, where large volumes of unstructured and varied types of data are being generated at high velocity, to be processed at (soft) real time. The article reviews the current state-of-the-art in big data processing for anomaly detection in smart manufacturing. The review covers various aspects such as data collection, data processing, anomaly detection, and real-time monitoring. The current paper also proposes a novel stateful data streaming computational model for big data processing in smart manufacturing units which conceptually lays the foundation on top of which any discrete anomaly detection engine would be able to work. The proposed architecture has several benefits, including its ability to handle the large volume, velocity, and variety of data generated in smart manufacturing. The architecture can be applied to various smart manufacturing applications, including predictive maintenance, quality control, and supply chain optimization. It is expected that this proposed architecture will pave the way for the development of more efficient and effective smart manufacturing systems in the future.
References
Agrawal, S., & Agrawal, J. (2015). Survey on anomaly detection using data mining techniques. Procedia Comput Sci., 60, 708–713. https://doi.org/10.1016/j.procs.2015.08.220
Apache flink. https://flink.apache.org/flink-architecture.html
Bajo, J., & Rodríguez, J.M.C. (2017). Neural Networks in Distributed Computing and Artificial Intelligence. Neurocomputing, pp. 272. https://doi.org/10.1016/j.neucom.2017.06.022
Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: a survey. ACM Comput. Surv., 41(3), 15. https://doi.org/10.1145/1541880.1541882
Corizzo, R., Ceci, M., & Japkowicz, N. (2019). Anomaly detection and repair for accurate predictions in geo-distributed big data, Big. Data Res., 16, 18–35. https://doi.org/10.1016/j.bdr.2019.04.001
DeLaus, M.D. (2019). Machine Learning for Automated Anomaly Detection in Semiconductor Manufacturing - Master’s thesis at Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology.
Ferreiro, S., Konde, E., Fernández, S., & Prado, A. (2016). Industry 4.0: predictive intelligent maintenance for production equipment. European conference of the Prognostics and Health Management Society, pp. 1–8.
Gökalp, M. O., Kayabay, K., Akyol, M. A., Eren, E. P., & Koçyigit, A. (2016). Big Data for Industry 4.0: A Conceptual Framework. 2016 International Conference on Computational Science and Computational Intelligence (CSCI). IEEE, pp. 431-434. https://doi.org/10.1109/CSCI.2016.0088
Goldstein, M., & Uchida, S. (2016). A comparative evaluation of unsupervised anomaly detection algorithms for multivariate data. PLoS ONE, 11(4), 0152173. https://doi.org/10.1371/journal.pone.0152173
Gölzer, P., Cato, P., & Amberg, M. (2015). Data Processing Requirements of Industry 4.0 - Use Cases for Big Data Applications. Twenty-Third European Conference on Information Systems (ECIS), Münster, Germany, 2015. ECIS 2015 Research-in-Progress Papers, pp. 61. http://aisel.aisnet.org/ecis2015_rip/61
Japa, A., & Shi, Y. (2020). Parallelizing the Bounded K-Nearest Neighbors Algorithm for Distributed Computing Systems”. 2020 10th Annual Computing and Communication Workshop and Conference (CCWC), pp. 0038-0045. https://doi.org/ 10.1109/CCWC47524.2020.9031198
Jindal, N., & Liu, B. (2007). Review spam detection. In: Proceedings of the 16th International Conference on World wide web. ACM., pp. 1189–90. https://doi.org/10.1145/1242572.1242759
Kamat, P., & Sugandhi, R. (2020). Anomaly Detection for Predictive Maintenance in Industry 4.0- A survey. E3S Web of Conferences 2020. https://doi.org/10.1051/e3sconf/202017002007
Latinovic, T., Preradović, D., Barz, C. R., Vadean, A. P., & Todić, M. (2019). Big Data as the Basis for the innovative Development Strategy of the Industry 4.0. IOP Conference Series: Materials Science and Engineering, 477, 012045. https://doi.org/10.1088/1757-899X/477/1/012045
Li, X., Zhang, W., Ding, Q., & Sun, J. Q. (2020). Intelligent rotating machinery fault diagnosis based on deep learning using data augmentation. Journal of Intelligent Manufacturing, 31, 433–452. https://doi.org/10.1007/s10845-018-1456-1
Lindemann, B., Fesenmayr, F., Jazdi, N., & Weyrich, M. (2019). Anomaly Detection in Discrete Manufacturing Using Self-Learning Approaches. Benjamin Lindemann et al. Procedia CIRP, 79, 313–318. https://doi.org/10.1016/j.procir.2019.02.073
Liu, J., Guo, J., Orlik, P.V., Shibata, M., Nakahara, D., Mii, S., & Takac, M. (2018). Anomaly Detection in Manufacturing Systems Using Structured Neural Networks. IEEE World Congress on Intelligent Control and Automation 2018, pp. 1-6. http://dx.doi.org/10.1109/WCICA.2018.8630692
Navia-Vazquez, A., Gutierrez-Gonzalez, D., Parrado-Hernandez, E., & Navarro-Abellan, J.J. (2006). Distributed Support Vector Machines. August 2006 IEEE Transactions on Neural Networks, 17(4), 1091-1097. https://doi.org/10.1109/TNN.2006.875968
O’Donovan, P., Leahy, K., Bruton, K., & Dominic T. J. O. (2015a). Big Data in Manufacturing: A Systematic Mapping Study. Journal of Big Data, 2, 20. https://doi.org/10.1186/s40537-015-0028-x
O’Donovan, P., Leahy, K., Bruton, K., & O’Sullivan D. T. J. (2015b). An Industrial Big Data Pipeline for Data-driven Analytics Maintenance Applications in Large-scale Smart Manufacturing Facilities. Journal of Big Data, 2, 25. https://doi.org/10.1186/s40537-015-0034-z
Parthasarathy, S., Ghoting, A., & Otey, M. E. (2007). A survey of distributed mining of data streams. In: Data streams. Springer, pp. 289–307. https://doi.org/10.1007/978-0-387-47534-9_13
Patcha, A., & Park, J. M. (2007). An overview of anomaly detection techniques: existing solutions and latest technological trends. Comput. Netw., 51(12), 3448–3470. https://doi.org/10.1016/j.comnet.2007.02.001
Phuyal, S., Bista, D., & Bista, R. (2020). Challenges, opportunities and future directions of smart manufacturing: A state of art review, Sustain. Futures, 2. https://doi.org/10.1016/j.sftr.2020.100023 100023
Pittino, F., Puggl, F., Puggl, M., Moldaschl, T., & Hirschl, C. (2020). Automatic Anomaly Detection on In-Production Manufacturing Machines Using Statistical Learning Methods. Sensors, 20, 2344. https://doi.org/10.3390/s20082344
Rivetti, N., Busnel,Y., & Gal, Y. (2017). Grand Challenge: FlinkMan – Anomaly Detection in Manufacturing Equipment with Apache Flink. 11th ACM International Conference on Distributed and Event-based Systems, Jun 2017, Barcelone, Spain. pp. 274-279. https://doi.org/10.1145/3093742.3095099
Spirin, N., & Han, J. (2012). Survey on web spam detection: principles and algorithms. ACM SIGKDD Explor. Newsl., 13(2), 50–64. https://doi.org/10.1145/2207243.2207252
Tamboli, J., & Shukla, M. (2016). A Survey of outlier detection algorithms for data streams. In: Computing for sustainable global development (INDIACom), 2016 3rd International Conference on. IEEE, pp. 3535–3540.
Tang, Z., Chen Z., Bao, Y., & Li, H. (2018). Convolutional neural network‐based data anomaly detection method using multiple information for structural health monitoring. Structural Control Health Monitoring, 26, e2296 http://dx.doi.org/10.1002/stc.2296
Tatu, A., Maaß, F., Färber, I., Bertini, E., Schreck, T., Seidl, T., & Keim, D. (2012). Subspace search and visualization to make sense of alternative clusterings in high-dimensional data. In: Visual analytics science and technology (VAST), 2012 IEEE Conference, pp. 63–72. https://doi.org/10.1109/VAST.2012.6400488
Wang, K. (2016). Intelligent Predictive Maintenance (IPDM) system – Industry 4.0 scenario. WIT Trans Eng. Sci., 113, 259–268. https://doi.org/10.2495/IWAMA150301
Yan, J., Meng, Y., Lu, L., & Li. L. (2017). Industrial big data in an industry 4.0 environment: challenges, schemes, and applications for predictive maintenance. In IEEE Access, 5, 23484-23491. https://doi.org/10.1109/ACCESS.2017.2765544
Zheng, J., Li, J., Liu, C., Wang, J., Li, J., & Liu, H.(2022). Anomaly detection for high dimensional space using deep hypersphere fused with probability approach. Complex & Intelligent Systems, 8, 4205–4220.
https://doi.org/10.1007/s40747-022-00695-9
Zope, K., Singh, K., Nistala S. H., Basak, A., Rathore, P., & Runkana, V. (2019). Anomaly Detection and Diagnosis in Manufacturing Systems: A Comparative Study of Statistical, Machine Learning and Deep Learning Techniques. Annual Conference of the PHM Society, Arizona USA, 2019. https://doi.org/11. 10.36001/phmconf.2019.v11i1.815
Żabiński, T., Mączka, T., Kluska, J., Madera, M., & Sęp, J. (2019). Condition monitoring in Industry 4.0 production systems - the idea of computational intelligence methods application. Procedia CIRP, 79 (3-4), 63-67. https://doi.org/10.1016/j.procir.2019.02.012
Zuo, L., Zhang, H., & Wang, H. (2021). Optimal subsample selection for massive logistic regression with distributed data. Comput. Stat., 36, 2535–2562. https://doi.org/10.1007/s00180-021-01089-0
Copyright (c) 2023 International Academic Publishing House (IAPH)
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
