Examining a generic streaming architecture for smart manufacturing's Big data processing in Anomaly detection: A review and a proposal

Milton Samadder; Anup Kumar Barman; Alok Kumar Roy

doi:10.52756/ijerr.2023.v30.019

Authors

Milton Samadder Department of Computer Science & Engineering, Central Institute of Technology, Kokrajhar, Assam-783370, India https://orcid.org/0009-0008-5261-9047
Anup Kumar Barman Department of Computer Science & Engineering Central Institute of Technology, Kokrajhar, Assam-783370, India
Alok Kumar Roy Department of Computer Science & Engineering, Bankura Unnayani Institute of Engineering, Subhankar Nagar, Pohabagan, Bankura, West Bengal, India

DOI:

https://doi.org/10.52756/ijerr.2023.v30.019

Keywords:

Big data processing, data streaming, Industry 4.0, message streaming, smart manufacturing, stateful computation

Abstract

The smart manufacturing industry has witnessed a rapid increase in data generation due to the integration of sensors, IoT devices, and other advanced technologies. With this huge amount of data, the need for efficient data processing methods becomes critical for identifying anomalies in real-time. With the rise of Industry 4.0 practices, digitally enabled manufacturing units are shifting their focus towards Smart Manufacturing paradigm for better productivity, throughput and increased business volume. Traditionally digital manufacturing units have considered different AI approaches like Neural Network, Statistical Methods, Deep Learning etc. to detect and predict anomalies in their production lines. But with the Smart Manufacturing ecosystem, a manufacturing unit must integrate manufacturing intelligence in real-time across entire production lines through sensor data of IOT devices. Hence the traditional anomaly detection systems fall short to respond well, under the changed scenario, where large volumes of unstructured and varied types of data are being generated at high velocity, to be processed at (soft) real time. The article reviews the current state-of-the-art in big data processing for anomaly detection in smart manufacturing. The review covers various aspects such as data collection, data processing, anomaly detection, and real-time monitoring. The current paper also proposes a novel stateful data streaming computational model for big data processing in smart manufacturing units which conceptually lays the foundation on top of which any discrete anomaly detection engine would be able to work. The proposed architecture has several benefits, including its ability to handle the large volume, velocity, and variety of data generated in smart manufacturing. The architecture can be applied to various smart manufacturing applications, including predictive maintenance, quality control, and supply chain optimization. It is expected that this proposed architecture will pave the way for the development of more efficient and effective smart manufacturing systems in the future.

References

Agrawal, S., & Agrawal, J. (2015). Survey on anomaly detection using data mining techniques. Procedia Comput Sci., 60, 708–713. https://doi.org/10.1016/j.procs.2015.08.220

Apache flink. https://flink.apache.org/flink-architecture.html

Bajo, J., & Rodríguez, J.M.C. (2017). Neural Networks in Distributed Computing and Artificial Intelligence. Neurocomputing, pp. 272. https://doi.org/10.1016/j.neucom.2017.06.022

Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: a survey. ACM Comput. Surv., 41(3), 15. https://doi.org/10.1145/1541880.1541882

Corizzo, R., Ceci, M., & Japkowicz, N. (2019). Anomaly detection and repair for accurate predictions in geo-distributed big data, Big. Data Res., 16, 18–35. https://doi.org/10.1016/j.bdr.2019.04.001

DeLaus, M.D. (2019). Machine Learning for Automated Anomaly Detection in Semiconductor Manufacturing - Master’s thesis at Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology.

Ferreiro, S., Konde, E., Fernández, S., & Prado, A. (2016). Industry 4.0: predictive intelligent maintenance for production equipment. European conference of the Prognostics and Health Management Society, pp. 1–8.

Gökalp, M. O., Kayabay, K., Akyol, M. A., Eren, E. P., & Koçyigit, A. (2016). Big Data for Industry 4.0: A Conceptual Framework. 2016 International Conference on Computational Science and Computational Intelligence (CSCI). IEEE, pp. 431-434. https://doi.org/10.1109/CSCI.2016.0088

Goldstein, M., & Uchida, S. (2016). A comparative evaluation of unsupervised anomaly detection algorithms for multivariate data. PLoS ONE, 11(4), 0152173. https://doi.org/10.1371/journal.pone.0152173

Gölzer, P., Cato, P., & Amberg, M. (2015). Data Processing Requirements of Industry 4.0 - Use Cases for Big Data Applications. Twenty-Third European Conference on Information Systems (ECIS), Münster, Germany, 2015. ECIS 2015 Research-in-Progress Papers, pp. 61. http://aisel.aisnet.org/ecis2015_rip/61

Japa, A., & Shi, Y. (2020). Parallelizing the Bounded K-Nearest Neighbors Algorithm for Distributed Computing Systems”. 2020 10th Annual Computing and Communication Workshop and Conference (CCWC), pp. 0038-0045. https://doi.org/ 10.1109/CCWC47524.2020.9031198

Jindal, N., & Liu, B. (2007). Review spam detection. In: Proceedings of the 16th International Conference on World wide web. ACM., pp. 1189–90. https://doi.org/10.1145/1242572.1242759

Kamat, P., & Sugandhi, R. (2020). Anomaly Detection for Predictive Maintenance in Industry 4.0- A survey. E3S Web of Conferences 2020. https://doi.org/10.1051/e3sconf/202017002007

Latinovic, T., Preradović, D., Barz, C. R., Vadean, A. P., & Todić, M. (2019). Big Data as the Basis for the innovative Development Strategy of the Industry 4.0. IOP Conference Series: Materials Science and Engineering, 477, 012045. https://doi.org/10.1088/1757-899X/477/1/012045

Li, X., Zhang, W., Ding, Q., & Sun, J. Q. (2020). Intelligent rotating machinery fault diagnosis based on deep learning using data augmentation. Journal of Intelligent Manufacturing, 31, 433–452. https://doi.org/10.1007/s10845-018-1456-1

Lindemann, B., Fesenmayr, F., Jazdi, N., & Weyrich, M. (2019). Anomaly Detection in Discrete Manufacturing Using Self-Learning Approaches. Benjamin Lindemann et al. Procedia CIRP, 79, 313–318. https://doi.org/10.1016/j.procir.2019.02.073

Liu, J., Guo, J., Orlik, P.V., Shibata, M., Nakahara, D., Mii, S., & Takac, M. (2018). Anomaly Detection in Manufacturing Systems Using Structured Neural Networks. IEEE World Congress on Intelligent Control and Automation 2018, pp. 1-6. http://dx.doi.org/10.1109/WCICA.2018.8630692

Navia-Vazquez, A., Gutierrez-Gonzalez, D., Parrado-Hernandez, E., & Navarro-Abellan, J.J. (2006). Distributed Support Vector Machines. August 2006 IEEE Transactions on Neural Networks, 17(4), 1091-1097. https://doi.org/10.1109/TNN.2006.875968

O’Donovan, P., Leahy, K., Bruton, K., & Dominic T. J. O. (2015a). Big Data in Manufacturing: A Systematic Mapping Study. Journal of Big Data, 2, 20. https://doi.org/10.1186/s40537-015-0028-x

O’Donovan, P., Leahy, K., Bruton, K., & O’Sullivan D. T. J. (2015b). An Industrial Big Data Pipeline for Data-driven Analytics Maintenance Applications in Large-scale Smart Manufacturing Facilities. Journal of Big Data, 2, 25. https://doi.org/10.1186/s40537-015-0034-z

Parthasarathy, S., Ghoting, A., & Otey, M. E. (2007). A survey of distributed mining of data streams. In: Data streams. Springer, pp. 289–307. https://doi.org/10.1007/978-0-387-47534-9_13

Patcha, A., & Park, J. M. (2007). An overview of anomaly detection techniques: existing solutions and latest technological trends. Comput. Netw., 51(12), 3448–3470. https://doi.org/10.1016/j.comnet.2007.02.001

Phuyal, S., Bista, D., & Bista, R. (2020). Challenges, opportunities and future directions of smart manufacturing: A state of art review, Sustain. Futures, 2. https://doi.org/10.1016/j.sftr.2020.100023 100023

Pittino, F., Puggl, F., Puggl, M., Moldaschl, T., & Hirschl, C. (2020). Automatic Anomaly Detection on In-Production Manufacturing Machines Using Statistical Learning Methods. Sensors, 20, 2344. https://doi.org/10.3390/s20082344

Rivetti, N., Busnel,Y., & Gal, Y. (2017). Grand Challenge: FlinkMan – Anomaly Detection in Manufacturing Equipment with Apache Flink. 11th ACM International Conference on Distributed and Event-based Systems, Jun 2017, Barcelone, Spain. pp. 274-279. https://doi.org/10.1145/3093742.3095099

Spirin, N., & Han, J. (2012). Survey on web spam detection: principles and algorithms. ACM SIGKDD Explor. Newsl., 13(2), 50–64. https://doi.org/10.1145/2207243.2207252

Tamboli, J., & Shukla, M. (2016). A Survey of outlier detection algorithms for data streams. In: Computing for sustainable global development (INDIACom), 2016 3rd International Conference on. IEEE, pp. 3535–3540.

Tang, Z., Chen Z., Bao, Y., & Li, H. (2018). Convolutional neural network‐based data anomaly detection method using multiple information for structural health monitoring. Structural Control Health Monitoring, 26, e2296 http://dx.doi.org/10.1002/stc.2296

Tatu, A., Maaß, F., Färber, I., Bertini, E., Schreck, T., Seidl, T., & Keim, D. (2012). Subspace search and visualization to make sense of alternative clusterings in high-dimensional data. In: Visual analytics science and technology (VAST), 2012 IEEE Conference, pp. 63–72. https://doi.org/10.1109/VAST.2012.6400488

Wang, K. (2016). Intelligent Predictive Maintenance (IPDM) system – Industry 4.0 scenario. WIT Trans Eng. Sci., 113, 259–268. https://doi.org/10.2495/IWAMA150301

Yan, J., Meng, Y., Lu, L., & Li. L. (2017). Industrial big data in an industry 4.0 environment: challenges, schemes, and applications for predictive maintenance. In IEEE Access, 5, 23484-23491. https://doi.org/10.1109/ACCESS.2017.2765544

Zheng, J., Li, J., Liu, C., Wang, J., Li, J., & Liu, H.(2022). Anomaly detection for high dimensional space using deep hypersphere fused with probability approach. Complex & Intelligent Systems, 8, 4205–4220.

https://doi.org/10.1007/s40747-022-00695-9

Zope, K., Singh, K., Nistala S. H., Basak, A., Rathore, P., & Runkana, V. (2019). Anomaly Detection and Diagnosis in Manufacturing Systems: A Comparative Study of Statistical, Machine Learning and Deep Learning Techniques. Annual Conference of the PHM Society, Arizona USA, 2019. https://doi.org/11. 10.36001/phmconf.2019.v11i1.815

Żabiński, T., Mączka, T., Kluska, J., Madera, M., & Sęp, J. (2019). Condition monitoring in Industry 4.0 production systems - the idea of computational intelligence methods application. Procedia CIRP, 79 (3-4), 63-67. https://doi.org/10.1016/j.procir.2019.02.012

Zuo, L., Zhang, H., & Wang, H. (2021). Optimal subsample selection for massive logistic regression with distributed data. Comput. Stat., 36, 2535–2562. https://doi.org/10.1007/s00180-021-01089-0