Skip to content
Licensed Unlicensed Requires Authentication Published by De Gruyter (O) September 27, 2019

Feature fusion to increase the robustness of machine learners in industrial environments

Merkmalsfusion zur Erhöhung der Robustheit von maschinellen Lernern in industriellen Umgebungen
Christoph-Alexander Holst and Volker Lohweg


Industrial applications put special demands on machine learning algorithms. Noisy data, outliers, and sensor faults present an immense challenge for learners. A considerable part of machine learning research focuses on the selection of relevant, non-redundant features. This contribution details an approach to group and fuse redundant features prior to learning and classification. Features are grouped relying on a correlation-based redundancy measure. The fusion of features is guided by determining the majority observation based on possibility distributions. Furthermore, this paper studies the effects of feature fusion on the robustness and performance of classification with a focus on industrial applications. The approach is statistically evaluated on public datasets in comparison to classification on selected features only.


Industrielle Anwendungen stellen besondere Anforderungen an maschinelle Lernalgorithmen. Verrauschte Daten, Ausreißer und Sensorfehler sind eine große Herausforderung für Lerner. Ein erheblicher Teil der Forschung im Bereich maschinelles Lernen konzentriert sich auf die Auswahl relevanter, nicht redundanter Merkmale. Dieser Beitrag beschreibt einen Ansatz zur Gruppierung und Fusion redundanter Merkmale, die vor dem Lernen und Klassifizieren ausgeführt werden. Die Gruppierung der Merkmale basiert auf einer korrelationsbasierten Redundanzmessung, während die Fusionierung von Merkmalen auf der Bestimmung einer über Möglichkeitsverteilungen ermittelten Mehrheitsbeobachtung basiert. Darüber hinaus untersucht dieser Beitrag die Auswirkungen der Merkmalsfusion auf die Robustheit und Leistungsfähigkeit der Klassifizierung. Der Ansatz wird anhand öffentlicher Datensätze im Vergleich zur Klassifizierung auf ausgewählten Merkmalen statistisch ausgewertet.

Funding source: Bundesministerium für Bildung und Forschung

Award Identifier / Grant number: 01IS18041D

Funding statement: This work was partly funded by the German Federal Ministry of Education and Research within the project ITS.ML, Grant number 01IS18041D.


1. M. A. Aizerman, E. M. Braverman and L. I. Rozonoer. Theoretical foundations of the potential function method in pattern recognition learning. Automation and Remote Control, 25:821–837, 1964.Search in Google Scholar

2. E. Alpaydın. Introduction to Machine Learning. The MIT Press, Cambridge, 2nd edition, 2010.Search in Google Scholar

3. B. M. Ayyub and G. J. Klir. Uncertainty Modeling and Analysis in Engineering and the Sciences. Chapman & Hall/CRC, Boca Raton, FL, 2006.10.1201/9781420011456Search in Google Scholar

4. J. Beyerer, J. Jasperneite and O. Sauer. Industrie 4.0. at – Automatisierungstechnik, 63(10), 2015.10.1515/auto-2015-0068Search in Google Scholar

5. F. Bocklisch and D. Hausmann. Multidimensional fuzzy pattern classifier sequences for medical diagnostic reasoning. Applied Soft Computing, 66:297–310, 2018.10.1016/j.asoc.2018.02.041Search in Google Scholar

6. S. F. Bocklisch. Prozeßanalyse mit unscharfen Verfahren. Verlag Technik, Berlin, 1st edition, 1987.Search in Google Scholar

7. L. Breiman. Bagging predictors. Machine Learning, 24(2):123–140, 1996.10.1007/BF00058655Search in Google Scholar

8. L. Breiman. Classification and Regression Trees. Routledge, New York, 2017.10.1201/9781315139470Search in Google Scholar

9. D. Dheeru and E. Karra Taniskidou. UCI Machine Learning Repository, 2017.Search in Google Scholar

10. T. G. Dietterich. Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation, 10(7):1895–1923, 1998.10.1162/089976698300017197Search in Google Scholar PubMed

11. A. Diez-Olivan, J. Del Ser, D. Galar and B. Sierra. Data fusion and machine learning for industrial prognosis: Trends and perspectives towards Industry 4.0. Information Fusion, 50:92–111, 2019.10.1016/j.inffus.2018.10.005Search in Google Scholar

12. H. Dörksen and V. Lohweg. Combinatorial refinement of feature weighting for linear classification. In Proceedings of the 2014 IEEE Emerging Technology and Factory Automation (ETFA), pages 1–7, 2014.10.1109/ETFA.2014.7005106Search in Google Scholar

13. D. Dubois, L. Foulloy, G. Mauris and H. Prade. Probability-possibility transformations, triangular fuzzy sets, and probabilistic inequalities. Reliable Computing, 10(4):273–297, 2004.10.1023/B:REOM.0000032115.22510.b5Search in Google Scholar

14. J.-F. Ehlenbröker, U. Mönks and V. Lohweg. Sensor defect detection in multisensor information fusion. Journal of Sensors and Sensor Systems, 5(2):337–353, 2016.10.5194/jsss-5-337-2016Search in Google Scholar

15. W. Elmenreich. An Introduction to Sensor Fusion, 2002.Search in Google Scholar

16. A. Fritze, U. Mönks, C.-A. Holst and V. Lohweg. An approach to automated fusion system design and adaptation. Sensors, 17(3):601, 2017.10.3390/s17030601Search in Google Scholar PubMed PubMed Central

17. S. Glock, K. Voth, J. Schaede and V. Lohweg. A framework for possibilistic multi-source data fusion with monitoring of sensor reliability. In World Conference on Soft Computing, 2011.Search in Google Scholar

18. I. Guyon, S. R. Gunn, M. Nikravesh and L. A. Zadeh. Feature extraction: Foundations and applications, volume 207 of Studies in Fuzziness and Soft Computing. Springer, Berlin Heidelberg, 2006.10.1007/978-3-540-35488-8Search in Google Scholar

19. D. L. Hall, J. Llinas and M. E. Liggins, editors. Handbook of Multisensor Data Fusion: Theory and Practice. The Electrical Engineering and Applied Signal Processing Series. CRC Press, Boca Raton, FL, 2nd edition, 2009.Search in Google Scholar

20. T. Hastie, R. Tibshirani, D. Botstein and P. Brown. Supervised harvesting of expression trees. Genome Biology, 2(1):research0003.1, 2001.10.1186/gb-2001-2-1-research0003Search in Google Scholar PubMed PubMed Central

21. N. Helwig, E. Pignanelli and A. Schütze. Detecting and compensating sensor faults in a hydraulic condition monitoring system. In SENSOR 2015, pages 641–646, Nürnberg, 2015. AMA Service GmbH.10.5162/sensor2015/D8.1Search in Google Scholar

22. A.-J. Hempel. Netzorientierte Fuzzy-Pattern-Klassifikation nichtkonvexer Objektmengenmorphologien. Doctoral thesis, Technische Universität Chemnitz, Chemnitz, 2011.Search in Google Scholar

23. C.-A. Holst and V. Lohweg. Improving majority-guided fuzzy information fusion for Industry 4.0 condition monitoring. In 2019 22nd International Conference on Information Fusion (FUSION). IEEE, 2019.10.23919/FUSION43075.2019.9011347Search in Google Scholar

24. Z. Hu and S. Mahadevan. Uncertainty quantification in prediction of material properties during additive manufacturing. Scripta Materialia, 135:135–140, 2017.10.1016/j.scriptamat.2016.10.014Search in Google Scholar

25. E. Hüllermeier. Fuzzy methods in machine learning and data mining: Status and prospects. Fuzzy Sets and Systems, 156(3):387–406, 2005.10.1016/j.fss.2005.05.036Search in Google Scholar

26. W. Jiang, C. Xie, M. Zhuang, Y. Shou and Y. Tang. Sensor data fusion with Z-numbers and its application in fault diagnosis. Sensors, 16(9), 2016.10.3390/s16091509Search in Google Scholar PubMed PubMed Central

27. W. Jiang, M. Zhuang and C. Xie. A reliability-based method to sensor data fusion. Sensors, 17(7), 2017.10.3390/s17071575Search in Google Scholar PubMed PubMed Central

28. M. Krüger. Gradual vs. binary conflicts in Bayesian networks applied to sensor failure detection. In 2015 18th International Conference on Information Fusion (Fusion), pages 66–73, 2015.Search in Google Scholar

29. P. Larrañaga, A. Ogbechie, J. Diaz-Rozo, D. Atienza Alonso, C. Bielza and C. Puerto-Santana. Industrial Applications of Machine Learning. Data Mining and Knowledge Series. CRC Press, Boca Raton, Florida, 2019.10.1201/9781351128384Search in Google Scholar

30. H. Li, H.-Z. Huang, Y.-F. Li, J. Zhou and J. Mi. Physics of failure-based reliability prediction of turbine blades using multi-source information fusion. Applied Soft Computing, 72:624–635, 2018.10.1016/j.asoc.2018.05.015Search in Google Scholar

31. J. Li, K. Cheng, S. Wang, F. Morstatter, R. P. Trevino, J. Tang and H. Liu. Feature selection: A data perspective. ACM Computing Surveys, 50(6):1–45, 2018.10.1145/3136625Search in Google Scholar

32. V. Lohweg, C. Diederichs and D. Müller. Algorithms for hardware-based pattern recognition. EURASIP Journal on Applied Signal Processing, 2004(12):1912–1920, 2004.10.1155/S1110865704404247Search in Google Scholar

33. R. C. Luo and M. G. Kay. Data fusion and sensor integration: State-of-the-art 1990s. In M. A. Abidi and R. C. Gonzalez, editors, Data Fusion in Robotics and Machine Intelligence, pages 7–136. Acad. Press, San Francisco, CA, USA, 1992.Search in Google Scholar

34. R. Maclin and D. Opitz. An empirical evaluation of bagging and boosting. In Proceedings of the Fourteenth National Conference on Artificial Intelligence and Ninth Conference on Innovative Applications of Artificial Intelligence, AAAI’97/IAAI’97, pages 546–551. AAAI Press, 1997.Search in Google Scholar

35. G. Mauris, V. Lasserre and L. Foulloy. Fuzzy modeling of measurement data acquired from physical sensors. IEEE Transactions on Instrumentation and Measurement, 49(6):1201–1205, 2000.10.1109/19.893256Search in Google Scholar

36. U. Mönks. Information Fusion Under Consideration of Conflicting Input Signals. Technologies for Intelligent Automation. Springer, Berlin, Heidelberg, 2017.10.1007/978-3-662-53752-7Search in Google Scholar

37. U. Mönks, D. Petker and V. Lohweg. Fuzzy-Pattern-Classifier training with small data sets. In E. Hüllermeier, R. Kruse, and F. Hoffmann, editors, Information Processing and Management of Uncertainty in Knowledge-Based Systems. Theory and Methods, pages 426–435, Berlin, Heidelberg, 2010. Springer Berlin Heidelberg.10.1007/978-3-642-14055-6_44Search in Google Scholar

38. M. Y. Park, T. Hastie and R. Tibshirani. Averaged gene expressions for regression. Biostatistics, 8(2):212–227, 2007.10.1093/biostatistics/kxl002Search in Google Scholar PubMed

39. H. Peng, F. Long and C. Ding. Feature selection based on mutual information: Criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(8):1226–1238, 2005.10.1109/TPAMI.2005.159Search in Google Scholar PubMed

40. V. Ricquebourg, L. Delahoche, B. Marhic, M. Delafosse, A.-M. Jolly-Desodt and D. Menga. Anomalies recognition in a context aware architecture based on TBM approach. In 2008 11th International Conference on Information Fusion, pages 1–8, 2008.Search in Google Scholar

41. H. Rinne. Taschenbuch der Statistik. Wissenschaftlicher Verlag Harri Deutsch GmbH, Frankfurt am Main, 4th edition, 2008.Search in Google Scholar

42. F. Shi, X. Su, H. Qian, N. Yang and W. Han. Research on the fusion of dependent evidence based on rank correlation coefficient. Sensors, 17(10), 2017.10.3390/s17102362Search in Google Scholar PubMed PubMed Central

43. M. Sokolova and G. Lapalme. A systematic analysis of performance measures for classification tasks. Information Processing & Management, 45(4):427–437, 2009.10.1016/j.ipm.2009.03.002Search in Google Scholar

44. A. Vergara, S. Vembu, T. Ayhan, M. A. Ryan, M. L. Homer and R. Huerta. Chemical gas sensor drift compensation using classifier ensembles. Sensors and Actuators B: Chemical, 166–167:320–329, 2012.10.1016/j.snb.2012.01.074Search in Google Scholar

45. K. Voth, S. Glock, U. Mönks, V. Lohweg and T. Türke. Multi-sensory machine diagnosis on security printing machines with two-layer conflict solving. In SENSOR+TEST Conference 2011, pages 686–691, Wunstorf, 2011. AMA Service GmbH.10.5162/sensor11/sp2.1Search in Google Scholar

46. L. A. Zadeh. Fuzzy sets. Information and Control, 8(3):338–353, 1965.10.1016/S0019-9958(65)90241-XSearch in Google Scholar

Received: 2019-02-28
Accepted: 2019-08-26
Published Online: 2019-09-27
Published in Print: 2019-10-25

© 2019 Walter de Gruyter GmbH, Berlin/Boston

Scroll Up Arrow