Interpretable machine learning with reject option

  • 1 CITEC Center of Excellence, Bielefeld University, Bielefeld, Germany
Johannes Brinkrolf
  • CITEC Center of Excellence, Bielefeld University, Bielefeld, Germany
  • Email
  • Further information
  • Johannes Brinkrolf received his master’s degree from Bielefeld University in 2016. From April 2016 to September 2016 he was a research assistant at the South Westphalia University of Applied Science. Since October 2016 he has been a PhD student at the Cognitive Interaction Technology Center of Excellence at Bielefeld University.
  • Search for other articles:
  • degruyter.comGoogle Scholar
and Barbara Hammer
  • Corresponding author
  • CITEC Center of Excellence, Bielefeld University, Bielefeld, Germany
  • Email
  • Further information
  • Barbara Hammer received her Ph.D. in Computer Science in 1999 and her venia legendi in Computer Science in 2003, both from the University of Osnabrueck, Germany. From 2000-2004, she was chair of the junior research group ‘Learning with Neural Methods on Structured Data’ at the University of Osnabrueck before accepting an offer as professor for Theoretical Computer Science at Clausthal University of Technology, Germany, in 2004. Since 2010, she holds a professorship for Theoretical Computer Science for Cognitive Systems at the CITEC cluster of excellence at Bielefeld University, Germany. Several research stays have taken her to Italy, the U.K., India, France, the Netherlands, and the U.S.A. Her areas of expertise include hybrid systems, self-organizing maps, clustering, and recurrent networks as well as applications in bioinformatics, industrial process monitoring, or cognitive science. She chaired the IEEE CIS Technical Committee on Data Mining in 2013/2014, and she is chair of the Fachgruppe Neural Networks of the GI and vice-chair of the GNNs. She has published more than 200 contributions to international conferences / journals, and she is coauthor/editor of four books.
  • Search for other articles:
  • degruyter.comGoogle Scholar


Classification by means of machine learning models constitutes one relevant technology in process automation and predictive maintenance. However, common techniques such as deep networks or random forests suffer from their black box characteristics and possible adversarial examples. In this contribution, we give an overview about a popular alternative technology from machine learning, namely modern variants of learning vector quantization, which, due to their combined discriminative and generative nature, incorporate interpretability and the possibility of explicit reject options for irregular samples. We give an explicit bound on minimum changes required for a change of the classification in case of LVQ networks with reject option, and we demonstrate the efficiency of reject options in two examples.

  • 1.

    Davide Anguita, Alessandro Ghio, Luca Oneto, Xavier Parra, and Jorge Luis Reyes-Ortiz, A Public Domain Dataset for Human Activity Recognition using Smartphones, in: 21st European Symposium on Artificial Neural Networks, ESANN 2013, Bruges, Belgium, April 24–26, 2013, 2013.

  • 2.

    Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok, Synthesizing Robust Adversarial Examples, CoRR abs/1707.07397 (2017).

  • 3.

    Michael Biehl, Barbara Hammer, Frank-Michael Schleif, Petra Schneider, and Thomas Villmann, Stationarity of Matrix Relevance LVQ, in: IJCNN, 2015.

  • 4.

    Michael Biehl, Barbara Hammer, and Thomas Villmann, Prototype-based models in machine learning, WIREs Cognitive Science 7(2) (2016), 92–111.

  • 5.

    Kerstin Bunte, Petra Schneider, Barbara Hammer, Frank-Michael Schleif, Thomas Villmann, and Michael Biehl, Limited Rank Matrix Learning, discriminative dimension reduction and visualization, Neural Networks 26 (2012), 159–173.

  • 6.

    Davide Castelvecchi, Can we open the black box of AI?, Nature 538 (2016), 20–23.

  • 7.

    A. Chandiok and D. K. Chaturvedi, Machine learning techniques for cognitive decision making, in: 2015 IEEE Workshop on Computational Intelligence: Theories, Applications and Future Directions (WCI), pp. 1–6, Dec 2015.

  • 8.

    C. Chow, On Optimum Recognition Error and Reject Tradeoff, IEEE Trans. Inf. Theor. 16(1) (2006), 41–46.

  • 9.

    B. Cline, R. S. Niculescu, D. Huffman, and B. Deckel, Predictive maintenance applications for machine learning, in: 2017 Annual Reliability and Maintainability Symposium (RAMS), pp. 1–7, Jan 2017.

  • 10.

    G. Ditzler, M. Roveri, C. Alippi, and R. Polikar, Learning in Nonstationary Environments: A Survey, IEEE Computational Intelligence Magazine 10(4) (2015), 12–25.

  • 11.

    L. Fischer, B. Hammer, and H. Wersing, Efficient rejection strategies for prototype-based classification, Neurocomputing 169 (2015), 334–342.

  • 12.

    Lydia Fischer, Barbara Hammer, and Heiko Wersing, Optimal local rejection for classifiers, Neurocomputing 214 (2016), 445–457.

  • 13.

    Alexander Geppert and Barbara Hammer, Incremental learning algorithms and applications, in: ESANN, 2016.

  • 14.

    Javier González-Jiménez, Javier G. Monroy, and José-Luis Blanco, The Multi-Chamber Electronic Nose–An Improved Olfaction Sensor for Mobile Robotics, Sensors 11(6) (2011), 6145–6164.

  • 15.

    I. J. Goodfellow, J. Shlens, and C. Szegedy, Explaining and Harnessing Adversarial Examples, ArXiv e-prints (2014).

  • 16.

    Radu Herbei and Marten H. Wegkamp, Classification with Reject Option, The Canadian Journal of Statistics / La Revue Canadienne de Statistique 34(4) (2006), 709–721.

  • 17.

    Joel Janai, Fatma Güney, Aseem Behl, and Andreas Geiger, Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art, CoRR abs/1704.05519 (2017).

  • 18.

    T. Kohonen, M. R. Schroeder, and T. S. Huang (eds.), Self-Organizing Maps, 3rd ed, Springer-Verlag New York, Inc., Secaucus, NJ, USA, 2001.

  • 19.

    A. L’Heureux, K. Grolinger, H. F. Elyamany, and M. A. M. Capretz, Machine Learning With Big Data: Challenges and Approaches, IEEE Access 5 (2017), 7776–7797.

  • 20.

    Zachary Chase Lipton, The Mythos of Model Interpretability, CoRR abs/1606.03490 (2016).

  • 21.

    Viktor Losing, Barbara Hammer, and Heiko Wersing, Interactive Online Learning for Obstacle Classification on a Mobile Robot, in: IJCNN, 2015.

  • 22.

    Viktor Losing, Barbara Hammer, and Heiko Wersing, Choosing the best algorithm for an incremental on-line learning task, in: ESANN, 2016.

  • 23.

    Viktor Losing, Barbara Hammer, and Heiko Wersing, Tackling heterogeneous concept drift with the Self-Adjusting Memory (SAM), Knowl. Inf. Syst. 54(1) (2018), 171–201.

  • 24.

    Anh Mai Nguyen, Jason Yosinski, and Jeff Clune, Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images, CoRR abs/1412.1897 (2014).

  • 25.

    Matthias Richter, Thomas Längle, and Jürgen Beyerer, Knowing when you don’t: Bag of visual words with reject option for automatic visual inspection of bulk materials, in: 23rd International Conference on Pattern Recognition, ICPR 2016, Cancún, Mexico, December 4–8, 2016, pp. 3079–3084, 2016.

  • 26.

    P. Schneider, M. Biehl, and B. Hammer, Adaptive relevance matrices in learning vector quantization, Neural Computation 21 (2009), 3532–3561.

  • 27.

    Shai Shalev-Shwartz and Shai Ben-David, Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press, New York, NY, USA, 2014.

  • 28.

    Chawin Sitawarin, Arjun Nitin Bhagoji, Arsalan Mosenia, Mung Chiang, and Prateek Mittal, DARTS: Deceiving Autonomous Cars with Toxic Signs, Cornell University Library, Report no. arXiv:1802.06430, 2017.

  • 29.

    J. Su, D. Vasconcellos Vargas, and S. Kouichi, One pixel attack for fooling deep neural networks, ArXiv e-prints (2017).

  • 30.

    Lucia Ureche, Keisuke Umezawa, Yoshihiko Nakamura, and Aude Billard, Task Parameterization Using Continuous Constraints Extracted From Human Demonstrations, IEEE Transactions on Robotics 31(6) (2015), 1458–1471.

  • 31.

    Kush R. Varshney and Homa Alemzadeh, On the Safety of Machine Learning: Cyber-Physical Systems, Decision Sciences, and Data Products, CoRR abs/1610.01256 (2016).

  • 32.

    Marten Wegkamp and Ming Yuan, Support vector machines with a reject option, Bernoulli 17(4) (2011), 1368–1385.

Purchase article
Get instant unlimited access to the article.
Log in
Already have access? Please log in.

Log in with your institution

Journal + Issues

AT – Automatisierungstechnik covers the entire field of automation technology. It presents the development of theoretical procedures and their possible applications. Topics include new discoveries about the development and application of methods. It presents the function, properties, and applications of tools and includes contributions from the worlds of research, academia, and industry.