Skip to content
Licensed Unlicensed Requires Authentication Published by Oldenbourg Wissenschaftsverlag August 31, 2019

Simulation-driven machine learning for robotics and automation

Simulations-basiertes maschinelles Lernen in der Robotik und Automatisierungstechnik
Mohamed El-Shamouty

Mohamed El-Shamouty received his B. Sc. and M. Sc. in Communication Engineering and Media Technology from German University in Cairo, in 2015 and University of Stuttgart, in 2018 respectively. Since 2018, he is a research associate in the robotics department at the Fraunhofer Institute for Manufacturing Engineering and Automation IPA with the focus of using Machine Learning for safe and intelligent collaborative robots in production.

, Kilian Kleeberger

Kilian Kleeberger received his B. Sc. and M. Sc. degree in production engineering from the Friedrich-Alexander-Universität Erlangen-Nürnberg. During his studies, he focused on automation engineering, image processing and robotics. Since 2017, he is a research associate in the robotics department at the Fraunhofer Institute for Manufacturing Engineering and Automation IPA. His research focuses on computer vision and robotics.

, Arik Lämmle

Arik Lämmle received his B. Sc. and M. Sc. degree in mechanical engineering from the University of Stuttgart. During his studies, he focused on industrial robotics, production processes and machine tools. Since 2017, he is a research associate in the robotics department at the Fraunhofer Institute for Manufacturing Engineering and Automation IPA. His main focus is the development of intelligent systems for autonomous execution of assembly operations using a combination of advanced simulation environments, artificial intelligence and industrial robots.

and Marco Huber

Marco Huber received his diploma and Ph. D. in computer science from the Karlsruhe Institute of Technology (KIT), Germany, in 2006 and 2009, respectively. From 2009 until 2011 he was with Fraunhofer IOSB, Karlsruhe, Germany, where he was leading a research group on computer vision and information fusion. He then worked as Senior Researcher with AGT International in Darmstadt until 2015. From April 2015 to September 2018, Marco Huber was responsible for product development and data science services of the Katana division at USU Software AG in Karlsruhe. Since October 2018, he is full professor for cognitive production systems with University of Stuttgart and also head of the Center for Cyber Cognitive Intelligence (CCI) at the Fraunhofer Institute for Manufacturing Engineering and Automation IPA. His research focuses on machine learning, sensor data analysis, and robotics for production.

ORCID logo EMAIL logo
From the journal tm - Technisches Messen

Abstract

Mass personalization—a megatrend in industrial manufacturing and production—requires fast adaptations of robotics and automation solutions to continually decreasing lot sizes. In this paper, the challenges of applying robot-based automation in a highly individualized production are highlighted. To face these challenges, a framework is proposed that combines latest machine learning (ML) techniques, like deep learning, with high-end physics simulation environments. ML is used for programming and parameterizing machines for a given production task with minimal human intervention. If the simulation environment realistically captures physical properties like forces or elasticity of the real world, it provides a high-quality data source for ML. In doing so, new tasks are mastered in simulation faster than in real-time, while at the same time existing tasks are executed. The functionality of the simulation-driven ML framework is demonstrated on an industrial use case.

Zusammenfassung

Die Massenpersonalisierung – ein Megatrend in der industriellen Fertigung und Produktion – erfordert eine schnelle Anpassung von Robotik- und Automatisierungslösungen an immer kleinere Losgrößen. In dieser Arbeit werden die Herausforderungen der roboterbasierten Automatisierung in einer hochgradig individualisierten Produktion aufgezeigt. Um diesen Herausforderungen zu begegnen, wird ein Framework vorgeschlagen, welches neueste Methoden des maschinellen Lernens (ML), wie beispielsweise Deep Learning, mit High-End-Physiksimulationen kombiniert. ML dient zur Programmierung und Parametrierung von Maschinen für eine bestimmte Produktionsaufgabe mit minimalem menschlichen Eingriff. Wenn die Simulationsumgebung physikalische Eigenschaften wie Kräfte oder Elastizität der realen Welt realitätsnah erfasst, stellt sie eine hochwertige Datenquelle für ML dar. Dabei werden neue Aufgaben in der Simulation schneller als in Echtzeit gemeistert, während gleichzeitig bestehende Aufgaben ausgeführt werden. Die Funktionalität des simulationsgesteuerten ML-Frameworks wird anhand eines industriellen Anwendungsfalls demonstriert.

Award Identifier / Grant number: 017-192996

Award Identifier / Grant number: Cyberprotect

Award Identifier / Grant number: NEU016/1

Funding statement: This work was partially supported by the Ministry of Economic Affairs of the state Baden-Württemberg (Zentrum für Cyber Cognitive Intelligence – Grant No. 017-192996 and Cyberprotect) and the Baden-Württemberg Stiftung (Deep Grasping – Grant No. NEU016/1).

About the authors

Mohamed El-Shamouty

Mohamed El-Shamouty received his B. Sc. and M. Sc. in Communication Engineering and Media Technology from German University in Cairo, in 2015 and University of Stuttgart, in 2018 respectively. Since 2018, he is a research associate in the robotics department at the Fraunhofer Institute for Manufacturing Engineering and Automation IPA with the focus of using Machine Learning for safe and intelligent collaborative robots in production.

Kilian Kleeberger

Kilian Kleeberger received his B. Sc. and M. Sc. degree in production engineering from the Friedrich-Alexander-Universität Erlangen-Nürnberg. During his studies, he focused on automation engineering, image processing and robotics. Since 2017, he is a research associate in the robotics department at the Fraunhofer Institute for Manufacturing Engineering and Automation IPA. His research focuses on computer vision and robotics.

Arik Lämmle

Arik Lämmle received his B. Sc. and M. Sc. degree in mechanical engineering from the University of Stuttgart. During his studies, he focused on industrial robotics, production processes and machine tools. Since 2017, he is a research associate in the robotics department at the Fraunhofer Institute for Manufacturing Engineering and Automation IPA. His main focus is the development of intelligent systems for autonomous execution of assembly operations using a combination of advanced simulation environments, artificial intelligence and industrial robots.

Marco Huber

Marco Huber received his diploma and Ph. D. in computer science from the Karlsruhe Institute of Technology (KIT), Germany, in 2006 and 2009, respectively. From 2009 until 2011 he was with Fraunhofer IOSB, Karlsruhe, Germany, where he was leading a research group on computer vision and information fusion. He then worked as Senior Researcher with AGT International in Darmstadt until 2015. From April 2015 to September 2018, Marco Huber was responsible for product development and data science services of the Katana division at USU Software AG in Karlsruhe. Since October 2018, he is full professor for cognitive production systems with University of Stuttgart and also head of the Center for Cyber Cognitive Intelligence (CCI) at the Fraunhofer Institute for Manufacturing Engineering and Automation IPA. His research focuses on machine learning, sensor data analysis, and robotics for production.

Acknowledgment

The authors would like to also acknowledge Ramez Awad, Xinyang Wu, Dennis Lamaj, Jiacheng Yang and Karin Röhricht for fruitful discussions and support with implementations.

References

1. S. J. Hu, “Evolving paradigms of manufacturing: From mass production to mass customization and personalization”, in CMS, Elsevier, 2013.Search in Google Scholar

2. G. Thomas, M. Chien, A. Tamar, J. Ojea, P. Aparicio, and P. Abbeel, “Learning Robotic Assembly from CAD”, in IEEE International Conference on Robotics and Automation (ICRA), 2018.10.1109/ICRA.2018.8460696Search in Google Scholar

3. W. Zou, M. Andulkar, and U. Berger, “Development of Robot Programming System through the use of Augmented Reality for Assembly Tasks”, in 50th International Symposium on Robotics, VDE, 2018.Search in Google Scholar

4. L. Wang, B. Schmidt, M. Givehchi, and G. Adamson, “Robotic assembly planning and control with enhanced adaptability through function blocks”, The International Journal of Advanced Manufacturing Technology, vol. 77, 2015.10.1007/s00170-014-6468-1Search in Google Scholar

5. A. Billard, S. Calinon, R. Dillmann, and S. Schaal, Robot Programming by Demonstration, B. Siciliano and O. Khatib (eds), Springer Berlin Heidelberg, 2008.10.1007/978-3-540-30301-5_60Search in Google Scholar

6. OpenAI, M. Andrychowicz, B. Baker, M. Chociej, R. Jozefowicz, B. McGrew, J. Pachocki, A. Petron, M. Plappert, G. Powell, A. Ray, J. Schneider, S. Sidor, J. Tobin, P. Welinder, L. Weng and W. Zaremba, “Learning dexterous in-hand manipulation”, 2018.10.1177/0278364919887447Search in Google Scholar

7. R. Selizki, Computer Vision: Algorithms and Applications, Springer, 2010.Search in Google Scholar

8. F. Vicentini, M. Giussani, and L. M. Tosatti, “Trajectory-dependent safe distances in human-robot interaction”, in IEEE Emerging Technology and Factory Automation (ETFA), 2014.10.1109/ETFA.2014.7005316Search in Google Scholar

9. S. James, P. Wohlhart, M. Kalakrishnan, D. Kalashnikov, A. Irpan, J. Ibarz, S. Levine, R. Hadsell, and K. Bousmalis, “Sim-to-real via sim-to-sim: Data-efficient robotic grasping via randomized-to-canonical adaptation networks”, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.10.1109/CVPR.2019.01291Search in Google Scholar

10. N. Fazeli, M. Oller, J. Wu, Z. Wu, J. B. Tenenbaum, and A. Rodriguez, “See, feel, act: Hierarchical learning for complex manipulation skills with multisensory fusion”, Science Robotics, vol. 4, no. 26, 2019.10.1126/scirobotics.aav3123Search in Google Scholar

11. L. Johannsmeier, M. Gerchow, and S. Haddadin, “A framework for robot manipulation: Skill formalism, meta learning and adaptive control”, arXiv:1805.08576, 2018.Search in Google Scholar

12. S. Phaniteja, P. Dewangan, P. Guhan, A. Sarkar, and K. M. Krishna, “A deep reinforcement learning approach for dynamically stable inverse kinematics of humanoid robots”, in ROBIO, 2017.10.1109/ROBIO.2017.8324682Search in Google Scholar

13. J. Hwangbo, J. Lee, A. Dosovitskiy, D. Bellicoso, V. Tsounis, V. Koltun, and M. Hutter, “Learning agile and dynamic motor skills for legged robots”, Science Robotics, vol. 4, no. 26, 2019.10.1126/scirobotics.aau5872Search in Google Scholar

14. S. Levine, P. Pastor, A. Krizhevsky, and D. Quillen, “Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection”, in International Symposium on Experimental Robotics (ISER), 2016.10.1007/978-3-319-50115-4_16Search in Google Scholar

15. T. Johannink, S. Bahl, A. Nair, J. Luo, A. Kumar, M. Loskyll, J. A. Ojea, E. Solowjow, and S. Levine, “Residual reinforcement learning for robot control”, arXiv:1812.03201, 2018.Search in Google Scholar

16. S. Stanev, R. Awad, M. Prieur, W. Walla, and J. Ovtcharova, “Production-oriented product validation method as support for the reuse of production lines in the automotive industry”, in 3rd International Conference on Changeable, Agile, Reconfigurable and Virtual Production (CARV), Herbert Utz Verlag, 2009.Search in Google Scholar

17. R. Awad, M. Fechter, and J. van Heerden, “Integrated risk assessment and safety consideration during design of hrc workplaces”, in 22nd IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), IEEE, 2017.10.1109/ETFA.2017.8247648Search in Google Scholar

18. E. Todorov, T. Erez, and Y. Tassa, “MuJoCo: A physics engine for model-based control”, in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2012.10.1109/IROS.2012.6386109Search in Google Scholar

19. T. Erez, Y. Tassa, and E. Todorov, “Simulation tools for model-based robotics: Comparison of bullet, havok, mujoco, ode and physx”, in IEEE International Conference on Robotics and Automation (ICRA), 2015.10.1109/ICRA.2015.7139807Search in Google Scholar

20. E. Rohmer, S. P. N. Singh, and M. Freese, “V-REP: A versatile and scalable robot simulation framework”, in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2013.10.1109/IROS.2013.6696520Search in Google Scholar

21. V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglo, H. King, D. Kumaran, D. Wierstra, S. Legg and D. Hassabis, “Human-level control through deep reinforcement learning”, Nature, vol. 518, no. 7540, pp. 529–533, 2015.10.1038/nature14236Search in Google Scholar

22. M. Andrychowicz, F. Wolski, A. Ray, J. Schneider, R. Fong, P. Welinder, B. McGrew, J. Tobin, P. Abbeel, and W. Zarembah, “Hindsight experience replay”, in 31st Conference on Neural Information Processing Systems (NIPS), 2017.Search in Google Scholar

23. M.-A. Zöller and M. F. Huber, “Survey on Automated Machine Learning”, arXiv:1904.12054v1, 2019.Search in Google Scholar

24. X. B. Peng, M. Andrychowicz, W. Zaremba, and P. Abbeel, “Sim-to-real transfer of robotic control with dynamics randomization”, in IEEE International Conference on Robotics and Automation (ICRA), 2018.10.1109/ICRA.2018.8460528Search in Google Scholar

25. J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, “Domain randomization for transferring deep neural networks from simulation to the real world”, in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2017.10.1109/IROS.2017.8202133Search in Google Scholar

26. K. Bousmalis, A. Irpan, P. Wohlhart, Y. Bai, M. Kelcey, M. Kalakrishnan, L. Downs, J. Ibarz, P. Pastor, K. Konolige, S. Levine, and V. Vanhoucke, “Using simulation and domain adaptation to improve efficiency of deep robotic grasping”, in IEEE International Conference on Robotics and Automation (ICRA), 2018.10.1109/ICRA.2018.8460875Search in Google Scholar

27. P. Dhariwal, C. Hesse, O. Klimov, A. Nichol, M. Plappert, A. Radford, J. Schulman, S. Sidor, Y. Wu, and P. Zhokhov, “Openai baselines”, https://github.com/openai/baselines, 2017.Search in Google Scholar

28. I. Caspi, G. Leibovich, G. Novik, and S. Endrawis, “Reinforcement learning coach”, https://doi.org/10.5281/zenodo.1134899, 2017.Search in Google Scholar

29. Y. Duan, X. Chen, R. Houthooft, J. Schulman, and P. Abbeel, “Benchmarking deep reinforcement learning for continuous control”, in 33rd International Conference on Machine Learning (ICML), vol. 48, JMLR, 2016.Search in Google Scholar

30. F. Aichele, B. Schenke, B. Eckstein, and A. Groz, “A framework for robot control software development and debugging using a real-time capable physics simulation”, in 47st International Symposium on Robotics (ISR), 2016.Search in Google Scholar

31. F. Khatami and C. Müller, “World Robotics 2018: Industrial Robots”, 2018.Search in Google Scholar

32. H. ElMaraghy and W. ElMaraghy, “Smart adaptable assembly systems”, CIRP, vol. 44, 2016.10.1016/j.procir.2016.04.107Search in Google Scholar

33. P. Tempel, F. Eger, A. Lechler, and A. Verl, “Schaltschrankbau 4.0: Eine Studie über die Automatisierungs- und Digitalisierungspotenziale in der Fertigung von Schaltschränken und Schaltanlagen im klassischen Maschinen- und Anlagenbau”, https://discover.eplan.eu/schaltschrankbau-studie-anwender, last accessed on July 2019.Search in Google Scholar

34. D. de Gregorio, R. Zanella, G. Palli, S. Pirozzi, and C. Melchiorri, “Integration of robotic vision and tactile sensing for wire-terminal insertion tasks”, IEEE Transactions on Automation Science and Engineering, vol. 16, no. 2, 2019.10.1109/TASE.2018.2847222Search in Google Scholar

35. P. Heisler, P. Steinmetz, I. S. Yoo, and J. Franke, “Automatization of the cable-routing-process within the automated production of wiring systems”, in Energy Efficiency in Strategy of Sustainable Production III, Applied Mechanics and Materials, Trans Tech Publications, 2017.10.4028/www.scientific.net/AMM.871.186Search in Google Scholar

36. K. Kleeberger, C. Landgraf, and M. F. Huber, “Large-scale 6D Object Pose Estimation Dataset for Industrial Bin-Picking”, in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019.10.1109/IROS40897.2019.8967594Search in Google Scholar

37. D. Kalashnikov, A. Irpan, P. Pastor, J. Ibarz, A. Herzog, E. Jang, D. Quillen, E. Holly, M. Kalakrishnan, V. Vanhoucke, and S. Levine, “QT-Opt: Scalable deep reinforcement learning for vision-based robotic manipulation”, in Conference on Robot Learning (CoRL), 2018.Search in Google Scholar

38. D. Quillen, E. Jang, O. Nachum, C. Finn, J. Ibarz, and S. Levine, “Deep reinforcement learning for vision-based robotic grasping: A simulated comparative evaluation of off-policy methods”, in IEEE International Conference on Robotics and Automation (ICRA), 2018.10.1109/ICRA.2018.8461039Search in Google Scholar

39. J. Mahler, J. Liang, S. Niyaz, M. Laskey, R. Doan, X. Liu, J. A. Ojea, and K. Goldberg, “Dex-Net 2.0: Deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics”, in Robotics: Science and Systems (RSS), 2017.10.15607/RSS.2017.XIII.058Search in Google Scholar

40. A. Depierre, E. Dellandréa, and L. Chen, “Jacquard: A large scale dataset for robotic grasp detection”, in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2018.10.1109/IROS.2018.8593950Search in Google Scholar

41. J. Redmon and A. Angelova, “Real-time grasp detection using convolutional neural networks”, in IEEE International Conference on Robotics and Automation (ICRA), 2015.10.1109/ICRA.2015.7139361Search in Google Scholar

42. S. Kumra and C. Kanan, “Robotic grasp detection using deep convolutional neural networks”, in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2017.10.1109/IROS.2017.8202237Search in Google Scholar

43. I. Lenz, H. Lee, and A. Saxena, “Deep learning for detecting robotic grasps”, in International Journal of Robotics Research (IJRR), 2015.10.1177/0278364914549607Search in Google Scholar

44. D. Morrison, J. Leitner, and P. Corke, “Closing the loop for robotic grasping: A real-time, generative grasp synthesis approach”, in Robotics: Science and Systems (RSS), 2018.10.15607/RSS.2018.XIV.021Search in Google Scholar

45. L. Halt, F. Nägele, P. Tenbrock, and A. Pott, “Intuitive constraint-based robot programming for robotic assembly tasks”, in IEEE International Conference on Robotics and Automation (ICRA), 2018.Search in Google Scholar

46. X. Zhang, A. S. Polydoros, and J. Piater, “Learning movement assessment primitives for force interaction skills”, arXiv:1805.04354, 2018.Search in Google Scholar

47. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms”, arXiv:1707.06347, 2017.Search in Google Scholar

48. W. Bauer, M. Bender, M. Braun, P. Rally, and O. Scholtz, “Lightweight robots in manual assembly—best to start simply”, Examining companies’ initial experiences with lightweight robots, Stuttgart, 2016.Search in Google Scholar

49. ISO 10218-1:2011, Robots and robotic devices – Safety requirements for industrial robots – Part 1: Robots, ISO, 2011.Search in Google Scholar

50. ISO 10218-2:2011, Robots and robotic devices – Safety requirements for industrial robots – Part 2: Robot systems and integration, ISO, 2011.Search in Google Scholar

51. N. Schaaf and M. F. Huber, “Enhancing Decision Tree based Interpretation of Deep Neural Networks through L1-Orthogonal Regularization”, arXiv:1904.05394, 2019.Search in Google Scholar

Received: 2019-05-15
Accepted: 2019-08-06
Published Online: 2019-08-31
Published in Print: 2019-11-26

© 2019 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 2.12.2022 from frontend.live.degruyter.dgbricks.com/document/doi/10.1515/teme-2019-0072/html
Scroll Up Arrow