Accessible Published by De Gruyter November 28, 2015

3Dsensation: optical technologies are the key for future man-machine interaction

Andreas Thoss

With 3Dsensation, the German Federal Ministry of Education and Research (BMBF) has initiated an interdisciplinary project to promote new technologies in the field of man-machine interaction. Professor Andreas Tünnermann, director of the Fraunhofer Institute for Applied Optics and Precision Engineering (IOF), in Jena, Germany, is the spokesperson for this Innovation Alliance and has provided information about the background and goals of the project in an interview with AOT publisher Andreas Thoss.

AOT: Professor Tünnermann, what is the 3Dsensation project about?

A. Tünnermann: One of the major challenges in the coming years is the secure communication between humans and machines in various living and production surroundings. The first condition is that the machines are able to detect data of their environments and interpret it appropriately. Optical sensors are very important for this.

Just like humans who essentially capture their environment through their eyes, machines need optical and electromagnetic sensors to produce pictures of their environments. But this will not be limited to cameras: it is more likely that this will involve sensor systems, which can capture distances – especially 3D information – and then process them. The aim of the 3Dsensation alliance is to develop such sensor systems and qualify them for the use in various industries.

In addition, the Innovation Alliance is concerned with how the interaction between man and machine must be designed to achieve a higher acceptance among users. The alliance explores these issues across all sectors in the areas of mobility, production, health, and safety. We have found through surveys in the submarkets that the scientific and technological challenges are comparable. In this respect, synergies should be achieved if this challenge is addressed across a variety of industries.

AOT: The Innovation Alliance is based on an invitation from the BMBF. Who, then, is behind 3Dsensation?

A. Tünnermann: In 2012, the BMBF started its program ‘Zwanzig20 – Partnership for Innovation’. The aim was to go beyond classical collaborative projects and try a sustainable approach across various branches. This involves inter-regional and cross-disciplinary collaborations. In our case, a consortium was established for a concept phase, to take up issues related to man-machine interaction. Following a 1-year design phase, a strategy paper was developed, and this was evaluated by an independent jury and was then incorporated into a funding recommendation to the BMBF.

The special feature of 3Dsensation is its enormous interdisciplinarity. Our alliance includes the work of scientists from neurology as well as those from physics and the social sciences. The BMBF had found that funding projects often were functional only along well-defined industry-specific scenarios. Once you try to operate across all sectors, it fails. Our alliance aims to find solutions for various industries and provides the foundation for sustainable cooperation between the various experts.

 More than 30 industrial enterprises and 20 research institutes currently cooperate within the 3Dsensation alliance (Photo: Fraunhofer IOF).

More than 30 industrial enterprises and 20 research institutes currently cooperate within the 3Dsensation alliance (Photo: Fraunhofer IOF).

With support from the BMBF, the alliance can bring together researchers for technology assessment with cognitive scientists and engineers, to come up with solutions that are feasible.

AOT: How is the alliance equipped?

A. Tünnermann: The consortium with partners from science and industry was awarded grants of 45 million Euros from the BMBF. Partners in the consortium include various scientific institutions, universities, and enterprises from industry. They are active in the fields of production, mobility, health, and safety. An important argument for the alliance is certainly that those who are involved in it are committed to providing substantial resources beyond the BMBF funding. Additional resources from the partners and other third-party funds in these fields are to be raised for the alliance. Overall, we want to expend more than 100 million Euros for issues of man-machine interaction in the areas such as 3D data acquisition, processing, interpretation, and reproduction in selected user scenarios.

AOT: What particular application areas do you have in mind?

A. Tünnermann: Prominent examples are in the field of autonomous mobility and include issues from production technology. A major issue is the reintegration of people in line manufacturing; for example, in the automotive industry. We are trying to realize a direct interaction between humans and robots, to combine the advantages of humans and machines symbiotically. More specifically, this will bring together the flexibility and creativity on the part of the people and the sheer force and endurance on the side of the machines in various processes.

An important issue is the question of how Germany will be established in the future, taking into account the demographic changes that will occur. The issues we see in this area include, for example, the mechanical support in the care sector. But supporting production workers is equally important. The aim is to enhance independence in different areas of work and life, regardless of the physical performance of humans.

 In its development project, 3Dsensation machines will receive the ability to incorporate visual recordings and interpret complex scenarios through innovative 3D technologies (Photo: Fraunhofer IOF).

In its development project, 3Dsensation machines will receive the ability to incorporate visual recordings and interpret complex scenarios through innovative 3D technologies (Photo: Fraunhofer IOF).

 Within 3Dsensation a new high-speed scanning system based on pattern projection was developed. It allows acquiring human body 3D scans with a frame rate of up to 1.5 kHz (Photo Fraunhofer IOF).

Within 3Dsensation a new high-speed scanning system based on pattern projection was developed. It allows acquiring human body 3D scans with a frame rate of up to 1.5 kHz (Photo Fraunhofer IOF).

AOT: What have you achieved so far?

A. Tünnermann: We have focussed on a crucial aspect in the first project. Within the framework of so-called ‘basic projects’, we have tried to capture requirement profiles across industries. Which requirements do arise in data acquisition, processing, and reproduction if we want to enable a secure interaction between man and machine?

One project has already been completed. We analyzed the natural gestures that people make and how these can be used to control machines. Another project involves the evaluation of about 20 different processes for 3D data acquisition based on defined inspection tasks. With the data that is obtained there, we can compare and evaluate these processes in terms of their usability.

This has provided a crucial basis for possible additional development projects. As an alliance, we also introduced so-called ‘inventor projects’. In these, creativity is strongly encouraged. Within the term of 1 year, and with a volume of about 100 000 Euros, partners can explore promising research approaches in the field of man-machine interaction, where these are not yet fully mature but have an enormous potential for innovation. If successful, we can use the results as a basis for possible collaborative projects with additional partners.

Examples of these types of inventive projects include a new concept for a 3D camera in mobile devices or a system for detecting the movement of lips in order to identify the spoken word.

AOT: What are the goals for the next 2 years?

A. Tünnermann: Within the alliance, we have a common roadmap, through which we want to explore the areas of data acquisition and sensor systems, as well as data processing. Moving forward, we will tackle collaborative projects, which will focus more heavily on the overall goal of the alliance, that is safe and efficient man-machine interaction. We have already defined ideas that clearly visualize the possibilities of direct interaction between man and machine. One idea is a mounting assistant that supports workers involved in production lines and in quality assurance tasks.

Essentially, we try to optimize the use of human creativity and flexibility on the one side and the strength and endurance of the machine on the other.

A highlight for us will be the innteract conference in Chemnitz in June 2016, where we will introduce and discuss various projects of the alliance. The event will also offer a wide range of opportunities for networking and partnering within the alliance and beyond.

3Dsensation alliance

More than 30 industrial enterprises and 20 research institutes currently cooperate within the 3Dsensation alliance. In addition to small and medium-sized businesses, there are large enterprises also supporting the project. The alliance’s main markets are production, health, mobility, and security. 3Dsensation pursues a transdisciplinary and inter-sectoral approach to research and brings together labor and creative studies, cognitive and brain sciences, natural and technical sciences, and law and social sciences. On the technical side, machinery should be given the ability to include visual recordings and the interpretation of complex scenarios. www.3d-sensation.de

Since his appointment in 1998 to a full Professorship in Applied Physics at the Friedrich Schiller University of Jena, Andreas Tünnermann has headed the Institute of Applied Physics as a director. In 2003, he was appointed Director of the Fraunhofer Institute for Applied Optics and Precision Engineering (IOF) and is a member of the governing board of the Helmholtz Institute Jena. During his career, he has received several awards including the Otto Schott Award from the Carl-Zeiss-Stiftung, the Berthold Leibinger Innovation Award, the Gottfried Wilhelm Leibniz Prize from the Deutsche Forschungsgemeinschaft, the Order of Merit from the Free State of Thuringia, the Thuringian Research Award and the ERC – Advanced Grant for the Development of Fiber Lasers.

Published Online: 2015-11-28
Published in Print: 2015-10-1

©2015 THOSS Media & De Gruyter