Study of industrial interactive design system based on virtual reality teaching technology in industrial robot

1 Introduction

Industrial design based on virtual reality (VR) technology is a product design theory and method based on modern information technology and integrating VR technology and modern advanced manufacturing technology [1]. Modern information technology is a technology based on computer to receive, store, transmit, and process all kinds of media information. VR is a man–machine interface technology that highly realistically simulates human behavior such as viewing, listening, and moving in the natural environment. In short, it is a computer system that can create and experience the virtual world [2]. Virtual implementation technology not only refers to those technologies wearing helmets and gloves but also includes all related technologies and methods with natural simulation and realistic experience. Its fundamental goal is to achieve real experience and human–computer interaction based on natural skills. Systems that can achieve or partially achieve such goals can be called VR systems. Modern advanced manufacturing technologies include Lean production: It requires simplifying the production process, reducing the amount of information, eliminating excessive production organization, and simplifying and standardizing the products and their production process as much as possible, as shown in Figure 1. Concurrent engineering is a way for designing goods and their associated processes (such as manufacturing and support) in a coordinated and parallel manner. Agile manufacturing: It selects partners to build a virtual firm based on competitiveness and reputation, divides work and cooperates, and works together with the same aim to improve overall competitiveness and react fast to meet user needs. Green manufacturing: It is a modern manufacturing mode that takes environmental effect and resource efficiency into account. Its goal is to minimize harmful effects on the environment and boost resource efficiency throughout the whole product life cycle, from design to manufacture, packaging, transportation, and scrap [3] not industrial design. Industrial design not only includes traditional engineering design and industrial component design but also includes consumer survey, public relations planning, market research, enterprise website design, ergonomics research, and maintenance to serve the whole process of product sales and production [4].

Figure 1

VR system flowchart.

VR technologies are used to help the learning–teaching procedure in educational articles for a long time [5]. With new technologies that are more efficient and cheaper, their usage has extended across all levels of education. Merchant et al. [6] offer a meta-analysis on the impact of VR-based tools of learning on the performance of university and high school students, which includes a meta-analysis of 69 research studies involving over 8,000 students. The findings of this study suggest that learning has improved, with specific gaming resources offering the highest benefits. Although this technology has various advantages, it also has a number of drawbacks. Students should get more familiar with environments of virtual world and enhance their technical abilities, according to Petrakou [7]. It would also be desired to address the technical challenges associated with these computer-generated environments. Dalgarno et al. [8] found that using immersive 3D virtual environments necessitates a high level of teaching and instructional support, making them challenging to use. Similarly, the value of VR environments in the field of engineering has been extended to engineering education of university, which has numerous VR projects.

Several research initiatives have been completed to provide methodological suggestions. It can be used in a variety of higher education settings, including engineering education. As a result, Bell and Fogler [9] outlined a set of recommendations for guiding educational approaches to learners depending on the instructor’s and student’s learning–teaching styles, which frequently differ. Students who used the VR module had superior outcomes, indicating stronger acquisition of the assessed competencies. Furthermore, because these learners indicated that the VR tool has helped their learning, the majority usage of technology is shifted to the motivation field. Similarly, Hashemipour et al. [10] introduced a module-based VR environment aimed for mechanical industry and engineering. The method’s usability is assessed using Kirakowski’s software usability measurement inventory technique [11], with five VR categories added: sensation of presence, obvious exit and entry points, orientation and navigation, realistic feedback, and faithful perspectives. Sutcliffe and Gault [12] designed criteria sets for calculating the VR application usability. There are 12 criteria that users must score and all are essentially directed on their interface.

Other university experiences [13] have been based on transdisciplinary ventures. Hafner et al. [14] recommended that course development is an essential step in learning, designing and developing virtual worlds. It also encourages the multiple abilities of development by presenting industrial projects among students of various degrees. The results demonstrated the improvement in breeding quality, as determined by an examination of the projects and questionnaire series.

This study’s major goal is to develop, construct, and examine a low-cost immersive VR model [15,16] to improve functional requirements of the system from the perspective of industrial robot teaching, training, and practical application. The research focuses on development modules of the system, which are industrial robot assembly module, industrial robot motion module, and typical application module of industrial robot.

The structure of this research article is presented as follows: Section 2 demonstrates the related work and background on VR in industry 4.0, teaching, training, and practical applications. Section 3 illustrates the proposed methodology of industrial robot teaching, training, and practical application. The proposed method’s prototype is constructed and described in Section 4. It also includes the results of the tests conducted to guarantee that the approach is correct. Section 5 shows the conclusion and outlook.

2 Literature review

Aided assembly, self-configured workstation layout, intelligent storage management, late customization product and process traceability, adjustment systems, and assembly control systems are all possible options within industry 4.0 [5]. Industrial robot assembly and adjustment system concepts are used in this study, which helps in industrial robot teaching, training, and practical applications. For this, technologies based on internet of things are used, such as putting sensors in every workstation, tool, and product and allowing them to communicate with one another, VR is an example of operator support. VR creates virtual environments in which users can interact with virtual items. As mentioned above, VR in industry 4.0, teaching, training, and practical applications is explored in this literature review.

Akbar et al. found that labor cost is increased with the continuous introduction of the concepts of “industry 4.0” and “made in China 2025.” The other automation strategies such as automatic production line have become one of the main production modes of factories. Author also suggested that more and more industrial robots have replaced the original workers [17].

Liu et al. analyzed the main reason for the shortage of industrial robotics talents. The first reason is the preliminary statistical data of the global industrial robot market that is released by the International Federation of Robotics at the China International Robot in 2018. The second reason is the Intelligent Manufacturing Development Forum that is published on July 9, 2019. The third reason is the slight growth of global industrial robot market in 2018 and high record point of sales volume. The last reason is the annual sales volume of industrial robots that is about 384,000 units [18]. Hong suggested that industrial robots have the advantages of good repetition, stability, high work efficiency, reliability, accuracy, and can work in high-risk environments. They are the essential part in the transition and upgrading of traditional manufacturing industries, particularly labor-intensive ones. However, industrial robots have high training cost, large occupation of space, and difficult training [19]. Sobri et al. studied that to train employees in the industry, stimulate the teenagers’ interest in robot research and development, and cultivate the reserve talents of industrial robot technology, various countries have developed a large number of industrial robots and simple development platforms for education. So that, trainers can understand and learn relevant technologies and development processes of industrial robots in teaching supplies [20]. Yu et al. studied that in the era of rapid development of computers, VR technology for short is a rapidly developing technology [21]. Bruno et al. proposed a software based on the principle of VR technology to build a virtual three-dimensional environment. After that, three-dimensional environment is transmitted to the user through the output device, so as to bring the user an experience that they can feel the reality without contacting the reality and can experience the reality of the virtual environment without leaving the home [22]. The advantages of VR technology are applied in the fields of product display, teaching, training, interior design, industrial simulation, military aerospace, and so on. Lindner et al. found that the initial application of VR is for the training of pilots and astronauts. With the rapid development of theoretical knowledge, social cognition, and science technology, VR technology is applied in the civil market [23]. So far, the VR technology mainly focuses on user interface design and production, perception research and application, hardware design manufacturing, and background software design. Wang et al. found that after entering this century, they have successively established VR training systems in space station, aviation, and other industries and gradually built a systematic VR education system [24]. Liu et al. observed that domestic VR technology has developed rapidly in recent years, and many universities and companies have made great progress. For example, Beijing University of Aeronautics and Astronautics has successfully built a distributed virtual environment, which can simulate the real environment [25]. The national optical disc Engineering Research Center of Tsinghua University successfully imported the panorama of the Potala Palace in Tibet into VR using QuickTime technology. The global VR technology has developed rapidly and made a major breakthrough. Many countries in the world are actively studying VR technology. The 3D virtual interactive system based on VR technology continues to innovate in the fields of industrial production, film and television, urban construction, and education, and its application has become more and more wide. Dinis et al. [26] engaged and motivated the students and also allowed them to understand the planning challenges that are typically limited by their prior knowledge. The goal of the project is to teach K-12 children about civil engineering and its importance in the society. In another article, Dinis et al. [27] developed a VR platform to teach civil engineering through VR games to the students of pre-university. The stimulated results show that VR is an important part of the education that allows the participant to connect appropriately with the platform with no previous experience. Valdez et al. [28] developed online electrical laboratories using the VR platform. These laboratories minimize the cost, time of teachers, and risk of experiments with dangerous strategies. For monitoring the construction progress, Sampaio and Martins [29] developed a VR model for engineering students using 3D models. Hurtado et al. [30] implemented a VR model for teaching robotics. The application included a built-in physics engine as well as haptic and visual feedback-based interaction. Robotic arms can be programmed to obey particular commands or controlled by a virtual pendant. Users trained on this system are better qualified for accomplishing tasks on real robots than their counterparts who are briefed with standard training materials, according to the results of using this model. The authors of refs 31 and 32 have also presented similar systems, with similar findings. Put et al. [33] designed a new VR system using image scanning-based 3D model. The basic aim of this study is to create a new system from the existing machinery of VR systems. According to the industry 4.0 concept, Zywicki et al. [34] described a method for learning the inner workings of an intelligent factory. Zhang and Liu [35] explored the characteristics of VR technology and its applications in PE. By playing a vital part in PE training, VR technology may better engage students in their initiative. The body movement capture tools, 3D locating, tracking systems, hand gesture, and other manual tools are the most common data gathering equipment. An open-ended platform, software application, and database models are included in the VR training system software. Melatti and Johnsen [36] offered a virtual toolbox for teachers to plan and deliver education that students may observe in the same virtual environment. A platform that closely resembles a classroom and provides the user with all transnational viewing angles. Students are seated in the virtual classroom’s center, with a PPT screen behind them. The benefits of this application are that virtual tools can be introduced to a software library on a regular basis. All teachers in the community can then use this library repository to create unique lectures by pulling tools into their VR Classroom.

3 Proposed methodology

With the development of industrial robot, more colleges and technical secondary schools have established the specialty of industrial robot technology. It is coupled with the proposal of the teaching concept of “VR integration,” which leads to the shortage of teaching supplies for industrial robot. The industrial robot assembly and adjustment system developed in this subject is mainly for colleges and technical secondary schools and other institutions engaged in industrial robot training. These institutions usually need not only a single industrial robot but also an industrial robot workstation with multiple functions such as robot body assembly, operation, programming, stacking, and handling [52]. Combined with the specific functional requirements, the industrial robot assembly and adjustment system based on VR developed in this subject needs to have the following basic functions.

3.5 Immersive effect

The traditional simulation system presents images and animation with flat panel display and takes touch screen, mouse, and keyboard as human–computer interaction equipment, which is difficult to provide users with a more real immersive experience [56]. The project uses VR equipment as an interactive tool. Using head-mounted display and hand-held controller to replace the interactive tools such as display, touch screen, mouse, and keyboard of traditional simulation software, the operator can personally complete the learning of industrial robot and effectively solve the problem of poor human–computer interaction of traditional simulation systems.

According to the functional requirements analysis of the above industrial robot assembly and adjustment system, this article designs the VR system into three modules: industrial robot assembly function, industrial robot basic teaching function, and industrial robot application. The functional module of VR industrial robot assembly and adjustment system is shown in Figure 2 [57].

Figure 2

Functional modules of industrial robot assembly and adjustment system.

At present, the necessary software tools for VR project development are 3D modeling software, VR development platform, and interactive equipment. The relationship between the three is shown in Figure 3.

Figure 3

Development tools.

Modeling is the basis of realizing VR technology. The quality of model creation directly affects the visual effect of users, and the creation cycle of model will also directly affect the cycle of project development. Common 3D modeling software include SolidWorks, CATIA, UG, Maya, 3DsMAX, etc. In the process of developing the VR model of industrial robot, SolidWorks is used to develop the basic model of VR environment, and 3DsMAX software is used to make the high–low model and normal map of the basic model. 1. In VR environment, the performance of object material is much higher than that in traditional simulation environment. This article uses Substance. Pannier to make physics-based rendering (PBR) material map, as shown in Figure 4.

Figure 4

Model development process.

For the VR industrial robot assembly and adjustment system, the interactive equipment must have more accurate positioning and large free moving space. To sum up, the system uses HTC VIVE as VR interactive equipment.

The fine details of the 3D model surface will be magnified in the VR scenario. The performance effect is higher when the model surface has more information [58]. If an object’s surface has more detailed features, then the model consumes more system resources. As a result, figuring out how to use fewer resources to occupy high-quality surface-detailed features necessitates theoretical texture mapping research to provide model mapping support. Texture mapping refers to defining the mapping relationship between image coordinates with object surface features and object coordinates, mapping them to the object, so that the object has the surface features on the image.

In the VR scene, the performance of the 3D model of the object closer to the reality and the surface detail characteristics of the object model are usually very complex, and model grid number is significantly increased, which occupies most of the resources of the running equipment, resulting in system jamming and other phenomena. The application of texture mapping technology can map the picture with the object surface characteristics to the object surface, so that the object has the detailed characteristics of the picture, that is, the model with lower mesh number shows the characteristics of the model with higher mesh number, so as to reduce the system consumption of resource and improve the running speed of the system [59].

Assuming that any point on the surface of object a is represented by g and h, then a is:

(1) A ( g , h ) = x ( g , h ) y ( g , h ) z ( g , h )

The points on the texture are mapped to A ( g , h ) in the way of (2):

(2) u = a s + b t + c , v = d s + e t + f .

4 Experiments and discussion

The development of VR project mainly includes two parts: the first is the creation of model and the second is interactive development. Model is the basis of interactive development. The visual effect of VR world is directly related to the quality of the model. The three-dimensional model of the system is divided into robot body, working platform, assembly tool, teaching pendant, and laboratory, as shown in Figure 5. The creation of the model in this article takes the industrial robot ontology as an example [60].

Figure 5

System model composition.

The system model can be divided into parts with accuracy requirements and parts without accuracy requirements according to whether there are accuracy requirements. For parts without accuracy requirements, such as motor cabinet, laboratory, and teaching pendant of workstation, 3DsMAX can be directly used to create the model [24].

• 3DsMAX modeling software is not applicable to parts with accuracy requirements, such as mechanical assembly modules of industrial robots.

  To solve this problem, SolidWorks is selected as the development of the basic model in the development process of this subject. Because of its extensive application foundation in the machinery industry and its powerful and convenient solid modeling function, the time cost for the development of the system model is significantly reduced [61,62]. In the VR environment, the details of the 3D model will be magnified and observed.

• All real materials are the key to affect the immersion. Especially in the VR application in the machinery industry, more real material performance is required.

To solve this problem, this article imports the basic model created by SolidWorks into 3DsMAX for high–low model making, UV spreading and normal baking, and then uses substance painter to make PBR materials. To sum up, the VR model development route of the project is shown in Figure 6 [63].

Figure 6

VR model development route.

Because most of the parts in the VR industrial robot assembly and adjustment system are standard parts, they can be called directly from the part library. For the modeling of nonstandard parts, first, the dimensions are obtained according to the solid model, and the 3D model is created with the commands of sketch drawing and feature editing in SolidWorks 3D software according to the dimensions. Finally, the assembly function of SolidWorks is used to assemble various parts in the environment. The modeling process is shown in Figure 7.

• In this article, Unity3D is used as the VR development platform, which is not compatible with the file format exported by SolidWorks.

  To solve the compatability problem, the model is exported from SolidWorks to standard tessellation language (STL) format file. STL format file refers to the three-dimensional light modeling file. It was originally formulated by an American company called 3D Systems in 1988. It is an interface protocol and serves the three-dimensional prototype manufacturing technology. The principle of the format file is to mesh the three-dimensional solid model in space and use the disordered spatial triangular pieces to approach the three-dimensional solid model infinitely. Because the file stored in STL format has simple structure and simple data processing, with the development of computer-aided design, it has developed into a common format for data conversion in various 3D modeling and manufacturing equipment. At present, almost all 3D modeling software can export STL format files.

• When SolidWorks exports STL format files, each component generates a separate file. 3DsMAX cannot batch import models without plug-ins when importing STL format. If each component is imported separately, the relative position relationship between each component will be lost and the development efficiency will be significantly reduced.

  To solve this problem, this article uses Maya 3D modeling software as a tool for batch importing STL format files. After importing, the relative position relationship between the components of the manipulator will be retained and then sent to 3DsMAX for subsequent processing.

• When the number of faces of the model is reduced, the operation performance of the system can be significantly improved, but the detailed performance of the model will be reduced with the reduction of the number of faces.

Figure 7

SolidWorks modeling flowchart.

To solve this problem, this project adopts the form of low face model (low model) combined with normal map to present the detailed features of high face model (high model). Before making the normal map, you need to obtain the low model and the original model [64,65]. In this article, the reduced model is the low model, and the original model is the high model. The process of baking normal mapping is shown in Figure 8.

Figure 8

Baking normal mapping process.

5 Conclusion

This article analyzes the functional requirements of the system from the perspective of industrial robot teaching, training, and practical application. The main development modules of the system are determined, which are industrial robot assembly module, industrial robot motion module, and typical application module of industrial robot. The overall framework design of the system is completed. Through the determination of the main development modules and the analysis of the current mainstream development platform, it is finally determined to use SolidWorks and 3DsMAX as three-dimensional model development tools. Unity3D is used as the VR development engine. HTC VIVE is used as a VR device. The overall scheme design of the system is completed. By studying the model and interaction theory in VR, it provides theoretical support for the development of 3D model in VR environment. It also introduces the development process of three-dimensional model in the virtual environment of industrial robot assembly and adjustment system and the production process of PBR (based on physical rendering) material in detail, completes the development of VR model, and builds the system model scene, which provides a three-dimensional model basis for the follow-up VR motion control. Through the research on the software and hardware configuration of developing VR system and the VR basic interaction such as pointer, transmission, and grasping mechanism, a set of C# script framework is designed to realize the assembly function of industrial robot ontology.