Learning gestures and procedures has always been a substantial issue for professionals who must acquire specific know-how. For instance, in medical field, the exclusive study of textbooks is not enough to acquire patient care skills. An experienced doctor must supervise an apprentice doctor: the novice will have the first experiences with a dummy and he will suture oranges to improve his own practice. In the same way, to learn how to shape wood, the apprentice carpenter must gain experience through many tests and attempts: in other words he has to be able to make mistakes to improve his technic. Nevertheless, some professions exist in which mistakes are not tolerated. This is the case, for instance, of airline pilots: for them a wrong manipulation could lead to death of the apprentice himself, and even of the people in the aircraft. Regarding the operators working in a factory producing nuclear energy is possible to realize that a haphazard handling could lead to a catastrophe. The same issue can be found in the case of pharmaceutical factories. Operators can not at all work in an aseptic environment before they have achieved their qualification. By then, the idea of producing a simulator to train and evaluate the professional skills of workers required to intervene in cleanrooms has been formulated. A cleanroom is an environment conditioned by an air filtration device in order to prevent the intrusion of particles. Parameters such as temperature, humidity and pressure are maintained at a specific level. Access to this environment demands a severe training. Workers must scrupulously observe the standards of hygiene, dress and behaviour.
This article presents the scientific work of design and development of a cleanroom simulator, LabQuest, resulting from the collaborative work of WhiteQuest (a French company) and LISEC (the Interuniversity Laboratory of Educational Sciences and Communication). Then, the scientific protocol of tests and results will be present and discuss in order to confirm the relevance of the simulator for pedagogical purposes.
Simulation and learning
Simulation is a learning strategy having as its main characteristic the possibility to reproduce, in a protected environment, specific aspects of real life. As Landriscina  describes, simulation is “an interactive representation of reality based on the construction of a model or a system in which you want to understand a given situation.”
Simulation is one of the basic methods of learning: it is used from elementary classes for solving algebraic problems but also in medical field, to teach and train nurses and doctors (using dummies for complex manipulations). In recent decades, the evolution of computer technologies has developed the virtual technics, which has made it possible to accurately reproduce real situations . Users can manipulate these environments and they will benefit from a successful learning.
The advantages of a simulator, compared to the reproduction of actions in real life, are numerous: (1) it permits to break free from problems of space-time, (2) it makes it possible to practice procedures without affecting the personal’s or other people’s safety (for instance, the simulators for airline pilots), (3) it permits to build virtual models and to test their feasibility and relevance .
Simulation and pedagogical engineering
The development of a simulator, as well as other learning tools, requires a complex pedagogical engineering that takes into account the fundamental learning aims and the available simulation techniques. In this study, we have developed a pedagogical engineering protocol to design a simulator able to train professional gestures in order to work in cleanrooms. This protocol, reproducible to deploy any new simulator, lays its foundations on the work activity analysis, developed by professional didactics.
Professional didactics has designed a method to analyse human activity in work attitude so as to develop the best strategies to train professional skills . To do this, it is first necessary to complete an inventory of professional actions  and standard procedures that are used in a specific profession. These actions are listed from the norms prescribed in the official protocols. Secondly, it is useful to compare these standards with the actions actually implemented by workers . The role of the representations that the same workers have on their own practice must not be underestimate. Often, these differ according to the role and hierarchical position of the person in the organization. This piece of information is essential material to complete a diagnosis of fundamental training contents to be integrated into the simulation software [6, 7].
Research questions and hypothesis
Today it is possible to observe that training and evaluation tools, available in pharmaceutical field, are not adapted to the needs of factories in which these methods are used.
The goal of training professional practices is not achievable with the existent methods. The pedagogical supports available give only an explanation of an ISO [11, 12] standard, most of the time too abstract and distant from the praxis of the employees. The result is a non-customized training lasting from 6 up to 8 weeks. By then, managers and trainers claim an approach more focused on practice and the repetition of professional practices in order to learn how to operate in a cleanroom. Nevertheless, there is no way a novice could directly access the cleanroom because of risks for his own health, for the environment and for the company, in case of error. These are all the reasons why the design of a simulator for training and assessment of GMP (good manufacturing practice) and professional practices is interesting.
By then, the hypothesis that the design of a simulator offers a pedagogical value not achievable with traditional methods. In other words, professionals trained with the LabQuest simulator should achieve a better performance than those trained with traditional methods (video, QCM, PPT).
Conception of the simulator
LabQuest is the result of a process based on two theoretical contributions: the first is the analysis of human activity at work and the second is instrumental conflict . The analysis of work activity provided the key that to identify, collect, analyse and select the fundamental contents to design the training tool. The instrumental conflict shaped the method of selection and adaptation of pedagogical contents to digital technologies in a bi-directional way: the pedagogical contents were selected according to available digital technologies and, in the same way, the choice of technologies was influenced by the pedagogical aims we wanted to achieve.
For the realisation of the simulator, five phases were necessary.
The first phase consists in identifying and compiling an inventory of the procedures, gestures and standards of the aseptic zone activity. Information has been collected from: (1) interviews with the leading personal of three sites of different sizes and manufacturing different products, the goal being to collect their expectations regarding the work of their employees, (2) the textbook analysis of Ref.  in order to derive standards and fundamental rules, (3) interviews with employees of the same production sites so as to have real workers’ testimonies.
In the second phase, a choice of actions, procedures and standards, that have to be integrated into the simulator has been accomplished. These contents must be in accordance with the available techniques. The simulator offers 10 professional gestures (dressing, surface cleaning, etc.) and 60 pedagogical contents.
A third phase consisted in designing the scenario based on the following requirements: (1) to reproduce a typical day production by defining the basic steps common to all factories, (2) to integrate the rules, standards and common gestures to all target factories, (3) to reproduce a common type-environment (Figure 1 and 2), (4) integrate professional practices that are also common to all target sites .
In the fourth phase, an algorithm of score was developed according to a prioritization of tasks (in order of importance) characterizing a procedure. Other factors are considered: the elapsed time, the number of hesitations and the progress in performance.
The last and fifth phase consisted in the realization of human-machine interface. It includes options to interact with objects, the environment and the avatar (Figure 3). The interface was designed in line with intuitiveness and simplicity criteria to reduce the difficulties related to the use of the simulator. For example, when an object is selected, it appears at the bottom of the screen in the virtual user’s hands; the displacement of an object is done by drag and drop to the place of destination . In this phase, numerous usability tests have been realised in order to improve the ergonomics of the simulator [6, 7].
The scientific protocol
In order to verify or disprove our hypothesis about a better efficiency of using LabQuest for training professional skills, a very sharp and articulated scientific protocol has been designed.
This article explains it in detail, highlighting the critical elements: indeed, we had to make very considered decisions not to generate biases in the results of the tests.
Execution of the tests (Figure 4)
45 people of different ages and professions participated in the test. However, was made sure that none of the subjects had ever worked in an aseptic area. Indeed, even a partial knowledge about how to behave in a sterile environment could influence the performance of the subjects (and bias the test). We decided not to test people working in cleanrooms to nullify the effect of possible parallel training that would have had a consequence on their performances.
An office in LISEC laboratory has been employed for the realisation of the tests. In this room, an environment such as the one that can be found in a sterile area was reproduced. The objects available were:
Disinfectant hand gel;
A water-alcohol solution for cleaning and disinfection of objects and surfaces;
Petri dishes 3 available inside the RABS;
A pen available inside the RABS;
A production file placed on the table;
A pen lying on the table;
A translucent surface that was laid horizontal.
The participants in whole sample were asked to realise the procedures that are normally employed in a cleanroom. For example, the bio-cleaning, disinfection of hands, the substitution process of the Petri dish. To carry out these procedures, subjects disposed of specific objects (see the list of materials). No further information was given.
At this point the sample was divided into two groups (LabQuest protocol and traditional protocol). During the training with LabQuest, the subject is conducted in the 3D environment by a path indicated by” pins.” Each “pin” represents a pedagogical content. After reading this content, the person can perform the action in the simulator. The group B is trained with a traditional protocol. This training consists in reading a document, in which information about the GMP (Good Manufactory Practices) had been integrated. We controlled that the information available in the document was the same as the one available in the LabQuest protocol.
The whole sample was asked to re-execute the same procedures proposed in the PRE-TEST phase, in the same conditions, without any variations to the first round. The aim was to observe any potential increase in performance.
Video of mistakes
To the whole sample it was asked to watch a video presenting the critical procedures characterizing a standard production day. Subjects were asked to comment in real time and report any errors perceived, done by the characters.
Observation methodology and coding of data
The execution of the tests was systematically recorded in order to save all data.
A camera recorded the procedures performed in the PRE-TEST and POST-TEST. At the same time, the researcher compiled an observation sheet to secure data in case there might have been an error in the starting of the recording, if the battery of the camera got discharged, etc.
A screen and audio recording system had been installed in order to record the LabQuest training protocol. In traditional training, only the audio was recorded, to have a record of comments made by the subjects.
A screen and audio recording system had been implemented so as to record the detection of mistakes made by people while watching the video.
All the collected data were inserted in the database software used for data analysis: SPSS. Every finished procedure was decomposed into a series of simple tasks that had been accomplished. Each task was transformed into a nominal variable: while observing the pre-tests and post-tests, the researcher classified the tasks as “well executed,” “poorly executed,” “well executed, but at the wrong time,” etc. Errors reported during the video broadcast were classified as “relevant error reported,” “relevant not reported error,” “non-relevant reported error” and “non-relevant not reported error.” It was made a difference between a relevant and non-relevant detected error. In fact, a “relevant error” is defined as a non-compliant gesture or behaviour according to GMP rules. For instance, touching the outward appearance of the uniform is recognised, by most of the people, as something not conform to the GMP. This idea is actually true. A “non-relevant error” is defined as a gesture or a behaviour witch is judged as non-compliant by a non-experimented person but, which is actually accorded with GMP rules. For example, most of the people not knowing the GMP think that putting the Petri dish against the breast is an error. On the contrary, this gesture is actually fundamental: taking a sample from the collaborators uniform so as to analyse its contamination grade.
In order to find a correlation between the results obtained in the POST-TEST and the protocol given (LabQuest or traditional), we applied a χ2 test.
All audio and screen recordings were transcribed to be used for qualitative analysis which will take place with a detailed analysis.
Preliminary observation and results
Is possible to obtain, in a very early process phase of analysis promising results. A correlation analysis has been realised on the whole sample (LabQuest and traditional protocol) with:
the data obtained in PRE-TEST;
the data obtained in POST-TEST;
the data obtained in a broadcast of the “video mistakes.”
First an omogeneity test has been made concerning the PRE-TEST performace. The aim of this test was to verify that subjects of both groups (LabQuest and Traditionnal) had the same level of knowlege concerning GMP in order to create equitable groups.
A χ2 test has been performed for the POST-TEST data. The aim was to find if there was a difference, between the two groups, concerning the performance realised after the training. Farther, a repeated measure ANOVA (Figure 8) has been realised in order to verify the causal relation between the training and POST-TEST performance.
A χ2 test has been performed for the “video mistakes” data. As before, the aim was to find if there was a difference, between the two groups, concerning the perception of errors that were committed by others peoples shown in the video.
More correlations analysis with the χ2 test has been realised in order to study the progression of washing hands and the growing procedures relates to the subjects’ activity in the simulator.
In the x-axis can be observed the list of the tasks composing the bio-cleaning procedure of a surface. In the y-axis can be observed the percentage of success in finalizing those tasks. The results show that people who have done training with LabQuest have a better performance in achieving the tasks composing the whole procedure than people who had a traditional training.
The x-axis lists all the errors that were detected during the broadcast of the video (errors are rated as “relevant – RE” and “irrelevant – IE”) (Figure 6). The y-axis presents the percentage of people who have detected the errors. The results show that people who did training with LabQuest have a better performance in the detection of relevant errors (RE) compared to people who made a traditional training. Regarding irrelevant errors (IE), we can observe that there is no significant difference between the two groups.
87.6% of the subjects, who completed the LabQuest protocol correctly, succeeded in accomplishing gestures and procedures in the POST-TEST phase, while 57.5% of the subjects trained with the traditional method succeeded. In particular, for the professional gesture of “bio-cleaning,” (Figure 5) 100% of the people succeeded after training with LabQuest, whereas 65% of those trained with the traditional protocol (p=0.003 χ2) did. Again, it is possible to observe that 78.3% of the subjects trained with LabQuest succeeded in precise gesture as “good positioning of a Petri dish lid, after opening it,” while 45.5% of the subjects trained with the traditional protocol (p=0.058 χ2) did.
Concerning the exercise of checking mistakes while watching the video, 89.3% of the subjects who completed the LabQuest protocol identified all non-compliant behaviours and mistakes, opposed to 52.2% of the subjects trained with the traditional protocol. Specifically, 100% of the subjects who completed the LabQuest protocol have detected serious non-compliances as “not closed out the Petri dish of the machine,” opposed to 58.8% of the subjects trained with the traditional protocol (p=0.002 χ2). 88.9% of the subjects who did the training with LabQuest identified a wrong bio-cleaning surface procedure, while 58.8% of the subjects who followed a traditional training (p=0.008 χ2) did. Again, 83.3% of the people trained with LabQuest reported the “self-touch error” (replacement of mask in the production area), opposed to 58.8% of the subjects trained with the traditional protocol (p=0.109 χ2 ns).
The pre-test and post-test performances of subjects were assigned a rating on a scale of 0–20, to compare the results between the two protocols (Figure 7).
On the chart, one can notice that both groups (LabQuest and traditional) had in the pre-test approximately the same result (LQ m=10.4; Trad m=10.04). On the contrary, is possible to observe that in the POST-TEST, the subjects who followed the LabQuest protocol obtained a best result compared to those who used the traditional protocol (LQ m=17; Trad m=14.65) (Figure 7). In addition, one can notice that the subjects who followed the traditional protocol were led to a worst result than those trained with LabQuest (the lowest grade relating to the traditional protocol is 10/20, while the one related to LabQuest is 14.5/20). No differences are noticed concerning the highest grades.
On the chart is possible to observe the performances obtained by subjects before (1=PRE-TEST) and after (2=POST-TEST) the training. The repeated measures ANOVA test shows a causality effect of the training LabQuest on the POST-TEST results. F [1, 39]=8.190, p<0.01 and eta-squared η2=0.174.
A χ2 test has been realised in order to check whether there were a correlation between training with LabQuest and improving the hand washing performance in the post-test.
Eight people out of 15 (53.4%) improved their POST-TEST performance after the display of the hand washing procedure in the simulator (Figure 9). This result is significant (p=0.068 χ2) if we use a 90% confidence interval, possible when we work with a sample of n=23.
Here (Figure 10), one can observe how people integrated rules during the training with the simulator.
During the test, 18 subjects out of 23 (78.2%) clicked on the “gowning procedure” content even if it was not required by the protocol. After that, 12 out of 18 people (66.7% of the people accessing this content and 52.2% of the total) corrected their gowning procedure performed after the training with the simulator (p=0.008 χ2).
Conclusion and perspectives
The results obtained in this preliminary analysis phase seem to promise the verification of the hypothesis that states a better efficiency of the LabQuest simulator compared to traditional methods. In particular it is possible to notice that the people who did the training with LabQuest have a better performance in the implementation of procedures. In addition, it has been noticed that people who have experienced the simulator develop a finer critical sense and a major confidence in the detection of errors and non-conformity observed in other people’s practices.
Further quantitative analysis will take place soon. This will be used to understand what impact the activity of learners in the simulated 3D environment has on the performance in real conditions. To make a practical example, it will be observed if the procedure of “substitution of Petri dishes in the machine” performed in the simulator is realised identically in a real situation.
A detailed analysis of the learning process will take place as part of a qualitative study. This analysis will seek to understand how a realistic simulation supports the learning process. The difficulties and advantages that this new pedagogical tool provides compared to traditional methods will be highlighted.
A new study in vivo will take place soon with an industrial partner. The aim will be to measure the performance progress of the people following LabQuest training. An evaluation of the influence of the simulator on the non-conformity or non-compliant behaviour but also the lot withdrawals will take place.
1. Landriscina F. La simulazione nell’apprendimento. Quando e come avvalersene. Trento: Erickson, 2009. Google Scholar
2. Guéraud V, Pernin JP, Cagnat JM, Cortés G. Environnements d, apprentissage basés sur la simulation. Sci et Tech Éduc 1999;6:95–141. Google Scholar
3. Samurçay R, Pastré P. (eds.) 2004. Recherches en didactique professionnelle. Toulouse: Octarès Éditions Google Scholar
5. Cizeron M. Le geste professionnel comme concept et outil d’analyse. M. Cizeron et N. Gal- Petitfaux. Analyse des pratiques: expérience et gestes professionnels. Presses de l’université Blaise Pascal, 2010:257–268.
7. Morcillo A, Lutz G, Amiel A, Camps J-F, Plégat-Soutjis F, Tricot A. 2003. Utilité, utilisabilité, acceptabilité : interpréter les relations entre trois dimensions de l’évaluation des EIAH. Presented at the Environnements Informatiques pour l’Apprentissage Humain 2003, ATIEF INRP: 391–402
9. Denami M, Marquet P. 2015. Un simulateur pour l’évaluation et la formation des compétences professionnelles des opérateurs en zone aseptique. Presented at the Environnements Informatiques pour l’Apprentissage Humain 2015, ATIEF. Agadir 3–5 June 2015.
10. Marquet P. Obstacles to use ICTs in training and consequences for the development of e-learning and m-learning. Educ Knowl Econ 2011;4:183–192. Google Scholar
14. Reski R. Development, genetic and molecular biology of mosses. Bot Acta 1998;111:1–15. Google Scholar
RABS (Resticted Access Barrier System): is an installation which is used in many industries, such as pharmaceutical, medical, chemical, etc. where clean air is needed. The RABS provides a physical barrier between workers and production areas .
Petri dish is a shallow cylindrical glass or plastic lidded dish that biologist use to culture cells, bacteria or mosses. In pharmaceutical factories it is included in a machine to control the contamination level of the air inside the machine .
About the article
Maria Denami is a pedagogical engineer working in WhiteQuest, a French company producing innovative softwares. She did her bachelor study in psychology, in Milan, and her Master thesis in Strasbourg, in the topic of learning with informatics technologies. She is now finishing her dissertation in Educational Sciences. Her study is about Serious Games and Simulator design. In this field, she has developed a new protocol in order to create pedagogical-virtual tools game-based. WhiteQuest has employed this same expertise in order to develop the simulator of clean rooms, today used by many important pharmaceutical factories.
Published Online: 2016-01-15
Published in Print: 2016-03-01
Conflict of interest statement: The author states no conflict of interest. She has read the journal’s Publication ethics and publication malpractice statement available at the journal’s website and hereby confirms that she complies with all its parts applicable to the present scientific work.