Developing a protocol and experimental setup for using a humanoid robot to assist children with autism to develop visual perspective taking skills

Abstract Visual Perspective Taking (VPT) is the ability to see the world from another person’s perspective, taking into account what they see and how they see it, drawing upon both spatial and social information. Children with autism often find it difficult to understand that other people might have perspectives, viewpoints, beliefs and knowledge that are different from their own, which is a fundamental aspect of VPT. In this research we aimed to develop a methodology to assist children with autism develop their VPT skills using a humanoid robot and present results from our first long-term pilot study. The games we devised were implemented with the Kaspar robot and, to our knowledge, this is the first attempt to improve the VPT skills of children with autism through playing and interacting with a humanoid robot.We describe in detail the standard pre- and post-assessments that we performed with the children in order to measure their progress and also the inclusion criteria derived fromthe results for future studies in this field. Our findings suggest that some children may benefit from this approach of learning about VPT, which shows that this approach merits further investigation.


Introduction
Autism Spectrum Disorder (ASD) is a lifelong developmental condition that affects how people perceive the world  and interact with others [1]. ASD appears in many different forms and varies in its degree of severity. One of the most common manifestations of ASD is an impaired ability for social communication and interaction [2]. In the late 1990s, K. Dautenhahn and I. Werry began pioneering studies investigating how robots could be used as assistive tools for children with ASD [3,4]. These early studies identified that robotics offers huge potential for some children with ASD, in part due to the safe and predictable nature of the interactions it facilitates. This area of research has attracted a lot of interest and since these first studies, there have been numerous subsequent investigations worldwide into the field of assistive robotics for children with ASD [5,6]. An area in which a significant amount of research has been carried out is the use of robots to encourage social interaction and collaborative play amongst children with ASD [7]. However, to date very little research has been conducted into the possibility of using robots to assist developing Visual Perspective Taking (VPT) skills of children with ASD. The aim of the research presented in this paper is to establish if humanoid robots such as Kaspar can assist children with ASD develop their VPT skills.

Background
Individuals affected by ASD often have great difficulty with Theory of Mind (TOM), i.e. the ability to understand that other individuals have their own thoughts, plans and perspectives; this also extends to the attitudes, beliefs, desires and emotions of others [2,[8][9][10][11][12][13]. Because children with ASD struggle to understand that other people do not have the same views and thoughts as themselves, it can cause them difficulty communicating and relating to other people socially. An aspect thought to be part of TOM is Visual Perspective Taking (VPT). VPT is the ability to view the world from another individual's perspective, considering what they see and how they see it [14]. To successfully perform VPT an individual must consider both spatial and social information. Spatial information required for VPT comprises of the current location of the viewer and the target in the environment in relation to one's self and another [15][16][17], whilst the social information utilized for VPT concerns the simultaneous representation of two differing perspectives, judging if and how another individual can see an object [18]. Hamilton defines two distinct levels of VPT [19]: the first level (VPT1), is understanding that other individuals have a different line of sight to oneself. The second level (VPT2) is understanding that two or more people viewing the same object from different points in space might see different things. There is some dispute amongst the scientific community as to the potential relationship between TOM and VPT, because both rely on simultaneous representation of two differing perspectives [18]. Some believe that TOM and VPT share common cognitive processes [19], whilst others suggest that they may be entirely separate [20], however the results of studies into VPT amongst children with ASD are somewhat inconsistent [19,[21][22][23]. Pearson et al. reviewed 13 different studies that focus on VPT and concluded that many children with ASD appear to be able to perform VPT1, but often struggle with VPT2 [24]. VPT also plays a role in joint attention, another area in which children with ASD often struggle and robots have been used to assist [25][26][27].
In recent years various robotic systems have been developed to research how they can be used to promote social interaction skills and mediate interaction for people with and without cognitive and/or physical impairments. Artificial pets such as the baby seal Paro [28,29], the teddy bear Huggable [30,31], the cartoon-like robot Keepon [31], and humanoid robots such as Nao [32][33][34], the robotic doll Robota [35][36][37] and the child-sized robot Kaspar [38] were designed to engage people in personal experiences stimulated by the physical, emotional and behavioural affordances of the robot. All of these robots have been used in research with children with ASD along with many others [7,39,40]. In Scasselati et al.'s review of Socially Assistive Robotics (SAR) systems designed for autism therapy, it is reported that although the robotic platforms vary significantly in terms of visual appearance and behaviour, they have been shown to evoke prosocial behaviours such as joint attention and imitation from many children with ASD [41]. Projects such as Aurora [42] and IROMEC [43] have also investigated how humanoids robots such Kaspar and Robota, and mobile robots such as the IROMEC robot can be used as social mediators to help facilitate and promote social communication and interaction amongst children with ASD. Children with ASD can be quite solitary and often will not engage in social or collaborative play activities with other children because of the difficulty they have in understanding the views, beliefs and intentions of others [44]. A study by Wainer et al. designed and tested a collaborative triadic interaction game using the Kaspar robot [45]. The games were designed to have pairs of children with ASD playing games with Kaspar. Each participant (including the robot) was an equal player in the game. In the study Kaspar operated completely autonomously, and used information on the state of the game and behaviour of the children to engage, motivate, encourage and advise pairs of children playing an imitation game. The study demonstrated how a humanoid robot can be used to foster and support collaborative play among children with ASD.
In this paper we present a new approach where we aim to help children with ASD in developing their VPT skills using the Kaspar robot. Using a robot to teach children about VPT has a distinct advantage in the fact that what the robot can see can be shown directly to the children using the cameras in the robot's eyes and a screen on which to present the robot's perspective/what the robot can see. To our knowledge this is the first time that this approach has been tried to help children learn about VPT. Although literature indicates that children with ASD can perform VPT1 tasks, VPT2 tasks are often much more difficult and sometimes impossible for these children. We therefore aimed to start with VPT1 tasks and increase the difficulty of the tasks to VPT2 over the duration of several sessions. It is believed that neuro-typical children develop VPT1 skills between the ages of 18-24 months [46][47][48][49], whilst VPT2 skills develop around the ages of 4 to 5 [50]. However despite this, the majority of the studies reviewed by Pearson et.al. worked with children over the age of 11, as the mental age of children with ASD is often lower than their chronological age [24]. In attempting to devise an approach teaching children with ASD about VPT, we developed games that included elements of well known children's games such as "I Spy" and "Hide and Seek", which the children could play with the Kaspar robot [51]. The games involve a number of different combinations of actions, starting with moving objects into and out of the robot's field of view (Figure 1), and even physically controlling the robot's line of sight.
The key to these games is giving the children the ability to see the world from the robot's perspective and to assist them in learning about VPT. The games were specifically developed to not be reliant on the child's ability to speak, which many children with ASD at the lower-functioning end of the spectrum have difficulty with, in order to maximise the number of children that could play and benefit from the games. The development of these games was based on a number of factors. The primary factor that we initially considered was the literature on VPT and how this related to our previous experience of developing scenarios for children with ASD using a humanoid robot [38,52,53]. Further to this we consulted teachers that specialised in working with children in special needs schools to gain feedback on the games we devised. To implement these games we used the humanoid robot Kaspar and a screen next to the robot to display what the robot can see through its eyes.

Ethics statement
This research was approved by the University of Hertfordshire's ethics committee for studies involving human participants, protocol numbers: acCOM SF UH 02069 and cCOM/SF/UH/02080. Informed consent was obtained in writing from all parents of the children participating in the study.

Initial proof of concept with three children with ASD
Because VPT is about understanding what other individuals can or cannot see, at the start of our investigations it was not clear whether the concept of using a screen to display "what Kaspar sees" would work, i.e. if this would be understood by children with ASD. Thus, this basic concept was tested with three children aged 3-5, diagnosed with ASD, in a special needs school. It was very important to first test the concept with low functioning children with ASD to establish if the children would understand the concept that what they see on screen next to the robot represents what the robot sees through its eyes. The children, who were already familiar with Kaspar due to previous studies in the school, engaged in free interaction play scenarios with the robot. But in addition to this the experimenter introduced several games that involved moving toys into and out of the robot's field of view, then asking the child if the robot could see the toys. Furthermore, the child was also encouraged to control the direction of the robot's head manually (by rotating Kaspar's head) so that the robot could see the toys that were placed in different locations in the room. The experimenter demonstrated to the children how to verify what the robot can and cannot see by looking at the screen. The three children that participated in this proof of concept study possessed different cognitive abilities. One child did not grasp the concept that the screen displays what the robot can see, whilst another child had difficulty understanding this concept initially, but began to understand towards the end of the session with the help of the experimenter. Figure 2 (left to right) shows step by step how the experimenter first demonstrates to the child that the screen displays what Kaspar can see, and then the child brings a toy into the robot's field of view and then verifies this on the screen situated next to the robot. The third child understood the concept very quickly and the experimenter had to engage the child in other games with the robot to keep the child engaged in order to sustain the interaction. Overall, this proof of concept study was encouraging, showing the potential of using the screen, while confirming the variety of abilities one can find in children diagnosed with ASD. One of the core lessons learnt in this proof of concept study was the importance of mirroring the image on the screen. If the image on the screen was not mirrored, objects in the field of view of the camera would appear to be moving in the opposite direction from the camera's perspective which would be very confusing for anyone interacting with the system, particularly young children. This is a lesson that was taken forward into all future studies conducted. Furthermore, all of the children that took part in this initial proof of concept study learned at very different rates. To address this issue, the games we developed for future studies gradually increased in difficulty. More details about the initial proof of concept study can be found in a previous article [54].

Figure 2:
The child is learning that the screen shows what the robot is seeing. The child will move the object into and out of the robot's view, and the robot will give positive feedback when it can 'see' the object -not too close to, and not too far away from its face.

Equipment and setup
Before developing the games for this work there were some technological considerations to take into account.
The games devised would all require the potential to apply a level of automation to them. Because this research is part of the EU Horizon 2020 BabyRobot project, a project which aims to develop semi-autonomous robotic systems that can work in real world settings and assist with real world problems, it was important to strike a balance between what will be useful to develop for teaching the children and what is possible to develop with the current technology and sensors available. As a result of this all of the games devised have the potential for technology to automate some aspects of the games. The standard layout can be seen in Figure 3. The cameras used to record the sessions had wide angled lenses to ensure that the child was always in view. The footage from the cameras was later used to verify how the children were progressing and also to review how we could improve the games for future studies. The screen was placed next to the robot in order for the child to see what the robot could see. There were some small variations on this setup with additional equipment being used and these changes are noted for each game. The Kaspar robot used for this work is a humanoid robot with 22 Degrees Of Freedom (DOF) capable of performing gestures with its arms (each arm has 5 DOF) and making expressions with its face. Importantly, the eyes of the robot have high definition webcams, one of which was used to feed a live steam of what the robot could see to the screen. The robot also has a speaker built into the head which facilitates the audio features of the robot. More details of the Kaspar robot can be found in a previous publication [55].

The scenarios
After the basic proof of concept test, a number of VPT activities were developed and implemented as interactive games. To facilitate learning about VPT, 9 games were devised to become incrementally more difficult. Because of the exploratory nature of this work, prior to this long-term study, the games went through several rounds of testing and revisions [56]. Additionally, there were also some necessary, slight modifications done during this study. We present the findings of this pilot-study and outline the changes made to the games throughout the study, providing details and a rationale as to why these changes were necessary. Following on from the initial proof of concept test the children would show Kaspar animal themed toys or images and in return Kaspar would make the sound of that animal and perform some gestures to accompany these sounds. We chose to use an animal theme because children are often interested in animals, regardless of their backgrounds and ages, and as a result it will be easier to maintain their engagement. Further to this the six animals chosen for use all had distinctive sounds which could be used as a sensory reward.
In order to ensure that the children were learning the skills that the games were targeting, a progression criterion was devised. To progress to the next game the child must first complete the game three times consecutively, unprompted and unaided by the experimenter, to demonstrate that they have an understanding of what is being taught to them. By taking this approach we can ensure that the child is developing the desired skills and we have a measure to establish how well the child is progressing.

The games
-Game 1: Show me an animal and I'll make the sound: The first game is a VPT1 exercise as the children learn that Kaspar has a different line of sight   from their own line of sight. This game involves the child freely showing Kaspar animal toys of the child's choice. In this game, Kaspar looks straight ahead not moving its head or eyes; the child therefore needs to locate and move the toys into Kaspar's field of view. This game allows the child to explore what happens when they move a toy into Kaspar's field of view, because the toy becomes visible on the screen that shows what the robot can see with its eyes. Once Kaspar can see the toy, the robot makes the sound of the animal being shown to it by the child, which serves as a reward for the child. -Game 2: I'll ask for the animal, you find me the animal: Building on game 1, rather than free exploration, the child now needs to find the animals that Kaspar asks for. Kaspar will ask the child to show it particular animals, and the child needs to find the corresponding animal and show it to the robot in an appropriate manner in order for Kaspar to make the corresponding As with game 2, Kaspar looks in one direction only and requests to see particular animals. However, in this task the child is given a cube with pictures of animals on the faces of the cube (Figure 4). When the child shows Kaspar the requested animal picture on the face of the cube, Kaspar makes the sound of that animal as a reward similarly to the previous games. It is important to note that the face of the cube that is towards the child is different from the face of the cube that is facing the robot. This game is classed as a VPT2 exercise because the robot and the child are looking at the same object but see different things. The child needs to understand that what he/she sees is not the same as what Kaspar sees. -Game 6: I spy with my little eye: This game is based on the well-known game "I spy". The toys are placed around the room (with sufficient spacing) and the child needs to work out and pick up the toy that Kaspar is referring to and show the toy to Kaspar. Unlike all the previous games, in this game the child will not have the assistance of the screen because we are beginning to try and encourage the children to work out what Kaspar can see without referring to the screen. This is a very important step because in real life interactions with other people the child cannot see what other people can see via a screen. In these games the screen is simply used as a stepping stone to help teach the children about VPT and at this stage we want to try and get the children to complete the game without the assistance of a screen. -Game 7: What can we see? This game is a VPT2 exercise in which a physical separator device is placed on the table between Kaspar and the child. As shown in Figure 5, the separator allows three positions: in the first position, the toy can be seen by both Kaspar and the child. In the second position the toy can be seen by Kaspar only, and in the third position the toy can be seen by the child only. In this game the child places one toy in the holder and the researcher rotates the holder into one of three positions before asking the child questions about the visibility of the object. As with game 6, the screen is not available to the child as a point of reference. -Game 8: Who can see what? Similarly to game 7, the child will answer questions on the visibility of toys placed in a holder, however in this game the child will place three toys into the holder and the holder has 3 different positions in terms of the toys' visibility to the robot and the child ( Figure 6). Again Kaspar asks the child questions about the visibility of the toys in the holder. -Game 9: Where will I look? This game is inspired by the well-established Sally-Anne test [8] that is used in developmental psychology to measure a person's social cognitive ability to attribute false beliefs to others. Often children with ASD struggle to view a situation from another person's perspective and realise that what they want, feel, know and think is different from another person's thoughts and feelings. In this game there are two boxes, a blue box and a red box; both have lids. The child has one toy and Kaspar asks the child to put it one of the boxes then place the lid on it whilst Kaspar watches. Next, the robot says it is tired and going to have a quick nap; Kaspar will then close its eyes. Whilst Kaspar's eyes are closed and the robot is "sleeping", the researcher encourages the child to move the toy into the opposite container and place the lid on it. The researcher then asks the child to wake Kaspar up to continue playing. When the robot wakes up, the researcher asks the child to point to where the robot would look for the toy. The child should point to the last place where Kaspar saw the object if they have developed TOM. Kaspar then states where it thinks the toy is i.e. where it last saw the toy. If the child does not identify this correctly the researcher explains to the child that the robot did not see the child move the toy and would have looked in the container that it last saw the toy in. This is to assist the child in learning about TOM and assess their progress.

Pilot-study with 12 children with ASD
As a pilot study, the objectives of this study were largely exploratory. Our main research question was to see if any differences in the pre-and post-testing of the children could be found following an intervention period with the robot Kaspar.
The main study was conducted in a local specialist secondary school in Hertfordshire for children with moderate learning difficulties. Twelve children aged between 11 and 14 years old participated in the study. The children varied in ability which was reflected by the range of their non-verbal mental age, which was between 6 and 14. Of the 12 children that took part 5 were female and 7 were male. 11 of the 12 children had been diagnosed with ASD whilst one child was diagnosed with global developmental delay which gave the child ASD traits.

Pre-and post-testing
In order to measure the potential impact of the games on the children, three tests were conducted with the children before and after all of their sessions interacting with the robot. These assessment tools were chosen because they have previously been used in the literature for children with ASD. Note: The pre-testing was conducted after the short-term pre-study pilot so 4 of the children had interacted with the robot before the pre-testing was conducted.

The Smarties test
The Smarties test is designed to establish if the child has a theory of mind by asking a series of questions about the contents of a Smarties tube (Figure 7) [57]. The tube is shown to the child then the child is asked "what do  you think is inside". Very often the child would say either "chocolate", "sweets", or "smarties". When the tube is opened the child sees that there are pencils inside rather than the sweets as they had expected. The pencils are then put back into the tube and the tube is closed. Once the tube has been closed the child is then asked what their teacher (someone who had not seen the pencils being put into the tube) would think is inside. If the child has a theory of mind they will answer smarties, chocolate or something to that effect, if they do not then the child will say pencils.

The Sally-Anne test
The Sally-Anne test is a well-known test, again designed to establish if the child has a theory of mind, in particular with regard to false beliefs ( Figure 8) [8]. The advantage of this test is that it is more accessible to non-verbal children because the children can simply point in order to answer questions rather than speak. Two dolls that look different are placed on the table, one is called Sally the other  is called Anne. The child has to confirm that they know which doll is called Sally at the beginning of the test. Sally has an empty basket, whilst Anne has an empty box. Sally places a ball into her basket whilst she goes out to play. Anne moves the ball from the basket into her box whilst Sally is out. The child then needs to indicate where the ball is and then where Sally left it. The child is finally asked where Sally will look for her ball when she is back. If the child says that Sally would look in the basket, this suggests that it is possible that the child attributed a false belief to Sally, i.e. that Sally would not have known where the ball is because she did not see it move. If, however, the child says that Sally would look in the box, this is an indication that they have not done this.

The Charlie test
The Charlie test is designed to examine the child's understanding of eye gaze [58], which is important for VPT (Figure 9). The child answers a number of questions which revolve around the concept of eye gaze. Figure 10.a shows an example question where the child is asked "which face is looking at you?", whilst the image is directly in front of the child. As the test becomes more complex the child is then presented with pictures of 4 different sweets and asked which one is their favourite (Figure 10.b). Once the child has selected a favourite then a face referred to as Charlie is placed in the middle of the sweets looking at  something different to what the child stated and an arrow is also placed on the sheet pointing at another selection that is not what the child stated or what Charlie is looking at. The child is then asked "'What is Charlie looking at?"'. If the child states the sweet that the face is looking at then this is coded as correct, if the child states the sweet they chose themselves, this is coded as an egocentric response. If the child states one of the other sweets this is coded as random. The Charlie test consists of 15 questions in total and similarly to the Sally-Anne test can be performed with a non-verbal child. Results from the Charlie test are particularly relevant to our research since it addresses eye-gaze which in turn is directly relevant to VPT tasks.

Results
In order to analyse whether the games that the children played with Kaspar were effective in improving their VPT and TOM abilities we have compared their performance in the pre-and post-assessment tasks (Smarties test, Sally-Anne test and most importantly in the Charlie test). No significant differences were found in the children's performance between the pre-and post-assessment sessions. The McNemar test has been utilised for analysing the results of the binomial data of the Smarties test (p>0.05) and the Sally-Anne test (p>0.05), whilst the Wilcoxon signed ranks test has been used to analyse success rates in the Charlie test (Z=-1.41; p>0.05), (Figure 12). Despite no significant differences being found, 7 out of the 12 children achieved a higher success level in at least one of the posttest assessments than in their pre-test assessments (Figure 11). Although only one child's performance improved statistically on the individual level, it is important to note that there were a relatively low number of children participating in this pilot-study. Furthermore, in order to find a statistically significant improvement in the children's pre-and post-assessment tasks the children would not only need to show very high performance in the post-test assessment but also a very low performance in the pretest assessment. For these reasons, we find it very important to note that 7 children showed an increased number of successful tasks in their post-test assessments and we consider that an improvement in the VPT abilities was achieved for these children, even though it was not statistically significant for most of them when analysed on the individual level. Importantly, since this was a pilot-study and the first of its kind, the initial goal of this study was to establish if the concept of the VPT games would work and identify the group of children with ASD that could benefit the most from our intervention with Kaspar, which is why there was no specific selection criteria for the children taking part in the study. Some of the children found certain games too difficult and did not manage to complete all of the games. Some other children were very capable and demonstrated a very high level of performance in the preassessments, meaning that they completed the games very easily with little effort. In both of these cases the children would not have benefited from the interactions with Kas-par in terms of increasing their VPT skills, and accordingly, there was no improvement in the post assessment results of these children which has certainly affected the results of our statistical comparison on the group level. Taking this into consideration when we compared the pre-and postassessment performance of only the seven children who actually seemed to have benefitted from the sessions with Kaspar, we found significant improvement in their success rates in the Charlie test (Z=-2.21; p=0.02), suggesting that their VPT abilities did improve over the sessions with Kaspar ( Figure 13). Since this was the first study of this kind and it was necessary to make several changes to the procedure during the course of the study, unfortunately we could not conduct an analysis of the children's progression data throughout the games. However, the above results derived from the pre-and post-assessment comparisons assisted us to identify a group of participants whose VPT and TOM abilities may potentially be improved upon by playing these games with Kaspar. This data coupled with the Autism Diagnostic Observation Schedule (ADOS) and IQ scores of the participants also helped us to define suitable inclusion criteria for a broader study in the nearfuture (see inclusion criteria below).

Inclusion criteria derived from results
To define an inclusion criteria for future studies further assessment of the children who participated in the study was conducted utilising two widely accepted measures, The Autism Diagnostic Observation Schedule (ADOS-2 Figure 14) [59] and the Leiter International Performance Scale (Leiter-3) [60,61]. ADOS has increasingly been considered as the gold standard observational instrument for use in the diagnosis and/or classification of autism and ASD. It involves direct observation and engagement of children for whom an ASD diagnosis is suspected. Its primary focus is on engagement and interaction between one examiner and the child, being evaluated through several playful activities (along with some conversational topics depending on the child's expressive language abilities) [62]. This test was administered by one of the authors of this paper who was qualified and had the appropriate training to do the ADOS assessment.
The Leiter-3 International Performance Scale is a test of nonverbal intelligence and cognitive abilities, designed to cover an age range of 3 to 75+ and to address specific disabilities. Neither the examiner nor the examinee is required to speak, and the latter does not need to be able to read or write either. The test is composed of engaging, game-like tasks that are designed to maintain the exami- nee's interest. The Leiter-3 assessment is a relatively quick and easy test to conduct, with objective scoring making it an efficient assessment [60,61]. Since the Leiter-3 is a completely nonverbal assessment, it is especially suitable for individuals with cognitive delays, speech or hearing problems, ADHD, and most importantly for the purpose of our work, assessment of individuals on the autism spectrum. It provides an IQ score, as well as a percentile and ageequivalent scores for each subtest. The Leiter-3 assessment was conducted by one of the authors of this paper who is a psychologist and qualified to perform this assessment.
Based on these assessments we concluded that participants who could benefit from the games with Kaspar had an ADOS comparison score of >6 out of 10 (meaning that their level of autism-spectrum related symptoms were "moderate" to "high"), while their non-verbal mental age was in the range of 6-14 (which was between 2 and 5 years lower than the participants' chronological age). These scores will be used as inclusion criteria in our future studies. It is also important though that these tests are applied in conjunction with our initial pre-assessments (Smarties test, Sally-Anne test and Charlie test), which have the capacity to identify whether there is room for improvement in the potential participants' VPT and TOM abilities. If the children already possess these skills, i.e. pass those tests easily, then they would not benefit from the games with Kaspar in terms of increasing their VPT skills.

Discussion
Since this pilot-study is the first of its kind there are a number of lessons that were learnt during the course of the study that will be taken into consideration when conducting future studies, and which are as follows: -Order of games: Very early in this study we realised that some of the children had great difficulty in choosing which toy to show Kaspar, even though initially it was thought that the children may find it easier to select a toy themselves than find the toy stated by Kaspar. As a result of this, after the first session the order of games 1 and 2 were swapped as were the order of games 3 and 4, so that the child would initially take instruction, then make their own choices. To be consistent with this change, all of the children experienced the original order in their first session and after this were presented with the games in the new order. In our next study we will modify the order of the games to ensure that the difficulty of the games is more reflective of how difficult the children found the games in the first pilot-study. -Progression criteria: In the pilot-study initially there was a progression rule in place that the children would have to complete a game successfully on three consecutive occasions before moving onto the next game. Because this pilot-study had no set inclusion criteria, the ability of the children that took part in this study varied greatly and as a result it was impossible to stick to this progression criteria as different children found different games difficult. For example, some children found physically manipulating the head of the robot to look at toys very difficult (in games 3 and 4), but found game 6 (the I-spy game) very easy. We therefore decided to be flexible with the progression criteria in the first study and use this as an exercise to re-assess and re-arrange the games for future studies and form an inclusion criteria to select the children that can benefit from the games in terms of increasing their VPT skills. By using an inclusion criteria in our next study, it will be possible to have a robust progression criteria that can be adhered to. -Level of assistance given: During the sessions with the children sometimes the teaching assistant and/or the experimenter would give the child a lot of assistance in completing the games. Again, this relates to the range of abilities of the participating children being so varied, but also relates to the human nature of wanting to help the child. We found that giving the children assistance during the games is perfectly acceptable and a very good method of teaching the children about VPT. However, it is important that the child can complete the objective of each game on their own without assistance of the researcher or teaching assistant. In our next study it will be important to commu-nicate this to any teaching assistants present with the children, and it is also important for the experimenter to remember this when assessing if a child has completed a game successfully or not. -Implementation of an additional game: As the children progressed to the final game some of them seemed to find the concept of the robot not being able to see while its eyes were closed difficult; as a result of this we introduced an additional game that we referred to as 8.5, as it sat between games 8 and 9. This game was played with the children if they were unable to successfully complete game 9. Although game 9 was a TOM based game it was believed that going back to a very basic level in game 8.5 may assist the children in learning about VPT and ultimately TOM. The objective of game 8.5 was to assess if the children could understand the difference between what Kaspar could see with its eyes closed, eyes open, eyes obscured with a sleeping mask and eyes obscured with its hands. This game was initially completed with the children with the assistance of the screen then once the child could demonstrate they understood this, the screen was switched off to ensure that the child did not rely on the screen. This game will also be included in our next study. -Child specific rewards: During the study, one of the children requested that Kaspar would sing a song that he had heard in a film he liked. In the next session this song was added to the robot's behaviours and used as a reward for the child when he completed a game and moved to the next level. This child's specific reward was very effective in motivating the child to successfully complete the games and was also adopted with another child in the study. Taking this into consideration, in the next study we will establish some child specific rewards as this has such a positive impact on the children's response to the games and engagement. -Use of the screen: Initially it was planned that the children would be allowed to use the screen up until game 5 but not in later games. During the study it became apparent that for the children to learn about the later games and concepts it would still be useful for them to be able to see the screen and learn what Kaspar can and cannot see. Therefore, we waivered this rule for the first study. However, in our next study the children will have access to the screen for all of the games but will also have to complete all of the games without the screen in order to have been considered successful in completing each game.

Conclusions
Despite the children in this study varying significantly in both their abilities and personalities, they all seemed to be able to engage in the majority of activities with the robot. Because this was the first study investigating if a robot could be used to assist children with ASD develop their VPT skills we did not have an inclusion criterion and therefore included a wide range of children from the ASD spectrum. This study has allowed us to both establish that using a robot to assist children with ASD develop their VPT skills and also to pinpoint which children are most likely to benefit from such sessions. Based on the IQ and ADOS assessments coupled with the pre-and post-test results we concluded that children who could benefit from the VPT games with Kaspar the most had an ADOS comparison score of >6 out of 10 (meaning that their level of autismspectrum related symptoms were "moderate" to "high"), while their non-verbal mental age was in the range of 6-14 (which was between 2 and 5 years lower than the participants' chronological age). Taking this into consideration when analysing the data from this study we found that the seven children who fitted this criterion had a significant improvement in their post assessment Charlie test scores which would indicate that the games did have some beneficial effect on the children's VPT abilities.

Future work
As this study was the first of its kind, we did not have any inclusion criteria which therefore made it impossible to stick to the progression criteria with some of the children. In our future studies, only the children who meet the inclusion criteria and thus have the potential to benefit from the intervention with Kaspar, will be included. Moving beyond this, because the research in this paper is part of the EU Horizon 2020 BabyRobot project, a project which aims to develop semi-autonomous robotic systems that can work in real world settings and assist with real world problems, a semi-autonomous implementation of the games will be implemented and tested in a school in the near future [63]. Further to this we plan to implement games where the robot learns from its interactions with children which is an approach that has been used by other researchers [64,65].