Categorizing Social Robots with Respect to Dimensions Relevant to Ethical, Social and Legal Implications

Ethical Issues

Autonomy is a central category when it comes to discussing the ethical implications of robots. In general, one could slightly change the well-known phrase to “with great autonomy comes great responsibility,” since the more autonomous a robot is, the more likely it is to need to make decisions that are morally relevant, such as deciding between helping a user complete an action that causes harm or disobeying the demands of the user [47]. Designing a robot with a high degree of autonomy but with no means to track morally sensitive situations and no capacity for moral reasoning or following moral rules can create highly problematic challenges [29]. Morally significant situations generally arise much earlier than the debate on autonomous driving or war robots sometimes suggests. In “weaker” scenarios, the robot need not make a life or death decision but rather – for example – decide how to act based on weighing various moral values [1]. And, even if the choices the system must make are not morally prohibited, they still often reflect a preliminary decision regarding ethically relevant values. A technical care system with a lower threshold for alarming relatives may, for example, be more secure but, on the other hand, more annoying, due to possible false positives and, in extreme cases, also lead to a problematic “normalization” of its user’s lifestyle [23].

Participatory Issues

A robot’s degree of autonomy has an implication on how it should be developed. Autonomy of a robotic system can have a positive effect on the productivity of a robot and can lead to the perception of it as a useful tool. It can also encourage the belief that the robot is smart and possesses an AI and therefore some sort of personality, but at the same time, it can be frightening for users, since its functionality can be difficult to understand [40], [52]. Autonomy of a robotic system has the effect that its users need not take care of the machine at all times; at the same time, this means that they are losing the control over it [41], [44]. This loss of control can result in a reduced understanding of what the robot is doing and a less accurate forecast of what it will do next. The robot becomes less transparent to the user.

This lack of transparency and understanding could lead to diminished trust for the robot and ultimately reduce its effective usage [3], [37]. To avert such a development, it is important to include the users in the development of the robot and tailor its abilities in accordance to the abilities that users expect. Also, limitations of the robot’s functionalities should be obvious, to avoid frustration and an extend of trust in the robot that is exaggerated.

Furthermore, it makes a difference whether the robot is used in public or non-public spaces. Public spaces are full of things and situations that are not in the control of the people in it, since they must be shared with others who might have their own agendas and bring things to these places that are not under our control. An autonomous robot might be more accepted in public spaces, since we are all used to being surrounded by situations and things that do not belong to us and because it is possible to retreat from public spaces to a private space or another public space.

The situation becomes a different one in private spaces, such as users’ homes. There, it is not possible for them to avoid confrontation. Users must interact with the robot and do not always know what happens with the data that is produced during the interactions, which might me quite sensitive data, like videos or microphone recordings. Therefore, it is crucial that what the robot is doing and planning to do is transparent.

To understand what users, expect from an autonomous robot and be able to design it in a manner that makes it transparent for them, it is crucial to gain a deep understanding of the stakeholders surrounding such systems. For this purpose, we propose the methods of Living Labs [10], a method that makes it possible to observe the usage of robots in real live environments over an extended period [10]. This method allows developers to gain deep insights regarding the usage of such systems and helps them understand what challenges such systems face.

Legal Issues (Privacy)

With a high level of autonomy, it can be presumed that the likelihood of personality infringements arises (e. g. if the robot acts fully autonomously, it makes decisions that are difficult or impossible for the user to predict and therefore lead to a lack of transparency). With self-learning social robots, the traceability of decisions is a possible challenge for practice. The General Data Protection Regulation (GDPR) only provides general regulations (in particular, Art. 22 GDPR) for the use of Big Data and artificial intelligence (AI), which in turn can be substantiated by national laws and the rules of conduct of professional associations (Art. 40 GDPR) [48]. To counteract these transparency problems, initial approaches already exist regarding how AI can be made comprehensible [49].

Concerning autonomous robots making their own decisions in the public sphere, the question is who is responsible for the data processing [28], [45], not only regarding potential liability issues associated with possible damage caused by the robot, but also regarding data protection responsibility. For example, if a care robot goes shopping for the user in a pharmacy, data processing may occur on the way to and in the pharmacy (another question is whether the robot is able/should be empowered to enter into a contract for its user at all). Those affected by data processing (passers-by on their way to, customers in or employees of the pharmacy) must know who to contact to obtain (further) information on the processing of personal data and to assert their rights in this regard. It should be noted that missing or incorrect information (in combination with a provision that depends on the will of the data subject, such as consent or the execution of the contract) could lead to unlawful processing [2], [42]. Who is responsible for data processing depends on who can decide on the means and purposes of the processing. An autonomous robot “decides” for itself, but only within the framework defined for it in advance.

In contrast to use in public spaces, the use of an autonomous robot in a private environment does not fall within the scope of the GDPR, under certain conditions (Art. 2 para. 2 lit. c GDPR – household exemption). One of those conditions is that the manufacturer/seller/programmer of the robot has neither access to the data nor any influence on the purpose and means of data processing. The user himself is responsible on his private property; he decides what, when and how the robot should do/process. If, in addition, no economic or professional connection with data processing exists, the GDPR would not be applicable (e. g. no data subject rights would have to be considered) [9].

Ethical Issues

A high degree of active emotionality means that technical systems can become objects of our empathy and that we may come to care about them. On the one hand, this need not necessarily be morally problematic, but it also does not automatically lead to an ethically improved technical system, on the other. Whereas interaction and communication with “emotionalized” robots can be more natural and fluent [31], it can also lead to a problematic manipulation of users. This might, for example, be the case with a robot that makes its users do things that they do not want to do by showing negative emotions (crying, whining) until users finally concede. Furthermore, the ability of an artificial system to read our emotional states raises ethical questions. For example, the extensive reading of our emotional state (also based on data that we would not initially associate with it) raises the question of whether we always want to be transparent for these systems.

Participatory Issues

The question of whether a robot should be emotional towards humans (active emotionalization) or should recognize the emotions of the humans surrounding it (passive emotionalization) is highly sensitive and must be answered by its users. In the case of recognizing someone’s emotions, sensitive information about the user can be revealed, which the user might not want to be spoken about by the robot or shared with others. In the case of a robot acting emotionally towards a user, some risk exists that it might be manipulative. For example, in the context of care homes and people with dementia, an emotional approach could be perceived as real and unique by the users, whereas in reality, it is a program played for every person who is in contact with the robot, an “industrialization” of personal care. While such a functionality could evoke the positive feeling of being recognized, the question remains whether this is something that is ethically good and desired by the caregivers and the family of the person. A similar situation could arise in any household. If the robot attempts to simulate emotions, a perceived relationship between human and robot could be stimulated, one which is based on program code. Again, this could be perceived positively by the user, yet it should be the user’s decision to what extent the robot should be simulating emotions.

To overcome this challenge, it might be useful to let the user engage in a kind of negotiation with the program. Therefore, an end-user development (EUD) program could be helpful [20]. This program would allow users to adjust the robot’s emotions, for example, in a way that the robot only shows the emotion of happiness or that it only shows specific emotional reactions to specific users who have authorized such behavior. In the context of passive emotionalization (reading emotions), this program could, for example, define what happens with the gathered data and whether the emotions should be read only in certain time slots, for example, during office hours or breakfast. Furthermore, it could be possible to adjust the reading of emotions to those emotions that other humans can also read but to forbid the reading of things that remain hidden to the eyes of humans, such as a person’s pulse or their typing speed. Through an EUD program, users would gain more control of the robot and be able to define the design features of the robot individually.

Legal Issues (Privacy)

One example for a robot with a high level of emotionality could be a robot that helps children with autism to interpret and learn emotions. Emotional manipulation to induce behavior could be an infringement of personality. Regarding the right of personality, it must be considered that, in the field of data protection, a data protection impact assessment must be conducted for such a robot, since “new technology” is used (Art. 35 para. 1 GDPR). This impact assessment identifies possible risks of data processing and tries to minimize the determined risks: The risk exists that the sensitive data of the “transparent patient” may be passed on to third parties (doctors, insurance companies, etc.) without authorization and that the patient may lose control of his health data [8], for example, if a doctor were to send pictures/sound recordings to colleagues for advice. If data were to be unintentionally disclosed, it could lead for example to a loss of confidentiality, health manipulation, physical and mental damage or commercial exploitation. The (health) facility may suffer the loss of reputation or financial damages. To prevent such negative outcomes, adequate technical and organizational measures (TOMs) need to be implemented. Furthermore, certain basic settings that specify maximum data protection (privacy by design/default) should be provided. Examples could be that the release for data transfer to third parties must be actively clicked on in each case [16]. Privacy by design/default could be realized, for example, if the AI generally conducts human-like emotional analyses. Only in case the user activates the advanced function the AI utilizes the full spectrum of what is technically possible to analyze emotions (e. g. analysis of data from facial expressions, posture, spelling or mouse movements).

Ethical Issues

As mentioned above, human–human interaction is often regarded as the paradigmatic background for creating social human–robot interaction. In cases in which the robot should not merely be a tool or service, the aim is often to design and develop the robot as a companion, friend or roommate. However, research regarding participatory design shows [7], [46] that the robot could be much more effective as a facilitator or mediator for human–human interaction than as a replacement for social interaction. Keeping this momentary state of art in mind, we should be careful not to create robots with only a very diminished capacity of social interaction and sell them as substitutes for real human–human interaction.

Furthermore, every social interaction is part of creating our individual dispositions and scripts of how we interact with other humans [24], [25]. From this perspective, the question arises regarding what types of effect (diminished) forms of social human–robot interactions have on our dispositions to interact with our fellow humans. Having a friendly robot companion at home that simulates various forms of social interactions but never objects or questions the orders of its users may have negative effects on how users are disposed to interact with others. One must also keep in mind that certain moral restrictions arise from empathizing with robots, even if one knows that they do not really have emotions and only simulate such [30].

Participatory Issues

The role of robots in the social lives of humans is difficult to determine. Some people accept the robot as a social agent, while others repel this concept. Acceptance as a social agent is highly dependent on the predetermined usage and the actual user. The intended role of the robot impacts how it should be developed. If the robot is designed as a functional tool that has specific tasks, it becomes less important to develop the robot applications together with its users, since its task would be to fulfill a specific duty, such as assembling car parts or mowing grass. However, if the robot is designed to have a social role, it becomes crucial to understand the users and their needs and limits.

To do so, it is important to consider the role that users would prefer their robot to occupy. Furthermore, it is important to understand how a robot should act in specific social situations. Determining this can be quite difficult, as some potential users have possibly never been confronted with a social robot. Yet, it is often just these groups that are the preferred targets for user research. Recently, for example, much research has been conducted regarding elderly people in interactions with social robots (e. g. [19] or [36]), as this group presumably needs socio-technical innovation but often have very limited encounters with robotic systems in their everyday lives.

It is crucial to work with users and develop these technical devices together with them to enhance their autonomy [29] and self-respect. To achieve a deep knowledge and understanding of how users interact with the robot, we propose working with design case studies [50], [51]. This method is divided into four phases. (1) The grounded design phase represents the first phase, in which it is important to get to know the users and their living spaces in depth. (2) Within the design phase, a certain use case is designed based on the knowledge gained and developed into prototype. (3) In the evaluation phase, the developed prototype is tested in the envisaged environment. During this time, developers observe the situation, as well as the user’s interaction with the robot, and interview the user to reassess whether the intended functionality has been matched. (4) In the iteration phase, based on the knowledge that has been gained in the third phase, the prototype is further developed. Once the adjustments are complete, the prototype is again tested in the envisaged environment and evaluated. This process of iteration is continued until the robot is sufficiently developed to work in a meaningful way, according to its intended social role towards its users.

Legal Issues (Privacy)

An example of a robot with a high level of sociality could be a care robot in a nursing home that tracks the interest and movement patterns of elderly people to determine the best time for each activity with the patient. Possible legal problems in relation to such a robot could be its approval according to the Medical Devices Act, as well as the violation of privacy or human dignity by the surveillance of the patient/user at all times to generate user profiles.

Considering data protection, it must be considered that a specific provision of the GDPR for creating user profiles applies to ensure lawful processing. This provision could be found either in Article 6, for general data, or Article 9, for sensitive data. However, it must be determined what sensitive or, respectively, health data constitutes. According to Recital 35 GDPR, health data “should include all data pertaining to the health status of a data subject which reveal information relating to the past, current or future physical or mental health status of the data subject.” Still, the question remains whether the combination of general data, such as the height, weight and sex of a person, is sufficient to represent such a sensitive data point. At least the body mass index could be determined with these parameters, which could therefore be interpreted as health data. It is therefore more appropriate to assume a broad interpretation of the term “health data”, since not much (combined) data is required to generate information on the health status of a person [18], [38]. It can therefore be assumed that sensitive data (health data) must be collected in the context of creating a user profile. The processing of health data is generally prohibited (Art. 9 para. 1 GDPR). If the robot is to process this nevertheless, an exception to this prohibition is required. This could be found, for example, within the framework of Art. 9 para. 2 lit. h GDPR. Accordingly, the prohibition does not apply if the processing is necessary for, for example, preventive health care, medical diagnostics or health care. In addition, processing must be conducted by or under the responsibility of qualified personnel, who must also be subject to professional secrecy (Art. 9 para. 3 GDPR). These conditions must be considering when determining which data should be collected for the user profile.

Alternatively, obtaining consent could also make the processing of health data lawful. To this end, the data subject must explicitly consent to the processing. Within the framework of consent, further requirements must also be fulfilled. For consent to be considered voluntary, for example, no imbalance should exist between the responsible person and the data subject. Therefore, the question of responsibility again arises here. Is the user himself responsible? Could any voluntariness at all exist in a nursing home–patient relationship [18]? It should also be noted that sector-specific regulations take precedence over the general regulations of the GDPR and the Federal Data Protection Act. Some factors are specific to the area of health data – it can be assumed, for example, that consent for the transfer of patient data can only be given voluntarily if this transfer is also permitted by/possible within the law [14]. In addition, the purpose of the processing must be stated so that an informed consent can be given. The question thus arises as to how precisely the purposes for data processing must be described in advance [18]. Health data is regularly used for a variety of purposes (some of them have not been established in the early stages of Big Data analyses). In addition to treatment, support, care and emergency assistance, purpose could also relate to health management, profitability control, quality assurance, pharmaceutical research or completely extraneous purposes, such as commercial advertising or official actions [48]. It should be noted that all applicable purposes (as well as the legal basis upon which the processing is based, according to Art. 13/14 GDPR) must be communicated to the data subject prior to the actual processing. A subsequent change of purpose is only possible under narrow conditions, which is why it is better to indicate too many possible/conceivable purposes, rather than too few, at the beginning of data processing. However, consent should also be formulated as narrowly and comprehensibly as possible to avoid overburdening the data subject.

Ethical Issues

Robots can help us to regain competences we have lost, enhance our competences or replace the need to act out our competences. In every case, some caution should exist regarding the dependence on a technical system that is created by introducing and using social robots. In the worst-case scenario, users unlearn important competences when they would otherwise still be able to perform the necessary actions. From an ethical and social perspective, it should be clarified which dependencies are created to ensure that these cannot be exploited.

Another ethically relevant aspect involves identifying competences before attempting to enhance or replace them. One lesson that can be drawn from the discussion on elderly care robots, which should be clear by now, is that the competences of caretakers lie not only in performing certain (mechanical/bodily) actions (bringing food) but also in engaging in social and emotional interactions (see, for example, [13], [34]). Neglecting the latter would entail focusing on only one part of the whole situation, and therefore one might be drawn to the conclusion that replacement without essential loss is possible. Seeing the various competences involved enables developers to set their aim straight: robots might be able to replace or help with heavy and toilsome work to make room for specific human competences.

Participatory Issues

Competences are crucial for every person. When it comes to the development of robotic systems, the questions always exist regarding which tasks the robot should fulfill and whether these tasks are performed solitarily or in cooperation with users [21], [35]. Quite regularly, these developments follow the paradigm of what is possible from a technical point of view, and often, this logic does not consider that people enjoy certain tasks or that they will feel reduced if these tasks are fulfilled by robots and they are thus condemned to do only that which the robot cannot. Therefore, it is important to understand the needs and preferences of the users and gain a deep knowledge on how such robotic systems are implemented in the intended context.

For example, a robotic system that helps users with their grocery shopping, by finding specific items and collecting them from the shelves, could lead to the loss of competence to go grocery shopping without assistance or be able to find products in a supermarket. It must be observed and, in a second step, negotiated whether this competence is something that users would like to keep or can accept the risk of losing.

To do so, it is necessary for stakeholders to participate in product development. Making stakeholders co-creators will drastically deepen understanding about them and be helpful in understanding what competences users would like to maintain and which are less important.