10 Brain-Machine Symbiosis

,


Introduction
A Brain-computer interface (BCI) establishes a direct communication channel between the human brain and a computer or an external device, which can be used to convey messages directly so that no motor activity is required (Wolpaw et al., 2002).The brain activity is acquired and in real-time analyzed to interpret the independent thought or action of the user, which can be transformed into a control signal.Particularly for people suffering from severe physical disabilities or those who are in a "locked-in" state, a BCI offers a possible new communication channel, but also augmenting or repairing human cognitive or sensory-motor functions.
Different types of BCIs exist and various methods can be used to acquire brain activity, but since the electroencephalogram (EEG) is the most practical modality (Mason et al., 2007) -if we want to bring BCI technology to a large population -this chapter will focus on EEG based BCIs only.Nevertheless we would like to say, that brain activity can be measured through non-electrical means as well, such as through magnetic and metabolic changes, which can be also measured non-invasively.Magnetic fields can be recorded with magnetoencephalography (MEG), while brain metabolic activity (reflected in changes in blood flow) can be observed with positron emission tomography (PET), functional magnetic resonance imaging (fMRI), and optical imaging (NIRS).Unfortunately, such alternative techniques require sophisticated devices that can be operated only in special facilities (except for NIRS).Moreover, techniques for measuring blood flow have long latencies compared to EEG systems and thus are less appropriate for interaction, although they may provide good spatial resolution.Besides EEG, electrical activity can also be measured through invasive means such as Electrocorticogram (ECoG) or intracranial recordings.Both methods require surgery to implant electrodes.The relative advantages and disadvantages of currently available noninvasive and implanted (i.e., invasive) methodologies are discussed in (Wolpaw et al., 2006).
Two different neurophysiological phenomena of the EEG can be used as input to a BCI.Either event-related potentials (ERPs) or event-related oscillatory changes in the ongoing EEG are analyzed.An ERP can be seen as an event-and time-locked response of a stationary system to an external/internal event, which is the response of the existing neural network.A significant number of reports are focused on the analysis of ERPs, including slow cortical potentials (Birbaumer et al., 1999), P300 component (a positive waveform occurring approximately 300 ms after an infrequent task-relevant stimulus) (Allison et al., 2007b, Donchin et al., 2000), steady-state visual evoked potentials (SSVEP) while looking at flickering lights (Gao et al., 2003).Event-related oscillatory changes on the other hand can be interpreted as result of changes in the functional connectivity within the neuronal network which are timebut not phase-locked.These internally triggered changes in the rhythmic components of the ongoing brain signal results in relative power increase or decrease, which can be associated with active information processing within these networks.For example, the imagination of different types of movements (MI) results in power changes over the motor cortex.
Most of the existing BCI applications are either software oriented, like mentally writing text via a virtual keyboard on a screen (Birbaumer et al., 1999), or hardware oriented, like controlling a small robot (Millán et al., 2004).These typical applications require a very good and precise control channel to achieve performances comparable to healthy users without a BCI.However, current day BCIs offer low information throughput and are insufficient for the full dexterous control of such complex applications, because of the inherent properties of the EEG.Therefore, the requirements and the skills don't match at all.Techniques like context awareness can enhance the interaction to a similar level, despite the fact that BCI is not such a perfect control channel (Tonin et al., 2011, Galán et al., 2008).In such a control scheme, the responsibilities are then shared between the user, who gives high-level commands, and the system, which executes fast and precise low-level interactions.
The classic user group in BCI research is severely disabled patients: persons who are unable to communicate through other means (Birbaumer et al., 1999).However, recent progress in the field of BCI technology shows that BCIs could also be helpful to less disabled users.New user groups are emerging as new devices and applications develop and improve.Rehabilitation of disorders has gained a lot of attention recently, especially for users with other disabilities such as stroke, addiction, autism, ADHD and emotional disorders (Allison et al., 2007b, Birbaumer & Cohen, 2007, Lim et al., 2010, Pineda et al., 2008).Furthermore, BCIs could also help healthy users in specific situations, such as when conventional interfaces are unavailable, cumbersome, or do not provide the needed information (Allison et al., 2007a).
Such passive monitoring offers potential benefits for both patients and healthy subjects.Furthermore, another area of research, interesting for healthy subjects, are BCI controlled or supported games; by augmentation of the operation capabilities or by allowing multi-task operations (Menon et al., 2009), or possible space applications (Millán et al., 2009).Another recent extension of BCI for healthy users is in the field of biometrics.Since the brainwave pattern of every person is unique, a person authentication based on BCI technology could use EEG measures to help authenticate a user's identity, either by mental tasks (Marcel & Millán, 2007) or reactive frequency components (Pfurtscheller & Neuper, 2006).
Many new BCI devices and applications have recently been validated mostly with healthy users, such as control of smart home or other virtual environment (Leeb et al., 2007b, Scherer et al., 2008), games (Lalor et al., 2005, Millán et al., 2008, Nijholt et al., 2008), orthosis or prosthesis (Müller-Putz & Pfurtscheller, 2008, Pfurtscheller et al., 2008), virtual or real wheelchairs (Leeb et al., 2007a, Cincotti et al., 2008, Galán et al., 2008), and other robotic devices (Bell et al., 2008, Graimann et al., 2008).We can even turn the BCI shortcomings into challenges (Nijholt et al., 2009, Lotte, 2011), by e.g.explicitly requiring a gamer to issue BCI commands to solve a task.Thereby far from perfect control 'solutions' are more interesting and challenging.These and other emerging applications adumbrate dramatic changes in user groups.Instead of being devices that only help severely disabled users and the occasional curious technophile, BCIs could benefit a wide variety of disabled and even healthy users.
Furthermore, when controlling complex devices via a BCI, the brain signals not only carry information about the mental task that is executed, but also about other cognitive processes that take place simultaneously.These processes reflect the way the user perceives the interaction and how much the device behavior truly reflects his/her intent.These cognitive processes can be exploited in a human-machine interaction, for allowing to recognize erroneous conditions during the interaction via the error potential (ErrP), or to detect the preparation or onset of motor actions, as well as identification of attentional and perceptual processes.

Applied Principles
We will first explain the underlying principles in this chapter, before moving towards examples of brain controlled devices and cognitive signals in the next chapters.

Brain-Computer Interface Principle
In the direct control examples presented later on (see section 10.3), a BCI based on motor imagery (MI) is used.MI is described as the mental rehearsal of a motor act without any overt motor output (Decety, 1996), which involves similar brain regions to those which are used in programming and preparing such real movements (Ehrsson et al., 2003, Jeannerod & Frak, 1999).The imagination of different types of movements (e. g. right hand, left hand or feet), results in an amplitude suppression (known as event-related desynchronization, ERD (Pfurtscheller & Lopes da Silva, 1999)) or in an amplitude enhancement (event-related synchronization, ERS) of Rolandic mu rhythm (7-13 Hz) and the central beta rhythm (13-30 Hz) recorded over the sensorimotor cortex of the participant (Pfurtscheller & Neuper, 2001).Therefore, the brain activity is acquired via 16 active EEG channels over the sensorimotor cortex.From the Laplacian filtered EEG, the power spectral density was calculated.Canonical variate analysis was used to select subject-specific features, which were classified with a Gaussian classifier (Galán et al., 2008).Decisions with low confidence on the probability distribution were filtered out and evidence was accumulated over time (see the basic principle in Figure 10.1).
Before being able to use a BCI, participants have to go through a number of steps to learn to voluntarily modulate the EEG oscillatory rhythms by performing MI tasks.Furthermore, the BCI system has to learn what the participant-specific patterns are, that can be used for that particular user for online experiments.If participants achieve good online control they are allowed to test the application prototypes (see Section 10.3).More details about the experimental paradigm, signal processing and machine learning (feature extraction, feature selection, classification and evidence accumulation) and the feedback are given in (Leeb et al., 2013).

The Context Awareness Principle
For example, let us consider driving a wheelchair in a home or indoor environment (scattered with obstacles like chairs, tables, doors …) that requires precise control to navigate through rooms.A context aware and smart wheelchair will help the user to navigate through.The user issues via the BCI the high level commands such as left, right and forward, which are then interpreted by the wheelchair controller based on the contextual information from its sensors, also called shared control in robotics.
Based on these interpretations, the wheelchair can perform intelligent maneuvers (e.g.obstacle avoidance, guided turnings).
Despite the low information transfer rate of a BCI, researchers have demonstrated the feasibility of mentally controlling complex robotic devices from EEG (Flemisch et al., 2003, Vanhooydonck et al., 2003, Carlson & Demiris, 2008).In the case of neuroprosthetics, Millán's group has pioneered the use of shared control that takes the continuous estimation of the operator's mental intent and provides assistance to achieve tasks (Millán et al., 2004, Tonin et al., 2010, Galán et al., 2008).Generally, in a context awareness framework, the BCI's outputs are combined with information about the environment (obstacles perceived by the robotic sensors) and the robot itself (position and velocities) to better estimate the user's intent (see Figure 10.2).Some broader issues in human -machine interaction are discussed in (Flemisch et al., 2003), where the H-Metaphor ("horse" metaphor) is introduced, suggesting that interaction should be more like riding a horse or controlling a horse carriage, with notions of "loosening the reins", allowing the system more autonomy.Context awareness is a key component of any future BCI systems, as it will shape the closed-loop dynamics between the user and the brain-actuated device so tasks can be performed as easily as possible and effectively.As mentioned above, the idea is to integrate the user's mental commands with the contextual information gathered by the intelligent brain-actuated device, so as to help the user to reach the target or override the mental commands in critical situations.In other words, the actual commands sent to the device and the feedback to the user will adapt to the context and inferred goals.Therefore, context awareness can make target-oriented control easier, can inhibit pointless mental commands (e.g.driving zig-zag), and can help determine meaningful motion sequences (e.g., for a neuroprostheses).Context awareness is helping on a direct interaction with the environment but is conveying a different principle than autonomous control.In autonomous control high-level commands which are more abstract (e.g.drive to the kitchen or the living room) are issued and then executed autonomously by the robotic device without interaction of the user, till the selected target is reached (Carlson & Millán, 2013), which could be suboptimal in cases of interaction with other people.
A critical aspect of context awareness for BCI is coherent feedback -the behavior of the robotic device should be intuitive to the user and the robot should unambiguously understand the user's mental commands.Otherwise, people find it difficult to form mental models of the neuroprosthetic device.The context awareness principle: The user issues high-level commands via a brain-computer interface mostly on a lower pace.The system is acquiring fast and precise the environmental information (via sonars, webcams…).The context awareness system combines the two information to achieve a path planning and obstacle avoidance, so that a control of the robotic device is possible (shared control) and e.g. the wheelchair can move forward, turn left or right.Modified from Rupp et al. (2014)

Hybrid Principle
Despite the progress in BCI research, the level of control is still very limited compared to natural communication or existing assistive technology products.Practical braincomputer interfaces for disabled people should allow them to use all their remaining functionalities as control possibilities.Sometimes these people have residual activity of their muscles, most likely in the morning when they are not exhausted.In such a hybrid approach, where conventional assistive products (operated using some residual muscular functionality) are enhanced by BCI technology, leads to what is called a hybrid BCI (hBCI).
As a general definition, a hBCI is a combination of different input signals including at least one BCI channel (Millán et al., 2010, Pfurtscheller et al., 2010).Thus, it could be a combination of two BCI channels but, more importantly, also a combination of a BCI and other biosignals (such as EMG, etc.) or special AT input devices (e.g., joysticks, switches, etc.).There exist a few examples of hybrid BCIs.Some hBCIs are based on multiple brain signals: such as MI for control and ErrP detection for correction of false commands (Ferrez & Millán, 2008b), or an offline combination of MI and SSVEP (Allison et al., 2010, Brunner et al., 2010).
Other hBCIs combine brain and other biosignals: switching an standard SSVEP BCI on/off via an heart rate variation (Scherer et al., 2007), or fusing electromyographic (EMG) with EEG activity (Leeb et al., 2011) so that the subjects could achieve a good control of their hBCI independently of their level of muscular fatigue.Finally, EEG signals could be combined with eye gaze (Danoczy et al., 2008) 2015) presented an hBCI framework, which was used in studies with non-impaired as well as end-users with motor impairments.

Direct Brain-Controlled Devices
In a traditional BCI fashion controlling complex devices such as brain-controlled wheelchair or mobile tele-presence platform in natural office environments would be a complex and frustrating task, especially since the timing and speed of interaction is limited by the BCI.Furthermore, the user has to share his attention between the BCI and the device, and also remember the place where he is and where he wants to go.In contrary, combing the above mentioned principles of BCI with context awareness and hybrid approaches allow subjects to control such complex devices easily.

Brain-Controlled Wheelchair
In case of brain-controlled robots and wheelchairs, Millán's group has pioneered the development of the shared autonomy approach to estimate the appropriate assistance which greatly improved BCI driving performance (Vanacker et al., 2007, Galán et al., 2008, Millán et al., 2009, Tonin et al., 2010).Although asynchronous spontaneous BCIs seem to be the most natural and suitable alternative, there are a few examples of synchronous evoked P300 BCIs for wheelchair control (Iturrate et al., 2009, Rebsamen et al., 2010), whereby the system flashes the possible predefined target destinations several times in a random order.The stimulus that elicits the largest P300 is chosen as the target.Then, the intelligent wheelchair reaches the selected target autonomously.Once there, it stops and the subject can select another destination -a process that takes around 10 seconds.
Here, we describe our recent work (Carlson & Millán, 2013), during which subjects controlled the movement of an electric wheelchair (InvaCare Europe) by thought.The wheelchair's turnings to the left and right are controlled via a 2-class BCI (see section 10.2.1).Whenever the BCI output exceeds the left or right threshold a command was delivered to the wheelchair.In addition, the participant can intentionally decide not to deliver any mental commands to maintain the default behavior of the wheelchair, which consists of moving forward and avoiding obstacles with the help of a shared control system using its on-board sensors.More details see (Carlson & Millán, 2013).
For controlling, the user asynchronously sent high-level commands for turning to the left or right (with the help of a motor-imagery based BCI) to achieve the desired goals, while short-term low-level interaction for obstacle avoidance was done by the context awareness (see Figure 10.3.a and section 10.2.2).In the applied context awareness paradigm, the wheelchair pro-actively slows down and turns to avoid obstacles as it approaches them.For that reason the wheelchair was equipped with proximity sensors and two webcams for obstacle detection.Using the computer vision algorithm described in (Carlson & Millán, 2013), we constructed a local 10 cm resolution occupancy grid (Borenstein & Koren, 1991), which was then used by the shared control module for local planning.Generally, the vision zone was divided into three zones.Obstacles detected in the left or right zone triggered rotation of the wheelchair, whereas obstacle in center (in front) slowed it down.We also implemented a docking mode, additionally to the obstacle avoidance.Therefore, we considered any obstacle to be a potential target, provided it was located directly in front of the wheelchair.Consequently, the user was able to dock to any "obstacle", be it a person, table, or even a wall.The choice of using cheap webcams and not using an expensive laser range-finder was taken to facilitate the development of affordable and useful assistive devices.If we want to bring the wheelchair to patients, the additional equipment should not cost more than the wheelchair itself.
In an experiment four healthy subjects (aged 23-28) participated successfully in driving the wheelchair (Carlson & Millán, 2013).The task was to enter an open -plan environment, through a narrow doorway, dock to two different desks, whilst navigating around natural obstacles and finally reach the corridor through a second doorway (see Figure 10.3.b).The experiment was performed twice, once with BCI control with the help of context awareness and once with the normal manual control, whereby the analog joystick was replaced by two discrete buttons.Across subjects, it took an average of 160.0 s longer to complete the task under the BCI condition.In terms of path efficiency, there was no significant difference among subjects between the distance traveled in the manual benchmark condition (43.1 ± 8.9 m) and that in the BCI condition (44.9 ± 4.1 m) (Carlson & Millán, 2013).The longer time is probably due to a combination of subjects issuing manual commands with a higher temporal accuracy and a slight increase in the number of turning commands that were issued when using the BCI, which yielded in a lower average translational velocity.Especially, inexperienced users had a bigger difference than experience ones.This is likely to be due to the fact that performing an MI task, while navigating and being seated on a moving wheelchair, is much more demanding than simply moving a cursor on the screen and when the timing of delivering commands becomes very important (Leeb et al., 2013).
We want to highlight that, in this study not only a complex task had to be performed, but also the potential stressfulness of the situation, since the user was colocated with the robotic device that he or she was controlling and was subject to many external factors.This means the user had to put trust in the context awareness system and expected that negative consequences (e.g. a crash) could result in the system failing (although an experimenter was always in control of a fail-safe emergency stop button).
In the future we are planning to add a start/stop or a pausing functionality for the movement of the robotic device, in parallel to the frequently-occurring commands of turning left or right.In the framework of a hybrid BCI, such rare start/stop commands could also be delivered through other channels such as residual muscular activity, which can be controlled reliably-but not very often, because of the quick fatigue.

Tele-Presence Robot Controlled by Motor-Disable People
Moving on from healthy people to motor-disabled end users as BCI participants, we present a study in which a tele-presence robot was remotely navigated within a natural office environment.The space contains natural obstacles (i.e.desks, chairs, furniture, people) in the middle of the pathways and some predefined target positions.Importantly, participants have never been in such an environment.The robot's turnings to the left and right are controlled via a 2-class BCI similar to the aforementioned study with the wheelchair.
The implementation of context awareness used the dynamical system concept (Schöner et al., 1995), to control two independent motion parameters: the angular and translation velocities of the robot.The systems can be perturbed by adding attractors or repellors in order to generate the desired behaviors.The dynamical system implements the following navigation modality.The default device behavior is to move forward at a constant speed.If repellors or attractors are added to the system, the motion of the device changes in order to avoid the obstacles or reach the targets.At the same time, the velocity is determined according to the proximity of the repellors surrounding the robot.The robot is based on Robotino™ by FESTO (Esslingen, Germany) a small circular mobile platform (diameter 36 cm, height 65 cm), which is equipped with nine infrared sensors that can detect obstacles up to ~30 cm distance and a webcam that can also be used for obstacle detection.Furthermore, a notebook with a camera is added on top of the robot for tele-presence purposes (see Figure 10.4.a), so that the participant can interact with the remote environment via Skype™.
Nine severely motor-disabled end-users, who had never visited the laboratory in person, were able to use such a tele-presence robot to successfully navigate around the lab (see Figure 10.4.b), whilst they were located in their own homes or clinics at distances of up to 550 km away (Leeb et al., 2015).In some extreme tests, a healthy subject was attending a conference in South Korea, where he demonstrated that he could use our motor imagery based BCI to reliably control the tele-presence robot, which was located in our lab in Switzerland.As before, the same paths were followed with BCI control and with manual control (i.e.button presses).Furthermore, context awareness was either applied or not.The time and number of commands needed were previously reported for healthy users (Tonin et al., 2010) and recently for patients (Tonin et al., 2011, Leeb et al., 2015).Remarkably, the patients performed similar to the healthy users who were familiar with the environment (91.5 ± 17.5 versus 102.6 ± 26.3 seconds).
Context awareness also helped all subjects (including novel BCI subjects or users with disabilities) to complete a rather complex task in similar time and with similar number of commands to those required by manual commands without context awareness.More details are given in (Tonin et al., 2010, 2011, Leeb et al., 2013, 2015).Thus, we argue that context awareness reduces subjects' cognitive workload as it: (i) assists them in coping with low-level navigation issues (such as obstacle avoidance and allows the subject to focus the attention on his final destination) and thereby (ii) helps BCI users to maintain attention for longer periods of time (since the amount of BCI commands can be reduced and their precise timing is not so critical).

Grasp Restoration for Spinal Cord Injured Patients
The restoration of grasp functions in spinal cord injured (SCI) patients or patients suffering from paralysis of upper extremities typically rely on Functional Electrical Stimulation (FES).In this context, the term neuroprosthesis is used for FES systems that seek to restore a weak or lost grasp function when controlled by physiological signals.Some of these neuroprostheses are based on surface electrodes for external stimulation of muscles of the hand and forearm (Ijzermann et al., 1996, Thorsen et al., 2001, Mangold et al., 2005).Others, like the Freehand system (NeuroControl, Cleveland, US), uses implantable neuroprostheses to overcome the limitations of surface stimulation electrodes concerning selectivity and reproducibility (Keith & Hoyen, 2002), but this system is no longer available on the market.
Pioneering work by the groups in Heidelberg and Graz showed that a BCI could be combined with an FES-system with surface electrodes (Pfurtscheller et al., 2003).In this study, the restoration of a lateral grasp was achieved in a spinal cord injured subject (see Figure 10.5.b).The subject suffered from a complete motor paralysis with missing hand and finger function.The patient could trigger sequential grasp phases by imagining foot movements.After many years of using the BCI, the patient can still control the system, even during conversation with other persons.The same procedure could be repeated with another tetraplegic patient who was provided with a Freehand system (Müller-Putz et al., 2005).All currently available FES systems for grasp restoration can only be used by patients with preserved voluntary shoulder and elbow function, which is the case in patients with an injury of the spinal cord below C5.So neuroprostheses for the restoration of forearm function (like hand, finger and elbow) require the use of residual movements not directly related to the grasping process.To overcome this restriction, a new method of controlling grasp and elbow function with a BCI was introduced recently via pulse-width coded brain patterns for controlling sequentially more degrees of freedom (Müller-Putz et al., 2010).
BCIs have been used to control not only grasping but also other complex tasks like writing.Millán's group used the motor imagery of hand movements to stimulate the same hand for a grasping and writing task (Tavella et al., 2010).Thereby the subjects had to split his/her attention to multitask between BCI control, reaching, and the primary handwriting task itself.In contrast with the current state of the art, an approach in which the subject was imagining a movement of the same hand he controls through FES was applied.
Moreover, the same group developed an adaptable passive hand orthosis (see Figure 10.5.a), which evenly synchronizes the grasping movements and applied forces on all fingers (Leeb et al., 2010).This is necessary due to the very complex hand anatomy and current limitations in FES-technology with surface electrodes, because of which these grasp patterns cannot be smoothly executed.The orthosis support and synchronize the movement of the fingers stimulated by FES for patients with upper extremity palsy to improve everyday grasping and to make grasping more ergonomic and natural compared to the existing solutions.Furthermore, this orthosis also avoids fatigue in long-term stimulation situations, by locking the position of the fingers and switching the stimulation off (Leeb et al., 2010).(Pfurtscheller et al., 2003)

Cognitive Signals and Mental States
Previous sections illustrate how it's possible to control complex devices by the decoding of user's intention from the execution of specific mental tasks (i.e.motor imagery -complemented by contextual information in order to increase the robustness of the system.However, when controlling a BCI, brain signals not only carry information about the mental task that is executed, but also about other cognitive processes that take place simultaneously.These processes reflect the way the user perceives the interaction and how much the device behavior truly reflects his/her intent.In this section we present several ways to exploit these cognitive processes to overall humanmachine interaction.
In particular, we will review different potentials that convey useful information allowing to recognize erroneous conditions during the interaction, preparation of motor actions as well as attentional and perceptual processes.

Error-Related Potentials
Error recognition is crucial for efficient behavior in both animals and humans.Wealth of studies have identified brain activity patterns that are naturally elicited whenever a person commits an error in tasks requiring a rapid response (Falkenstein et al., 2000).Interestingly, similar potentials are also observed when the person perceives error committed by other person (van Schie et al., 2004) or even machines (Ferrez & Millán, 2008a).These error-related potentials (ErrPs) provide a mean to obtain information about the subject evaluation of the interaction.Allowing to synergistically incorporate detection of erroneous situations -decoded from the brain activity -into the control of the external device.Enabling us to correct these situations or even improve performance via error-driven adaptation.
Error-related potentials can be observed in the EEG signals over fronto-central areas.In the case of self-generated errors differences between the correct and error condition appear at about 120 ms after the action, while differential responses to external feedback appear at about 200-500 ms after the erroneous stimuli.Interestingly, these signals are naturally elicited during the interaction, therefore no user training is required.Moreover, they are rather stable across time (Chavarriaga & Millán, 2010) and similar waveforms appear across different tasks (Iturrate et al., 2014) and feedback modalities (Chavarriaga et al., 2012).
One of the first attempts to exploit these signals during human-computer interaction was proposed by Parra and colleagues (Parra et al., 2003).In their study the user performed a two-forced choice task and the EEG was decoded to detect the errorrelated pattern after incorrect presses.This automatic correction yielded to a performance improvement of around 20%.Later on, it was demonstrated that error-related potentials were also elicited in the frame of brain-computer interaction (Ferrez & Millán, 2008a).Importantly, it is possible to decode these potentials at a single trial basis, i.e. inferring whether a given trial correspond to the erroneous or correct condition.This paved the way for automatic correction of BCI commands Ferrez and colleagues demonstrate this in a MI-based BCI by simultaneous decoding of the BCI control signal (e.g.motor imagery) and the ErrPs (Ferrez & Millán, 2008b).Similar approaches have also been implemented for P300-based BCIs both in healthy and motor disabled subjects (Dal Seno et al., 2010, Spüler et al., 2012).
These systems allow for instantaneous response of the previous command.However, this does not prevent the same errors to appear in the future.An alternative approach is to exploit the ErrPs to drive the adaptation of an intelligent controller so as to improve the likelihood of generating correct responses (Chavarriaga & Millán, 2010, Llera et al., 2011, Förster et al., 2010).Based on the reinforcement learning paradigm, the ErrPs provide information akin to (negative) rewards that make it possible to infer control strategies that the subject considers as correct.In this approach the human is placed within a cognitive monitoring loop with the agent, thus making possible to tune the agent's behavior to the user's needs and preferences.

Decoding Movement Intention
The counterpart of interpreting the outcome of a specific action is the possibility of detecting the intention to move prior to the execution.This can help in the control of neuroprosthetics as early detection can reduce the delay between the motor intention and the device activation.
There has been evidence of preparatory and anticipatory activity since several decades.This includes seminal works showing slow deflections of cortical activity between timely separated contingent stimuli (i.e.contingent negative variation) (Walter et al., 1964) and lateralized slow cortical potentials preceding movements up to 1.5 s (Libet et al., 1982).However, only recently this type of signal has been successfully decoded for Non-invasive BCI applications.A key factor for this decoding is the appropriate selection of spatio-spectral features used for the decoding, as low-frequency EEG oscillations exhibit a large inter-trial variability (Garipelli et al., 2013).Regarding arm movements, Lew et al. used a center-out reaching task (see Figure 10.6.a) to show that onset of self-paced movement intention could be detected more than 460 ms before the movement, based on SCP off-line analysis (Lew et al., 2012) (see Figure 10.6.b).A similar approach was also used to predict intention both in movement execution and imagination (Niazi et al., 2011).In turn, oscillatory activity in the alpha and beta bands (8-30 Hz) was also shown to carry information that could be used to detect self-paced wrist extension movement (Bai et al., 2011).
However, the previous studies were performed with simplified protocols.So the question remains on whether similar signals can be observed and decoded in realistic conditions.Experiments in virtual environments and during car driving tasks suggest that this is the case (Chavarriaga et al., 2012, Khaliliardali et al., 2012) (see Figure 10.6.c).We have analyzed the EEG activity while drivers (N=6) perform selfpaced lane changes in a simulated highway.Using classifiers trained on segments corresponding to straight driving and steering actions, the onset of the steering actions was detected on average 811 ms before the action with a 74.6% true positive rate (Gheorghe et al., 2013) (see Figure 10.6.d).(Gheorghe et al., 2013)

Correlates of Visual Recognition and Attention
Human machine interaction implies a closed-loop where information flows to, from and between the user and the device she/he is interacting with.For this reason there is an increased interest in identifying brain correlates of perceptual and attentional processes.
A clear example of combining human and machine capabilities can be found on applications of image retrieval.As of today, computer algorithms are quite efficient at processing large amounts of data based on their low-level features, but are less suitable to handle their semantic content.These applications decode the EEG responses to visual stimuli so as to identify those images that are interesting to the subject.They constitute labeled exemplars that can be exploited by computer vision algorithms to retrieve similar images from a larger database (Sajda et al., 2010, Bigdely-Shamlo et al., 2008, Uscumlic et al., 2013).
Interaction with complex system necessarily implies to attend to different stimuli.In the case of vision, attentional deployment can be produced with or without changes in the gaze direction (i.e.overt and covert attention).While the first one can be easily detected using eye-tracking techniques, the second one requires the analysis of the brain patterns, as no observable behavior is produced.Neurophysiological studies have shown that modulation of α rhythms signal voluntary changes of covert visual attention.Furthermore, there seems to be a specialized topographical representation of the attended locations (Rihs et al., 2007).Based on this, the use of covert shifts of attention -decoded from the EEG α activity -has been proposed as an alternative BCI paradigm (Treder et al., 2011, van Gerven & Jensen, 2009).Interestingly, a new approach allows for a more detailed spatio-temporal characterization of these processes by analyzing narrow sub-bands of the α rhythms, as well as the inherent temporal variability of such signals (Tonin et al., 2012).Experiments with able-bodied subjects over two days resulted in a mean on-line accuracy across the group of 70.6 ± 1.5% (N=8), and 88.8 ± 5.8% for the best subject (Tonin et al., 2013).Interestingly, these subjects that present a drop in performance also exhibited changes in the δ,θ, and α over frontal cortices.These changes have been previously reported as putative correlates of fatigue, increased workload or drowsiness (Borghini et al., 2012, Brouwer et al., 2012).This evidences the possibility of simultaneous monitoring of multiple cognitive signals.

Discussion and Conclusion
In this book chapter we gave a broad overview of the functionality of a brain-computer interface and showed possible scenarios and applications.We put emphasis on control of robotic devices (like wheelchair or tele-presence robots), motor substitution by functional electrical stimulation, decoding of erroneous decisions, intention of movements or actions, mental state monitoring of attention and visual recognition.
We presented examples of how healthy and disabled participants can use successfully the BCI to control robotic devices.Especially with the help of context awareness some of the BCI limitations can be overcome and the path in developing more practical BCIs is opened, especially towards the control of mobility (either a tele-presence robot or a wheelchair).We showed results from healthy users and end-users with disabilities, which were able to perform a rather complex navigation task.Remarkably, although the patients had never visited the remote location where the tele-presence robot was operating, their performances were similar to a group of healthy users who were familiar with the environment.Furthermore, the help of context awareness allowed all subjects to complete task in similar time and with similar number of commands to those required by manual commands without context awareness.Thus, we argue that context awareness reduces subjects' cognitive workload as it: (i) assists them in coping with low-level navigation issues (such as obstacle avoidance and allows the subject to focus the attention on his final destination) and (ii) helps BCI users to keep attention for longer periods of time (since the amount of BCI commands can be reduced and their precise timing is not so critical).
First steps are done in establishing the possibility to give the control back to patients, not only over robotic devices but directly of simple grasp patterns.Braincontrolled functional electrical stimulation of the muscles together with orthotic devices will allow a self-paced long term use.However, BCIs are not ready yet for an independent use at home (Leeb et al., 2013) and some gaps for usability and reliability have to be addressed.
The exploitation of cognitive signals, which carry valuable information in parallel to the mental task being performed, is pushing BCIs to the next level.Information like how the interaction is perceived or about erroneous decisions, or even about the attention to the task, will allow us to modulate the level of support which is needed for BCI control, or value the received signals.We can envision that a future car, could modify its behavior depending on the state of the user, it could help more or less, or even become more proactive in case the attention of a user is going down, or he is getting more and more distracted.
We expect even faster progress in the next years, since the BCI field is still gaining attention from funding agencies and companies.More practical and powerful tools for disabled people will develop.Furthermore, BCIs can benefit from other signals and human-computer interaction techniques, and vice-versa.BCIs can be used to extract cognitive-relevant information to improve standard interactions, which is becoming increasingly interesting for healthy users.
A further future key component for a successful application of BCI devices is the availability of dry and wireless electrodes systems with a sufficient data quality.First systems are appearing on the market, but long term studies are still missing.Furthermore, more translational studies involving end-users at their homes are needed to address the problems which are arising from application control outside laboratory environments.Adopting a user-centered iterative approach (for the end-users as well as for the healthy population) will allow addressing the specific needs and requirements of the different future user groups.

Figure 10 . 1 :
Figure 10.1: Basic principle of a BCI: The electrical signals from the brain are acquired, before feature -characteristic with the given task-are extracted.These are then classified to generate action, which are controlling the robotic devices.The participant immediately either sees the output of the BCI and/or the generated action

Figure
Figure 10.2:The context awareness principle: The user issues high-level commands via a brain-computer interface mostly on a lower pace.The system is acquiring fast and precise the environmental information (via sonars, webcams…).The context awareness system combines the two information to achieve a path planning and obstacle avoidance, so that a control of the robotic device is possible (shared control) and e.g. the wheelchair can move forward, turn left or right.Modified fromRupp et al. (2014)

Figure 10 . 3 :
Figure 10.3:(a) Picture of a healthy subject sitting in the BCI controlled wheelchair.The main components on our brain-controlled robotic wheelchair are indicated with close-ups on the sides.The obstacles identified via the webcams are highlighted in red on the feedback screen and will be avoided by the context awareness system.(b) Trajectories of a subject during BCI control reconstructed from the odometry.The start, end and target positions as well as the BCI triggered turnings are indicated.Modified fromCarlson & Millán (2013)

Figure 10
Figure 10.4:(a) A tetraplegic end-user (C6 complete) demonstrates his acquired motor imagery skills, manoeuring the brain-controlled tele-presence robot in front of participants and press at the "TOBI Workshop IV", Sion, Switzerland, 2013.(b) Layout of the experimental environment with the four target positions (T1, T2, T3, T4), start position (R)

Figure 10 . 5 :
Figure 10.5:(a) Picture of BCI subject with an adaptable passive hand orthosis.The orthosis is capable of producing natural and smooth movements when coupled with FES.It evenly synchronizes (by bendable strips on the back) the grasping movements and applied forces on all fingers, allowing for naturalistic gestures and functional grasps of everyday objects.(b) Screen shot from the pioneering work showing the first BCI controlled grasp by a tetraplegic patient(Pfurtscheller et al., 2003)

Figure 10 . 6 :
Figure 10.6: EEG correlates of movement intention: (a) Decoding of movement related potentials in both able-bodied and stroke subjects.(b) Single trial analysis of EEG signals in a center-out tasks yield recognition about chance level at about 500 ms before the movement onset (green line), earlier than any observable muscle activity (magenta line) (Lew et al., 2012).(c) Car driving scenario.Lowfrequency oscillations (<1 Hz) reflect preparatory activity up to 1 s before steering actions in a car simulator.(d) As shown in the topographic representation this activity appears over central midline areas, consistent with movement-related potentials reported in simpler tasks(Gheorghe et al., 2013) . Pfurtscheller et al. (2010) recently reviewed preliminary attempts, and feasibility studies, to develop hBCIs combining multiple brain signals alone or with other biosignals.Millán et al. (2010) reviewed the state of the art and challenges in combining BCI and assistive technologies and Müller-Putz et al. (