Abstract
The idea of connecting the human brain to a computer or machine directly is not novel and its potential has been explored in science fiction. With the rapid advances in the areas of information technology, miniaturization and neurosciences there has been a surge of interest in turning fiction into reality. A Brain-Computer Interface (BCI) provides a new communication channel between the human brain and the computer. It is a. method of communication based on voluntary neural activity generated by the brain and independent of its normal output pathways of peripheral nerves and muscles thus it implements the principle of “Think and make it happen without any physical effort”. This technology will be extremely valuable to people with devastating neuro-motor handicaps as they offer new augmentative communication technology to those who are paralyzed. Over the past decade, productive BCI research programs have arisen, facilitated and encouraged by new understanding of brain function, by the advent of powerful low-cost computer equipment, and by growing recognition of the needs and potentials of people with disabilities. The technology driving this breakthrough in the Brain Computer Interface field has a myriad of potential applications, including the development of human augmentation for military and commercial purposes. Many of these systems are being improved and will soon be of value to many different people in a wide variety of environments and situations.
Introduction
A group of technologies exploring the possibilities of alternate control interfaces using the brain as the initial signal generator are called brain computer interfaces or BCI. A BCI is a system that acquires and analyzes neural (brain) signals with the goal of creating a high bandwidth communications channel directly between the brain and the computer.
To better understand BCI one must understand the technology that comes together to create all of the different BCI systems. There are a few basic components to all brain computer interfaces, and they are signal capture system, signal processing system, pattern recognition system, and device control system. Each system must have a way to gather and hold data in order to respond to humans’ commands.
The signal capture system includes the electrodes themselves & the isolated electronic amplifiers. The signal is obtained by any brain function mapping technique such as EEG (Electro encephalogram), MEG (Magneto encephalogram), PET (Positron Emission Tomography) or FMRI (Functional MRI). Generally, EEG is preferred to measure brain activity. It is proved that according to different brain activities, EEG patterns will be different.
The signal processing system often used to be on a dedicated DSP board but now PCs are fast enough to do everything on the main processor. The algorithms that are implemented are Fast Fourier Transforms (FFTs) for spectral estimation, band pass filtering and Autoregressive (AR) modeling for linear prediction of the signal. AR models can also be used to derive spectral information.
The pattern recognition system often used to be composed of a linear classifier such as a logistic discriminant or a classical nonlinear classifier such as the Bayes quadratic classifier or linear vector quantiser (LVQ). Nowadays, neural networks are most commonly used.
Interfaces have been developed to control many different devices in device control system. Various software or tracking technology can be used to control the motion of output device.
The display unit can be auditory, tactile or visual but there must be a way to show the data to the user so that they may respond and interact with the technology. While existing technologies are still available to control a BCIs improve upon the computer interface to allow even the most severely handicapped to communicate with a computer.
Thus, Brain computer interface is the developing technology that can provide a new way of communication and control for paralyzed persons. It is a powerful technology that uses brain computer interface.
Block Diagram
For measuring brain function, neuroimaging modalities such as fMRI, EEG and MEG are providing clinicians and neuroscientists with a variety of powerful tools. Without a doubt EEGs have been the best tool so far for this type of research. From the different parts of the brain such as frontal, occipital, parietal & cortical different brain activities are measured with either invasive or non-invasive real time techniques.
After obtaining EEG signals, they are applied to signal processing unit, which includes amplifier, special function filters, ICA components (artifact rejection), ADC etc.
Now our task is to classify different EEG patterns according to its features such as frequency and amplitude in different states of consciousness like alertness, lethargy and dreaming. Our approach is generally based on an artificial neural network that recognizes and classifies different brain activation patterns associated with carefully selected mental tasks. Then the classified signal is translated into the control command signal using software to perform mental recognized task and is applied to the control device.
By watching the control action of the device on the computer screen, visual feedback from the eye is given to brain and the next control action can be decided by the user.
This whole close loop system is known as brain computer interface.
EEG Acquisition Techniques
Now, let’s understand every part of the block diagram of brain computer interface in detail. EEG can be obtained from the brain by either of two techniques.
[1] Invasive technique
[2] Non-invasive technique.
invasive technique
In this technique, EEG can be obtained either by using needle electrode or by implanting a chip (microelectrode) in the brain.
The brain chip is nearly of four-millimeter square chip, which is placed on the surface of the motor cortex area of the brain, contains 100 electrodes each thinner than a hair which detect neural electrical activity. The sensor is then connected to a computer via a small wire attached to a pedestal mounted on the skull. It develops a fast, reliable and unobtrusive connection between the brain of a severely disabled person and a personal computer.
Non-invasive Techniques
Many BCI technologies are striving to be non-invasive, as many humans do not feel comfortable and cannot afford to surgically implant devices in the skull. In the non-invasive technique, users wear an electrode cap that detects electroencephalographic (EEG) activity from the scalp and records specific brain waves. This technique allows detection of brain activity without any surgery or implantation. It has been widely assumed that only invasive devices could control complex movements, such as operating a word processing program or a motorized wheelchair by thought alone. The results show that people can learn to use scalp-recorded electroencephalogram rhythms to control rapid and accurate movement of a cursor in two directions.
When needle electrodes or electrode cap is used to obtain EEG from the scalp surface, we will be able to get overall brain activity. Well known free running EEGs include:
- Delta (1-4 Hz), found in deep sleep
- Theta (4-8 Hz), found in sleep, meditation, hypnosis
- Alpha (8-14 Hz), indicate relaxation and closed eyes
- Mu (8-14 Hz), largest when individual is not moving
- Beta (non specific higher frequencies), indicate alertness
- Event related potentials (ERPs): Brain’s response to a specific event, such as a tone or flash
Commonly studied ERPs include the P300 and N400.
- Spontaneous or free-running EEG: Naturally produced, rhythmic brainwaves; do not ………require outside activity.
There are some limitations of macroelectrode (electrode cap) recording from the scalp:
- Scalp smears electrical signal
- Only measures neurons near the scalp
- Only measures neurons perpendicular to scalp
- Neurons aligned opposite each other cancel each other
- Neurons must be active in synchrony to be detected
Macroelectrodes only measure the coordinated activity of many millions of neurons so we can’t obtain brain signals from any particular area of the brain, which is sometimes necessary in brain computer interface so that different signals can be used to control different devices. We can also control the devices even by using overall brain activity signal but it provides limited applications because of very few different types of patterns that are available whereas microelectrodes only measure the activity of one of very few neurons that means the signals obtained using brain chip we can obtain brain signal generated from any particular area which is just impossible using electrode cap. From different parts of brain different type of patterns can be obtained so we can classify these signals and control different devices by assigning different activities to the different patterns of brain signals.
“The impressive non-invasive multidimensional control achieved in the present study suggests that a non-invasive brain control interface could support clinically useful operation of a robotic arm, a motorized wheelchair or a neuroprosthesis,” said the researchers.
EEG Classification
EEG classification can be done by various methods, which contains Local Neuron Classifier (LNC), Linear Vector Quanticer (LVQ), RF, SVM etc. But the simpler and most efficient method of EEG classification is Multi-Layer Perceptron Neural Network (MLP-NN) based method. This method is shown in the figure above and different steps to understand the procedure are as described below.
- Acquire raw EEG data. Filter the EEG channel using a bandpass filter between 4 and 25 Hz.
- Use Morlet Wavelets to extract local frequency information. Compute their absolute value. These are the feature channels.
- Feed a two layer feed forward neural network with the frequency information and an additional time channel (restarts at zero at the begin of every trial). The neural net has two layers: the first weight layer uses the tanh activation function, the second a normal logistic activation. The net is trained using the cross-entropy error as the optimization criterion. The output of the neural network is the estimated instant classification.
- The final classification is obtained after performing a weighted time integration of the instant outputs, where individual weights are higher for low entropy outputs.
Signal Processing
One of the most significant obstacles that must be overcome in pursuing the utilization of brain signals for control is the establishment of a signal processing method that can extract event related information from a real-time EEG. Lab must be specialized in advanced, real-time statistical signal processing techniques, including robust, time-series methods, pattern recognition methods, and various custom and standard transformations (including Wavelet Transforms and Time-Frequency Transforms) for data analysis.
Amplification and Filtration
The EEG signal obtained from electrode cap must be amplified before further processing. Many EEG frequencies are not of interest because they do not provide the information about cognitive processes (involving psychological result of learning and reasoning).These frequencies are filtered out of the data very early in the recording process. In order for the signals to be filtered it is important to find a reference point that helps to represent better the brain activity coming from the motor-related mental tasks.
This can be accomplished using spatial filtering. Spatial filters are used when we need to rely on nearby, adjacent, values to estimate the value at a given point. Filters take out variability in a data set while retaining the local features of data. By varying the size of the filter, features in the data that vary at different spatial scales can be differentially removed.
After filtering surface laplacian technique is applied. Surface Laplacian is a technique that has been utilized to improve the spatial resolution of EEGs and even MEGs. By examining and understanding spatial filters, which describe the relationship between cortical current sources and the surface Laplacian, the amount of improvement to the spatial resolution afforded by the surface Laplacian can be investigated. The surface Laplacian spatial filters extend into higher spatial frequencies than do raw signal spatial filters, particularly for EEG Laplacian spatial filters, indicating that substantial improvement in spatial resolution is possible. However, the response of the surface Laplacian operation to the nature and amount of noise in the raw EEG and MEG signals is of paramount importance. Spatially correlated noise, coupled with uncorrelated noise, requires additional regularization of inverse spatial filters resulting in a decrease in spatial resolution. Substantial improvements in spatial resolution may be obtained using the surface Laplacian techniques as long as correlated noise levels are small and raw signals have relatively high signal-to-noise ratios. The next step is to band-limit the signal using a second order butterworth bandpass filter to obtain a smooth response the cut-off frequency.
EEG artifact removal using ICA technique
Severe contamination of EEG activity by eye movements, blinks, muscle, heart and line noise is a serious problem for EEG analysis. We propose to apply ICA (Independent component analysis) to multichannel EEG recordings and remove a wide variety of artifacts from EEG records by eliminating the contributions of artifactual sources onto the scalp sensors. Our results show that ICA can effectively detect, separate and remove activity in EEG records from a wide variety of artifactual sources, with results comparing favorably to those obtained using regression-based and Principal Component Analysis method.
Extracting single-trial evoked responses from spontaneous EEG
In single stimulus epochs the evoked response activity may vary widely in both time course and scalp distribution. The major difficulty in comparing single trials is that the spontaneous EEG activity may obscure response-evoked activity, since spontaneous EEG is typically much larger than the evoked response. ICA constructs spatial filters that can separate ERPs from EEG and artifactual sources.
All of the technologies are attempting to improve upon the current methods to increase signal-to-noise ratio (SNR), signal-to-interference ratio (SIR)) as well as optimally combining spatial and temporal information to transmit the most accurate information possible.
Control Interface & Control System
A group of technologies are exploring the possibilities of alternate control interfaces using the brain as the initial signal generator. The EEG signals obtained after classification is used to control the output device. The control can be achieved by using either software technique or tracking system.
These signals are fed into the computer. The signals are then interpreted by the software inside the computer to determine when the subject is attempting to activate the control device. The experimental control system is configured for the particular task being used in the evaluation. All our control programs are generated by Real Time Workshop from Simulink models and C/C++ using MS Visual C++ 6.0. Analysis of data is mostly done within Matlab environment.
Many brain computer interfaces achieve their unique level of control through the use of tracking. Tracking is a way to capture the motion of humans. Researchers can track body movement, eye movement or electrical signals from the brain. Tracking technology always requires software to interpret the data collected by the tracking device. Tracking can be done through a number of technological innovations. Most of these tracking technologies were developed for use in human computer interfaces. One of the tracking technologies generally used in brain computer interface is electromagnetic tracking technology.
Electromagnetic tracking technology can monitor the orientation of the user’s head and hand. The system emits an electromagnetic field, and a sensor reflects the field. When the sensor is moved it detect different magnetic fields that encode its position and orientation. The decoded signals are relayed to the playback until. The latency on electromagnetic systems is very low and it allows for large areas to be monitored in terms of movement. There are many other tracking technologies used in brain computer interface other than electromagnetic tracking technology are
- Mechanical tracking technology
- Optical tracking technology using infrared video camera
- Ultrasonic tracking technology
- Eye tracking technology
Feedback
The process called neurofeedback involves connecting electrical impulses from the user’s brain to the computer and back again by visual feedback by watching the control action on the screen of the computer monitor.
In the case of visually impaired person, a video camera device could send signals through a transdermal connector to the array, which with appropriate signal modification could transmit the correct series of electrical stimuli to the retinal axons and cause the brain to “see” the image transmitted from the camera.
Training
An interesting question for the development of a BCI is how to handle two learning systems: The machine should learn to discriminate between different patterns of brain activity as accurate as possible and the user of the BCI should learn to perform different mental tasks in order to produce distinct brain signals.
BCI research makes high demands on the system and software used. Parameter extraction, pattern recognition and classification as well as the generation of neurofeedback for a successful training of the user has to run in real-time.
The whole training process can be described stepwise as below.
Step 1 (initial training):
Based on a cue (arrow on the screen pointing to the left or to the right) the subject performs left and right-hand movement imageries (duration 3-4 seconds). To train the classifier between 40 and 160 trials are recommended. EEG should be recorded from electrode positions.
Step 2 (offline analysis and classifier generation):
Alpha and beta bandpower parameters for both EEG channels are computed to build the feature matrix. Multi-Layer Perceptron Neural Network (MLP-NN) is used for classification and cross-validation shows the usability of the best classifier.
Step 3 (training with neuro-feedback):
If cross-validation results yield a classification error below approx. 20 %, the classifier can be used to generate neuro-feedback for further training. For this case data are online classified and the result is graphically presented to the subject on the screen of the monitor and according to that subject will generate control action that will eliminate the classification error.
Step 4 (classifier update):
The continuous feedback should help the subject to train the motor imageries leading to a correct classification. To improve the performance the classifier should be updated after some successful sessions. A new classifier can also be computed from the data of a feedback session. Offline analysis of the recorded data supports feature optimization.
The four steps described above are used to improve the “feature set” of the data being fed into the neural network.
Some of the most commonly used strategies to realize a BCI are:
- Imagery of movements of different limbs cause changes in oscillatory EEG activity over the areas of the central cortex. These changes can be classified by weighting spectral parameters of different frequency bands for different electrode positions.
- Slow shifts of cortical potentials occur when a subject performs an imagery of expecting an event (like waiting for a traffic light turning to green). The resulting DC-shift can be used for biofeedback to improve the training effects and to generate a control signal for communication.
- Also, other mental tasks such as mental arithmetic, mental cube rotation or attention versus relaxation are used to produce characteristic changes of EEG patterns. One attempt has also been not to guide the subjects with any strategy but use specific EEG-biofeedback, so that the user attempts to find his/her own strategy for producing the required changes in the EEG.
- Another method uses steady-state visually evoked potentials (SSVEP) from flickering light sources. Directing attention to a source with a specific flicker frequency enlarges evoked components in the EEG with the same frequency.
It can be stated that none of all the methods used in BCI research yields perfect results, but the performance was significantly improved by new parameter-extraction algorithms and pattern-recognition/classification methods. The usability of a BCI has to be evaluated with respect to the following aspects:
- Accuracy
(classification error, hits vs. false, false positives, …)
- information transfer
(decision speed, bit/min, …)
- number of classes
(idling vs. activation of 1 class, 2 or more different classes, …)
- operation mode
(synchronous: predifined decision intervals, asynchronous: free decicion time)
- intended application
(spelling device, control of orthotic/prosthetic device, environmental control)
Current Applications
Cursor control using BCI
- Cursor control is based on changes in the subject’s mu rhythm, an 8-12 Hz rhythm found over primary somatomotor areas and detectable in almost all adults.
- Vertical movement is based on the sum of RH (right hemisphere) plus LH (left hemisphere) mu amplitude. A larger sum produces upward movement, while a smaller sum produces downward movement.
- Horizontal movement is based on the difference of RH minus LH mu amplitude. A larger difference produced rightward movement, while a smaller difference produced leftward movement.
Virtual Keyboard
This is the example of virtual keyboard in which the EEG cap is placed on the head of user. By thinking about left and right hand movement the user controls the virtual keyboard with her brain activity and accordingly the letters are typed in the box shown below the keyboard in the computer. By using this technique, paralyzed people can compete with healthy persons as well.
Some other applications other than the two explained above are:
- Use brain signals as Morse code
- Used for paralyzed persons
- As Game controller (Pac man, pong) in entertainment world
- Control of augmentive/assistive devices- Operation of physical devices such as mobile robots, wheelchairs
- Control of prosthetic devices
- It can be used for epilepsy control.
Advantages of Brain Computer Interface
- Easy to Use
- Natural, intuitive, direct, hands-free
- Functional Flexibility
- Works when hands, eyes, or voice are damaged or busy
- Information Content
- Faster access/retrieval and higher density
- Customized to each user
- Provides security/secrecy
- Excellent interactive tool
Disadvantages of Brain Computer Interface
- The speed of communication on computer is very limited compared to the brain.
- BCIs are very susceptible to artifacts. Hence BCIs can only be used in very limited situations.
- EEG recording produces a lot of noise.
- Highly relevant pattern recognition techniques are currently being invented and modified.
- More EEG research is necessary to determine where and how electrophysiological states correlate with mental states, and how this varies in different subjects, recording conditions, time of day, fatigue, experience, and more.
- EEG caps are expensive, look silly, and require an experienced human to prep.
- Other EEGs recording equipment (such as amplifiers and filters) is expensive, complicated, and not very portable.
- It often takes a long time to train the user to interact with the BCI properly.
- It often takes a long time to train the BCI to interact with the user properly.
- Difficulty with a microelectrode array is that it probably would only be able to provide black and white vision.
- Braingate implant may include possibility of infection, bleeding, stroke & pain.
Future
The most important issue in future Brain computer interface research will be how to assist people to accessing, managing, and understanding the vast amount of data and information that is available to them
A scientist named Nicolelis and his team are confident that in five years they will be able to build a robot arm that can be controlled by a person with electrodes implanted in his or her brain. Their chief focus is medical they aim to give people with paralyzed limbs a new tool to make everyday life easier.
Conclusion
Although far from mature, progress is being made in this field of biomedical engineering. Defining the self in the postmodern world through the brain computer interface is a new challenge. When a high-resolution brain computer interface (BCI) is successfully completed, there will be profound benefits to be made in almost every disability related field. “One day people may be capable of communicating through thought processes”.
