by Oleg Artamonov
11/17/2008 | 11:13 AM
New input devices, radically different from the mouse and keyboard, emerge from time to time. Voice input was discussed widely once and everyone thought then that computers would soon learn to recognize not only individual commands but even coherent speech and we would give up our keyboards and begin to irritate our colleagues by mumbling incessantly.
But years have passed and voice-based control has not become a mass or even notable tool as it encountered a technical obstacle: a powerful artificial intelligence is necessary to decipher human speech in real time considering the lack of pauses between words, the difference in pronunciation and intonation, the changing tempo and other peculiarities.
If even this has proved to be impossible, the idea of controlling the computer with the power of thought seems to belong with sci-fi novels or, at least, with very specific research laboratories. The phrase “mind-based control” is likely to provoke the image of a helmet stuck with electrodes and connected to the computer with a thick bunch of cables.
Therefore the quick and ubiquitous introduction of the NIA device from OCZ has been a real surprise. NIA stands for Neural Impulse Actuator. Neural impulses are the electric signals of our nervous system that facilitate the processing of information in the brain and transfer commands to the various muscles of the body. Thus, the NIA is supposed to register such impulses and translate them into computer-controlling commands.
And there are no helmets, no cables. Just a neat headset. Its system requirements are limited to one free USB port.
I want to say it right away that the NIA can’t read your thoughts in a literal sense. The human brain is so sophisticated that the electrical noise it produces at work cannot be fully decoded with current technologies. Even the standard method of recording an electroencephalogram that gives but a basic notion of what’s going on in the brain requires as many as 19 electrodes to be placed around the patient’s head. Perspective innovations that are intended to help paralyzed people to truly control a computer with their mind are often based on chips and electrodes implanted into the brain and gathering information about activity of its parts.
So what can the NIA read if it has only three sensors and does not need an experienced neurosurgeon to work? Alas, the name of the article may be somewhat misguiding. It is not thoughts and even not images. The main source of signals for the NIA is the neural impulses of mimic and eye muscles. Besides, the device can sense the alpha and beta rhythms of the brain that provide a general notion of its activity. Alpha rhythms dominate a relaxed state with the eyes closed whereas beta rhythms dominate during active mental activity.
The NIA is shipped in a neat white cardboard box that shows a picture of the case that accommodates an amplifier, ADC and USB interface, but not a picture of the headset proper. So, the box gives no idea as to how this advanced biotechnology for games is supposed to work.
Opening the box up, you can see the neatly packed black interface case pictured on the box and the main component of the NIA – a headset with sensors.
Besides that, the box contains a USB cable, user manual, and CD with drivers. I advise you to download the latest version of the driver from the OCZ website as it is updated quite frequently.
The NIA sensors are placed on a rubber headset that you put on your head and fasten with a couple of laces at the nape of the neck.
This is a universal but not very handy fastening: if you’ve got a chair with a tall back, you won’t be able to lean back in it because the hard ball of the laces lock will press into the back of your head.
There are three sensors on the front part of the headset. These are hard plastic frames that press tight against the skin of your forehead. The headset doesn’t incorporate any electronics – it is all accommodated in the interface unit.
The finish quality of the plastic isn’t high. The photograph shows ragged edges although the interior surface of the sensors is smooth. The raggedness doesn’t tell on the user’s convenience, though. It only affects the appearance of the device.
The electronic components – amplifiers receiving the signal from the sensors, an analog-to-digital converter and a USB interface – are all accommodated in a small and neat box, just over 10 centimeters long, which is made from anodized aluminum. This box doesn’t need additional power. The NIA is powered by the USB port it is connected to.
The headset is connected to the electronics box with a 3-pin connector. The cable is 1.4 meters long, thin and flexible.
On the bottom panel of the case there are rubber feet, vent holes and a label that covers the screws you need to unfasten to take the device apart. When the NIA is turned on, you can see a red LED shining through the slits. I don’t understand why the manufacturer didn’t make this LED more conspicuous and external.
The internal design of the NIA is neither too sophisticated nor uncommon. In the left part of the PCB there is a Microchip PIC microcontroller. The center of the PCB is occupied by an Analog Devices ADUM1300 isolator and a Fairchild HCPL-2631 optocoupler. A BurrBrown PCM1803A analog-to-digital converter, a Texas Instruments OPA4340 operation amplifier and a few smaller chips reside in the right part of the PCB.
Take note that the electrical circuit of the headset is separate from the computer’s power supply, obviously for safety reasons. The opamp and the ADC are powered by a dedicated switching power supply (its choke can be seen in the center of the PCB in the bottom part of the photo). The digitized data is transferred through the ADUM1300 isolator which is rated for a voltage up to 2.5kV. Thus, OCZ’s engineers have done everything necessary for the headset to be free from dangerous voltages. The headset is electrically isolated from the USB port as well as from the computer at large.
Interestingly, the back part of the PCB has a caption that reads “Powered by Brainfingers technology.” This sheds some light on the origin of the NIA. Brainfingers is the trademark of Brain Actuated Technologies, a developer of “brain interfaces” for people with various types of mental disorders. OCZ must have got interested in the opportunity to commercialize this technology and now offers a mass-market device based on it.
I performed most of my tests of the NIA on the following PC configuration:
Since I had begun my tests before the release of the 64-bit driver, I also checked the NIA out under 32-bit Windows Vista and Windows XP. So I can tell you that the 64-bit driver has no problems in comparison with the 32-bit one, but the latest versions of the driver ensure a much better control than version 1.000 you can find on the included CD. I connected my NIA to four different computers, two desktop PCs and two notebooks, and had no problems with installation and setup.
From the OS’s point of view, the NIA is a regular Human Interface Device like keyboard or mouse. However, you must install special software for the NIA to work. This software allows to calibrate the sensors and assign their signals to the movements of joysticks or to the pressing of buttons.
The program incorporates an extended user manual the first part of which teaches you the right way of wearing the headset. It is easy, actually: you just have to avoid your hair to get under the sensors. You don’t have to tighten the headset too much. It should only have good contact with the skin of your forehead.
Next goes a detailed narration about the device’s capabilities and setup tips. The text shown above makes it clear that setting the device up is not a straightforward process. The text describes the situation when the Jump and Move Back triggers react simultaneously and you can’t jump over a gap without falling into it the next moment. The developer suggests that you enable a 1-second delay for the Move Back trigger to give yourself enough time to make the jump. However, it means that you won’t be able to move back quickly. I will discuss the problem of multiple triggers reacting simultaneously below.
The next step is about calibrating the sensors. Put the headset on and relax. If everything’s normal, the yellow graph in the screen must get below the green graph. It means that when you are motionless, the signal from the sensors is close to zero.
A few problems can arise here.
First of all, the sensors need a good contact with the skin which depends on the state of the skin. If your skin is too dry, the signal will be bouncing up and down or even keep always at the maximum level. The developer recommends applying some moisturizing cream to your forehead to solve this problem. In my personal case the contact depended on the weather, time of the day or my mood. It is hard to pinpoint the reason but at one moment I could calibrate the NIA at first attempt but at others I had to use moisturizing cosmetics. By the way, in medicine the sensors are covered with a special gel when recording an electroencephalogram.
Second, all the three sensors must be pressed tight to your skin. This is always the case with the right and left sensors, but the position of the middle sensor depends on your head. If your superciliary arches protrude but a little, the middle sensor won’t touch the skin. You can correct this by shifting the headset up a little, but your eye movements will then be registered by the sensors poorly.
I personally couldn’t find a position for the headset in which the latter would register the horizontal movements of my eyes well enough (I either could not calibrate the NIA or the headset proved to be too high, producing a weak signal), but my wife did this at first attempt. The reason could be seen easily when I put the headset off: the right and left sensors left a clear print on the skin while the middle sensor left no trace at all.
Third, for the NIA to work you computer must be grounded or independent of the power grid. Otherwise the signal is bouncing up and down all the time. I solved this problem with my desktop PC by using a 3-wire power plug with grounding. With a notebook, which didn’t have a grounding contact in its power plug, the NIA worked normally only when the notebook was powered by its battery or grounded with a separate wire. It may also help to put your hand on the NIA’s metallic case – to ground yourself, so to say.
The developer also warns that there should not be electromagnetically noisy devices near the NIA. All interference will be registered by the NIA’s sensors.
If you click in the bottom right corner of the program window, you will open up a screen where you can adjust the sensitivity of the NIA’s sensors. Well, if you’ve got serious problems with calibration, such adjustment won’t help much. The reduction of the sensitivity level will eliminate the electromagnetic interference but will also make the NIA insensitive to your neural impulses.
During the calibration process you are going to look at a gyroscope rotating on the screen for a dozen seconds. You can even defocus your eyes. The main thing is to relax your mimic and eye muscles and not move them at all. The calibration over, the NIA will draw a red line corresponding to zero level. If this line is below the green one, the calibration has been successful and you can proceed to training. If the red line is high and the yellow line has sudden spikes, you shouldn’t try to use the NIA because you won’t be able to control anything with it without successful and stable calibration.
Next you can press the Brainfingers button and look at the signals from the NIA’s sensors sorted by type but not yet assigned to specific buttons or joysticks. These are Glance (eye movements), Muscle (mimic muscles), and alpha and beta rhythms of the brain.
In case you don’t know, alpha rhythms of the human brain are electromagnetic radiation with a frequency of 8-12Hz that can be observed when the brain is awake but the eyes are closed. Alpha rhythms get weaker during sleep or slumber as well as when the eyes are open.
Beta rhythms correspond to strong mental activity, to the state of anxiety or concentration. Beta rhythms have frequencies above 12Hz.
It is obvious that controlling something with your brain rhythms is more difficult than with muscles. To change their level you have to learn to relax, even to slip into a state of meditation, and to concentrate quickly. The result is predictable: judging by the NIA-related forums, only few users are able to consciously change the level of alpha or beta waves even after several days of training. My training was fruitless, too.
Well, the simplest exercise doesn’t require that. Let’s press the Practice button.
The NIA offers a simple practical exercise: a game that had appeared electronically long before the arrival of the PC. It is ping-pong. The screen displays two rackets and a ball bouncing between them. The computer controls the right racket, and you must move the left racket using the NIA.
Of all of the NIA’s capabilities this game makes use of only one vertical joystick that is controlled by the signals of the mimic muscles. That’s just what you need for a first training. If the above-described calibration has been successful, the racket is at the bottom of the screen when your muscles are relaxed. As soon as you wrinkle your forehead, the racket will jump up.
The exercise seems to be pointless at first. You try to wrinkle your forehead at such a moment that the racket got under the ball while flying from the bottom to the top of the screen. You seldom succeed.
Then you manage to wrinkle your forehead with as much force as is necessary for the racket to hang somewhere in the middle of the screen. You lose control from time to time, letting the racket fall down or jump up. You do this most frequently when the ball is just about to hit the racket.
Then you train yourself to move each eyebrow independently for the necessary angle to shift the racket vertically more or less well. And you hit the ball with an 80-percent precision. The main thing is not to think that this is the most you can get from the NIA and not throw it away.
It is hard to spot the moment of getting to the next level. It happens somewhere in your unconscious, which is a real shock. At some moment your brain transitions from the commands “Lift the eyebrow” and “Lower the eyebrow” to the command “Move the racket there.” You don’t think about what muscles of your finger to relax and strain in order to scroll this article with your mouse, do you? You just think “I have to move the wheel” and the brain does the job automatically.
It’s the same thing with the NIA. You just think about moving the racket up and the racket moves up! The people around you see that you have strained your forehead muscles a little, but you don’t perceive this as the straining of some muscles. You perceive this as the movement of the racket on the screen.
That’s a fantastic moment indeed. Once this state is achieved, you realize that the NIA is not so far from reading your thoughts. It doesn’t read them, actually. But you do get the feeling that it does! You are not controlling your forehead muscles – you are controlling the racket on the screen, folding your hands on your chest, leaning back in your chair and grinning at the people around. This feeling can’t be described – you have to feel it yourself. The NIA is far different from any automatism you can achieve with a keyboard, mouse or any other advanced input device.
And while you keep on training, you will learn to exert very little effort. A very weak movement of your facial muscles will be enough for a successful command.
The program is pompous but laconic about your first win at ping-pong. Next you can proceed to more sophisticated games. The NIA can translate neural signals into keystrokes and mouse movements in them.
The program has a number of ready-made game profiles and offers the opportunity to create your own profile.
Let’s see how your custom profile can be created. This profile defines what commands the different types of NIA signals (mimics, eye movements, and alpha and beta rhythms of the brain – if you’ve learned to control them) are translated into.
The first type of commands is called Switch Control. These are binary controls that can take the value of 0 or 1. Most profiles have only one event assigned to this control – the press of the left mouse button – but you can assign three events simultaneously if you want. For example, a shot (left mouse button) while jumping (spacebar) over your head (not in all games). You can’t specify the source signal of the control, but the developer says it is the blinking of your eyelids. They say that an ordinary person’s reaction and pressing of a mouse button takes about 250 milliseconds whereas the blinking of the eyes helps reduce this lag to 100-150 milliseconds. In other words, if your mouse-wielding opponent encounters you in an online shooter, he will have time to blink but not to shoot!
The other type of controls is called Joysticks. The NIA supports as many as four of them. You can specify the signal each joystick reacts to: mimic muscles, eye movements or brain rhythms.
After you have specified the control signals, you can set the joysticks up. Depending on the signal level, the respective slider can move from top to bottom position of the rule you can set up to four marks or triggers on. Each trigger corresponds to one, two or three commands (events).
The screenshot above shows two triggers defined for the joystick. There is no action when the signal level is zero – the slider is below the Z2 mark. At medium signal level the slider gets above the Z2 mark for which one command is assigned – the pressing and holding of the W button which corresponds to forward movement in the standard gaming WASD layout. I have previously selected mimic muscles as the signal source for this joystick, so now I will be able to move my hero forward in games by slightly raising my eyebrows.
The other trigger, Z4, corresponds to the maximum of signal. It is assigned the S button, i.e. backward movement, and the delay+hold mode. This tricky combination is necessary when a brief strong signal is assigned to a jump in another control. If the Z4 event had no delay, I would step backward together with making a jump in games. With the delay enabled, a brief rise of the eyebrows will do a jump but the backward movement won’t be enabled.
This trick has one obvious drawback – backward movement will always be performed by your game character with a delay.
Although called a joystick, this control element has only three states (idle, Z2 and Z4). The precise position of the slider between these marks is unimportant. Its crossing a specific mark is what matters.
You can set three parameters for each joystick which define its reaction to signals from the NIA’s sensors.
The horizontal joysticks registering your eye movements are somewhat different. Their idle point is in the middle. For the vertical joystick every nonzero signal from the NIA’s sensors means upward movement, but the horizontal joystick distinguishes the direction, right or left.
You can set two triggers for each direction, corresponding to weak and strong signals. In my example, only weak signals are used. I assigned them to the presses of the A and D buttons, i.e. to the character’s moving left and right. Like with other control elements, you can assign up to three buttons to each trigger. For example, you can enable the R2 and L2 triggers that correspond to maximum signal and make them move your character right and left together with a jump. Then, you will be able to move your character into the necessary direction by simply looking into the left or right corner of the screen. But if you move your eyes quickly, you’ll do the same thing with a jump.
Thus, the current version of the NIA software does not allow to control an onscreen pointer or emulate a joystick sensitive to the stick tilt. In fact, every available control element is discrete. You can just assign up to four different events (keystrokes or key combinations), depending on the signal level, for each of the so-called joysticks. You can also specify if it is a single stroke or the button must be held pressed. And you can also enable a delay before the reaction when necessary. Thus, the NIA can’t replace a mouse in games but can be used instead of a few buttons.
Having set the joysticks up, you can bind your profile to a game.
Every profile is saved on your hard disk. You have to select the necessary one before launching the game.
When you select a profile, the NIA prompts you to check it out. The screenshot above shows the emulation of a mouse click. The yellow line corresponds to the signal from the sensor that registers the blinking of your eyelids. The horizontal dashed line corresponds to the threshold level after which the event occurs. You can check out and adjust all the joysticks set up in this profile in the same way.
The last screen of the NIA program reminds you the shortcuts: Ctrl+F12 to enable NIA-based control in the game and Ctrl+F11 to enable sounds for the joysticks’ triggers. Now you can press the Launch button and start the game and press Ctrl+F12.
Well, the theory is okay, but the practice was a disappointment.
As I wrote above, the shape of my head doesn’t provide a proper contact with the middle sensor, so I had to move the headset higher up my forehead. It was thus farther from the eyes and could not register their movements. The signal proved to be too weak to move the horizontal joystick. And when I increased amplification, the joystick would jump chaotically left and right reacting to electromagnetic interference.
My wife was more successful. The NIA registered her eye movements easily and identified the direction well enough. However, she had to slant her eyes too much to move the joystick’s slider, which was somewhat uncomfortable. This problem could be eliminated by adjusting the sensitivity, though.
Another problem was the spontaneous reaction of the trigger that was responsible for the left mouse button (this button corresponds to a shot in most of shooters). This trigger was supposed to work at a blink of the eyelids. In fact, you have to actually shut your eyes to make the trigger work (otherwise the natural blinking would have been registered as well). And the trigger would often react not to the blinking of the eyes but to sudden movements of the mimic muscles. I didn’t manage to set the sensitivity thresholds in such a way as to avoid accidental shots.
The overall result is not encouraging. I could not get as much in-game control with the NIA as with an ordinary keyboard and mouse. The NIA is not very good at differentiating various signals and may confuse a shot with a jump and a jump with a step back. You also have to restrain your emotions as frowning may provoke a real squall of commands! The headset also feels uncomfortable on the head after a while, but you cannot adjust it as this will result in half a dozen false commands or even in total loss of control.
The OCZ NIA is surely a highly exciting device. It is indeed a new type of computer input devices and it really works. I don’t regret any second of my time I spent to test the NIA.
Although the NIA cannot read your thoughts, being limited to reading the neural impulses of your facial muscles, you do get the incredible feeling of controlling the computer with the power of your mind in simple games after a period of training. Your brain gets used to the NIA and you catch yourself realizing that you are controlling not the movements of your eyebrows but the position of the ping-pong racket on the screen.
Unfortunately, it proves to be more difficult to achieve high control precision in more complex games when all the NIA-accessible types of neural signals are at work and there are multiple events defined. Right now the NIA is not good enough at telling apart different types of neural impulses, and you have to adjust the sensors’ sensitivity meticulously or resort to such tricks as deliberate delays.
The NIA’s operation depends not only on your training but also on your skin and facial features. The NIA’s sensors do not work well on dry skin, requiring moisturizing cream. People with protruding superciliary arches have to forget about the horizontal joysticks because it is going to be hard for them to position the headset in such a way that it registered the eye movements and ensured a proper contact of the middle sensor with the skin.
Thus, the OCZ NIA is the most exciting device to experiment with, but its practical value in games is questionable. Its usefulness depends on the specific person much more than with any other type of input device. Of course, the NIA will come in handy for people with disabilities who find it difficult to use an ordinary mouse or keyboard, but I’m afraid that’s the only serious application of this device as yet.
There are quite many people writing positive reports about the NIA at forums. But there is no guarantee that it will suit you personally just as well. And the NIA is rather too expensive - about $130-$150 - to be bought just as a hi-tech toy.
Here is my wish-list for the developers. I would want the next version of the NIA to have a different design of the headset so that the sensors always had good contact with the skin. The NIA should also get better protection against EMI. And its software should improve its recognition of various types of neural signals.
I guess I should also mention two competing solutions that use a similar principle. One is the Emotiv EPOC device from the Australian Emotiv Systems. It is indeed like those devices you see in sci-fi movies as it has as many as 16 sensors placed all around the user’s head. And the other is the single-sensor NeuroSky that can register the general state of the brain: concentration, relaxation, irritation. The EPOC is expected to come out in 2009 as a standalone device whereas the NeuroSky is going to be supplied to the developers of toys, game consoles and other products for integration into them.