Immersive virtual reality has barely cleared the launch pad, but a new technology promises to redefine the VR experience. It’s called full dive virtual reality and it is straight out of science fiction — literally. Full dive virtual reality technology pushes VR beyond immersive headsets and makes the user one with the machine through the use of a Brain Machine Interface, or BMI. You read that right. The age of the cyborg is nearly upon us... almost.
Yes, advances are being made across a wide range of technologies that will allow users to control computers with mere thought, and to receive data from the computer directly into the brain. A true full immersion virtual reality experience is the new holy grail. But is full dive virtual reality possible now, or shall we check back in a few decades? More to the point for gamers, if full dive virtual reality is soon to transition from science fiction to the inferior world known as reality, when will full dive virtual reality gaming become more than a distance dream?
Before we explore those questions, let’s step back into sci-fi and remind ourselves exactly what we hope FD VR will do for us.
Origins of full dive Virtual Reality
The term originated from a Japanese light novel anime series, Sword Art Online, written by Reki Kawahara in 2009. In the series, a Virtual Reality Massively Multiplayer Online Role-Playing Game (VRMMORPG) called Sword Art Online, or SAO, is released in the year 2022. 10,000 players don their “NerveGear” BMI helmets and begin to play.
Players soon realize that the game developer has locked them into the game and that any attempt to logout or end the game will be fatal to their real-life bodies. The only way to exit the game, and to survive, is to successfully reach the 100th floor of the castle and defeat the final foe.
Not exactly the pitch you want to give investors, to be sure, but there are some good things there we can take away. First, the story popularized the concept of VRMMORPG. Gamers who weren’t thinking of engaging untold minions in gameplay suddenly started dreaming. Second, the concept of a helmet creating a non-invasive neural interface between man and machine wetted the appetite for those wanting a more seamless way to interact with computers. Not that the idea of BMIs was new, but seeing the concept demonstrated in such a rich storyline made the idea all the more tempting.
Examples of Full Dive Virtual Reality in Use
Back in the real world, real and exciting things are happening. Full dive virtual reality technology is rapidly stepping off the pages of science fiction and into the lab.
Recent advances in technology can sometimes be appreciated more when contrasted with earlier successes and failures.
In 2013, Harvard University broke ground for FD VR advances by conducting a human- brain-to-mouse-brain experiment in 2013. In this experiment, researchers used an electroencephalogram (EEG) brain-brain-interface (BBI) to detect the thought patterns of a researcher. By controlling his thoughts, the researcher was able to control the movement of a rat’s tail.
There were two reasons why this experiment was noteworthy. Not only did it demonstrate that human thought could be correctly interpreted by computer and used to control a rat’s brain, but the experiment was non-invasive for both the researcher and the rat. While researchers around the world have be able to control rats’ brains through the use of implanted electrodes, this rat suffered no such indignity. Non-surgical focused ultrasound (FUS) was used to impart the control signals to the rat’s brain, a technology we will elaborate on shortly.
The Paralyzed Walk
Fast-forward to 2015. Researchers at the University of California at Irvine, last year, also used an electroencephalogram machine to detect human thought. This time, however, those thoughts did not make it possible to control a rat’s tail, but to help a paraplegic man walk for the first time in five years. The man, who had suffered a spinal cord injury, walked nearly four meters with the assistance of the EEG device and advanced software.
The study involved placing electrodes on the patient’s head and on his legs. By detecting and interpreting signals from the man’s brain and sending those signals to his legs, the system bypassed the damaged spinal cord and allowed him to once again control the movement of his legs.
The experiment required the patient to undergo 19 weeks of training prior to taking his first step, but the encouraging results proved the concept was valid.
These two experiments may not be encouraging to those who had hoped for more, but consider this. Within two years, we have seen the technology advance from wiggling a rats tail to allowing a paralyzed person to actually walk across the floor. Such progress should be exciting, indeed, for anyone interested in full dive virtual reality development.
Technologies involved in Full Dive Virtual Reality
Full dive is a term that has been used by some game developers to imply that their games provide a more-immersive VR experience. By using advanced full-body motion sensors and high definition spacial audio, along with the VR headset, players do enjoy more immersive gameplay than ever before. But this is not true full dive.
Full dive, in the truest sense, goes beyond external viewers and sensors for user interaction and provides direct interaction between the user’s brain and the computer. Full dive is not so much full body immersion as it is full mind immersion.
Physical connections to the brain are not necessarily required, but there must be interpretation of the user’s thoughts by the computer, and — more importantly — the computer must be able to send sensory data directly into the user’s central nervous system. Full dive is, essentially, a brain-to-computer interface.
BCIs have been around, in various experimental stages, for years. What makes full dive a new concept is that it has the added component of virtual reality. Regardless of whatever advancements in BCIs may have been made until now, VR is a game changer (no pun intended).
Within the scope of what full dive is, or can become, is the question of just how much interaction between user and machine is possible. Two levels of interaction are theoretically achievable: Mobile and Immobile.
Mobile and Immobile Full Dive
Mobile full dive, as the name suggests, involves the user physically moving his or her body with their movements being detected by external sensors, not unlike current VR technology. For lack of refined methods of detecting and interpreting thoughts, physical action provides the most reliable means of providing data to the computer. In order for the experience to be FD, the sensory data returned by the computer must be fed directly into the user’s brain or central nervous system; otherwise, we just have a conventional VR experience.
Immobile full dive, on the other hand, means just what it says: the user is immobile and the FD experience is entirely cerebral. No physical movement is necessary, as the computer can translate the user’s thoughts with adequate precision to control the program. While conventional VR gear could still be used — and probably will be for the foreseeable future, full dive will ultimately involve bypassing the user’s five senses by sending sights, sounds, smells, tastes, and tactile sensations directly into the brain.
In theory, a full full-dive experience would disable the user’s external sensory awareness, replacing it with virtual awareness for the duration of the HD VR session. Likewise, the user would have no need to move their body, as their thoughts would control their virtual bodies within VR space. There is, admittedly, a creep factor to being so totally immersed that one’s own physical body becomes obsolete. Some might even question whether we really want to go there. But, of course, we will.
Studies such as the Harvard experiment show exciting progress, there must be greater even advances in order for FD to become a reality. The U.S. Defense Advanced Research Project Agency (DARPA) may provide a key component.
DARPA plans to spend $60 million (USD) over the next four years to develop a high-resolution, wide-bandwidth intracranial electrode array for recording and stimulating brain activity. The DARPA Stentrode (stent and electrode) has been successfully tested in sheep and is injected into a blood vessel where it records and stimulates neural activity. Experiments with humans is scheduled for 2017.
The minimally invasive device may be the closest thing we have, yet, to a brain modem. And while the ultimate goal is to enjoy a full dive experience without the need for implants of any sort, the lessons learned by the Stentrode will help us to get there.
One secret to advancing any technology is to find what works and keep doing it. As we discussed, EEG systems were successful in the 2013 rat-tail experiment, and they were successful in the 2015 University of California at Irvine experiment where a paraplegic walked. As promising as the DARPA device may be, the non-invasive EEG holds, perhaps, greater promise for developers wishing to advance full dive technology.
EEG has already moved from the lab to commercial applications. One company, Emotive, has penetrated the market with 5-channel and 14-channel “neuroheadsets” and advanced EEG software. Emotive is developing applications for its products in the fields of “academic research, advertising and media, education and training, mobility, defense, communication, automotive and IoT (Internet of Things) development.”
Clearly, EEG is an excellent foundation upon which to advance full drive technology.
Focused Ultrasound (FUS)
In terms of brain interfaces, focused ultrasound is a means of stimulating targeted regions of the brain by bombarding them with ultrasonic energy of a particular frequency and pressure level. In the case of the rat tail experiment, the motor cortex of the rat’s brain was stimulated with 350 KHz from a transducer placed on the rat’s head.
While FUS is primarily a therapeutic tool used by healthcare professionals for treating certain health conditions, Harvard’s out-of-the-box thinking adapted it for their brain interface. This is exactly the kind of creativity that will lead innovators to push full dive closer to reality.
Full Dive Virtual Reality in Gaming
To see how FD VR is affecting gaming, we need to return to the Sword Art Online anime series. Not only did SAO inspire gamers to crave a full dive virtual reality gaming experience — fatalities notwithstanding, but it started some big wheels turning among developers.
Although SOA creator Reki Kawahara has authorized a series of PlayStation games, IBM decided to take it, and VR gaming, to a whole new level.
In March of 2016, IBM performed an alpha test of a VRMMORPG platform that also gave a nod to gamers’ quest for FD VR games. Using its powerful “Cognitive System” supercomputer, known as Watson, and its SoftLayer cloud-based server platform, IBM brought massive processing power and VR interaction to Sword Art Online, albeit with significantly less realism than the fictional anime version.
The game, titled Sword Art Online: The Beginning, accommodated 208 alpha testers, selected by lottery from the 100,000 players who had applied. It’s not the 10,000 that the fictional SAO let in — and would not let out, but it’s a fine start. And the best news is that all 208 lived to play another day.
Like any self-respecting high-tech anime game test, the experiment was performed in an undisclosed lab located somewhere in Tokyo.
The test restricted multiplayer involvement to four players at a time, but this was probably done to allow easier analysis, rather than because of processing limitations.
VR gear included the full gamut of high-tech devices now available:
- The Oculus Rift headset
- The OVRVision and Leap Motion for hand movement detection
- The MS Kinect 2 for detecting body movement
- Special foot sensors that eliminated the need for controllers
Using a 3D scanner, players’ likenesses were used to create realistic avatars. It was as close as you can get total immersion using current technology.
Players were immersed into the town square of the City of the Beginning, where they could manipulate virtual objects and begin their battle. In addition to the three other participants, players battled Gleam Eyes, a character from the original story.
To date, there are no TRUE full dive games, but the IBM test took us a step closer. It demonstrated that a big player is at least interested in working to develop the processing power that will be needed to power HD VR games.
AppReal: Your Full Dive Virtual Reality Development Partner
Full dive virtual reality development isn’t for the faint of heart. Success requires a solid foundation in the very latest virtual reality technology, and the ability to transform theory into workable solutions. Pioneers in full dive development must go beyond adopting the latest technology; they must be able to do real science.
AppReal is a virtual reality development company specializing in programming, VR production, mobile app development, and custom development of virtual reality solutions. We have the experience and technology needed to bring clarity and direction to your full dive vision. Together, we can transform your ideas into reality. Why not call us today to arrange a free consultation?