Virtual Reality is quickly becoming an actual reality. The new head mounted gear, such as the Oculus Rift and Morpheus, all promise a level of immersion the likes we have never witnessed before. Yet, there is something missing. Even with all the fanciest headsets and improved graphics, you are still bound to normal reality by the simple fact that you still have to manipulate a controller or keyboard to actually do anything inside VR.
Next Gen GTA V Will Have First Person Mode
That is where NimbleVR comes in. The five person San Francisco start-up wants to bring your physical hands into Virtual Reality. They are well on their way to doing this with the prototype of their Nimble Sense, a hands-free VR add-on.
I got a chance to sit down with NimbleVR co-founder Robert Wang while he was here in New York and get a hands-on (or should I say 'hands-in') demo of the Nimble Sense.
Apple Watch Prices Revealed: From $500 To $5000
Wang is staring down a very long day of one-on-one demoing. Now, at 10:30 in the morning, he appears chipper and excited. "Virtual reality is really great and it makes people feel like they're in a different world," he starts, "but it would be really nice to actually reach out and interact with that world as opposed to a controller, keyboard or mouse. So we're developing a technology that can track your hands and fingers just using a camera."
"[The Nimble Sense is] actually a depth camera. Regular cameras see color, this camera sees distance. It's actually very similar to something like the Kinect." He slips the Oculus Rift DK 2 on to begin the demonstration. The device will work for all head mounted sets, like the Morpheus, but the particular demos today were made for the Rift in particular. They almost all feature object manipulation and are suited towards exploration games and I had visions of Oculus Rift optimized The Room-style exploration/horror games.
"I'll show you things that may seem a little obvious about stuff you want to do in virtual reality, but there's not really a good way to do that right now." Wang loads up the first of several demo levels. It's a colorful, bright and simple scene with a few dice and a cup sitting on a table. "When I have this on, I can look around in 3D and see my hands and I can grab something by closing my hands around it basically."
After a little hesitation (the hardware is still in a prototype phase and is thus very sensitive with regards to distance) Wang picks a die up and tosses it. It doesn't roll, but we're not interested in the physics of the program; the mere fact that he just picked up a digital object with his real hand was fantastic enough.
"You can grab something and you can look at it and you can rotate it in front of your face to get a better look at it," he adds, doing just that.
Finally, my turn. I slid the headpiece over my face. Wang fired up the 'movie theater' application, which places the user inside a digital cinema, complete with screen and chairs. "So you're watching a movie and let's say you want to pause the movie," he says. "Right now, you have to take the Rift off and hit the space bar, and then put the rift back on or do something else. So this is one of those things it's kind of nice to just be able to reach up with your hand."
Hitting the pause button is tricky at first. The Nimble Sense successfully recreates the distance between your head and your hands, so you at least have that innate sense of proportions and distance to rely on. But if you're unfamiliar with VR headsets in general, and with the Nimble Sense in particular, knowing how far out to stretch in order to hit the buttons is a bit hard to grasp at first. After a little while though, I'm starting and stopping the pleasant sunset movie with a flick of the wrist.
"One thing we can do without any of our tracking is enable social experiences," Wang explains, "one of the things Facebook is interested in is just being able to bring people's bodies - bring people's everything - into virtual reality so you can have a more direct social experience. Originally in the theater demo we were going to tell a story about a long distance relationship; like you're watching a movie with your long distance girlfriend, you can play with the interface and you can also see her." Admit, you know immediately where your first thoughts went when you heard that.
However, the first thing we think of when confronted with the notion of Virtual Reality is First Person Shooters. Getting over the burden of weapons without using a controller was tricky, but NimbleVR, in conjunction with several dev studios, have worked around that in their demo. The answer? Magic.
The next stage put me in a graphically rich environment in which the act of opening and closing my fingers produces a lightning bolt effect, "like Emperor Palpatine," Wang says. I will admit, it was ridiculously fun zapping the poor shmucks who got in my way. Moving would be an issue, but a simple rail shooter mechanic could make it work for the time being.
The mind is a tricky thing. Reaching out and touching things in the Rift is peculiar because while my brain processes the images, the lack of feeling is disconcerting. There is no weight to anything in the world. "Optical tracking of the hands is step one." Wang says, when I asked about ever utilizing pressure sensors or some type of feedback accessory. "We'd love to work with a sensor-fusion approaches, where you have different sensors or devices that provide more feedback. Maybe different sensors that make this even faster, lower latency and sensors that could even increase the accuracy of this thing."
The current model refreshes every 20 milliseconds and has a viewing angle of about 110 degrees, or roughly the distance in front of your face. Its applicable area for optimal hand-scanning falls at about 90 degrees, making stretching or wide hand movements difficult-to-impossible to render.
The Nimble Sense is not a perfect piece of technology. My experience with the dice demo was a bit frustrating, due to the lack of pressure and fine-tuning while picking them up (it felt like you have to almost close your hand and it required the use of one's index finger). But the Kinect or Wii U controls are not perfect, either. Incredibly fast shaking won't be rendered, so finesse movements, such as playing a digital piano for instance, are a long way off. But the Nimble Sense creates an unprecedented level of immersion, and any sensitivity tweaks can be fixed by the time VR comes to market.
"Virtual reality is still trying to find its niche. Maybe it's these exploration games, maybe it's social, maybe it's something else, like watching movies in VR. This is early days, nobody knows right now. People are trying to figure it out. For us, everything's kind of a moving target. We don't know exactly what experience we should be building for."
He adds, "But we do think that very fundamentally, we want to be able to bring your hands and even maybe parts of the rest of your body into VR with you. That's gonna change the experience." he adds.
The Nimble Sense Kickstarter campaign was launched about a week ago and has already exceeded its goals. You can still donate to the fund so the company can reach their stretch goals.
I had an excellent time with the Nimble Sense. The games were rudimentary, but there was a sense of immersion present I had never experienced in gaming before. Our hands are our primary tool in video games; by removing the need for the controller, you can open up a whole new (virtual) reality of gaming. I look forward to seeing what the full release version of the Nimble Sense can accomplish.