Explained: How does virtual reality work?
How does virtual reality work? How does a
virtual reality headset
Make you think you're sitting in a spaceship in a galaxy far, far away when you're actually about to bump into the kitchen counter? Well, with the expanding army of reality devices virtual, we will explain how they actually work.
Although the devices generally take the same form, the way they project imagery before our eyes varies widely. The tastes of the
HTC Vive
and
Oculus Rift
provide PC-based operations, although major players such as Google and Samsung offer more affordable smartphone headsets.Sony has also managed to break into the console scene with its
PlayStation VR
.
Standalone VR is something you'll hear more about - in 2018 Oculus will launch the
Oculus Go
, and Lenovo's standalone Daydream headset is also expected.
Essential Reading
:
Best VR Games for 2017
Once your headset and power source are secure, some sort of input is also required to connect - be it via head tracking, controllers, hand tracking, voice, buttons of device or touchpads.
Total immersion is the goal of everyone who makes a VR headset, game or application - to make the virtual reality experience so real that we forget about the computer, headset and accessories and act exactly as we do. would do in the real world. So how do we get there?
The basics
VR headsets like Oculus Rift and PlayStation VR are often referred to as HMDs, which simply means they're head-mounted displays. Even without audio or manual tracking, hold Google Cardboard to place your smartphone screen in front of your face can be enough to half immerse you in a virtual world.
The goal here is to create what appears to be a life-size 3D virtual environment without the limitations we usually associate with TV or computer screens, so whatever you look like, the screen mounted on your face follows you. It's different
augmented reality
, which overlays graphics on your view of the real world.
The future:
Virtual reality versus augmented reality
Video is sent from the console or computer to the headset via an HDMI cable in the case of headsets such as the HTC Vive and Rift. For the Google Daydream headset and the
Samsung Gear VR
, it is already on the smartphone inserted into the helmet.
VR headsets use either two streams sent to one screen or two LCD screens, one per eye. There are also lenses that are placed between your eyes and the pixels, which is why the devices are often referred to as goggles. In some cases , these can be adjusted to match the distance between your eyes, varying from person to person.
These lenses focus and reshape the image for each eye and create a stereoscopic 3D image by tilting the two 2D images to mimic how each of our two eyes sees the world very slightly differently. Try closing one eye then the another to see individual objects dancing from side to side and you get the idea behind it.
Get your next VR fix
Inside Europe's first IMAX VR Center
Strap in tight for Manchester's new VR venue
How to Start Playing VR Games on Vive and Rift
Virtual reality specialist Catherine Allen on building the future of the industry
Best Oculus Rift Games
Our favorite Rift games, including Touch-enabled titles
Best iPhone VR headsets: Budget options for Apple smartphone users
The best options for iOS VR
The best PS VR games
Big names and indie titles for Sony's PS4 headset
An important way for VR headsets to increase immersion is to increase the field of view, i.e. the width of the image. A 360 degree display would be too expensive and unnecessary. High-end helmets make do with a 100 or 110 degree field of view, which is wide enough to do the trick.
And for the resulting image to be compelling, a minimum frame rate of around 60 frames per second is needed to avoid stuttering or users feeling sick. The current crop of VR headsets goes way beyond that. - Oculus is capable of 90fps, for example, while Sony's PlayStation VR manages 120fps.
Head tracking
Head tracking means that when you're wearing a VR headset, the image in front of you changes as you look up, down, and side to side or tilt your head. A system called 6DoF (six degrees of freedom) plots your head along your X, Y, and Z axes to measure forward and backward, side-to-side, and shoulder-to-shoulder head movements, also known as pitch, yaw and roll.
Practice
:
Google Daydream View review
There are a few different internal components that can be used in a head tracking system, such as a gyroscope, accelerometer, and magnetometer. Sony's PSVR also uses nine LEDs dotted around the headset to provide head tracking. 360 degrees through an external camera monitoring these signals, while the Oculus has 20 less bright lights.
Head tracking technology needs low latency to be effective - we're talking 50 milliseconds or less or we'll detect the lag between when we turn our head and when the VR environment changes. Oculus Rift has incredibly minimized lag of just 30ms. Lag can also be an issue for all motion tracking inputs, such as PS Move-style controllers that measure our hand and arm movements.
Finally, headphones can be used to increase the sense of immersion. Binaural or 3D audio can be used by app and game developers to leverage the head-tracking technology of VR headsets to take advantage of it and give the user the impression that the sound is coming from behind, next to them or far away.
motion tracking
Head tracking is one of the big advantages of high-end headsets over other types of cardboard.
mobile VR headsets
But the big players in VR are still working on motion tracking. When looking down with a VR headset, the first thing you want to do is see your hands in virtual space.
For a while we've seen the Leap Motion accessory - which uses an infrared sensor to track hand movements - attached to the front of Oculus dev kits. We've also been experimenting with Kinect 2 cameras to keep up with our fidgeting bodies. But now we have some interesting input options from Oculus, Valve and Sony.
Oculus Touch
is a set of wireless controllers designed to give you the feeling of using your own hands in virtual reality. You grab each controller and use buttons, joysticks and triggers during VR games. So, for example, to shoot with a gun, you pull the trigger. There's also an array of sensors on each controller to detect gestures like pointing and waving.
Head-to-head head-to-head:
HTC Vive vs. Oculus Rift
It's a setup quite similar to Valve's Lighthouse position tracking system and HTC's controllers for its Vive headset. It involves two base stations around the room that scan the area with lasers. These can detect the precise position of your head and both hands based on when they hit each photocell sensor on the headset and around each wearable controller. Like Oculus Touch, these also feature physical buttons and amazingly, you can have two Lighthouse systems in the same space to track multiple users.
Other input methods can include anything from connecting an Xbox controller or joystick to your PC, voice commands, smart gloves, and treadmills like the Virtuix Omni, which let you Simulate walking in a VR environment with smart in-game redirects.
And when it comes to tracking your physical location in a room, Oculus now delivers an experience on par with the HTC Vive, which it didn't at the Doorstep. Rift owners now have the ability to buy a third sensor for $79 and add more coverage to their VR play area.
The problem, however, is that it's still not on par with HTC. While two SteamVR sensors for the HTC Vive can offer up to 225 square feet of tracking gaming space, two Constellation sensor cameras from Oculus n only offer 25 square feet of coverage, with a third camera sending the recommended space up to 64 square feet. This may change with
Oculus Santa Cruz
, the company's premium standalone headset.
Sony is also hunting in this area, if a recent patent is anything to go by. The filing details a light-and-mirror-based VR tracking system that uses a beam projector to determine player position, though if such functionality would appear on the current device or the second iteration of PSVR (or not at all) is all speculation about this to arrange.
eye tracking
Eye tracking may be the final piece of the VR puzzle. It's not available on the Rift, Vive, or PS VR but it will be in
The very promising VR headset from FOVE
.We have also seen
Tobii's eye tracking technology in action
in an HTC Vive. So how does it work?
Well, an infrared sensor monitor puts your eyes inside the headset, so FOVE knows where your eyes are looking in VR. where you look - is to make the depth of field more realistic.
In standard VR headsets, everything is perfectly sharp, which is not how we are used to experiencing the world. If our eyes look at an object in the distance, for example, the foreground becomes blurred and vice versa. following our eyes, FOVE's graphics engine can simulate this in 3D space in VR. That's right, blur can be good.
Headsets still need high-resolution screens to avoid the effect of looking through a grid. Also, what our eyes focus on should look as close to the real thing as possible. Without eye tracking, with full focus when you move your eyes - but not your head - around a scene, simulation sickness is more likely. Your brain knows something isn't right.