Explained: How does VR actually work?

How does virtual reality work? How a

virtual reality headset

does it make you think you're sitting in a spaceship in a galaxy far away when you're actually about to smash into the kitchen counter? Well, with the army of virtual reality devices expanding, we'll explain how they actually work.

Although the devices generally take the same shape, the way they project the image in front of our eyes varies greatly. The tastes of

HTC Vive

and

Oculus Rift

provide PC-based operations, although major players such as Google and Samsung offer more affordable smartphone headsets. Sony has also managed to break into the console scene with its

PlayStation VR

.

Standalone VR is something you'll hear more about too - in 2018 Oculus will launch the

Oculus Go

, and Lenovo's standalone Daydream headset is also expected.

Essential Reading

:

Best VR Games for 2017

Once your headset and power source are secure, some sort of input is also required to connect - whether through head tracking, controllers, hand tracking, voice, buttons on device or touchpads.

Total immersion is the goal of everyone who creates a VR headset, game or application: to make the virtual reality experience so real that we forget about the computer, headset and accessories and act exactly as we do. would do in the real world. So how do you get there?

The basics

VR headsets like Oculus Rift and PlayStation VR are often referred to as HMDs, which simply means they're head-mounted displays. Even without audio or manual tracking, holding GoogleCardboard to place your smartphone screen in front of your face can be enough to half-immerse you in a virtual world.

The goal here is to create what appears to be a life-size 3D virtual environment without the limitations we usually associate with television or computer screens. So no matter how you look, the face-mounted display follows you. This is different from the

augmented reality

, which overlays graphics on your view of the real world.

The future:

Virtual reality versus augmented reality

Video is sent from the console or computer to the headset via an HDMI cable in the case of headsets such as the HTC Vive and Rift. For Google Daydream headset and

Samsung Gear VR

, it is already on the smartphone inserted into the helmet.

VR headsets either use two streams sent to one. or two LCD screens, one per eye. There are also lenses that sit between your eyes and the pixels, which is why the devices are often referred to as glasses. In some cases, these can be adjusted to match the distance between your eyes, which varies from person to person.

These lenses focus and reshape the image for each eye and create a stereoscopic 3D image by tilting the two 2D images to mimic how each of our two eyes sees the world a little differently. Try closing one eye then the other to see individual objects dancing from side to side and you get the idea behind it.

Get your next VR solution

Inside Europe's first IMAX VR Center

Strap on tight for Manchester's new virtual reality site

How to Start Playing Virtual Reality Games on Vive and Rift

VR specialist Catherine Allen on building the future of the industry

Best Oculus Rift Games

Our favorite Rift games, including Touch-enabled titles

Best VR headsets for iPhone: Budget options for Apple smartphone users

Best PS VR Games

Big names and indie titles for Sony's PS4 headset

An important way for VR headsets to increase immersion is to increase the field of view, i.e. the width of the image. A 360 degree display would be too expensive and unnecessary. Most high-end headsets make do with a 100 or 110 degree field of view, which is wide enough to do the trick.

And for the resulting image to be compelling, a minimum frame rate of around 60 frames per second is needed to avoid stuttering or users feeling sick. The current crop of VR headsets go well beyond that - Oculus is capable of 90fps, for example, while Sony's PlayStation VR manages 120fps.

Head tracking

Head tracking means that when you're wearing a VR headset, the image in front of you changes as you look up, down, and side to side or tilt your head. A system called 6DoF (six degrees of freedom) maps your head along your X, Y, and Z axes to measure head movement forward and backward, side-to-side, and shoulder-to-shoulder , also known as pitch, yaw, and roll. p>

Getting started

:

Google Daydream View review

There are various internal components that can be used in a head tracking system, such as a gyroscope, accelerometer, and magnetometer. Sony's PSVR also uses nine LEDs dotted around the headset to provide 360-degree head tracking through an external camera monitoring those signals, while the Oculus has 20 less bright lights.

Head tracking technology needs low latency to be effective - we're talking 50 milliseconds or less or we'll detect the lag between when we turn our head and when the VR environment changes. The Oculus Rift has an incredibly minimized lag of just 30ms. Lag can also be an issue for any motion tracking inputs, such as PS Move-style controllers that measure our hand and arm movements.

Finally, headphones can be used to increase the feeling of immersion. Binaural or 3D audio can be used by app and game developers to leverage the head-tracking technology of VR headsets to take advantage of it and give the user the impression that the sound is coming from behind, to the side or far away.

motion tracking

Head tracking is one of the big advantages of high-end headsets over other Cardboard headsets

mobile VR

. But the big VR players are still working on motion tracking. When looking down with a VR headset, the first thing you want to do is see your hands in virtual space.

For a while we've seen the Leap Motion accessory - which uses an infrared sensor to track hand movements - attached to the front of Oculus dev kits. We also experimented with Kinect 2 cameras to track our moving bodies. But now we have some cool input options from Oculus, Valve, and Sony.

Oculus Touch

is a set of wireless controllers designed to make you feel like you're using your own hands in VR. You grab each controller and use buttons, thumbsticks, and triggers during VR games. So, for example, to fire a gun, you pull the trigger. There's also an array of sensors on each controller to detect gestures like pointing and waving.

One-on-one headset:

HTC Vive vs. Oculus Rift

Other input methods can include anything from connecting an Xbox controller or joystick to your PC, voice commands, smart gloves, and treadmills such as the Virtuix Omni, which allow you to simulate walking in a VR environment with smart in-game redirects.

And when it comes to tracking your physical location in a bedroom, Oculus now delivers an experience to match the HTC Vive, something it didn't at the door. Rift owners now have the option to purchase a third sensor for $79 and add more coverage to their VR play area.

The problem, however, is that it's still no match for HTC. While two SteamVR sensors for the HTC Vive can provide up to 225 square feet of tracked gaming space, two Constellation sensor cameras from Oculus only provide 25 square feet of coverage, with a third camera sending the recommended space up to 64 square feet. This could change with

Oculus Santa Cruz

, the company's premium standalone headset.

Sony is also researching this area, if a recent patent is anything to pass. The filing details a light-and-mirror-based VR tracking system that uses a beam projector to determine player position, though whether such a feature would appear on the current device or the second iteration of PSVR (or not on the everything) is all speculative about this scene.

eye tracking

Eye tracking may be the final piece of the VR puzzle. It is not available on the Rift, Vive or PS VR but it will be in

the very promising VR headset from FOVE

. We have also seen

Tobii's eye tracking technology in action

in an HTC Vive. So how does it work?

Well, an infrared sensor monitors your eyes inside the headset so FOVE knows where your eyes are looking in VR. The main benefit of this - aside from allowing game characters to react more accurately to where you're looking - is to make the depth of field more realistic.

In standard VR headsets, everything is in pin-sharp focus which isn't how we're used to experiencing the world. If our eyes look at an object in the distance, for example, the foreground becomes blurred and vice versa. By following our eyes, FOVE's graphics engine can simulate this in 3D space in VR. It's true, blur can be good.

Headsets still need high resolution screens to avoid the effect of looking through a grid. In addition, what our eyes focus on should resemble reality as closely as possible. Without eye tracking, with full focus as you move your eyes - but not your head - around a scene, simulation sickness is more likely. Your brain knows something is wrong.