This is part one of a 3-part series that will guide you through the world of Virtual Reality (VR). The guide is design to explain what these new immersive VR technologies are, how to experience them, and how to create your own VR experiences.
Part ONE: Understanding the Terminology
These are some of the major terms you will need to be familiar with to understand Virtual Reality and their related technologies.
Virtual Reality (VR):
VR is an interactive computer-generated environment that replaces a user’s real-world environment. The “Virtual” part of “Virtual Reality” comes from the fact that the environment that you are experiencing was created, and thus a virtual version of reality. Creating a VR environment is done by using stereo image displays, generally contained in a headset also known as a Head Mounted Display (HMD). The stereo image is an image that is matched to the angle of the human eye, so one image is slightly different than the other, just like looking through one eye then the other. Ideally stereo headphones, for listening to the audio, are worn or incorporated in the HMD. Sound is just as important as vision in an environment and can make experiences very immersive, just ask anyone that’s watched a surround sound movie in an Imax movie theater. An important note is that the image or video along with the sound change based on where you are looking. If a VR video is playing you can look around the entire environment while wearing a headset, and hear sounds coming from their direction. This creates the illusion of being inside of the computer-generated environment and the reason why these experiences are called “immersive”.
[Image: Oculus Go VR headset. Source: Oculus]
Augmented Reality (AR):
AR is a combination of computer-generated content on top of a real-world layer, thus augmenting your reality (AR) is implemented in one of two ways: Smartphone AR uses the camera of your smartphone to show you the real world on the smartphone screen and then overlays or augments that video with other computer-generated content. For example, you could be pointing your phone’s camera at a box of cereal in a store, and the AR layer would generate a graphic of a coupon over that box to show you that it’s on sale. In the real world there isn’t a coupon floating in there over the box, just on the mobile phone screen. The other option is to use a headset-based AR which is a head mounted display that lets you see the real world thru an optical device (semi reflective surface, transparent LCD, etc.) that also lets you see the overlay information. Think of a wearing a pair of glasses where you see normally, but you look at the cereal and a coupon starts floating above it, but it’s actually an image projected onto the glass surface for your eye to pick up along with what you are looking at.
Mixed Reality (MR):
MR is an interactive computer-generated experience that mixes or blends the user’s real-world environment and digitally-created content together, where both environments can coexist and interact with each other. This differs from Augmented Reality in that the digital content is aware of the real world and interacts with it. In this case the coupon doesn’t float on top of the cereal box, you would actually see the cereal box open and the coupon would fly out. This can only be done because the camera “sees” the cereal box, knows what it is, knows how it opens, and can add animation that actually changes the reality that everyone else sees with a computer-generated reality of that box. In the real world the cereal box hasn’t changed, only you get to experience a dancing singing coupon jumping out of the box. The computer world has added to the real world, and the result is a mixed reality experience. Now there isn’t an exact amount of context awareness that makes one experience AR and another MR. However, if the virtual objects look and act as if they are a part of the real world it can be said to be Mixed Reality. It’s important to note that different people define these things differently, sometimes the difference between the two can be so slight that it could be put in either category, or even both.
[Image: Magic Leap One MR headset. Source: Magic Leap]
3 Degrees of Freedom (3DoF):
That’s a lot of say and is usually used in it’s short form of “three dof (dauf)”. This technology gives the ability of a head mounted display or controller to track your rotational movements along the X (roll), Y (pitch), and Z (yaw) axes. In less technical terms the headset knows how you are moving your head and where you are looking so it can adjust for your head movements via 3 inputs.
6 Degrees of Freedom (6DoF):
This uses the same naming convention as 3DoF and is said as “six dof”. 6Dof is the ability of a head mounted display or controller to track both your rotational and translational movements in space. So, in addition to the 3 degrees of freedom tracked for rotational movements, 6DoF also measures translation across the X (forward, backwards), Y (left, right), and Z (up, down) axes. This technology allows you to look at a picture of the box of cereal, move your head to the right and peek, and you can actually see what’s around the back of the box…just like in the real world. However, the photo hasn’t changed, you’ve been able to see a different part of it.
[Image: 6 Degrees of Freedom. Source: Wikipedia]
360 Video:
A video recording where a full spherical view is recorded of everything going on around the camera. 360 videos are shot using an omnidirectional camera with multiple lenses or a collection of standard cameras. The video will allow users to see everything a real-world user would see and look around while the video is playing. The user looks up, down, left, right, front, back, and all the other variations. If the you title your head to the side in a regular 360 video you won’t be able to see around objects, like you would with 6DoF. Instead you would tilt the video with you.
[Image: Vuze XR in 360 Video mode]
Virtual Reality Video (VR Video):
VR Video is usually defined as a 3D stereoscopic video in either 360 degrees or 180 degrees. This definition is talking only about 360 degree VR video. A 360 video in 2D is a recording of two image spheres from two lenses, a front and back. A 360 3D video provides two images for each eye that let you see the 360 videos in 3D. This is done in one of two ways. Either by having multiple pairs of cameras facing in the same direction with left eye cameras used to make one image sphere and right eye cameras making the other, or by using photogrammetry (the science of making distance measurements from multiple photographs) techniques to generate a depth map that is used to warp the regular 360 info into two 360s one for each eye. The end result is that your brain “feels” depth when looking at a 3D 360 video and you can look around in all directions, reliving the moment.
[Image: Vuze+ VR Video Camera]
VR180:
This format of VR video is 180 degrees. It is a video recording using two cameras facing in the same direction that each record a 180-degree half sphere image, one for each eye. When viewed in a head mounted display the user sees a 3D scene they can look around in like a VR Video but limited to 180 degrees, much like what you would experience standing with your back to a black wall. This makes file sizes smaller than a 360 VR video and allows the video creator the ability to frame shots while keeping unwanted objects out of the shot, namely themselves.
[Image: Vuze XR in VR180 Mode]
Now that we have covered the basic terms it is my sad duty to inform you that these terms are confused and misused by the media and major brands all the time. Smart glasses like Google Glass are referred to as AR glasses all the time, but they don’t fit the standard definition of AR since there is no augmentation of the reality that you are seeing. They simply provide an information layer which could be done by walking around with a clear tablet or other smart glasses like device. Smart Glasses primarily function as a head-mounted ‘heads-up-display’ and do not augment reality anymore than looking at a screen does. Microsoft Mixed Reality headsets are not Mixed Reality at all but are 6DoF Virtual Reality Headsets. Microsoft justifies this by saying that everything from reality to the virtual is on “The Mixed Reality Spectrum”. Not exactly helpful in a new industry where there is already enough confusion.
This wraps up the introduction to VR and related technologies. Part TWO will cover how to view VR content, from watching 360 videos on a website thru properly experiencing VR content with an HMD.