This in-depth Tutorial covers What Is Virtual Reality And How Does It Work? You will learn about the History, Applications & Technology behind Virtual Reality:
This virtual reality tutorial looks into the introduction of virtual reality, including what it is, how it works, and its major applications.
We will learn about VR hardware and software that enable virtual reality as technology then we will delve deeper into the details of virtual reality headsets and how they function.
What You Will Learn:
Virtual Reality Tutorial
Let us take an example to start understanding the basics.
The below image is a demo setup with a virtual reality head-mounted display steering wheel. The user feels immersed in a car, driving.
[image source]
Virtual reality is a technology that attempts to regenerate computer images and videos to produce real-life visual experiences that are beyond those achieved on the ordinary computer monitor and phone. VR systems do so by using computer vision and advanced graphics to generate 3D images and video by adding depth, and by reconstructing the scale and distances between static 2D images.
The user must be able to explore and control these 3D environments using VR headsets lens and controllers which might have sensors on them for users to be able to experience the VR content.
For instance, click here for the video that allows you to experience Abu Dhabi in 3D while wearing a VR cardboard headset or directly on your P.C. monitor without a VR headset.
Simply click on the video and put your phone inside your VR headset. If you are not using headsets, simply look for the arrows <> inside the video to browse the video in 3D. You can look anywhere around you as you use the headset or the arrows to browse the video in 3D.
This is an example of a video taken with VR cameras or 3D cameras. However, modern VR is more advanced than 3D, allowing the user to immerse their five senses into their VR experiences. It also dwells on real-time tracking to enable using VR in real-time explorations.
The below example is of a user using VR glasses or a headset. What she actually sees is shown on the right-hand side.
[image source]
(i) In effect, virtual reality is all about using a device like a special 3D video or image camera to create a three-dimensional world that a user can manipulate and explore later or in real-time using VR headsets and lenses, while feeling he or she is in that simulated world. The user will see a life-size image and the resulting perception is that they are part of that simulation.
Here is a video reference: Virtual Reality Demo
(ii) VR hardware and software will help generate or create computer-generated 3D images and video and this output is cast to a lens mounted on goggles or headset. The headset is strapped onto the user’s head over the eyes, such that the user is visually immersed in the content they are viewing.
(iii) The person viewing the content can use gaze for the gesture to select and browse through the 3D content or can use hand controllers such as gloves. The controllers and gaze control will help track the movement of the user’s body and to place the simulated images and videos in the display appropriately such that there will be a change in perception.
By moving your head to look left, right, up, and down, you can replicate these motions inside of VR because the headset has head motion or tracking sensors by either tracking the eye or head. Sensors on controllers can also be used to collect stimuli response information from the body and send it back to the VR system to improve the immersion experience.
The below image is an example to understand the sense of touch and feel in VR: A user using VR gloves and a hand avatar to browse through and interact with VR content. The glove transmits the motion from the hand to the VR computing or processing unit or system and reflects action on the display. The VR will also transmit the stimulus back to the user.
[image source]
(iv) Hence, it has two important things; computer vision to help understand objects and position tracking to help track user movement to place the objects effectively on the display and to change the perception so that the user can “see the world”.
(v) It also comprises other optional devices such as audio headphones, cameras, and sensors to track user movements and feed it to a computer or phone, and wired or wireless connections. These are used to improve user experience.
Virtual reality has diverse applications. While most of the applications dwell on gaming, it is also finding its usage in medicine, engineering, manufacturing, design, education and training, and many other fields.
VR Training in Medicine:
[image source]
Introduction To Computer Graphics And Human Perception
The below image explains the general organization of human perception:
[image source]
(i) It is possible to avoid side effects on human perception while deriving maximum benefits from VR perception. This is possible with an in-depth and complete understanding of human body physiology and optical illusions.
(ii) Our human body perceives the world through body senses that respond differently to different stimuli. Mimicking human perception in virtual reality requires knowledge of how to fool the senses to know what are the most important stimuli and what is acceptable quality for subjective viewing.
Human vision provides the most information to the brain. It is then followed by hearing, touch, and other senses. The proper functioning of a VR system requires one to know how to synchronize all stimuli.
The below image explains that the light sensors are employed to sense the light reflected from the eye and once the light is absorbed by the pupil, the pupil’s position affects the light reflected back by the eye and sensed by the photodiode.
(iii) Virtual reality simply tries to simulate human perception (the brain’s interpretation of the senses) in the real world. The 3D VR environments are not only designed to look like the real-world but also one that gives the experience of it. In fact, VR is considered immersive when the simulated and the real world are as much similar as possible.
(iv) Although to some extent, the simulation may be wrong such that the experiences are enjoyable, the brain may not be tricked this way. In other cases, it means the simulation is so wrong to the extent that the user experiences cyber-sickness whereas VR tricks the brain into feelings of motion-sickness.
Motion sickness is the queasy feeling that some people get in a car, plane, or boat. It happens when the simulated and real world are different and the perception is, therefore, confusing to the brain.
What Is Virtual Reality & The Technology Behind It
Here is a video for your reference:
Virtual reality is a technology that simulates vision to end up with a 3D environment in which a user appears to be immersed while browsing through it or experiencing it. The 3D environment is then controlled in all 3D by the user who is experiencing it. On the one side, the user is creating 3D VR environments and on the other end, he is experiencing or exploring them with appropriate devices such as VR headsets.
Some devices like controllers allow the user to control and explore the content.
Creating the content starts with an understanding of computer vision, the technology that enables phones and computers to process images and videos so that they can understand them the way a human visual system does.
For instance, devices using this technology will interpret images and videos using image location, surroundings, and appearance. This means using devices such as a camera but also along with other technologies like artificial intelligence, big data, and a vision processing unit.
Artificial Intelligence and Machine Learning may rely on pre-processed image and video data (large amounts of data or big data) to identify objects in the environment. The camera will use blob detection, scale space, template matching, and edge detection or a combination of all these to make this possible.
Without going into details, for instance, edge detection generates an image by detecting points where brightness will drastically drop or stop altogether. Other methods use other techniques to identify an image.
(i) Virtual reality headsets attempt to help a user enjoy an immersive 3D environment by putting a screen in front of the user’s eyes to eliminate their connection with the real world.
(ii) An autofocus lens is placed between each eye and the screen. The lenses are adjusted based on the movement and positioning of the eyes. This allows tracking of the user movement vis-a-vis the display.
(iii) On the other end is a device such as a computer or mobile device that generates and renders the visuals to the eye through the lenses on the headset.
(iv) The computer is connected to the headset via an HDMI cable to deliver visuals to the eye through the lenses. When using a dedicated mobile device to deliver the visuals, the phone may be mounted directly on the headset such that the lenses of the headset simply lie over the mobile device’s display to magnify the images or sense the movement of the eyes with respect to the mobile device’s image and to finally create the visuals.
The below image is of a user using a high-end HTC VR headset tethered to the PC via an HDMI cable. We have untethered, tethered, and even wireless options.
[image source]
High-end VR devices like the one in the image above are expensive. They give high-quality immersive experiences because they use lenses and computers and advanced visual methodologies.
Click here for a video for a detailed look into HTC Vive high-end VR headset.
For low-end and cheaper Google and other cardboard VR headsets, they use a mobile device. The phone is usually removable from the headset mount. Low-end VR headsets called cardboards are much cheaper because they just have a lens and require no advanced material in the making.
The below image is of a Cardboard VR headset. A user inserts their phone inside the cardboard headset to lock out their eye from the rest of the world, clicks on a VR application that hosts virtual reality content, and they can enjoy VR at a cost of below $20.
[image source]
Google Cardboard VR headset with a controller:
[image source]
(v) For the mid-range headsets such as Samsung Gear VR, the headset is designed such that it has a computer device size of the phone integrated with a lens and which won’t come out. These are portable and mobile and provide the best freedom for using VR content. A user will simply buy the headset, connect to the internet, browse through VR content such as games or downloads, and then explore it in VR.
Samsung Gear VR:
[image source]
(vi) Each virtual reality headset and visual generation event in each virtual reality system tries to improve the quality of the visuals by playing around with a number of factors among them.
These factors are enlisted below:
#1) Field Of View (FOV) or the viewable area, is the extent to which the display will support the movement of the eye and head. It is the degree to which the device will contain the virtual world in front of your eyes. Naturally, a person is able to see about 200°-220° around them without moving the head. It would result in the feeling of nausea if the FOV results in a misrepresentation of information to the brain.
Binocular FOV and Monocular FOV:
[image source]
#2) The frame rate or the rate at which the GPU can process the visual images per second.
#3) The screen refresh rate which is the pace of displaying the visual images.
(vii) A FOV of at least 100, a frame rate of at least 60fps, and a competitive refresh rate are required on the minimum end to give the least of VR experiences.
(viii) Latency is a very important aspect related to the refreshing rate. For the brain to accept that the visual image generated on the screen is related to the head movement, the latency must be low to deliver the visual almost immediately. For Example, a lag of 7- 15 milliseconds is considered to be the ideal.
Who Can Use VR?
It depends on the needs. One can use it for entertainment like playing VR games, for training, attending virtual company or hangout meetings and events, etc. For a consumer of VR content, the first thing that you should think about is what type of virtual reality headset to buy.
Will it work with a phone, P.C., or what else? Content can be accessed online on media platforms hosting VR content or should be downloaded for offline usage?
Click here for a detailed guide on buying a virtual reality headset.
If you are a company, group, or institution intending to take advantage of the immersive benefits of virtual reality in your advertising campaign, training, or other applications, there may be more factors to consider including developing your own VR app and content.
In this case, you want to come up with good VR content that influences your viewers and that they can watch using as many VR headsets as there can possibly be. You might want just a sponsored and branded immersive VR video and post it online on YouTube and other places.
You may also develop a dedicated VR app for your company – possibly that works on Android and many other VR mobile and P.C. and non-P.C. platforms – that will host lots of your VR content and ads, which customers can find out and watch. You may also come up with a branded VR headset alongside your branded VR content.
If you are a developer willing to develop for VR, you might look into buying headsets that support SDK and other development tools. Then get a good grasp of the standards and what platforms are used to develop for VR.
History Of Virtual Reality
| Year | Development |
|---|---|
| 19th Century | 360 degree Panoramic Paintings: filled the viewer’s field of vision creating immersive experiences. |
| 1838 | Stereoscopic photos and viewers: Charles Wheatstone showed viewing 2D images side by side with stereoscope added depth and immersion. Brain combines them into 3D. Found application in virtual tourism |
| 1930s | The idea of Google-based VR world using holographics, smell, taste and touch; through Stanley G. Weinbaum’s short story titled Pymalion’s Spectables |
| 1960s | First VR head-mounted display by Ivann Sutherland. It had specialized software and motion control and was used for training as a standard. The Sensorama by Morton Heilig was used to immerse the user in a riding bike experience on the streets of Brooklyn. The single-user entertainment console produced stereoscopic display, stereo sound, odor via odor emitters, had fans, and a vibrating chair. |
| 1987 | Jaron Lanier coined the word virtual reality. He was the founder of Visual Programming Lab (VPL). |
| 1993 | Sega VR headset announced at the Consumer Electronics Show. Meant for Sega Genesis console, it had an LCD screen, head tracking, and stereo sound. 4 games developed for it but never went beyond prototype. |
| 1995 | The first-ever portable console with true 3D graphics for gaming, the Nintendo Virtual Boy (VR-32). Lacked software support and uncomfortable to use. VR debuted in the public arena. |
| 1999 | Wachowiski siblings’ film The Matrix had characters living in simulated world depicting VR. VR entered mainstream as a result of the movie’s cultural impact. |
| 21st Century | Boom of HD display and 3D graphics-capable smartphones make it possible for lightweight, practical and accessible VR. Consumer VR in the video game industry. Depth sensing cameras, motion controllers, and natural human interfaces enabled better human-computer interactions. |
| 2014 | Facebook bought Oculus VR, developed VR chat rooms. |
| 2017 | Multiple VR devices in commercial and non-commercial applications High-end P.C.-tethered headsets, smartphone VR, cardboards, WebVR, etc. |
| 2019 | Wireless high-end headsets |
VR appears to be developed hand-in-hand with Augmented Reality technology.
Development of AR technology.
[image source]
Application Of Virtual Reality
| Application | Explanation/description | |
|---|---|---|
| 1 | Gaming | It was and still is the most traditional application of VR. Used to play immersion games. |
| 2 | Workplace collaboration | Employees can collaborate on assignments remotely with the feeling of presence. Beneficial for demo tasks where visuals are critical to understanding and completion of tasks. |
| 3 | Pain management | VR visuals help distract patient's brains to confuse pain pathways and from suffering. For soothing patients. |
| 4 | Training and learning | VR is good for demo and demonstration for instance demo of surgical procedures. Training without exposing the lives of patients or trainees to danger. |
| 5 | Treatment of PTSD | Post-experience trauma is a common disorder among combat soldiers and also other people who undergo petrifying experiences. Using VR to re-liven experiences can help medical experts understand patients’ conditions and device ways of solving the problems. |
| 6 | Autism management | VR helps boost patients’ brain activity and imaging to help them deal with autism, a condition that impairs reasoning, interaction, and social skills. VR is used to introduce patients and their parents to different social scenarios and training them on how to respond. |
| 7 | Managing and treating social disorders | VR is applied in the monitoring of anxiety symptoms such as breathing patterns. Doctors can give anxiety medication based on those outcomes. |
| 8 | Therapy for paraplegics | VR is used to provide paraplegics to experiences the thrills of different environments outside their confinements, without having them travel to experience the thrills. For instance, it has been applied to help paraplegics regain control of their limbs. |
| 9 | Leisure | VR is widely applied in tours and tourism industry such as virtual exploration of travel destinations to help travelers make choices before doing the actual visits. |
| 10 | Brainstorming, forecasting, | Businesses can test new creative ideas before launching them, discuss them with partners and collaborators. VR can be used to experience and test new designs and models.VR is very useful in testing car models and designs, with all car manufacturers having these systems. |
| 11 | Military training | VR helps simulate different situations for training soldiers on how to respond in different situations. Training without putting them in danger while saving costs. |
| 12 | Advertising | VR immersive ads are very effective in and as part of an overall marketing campaign. |
Virtual Reality And Gaming
Click here for The Survios Virtual Reality Game Demo
Gaming is probably the oldest and most mature application of virtual reality. For instance, the revenue and its future forecast for VR gaming have been rising, expected to rise to beyond $45 billion in 2025. Even VR gaming is hard to differentiate from some medical and training VR applications.
Click here to see the Iron Man VR demo
The below image shows that the user explores scenes in the Half-Life Alyx VR game:
[image source]
Virtual Reality Hardware And Software
Virtual Reality Hardware
Organization of VR technology:
[image source]
VR hardware is used to produce stimuli to manipulate the VR user’s sensors. These can be worn on the body or used separately away from the user.
VR hardware uses sensors to track motions, for example, the user’s button presses, and controller movements such as hands, head, and eyes. The sensor contains receptors to collect mechanical energy from the user’s body.
The sensors in the hardware convert energy it receives from a hand movement or button press to an electrical signal. The signal is fed into a computer or device for action.
VR Devices
- These are the hardware products that facilitate VR technology. They include a personal computer, which is used to process inputs and outputs from and to users, consoles, and smartphones.
- Input devices include VR controllers, balls or tracking balls, controller wands, data gloves, trackpads, on-device control buttons, motion trackers, bodysuits, treadmills, and motion platforms (virtual Omni) that employ pressure or touch to produce energy that is converted to a signal to make selection possible from user to 3D environment. These help users navigate the 3D worlds.
- The computer must be able to render high-quality graphics and usually employs Graphics Processing Units for the best quality and experience. The Graphics Processing Unit is an electronic unit on a card that takes data from the CPU and manipulates and alters memory in order to accelerate the creation of images in a frame buffer and to the display.
- Output devices include visual and auditory or haptic displays which stimulate a sense organ and present the VR content or environment to the users to generate a feeling.
Virtual Reality Headsets
Comparison of different VR headsets, types, cost, kind of position tracking, and controllers used:
[image source]
A virtual reality headset is a head-mounted device used to provide virtual reality visuals to the eye. A VR headset comprises a visual display or screen, lenses, stereo sound, head, or eye motion tracking sensors or cameras for the same reason. It also sometimes comprises integrated or connected controllers that are used to browse through the VR content.
(i) The sensors used for sensing eye or head motion and tracking may include gyroscopes, structured light systems, magnetometers, and accelerometers. Sensors may be used to reduce the rendering load in addition to ad delivery for advertising. For instance, in reducing the load, the sensor is used to track the position where a user is gazing and to then reduce rendering resolution away from the user’s gaze.
(ii) Image clarity is determined by camera quality but by also the display resolution, optic quality, refresh rate, and field of view. The camera is used to also track motion for instance for room-scale VR experiences where the user moves around in a room while exploring virtual reality. However, sensors are more effective for this because cameras usually give a larger lag.
(iii) With P.C. – tethered VR headsets where the ability to roam free in space as you explore VR environments is a major concern. Inside-out and outside-in tracking are two terms used in VR. Both cases refer to how the VR system will track the position of the user and accompanying devices as they roam in a room.
Inside-out tracking systems such as Microsoft HoloLens use a camera placed on the headset to track the position of the user with respect to that of the environment. Outside-in systems such as HTC Vive use sensors or cameras placed in the room environment to determine the position of the headset in relation to the environment.
(iv) Usually, VR headsets are divided into low-end, mid-range, and high-end virtual reality headsets. Low-end includes the cardboards used with mobile devices. Mid-range includes the likes of Samsung mobile VR Gear VR with a dedicated mobile computer device and PlayStation VR; while high-end devices include the likes of P.C.-tethered and wireless headsets like HTC Vive, Valve, and Oculus Rift.
Recommended Reading ==> Top Virtual Reality Headsets
VR Software
- Manages the VR input/output devices, analyzes the incoming data and generates proper feedback. The inputs to the VR software must be on time and the output response from it should be prompt.
- A VR developer can build his/her own Virtual World Generator (VWG) using a software development kit from a VR headset vendor. An SDK provides basic drivers as an interface to access tracking data and call graphic rendering libraries. VWG can be ready-made for particular VR experiences.
- VR software relays the VR content from the cloud and other places via the Internet and helps to manage the content.
Virtual Reality Audio
Some headsets incorporate their own integrated audio headsets. Others provide the option of using headphones as add-ons. In virtual reality audio, a 3D illusion to the ear is achieved by using positional, multi-speaker audio – usually called positional audio. This gives a user some clues to get their attention, or even provides the user some information.
This technology is also now common in home theater surround sound systems.
Recommended reading =>> Top Home Theaters in India
Conclusion
This in-depth virtual reality tutorial introduces the idea of Virtual Reality, commonly known in short as VR. We dived deeper into how it works, including the details of producing 3D visuals inside computer and phone environments. These computer processing methods include the latest ones such as AI, which, in VR, processes graphics and images based on a trained machine memory based on big data.
We also learned how the headset lenses work together with the eye using light that comes to and from the eye to produce these virtual graphical illusions.
In this virtual reality tutorial, we have also considered the factors that influence the quality of experiences of VR by the user, and how they can be improved. We then delved into the applications of VR, among them gaming and training.
Finally, this virtual reality tutorial looked at the components of a virtual reality system, including the headset and all of its components, the GPU, and other auxiliary devices.