Have you ever been so engrossed in a book or a movie that hours seem to fly by? You were so engaged with what you were reading or seeing that you forgot about yourself and the physical world you live in. You were fully immersed in a non-physical, imaginary world. Great works of creativity, whether books, movies, paintings, or other forms of art, are capable of moving people away from their own selves into the world that the artist has created.
VR, too, relies on immersion, but not quite the same type of immersion as you’re used to. VR is at its best when it transports the user to a non-physical world. The difference with the immersion you feel when watching a movie is that VR has you believe it’s really you who’s in this new world. You don’t forget about yourself. You’re still in control, but you’re somewhere else now. This feeling is called presence.
This article will talk about the mix of technologies that help VR games and apps achieve that sense of presence. It will discuss stereoscopic displays, field of view, frame rate and refresh rate, and foveated rendering. These are all technologies that use your sense of sight to achieve presence. Of course, that’s only part of the equation. Excellent audio and even haptics are incredibly important to increase your sense of presence in VR too. But sight is the one we’ll focus on in this article.
Stereoscopic Displays
Let’s first acknowledge that VR doesn’t need photorealism to achieve a sense of presence. But it does need tailored content that’s different from the content that we’re usually served on our mobile and laptop devices. That’s because a stereoscopic display emulates how human vision works.
Our two eyes are a few inches apart from one another. This small separation means that each eye sees the world from a slightly different perspective. You can try this out right now: hold your thumb in front of you and look at it with one eye. Now switch to the other eye. It’ll seem as if your thumb has jumped a bit from its previous position. The closer you move your thumb to your eyes, the more pronounced this jump will be.
This jump in perspective is how your brain determines how far an object is from you. It’s one of the ways we perceive depth. As such, stereoscopic displays present a slightly different view of a virtual scene to each eye. It’s what creates depth and how we feel as if we’re really somewhere else. If it weren’t for stereoscopic displays, there would be no VR.
Field of View
We see much more of the world than what our eyes are focusing on. The extent of our observable world is our field of view (FOV). Monocular FOV is the FOV for one of our eyes. Binocular FOV is where the FOV of both our eyes overlap. A human’s total viewable area is anywhere from 200° to 220°. Our binocular FOV is between 100° to 120°.
The wider the FOV of a VR display, the more realistic it will seem and the more present you will feel. The closer the lens of a head-mounted display (HMD) is placed to your face, the wider your FOV will become. However, the image will become increasingly distorted too. While techniques such as barrel distortion can correct distorted images to a certain degree, HMD manufacturers still need to find the right balance between placing their lens close enough for a wide FOV, but far enough so the image doesn’t look distorted.
Frame Rate and Refresh Rate
Anything that you see on a screen that seems to have natural motion is really just one still image after the other, but shown so rapidly it seems continuous. Each still image is called a frame. Frame rate is how rapidly those images are presented to the viewer. This is generally the work of a PC’s CPU and GPU. The more powerful those two, the more capable they’ll be to generate more frames per second (FPS).
The general consensus is that VR should be presented to us at 90 FPS. This will provide the smoothest and most immersive experience. Anything below 90 FPS will come across as somewhat jarring, and it’ll increase the chances of simulation sickness.
Refresh rate, however, has more to do with the display than it does with the CPU and GPU of your device. It’s measured in Hertz (Hz) and it explains how often your screen can redraw itself entirely. The Oculus Quest has a refresh rate of 72 Hz, which means it can refresh itself 72 times per second.
A display’s refresh rate usually limits a device’s frame rate. This is relevant for VR HMDs that are still hooked up to a PC. The display of the Oculus Rift, for example, has a refresh rate of 90 Hz. Even if your PC is powerful enough to produce 120 FPS, you’ll be capped at 90 frames per second because of the display’s refresh rate (if the two don’t match, you’ll experience something called screen tearing, where the images you see will seem torn apart).
Foveated Rendering
Our eyes are very good at seeing the details of what we’re focusing on. However, everything outside that narrow focus field, i.e. everything in our peripheral vision, will seem blurry. An upcoming graphics rendering technique that simulates this visual phenomenon is called foveated rendering.
Not only will foveated rendering make VR games and apps feel more realistic, it’ll also drastically reduce the workload on standalone VR devices or on the devices HMDs are hooked up to. After all, the image quality of everything in the peripheral vision can be drastically reduced, saving power and memory.
The problem, however, is that foveated rendering requires eye-tracking technology to figure out what our eyes are focusing on. At the time of writing, that technology isn’t available yet. However, the major VR manufacturers are actively working on devices with eye-tracking capabilities.
In Conclusion
To summarize: stereoscopic displays present a different viewpoint to each of our eyes. Barrel distortion on curved lenses allows for a wider FOV. Frame and refresh rate make it seem as if we’re in a world with continuous motion. Foveated rendering blurs what we’re not looking at.
All these technologies have us believe that we’re in a different world. It’s what makes VR so effective and why it has applications in so many different industries. No other medium comes close to achieving this level of presence.
