Until a few years ago, not many people in games had heard of ray tracing. In a short period of time, this technology has gone from something that only a few specialist people knew or cared about to something that gamers not only are aware of, but want to see in more titles.
At its most basic form, ray tracing is simply shooting a ray from one point to another and seeing if it hits any objects. It’s used for all kinds of things, both in video games and other software, but in the case of graphics rendering it’s used to accurately simulate how light actually works in the real world.
“Look in a mirror and you’ll see your own reflection. That’s actually the light travelling from the sun, or a light bulb, and bouncing around until it hits you,” explains Phil Crabtree, technical director over at Kaizen Game Works. “The light beams then bounce off you and some of them will hit your mirror. Some of those will then bounce off towards your eyes for you to see your own reflection.
“Shadows are also influenced by the way light bounces around. The position and length of a shadow will grow or shrink depending on the angle of the light, and how dark or sharp that shadow is can depend on other light sources, too.
“Things get interesting when you consider that different surfaces respond to light differently too, so it’s not just the light simply bouncing off an object, but it can also involve how light travels through a surface – like a window – or how it scatters just under the surface of things like your skin.”
Without ray tracing, light in games traditionally is done ‘manually’. Sometimes developers would add light sources into the world, with the engine then figuring out what materials and surfaces would look like based on their proximity to this. This is limited method but a decent way of approximating what a scene or asset would look like that’s computationally very efficient. Ray tracing, by contrast, is an intense process that requires a lot of computational power to do all the correct calculations.
“Traditional methods of computing shadows or reflections are very different from how they work in real life, making it difficult to reproduce them in a physically accurate and photo-real way,” Vladimir Kajalin, principal rendering engineer for Crytek’s CryEngine, explains.
“But in the case of ray tracing, all those effects are by default computed in a similar way to how it works in real life. This allows not only the rendering of accurate-looking shadows or reflections but may also unexpectedly bring us some new advanced cool features for free – for example, caustics. We just need proper implementation of ray shading functions, and then our GI system will magically output a nice caustics effect exactly where you expect that in real life. All these kinds of details make the game scene look more complete and more believable, and of course, it all helps to immerse the player deeper into the game world and game story.”
Crabtree adds: “It’s always been very time-consuming to figure out all those light bounces. The actual process is fairly straightforward, you just need to draw a line at every possible angle from every light source.
“When it hits an object, you calculate how that object influences the light colour and strength, then calculate the angle it bounces off, and repeat until the light is completely absorbed or it hits a camera – the game equivalent of your eyes.”
As you can see, this process involves a lot of hard work. But there is a way to cut down the amount of effort required to achieve ray tracing; you reverse the process.
“Rather than calculate every possible light beam from every possible source, we instead work backwards and fire a beam from the camera that represents one pixel, and follow that until it hits a light source, or until we’ve decided we’ve had enough bounces,” Crabtree says. “This way you’re only calculating what the camera is actually seeing. Using this reverse method, you might still need to fake some information, like the level of ambient light, to help reduce the amount of bounces you need to do to get a usable image.”
The concept of ray tracing, and the tech to achieve it, isn’t all that new. It has been employed in tangential industries – for example, with CGI in some recent films. This still requires an awful lot of work; according to Pixar, it took them between 60 hours to 160 hours to render a single frame for 2019’s Toy Story 4, which employed ray tracing.
Films need to run at 24 frames per second and everything is pre-rendered. The process is different for video games, which need to process at least 30 frames per second in real time.
“Film CGI version will be more complex, but it shows the discrepancy between the two and why it’s not come to games sooner,” Crabtree says.
The recent boom in video games ray tracing has been, in part, due to graphics card company Nvidia’s RTX lineup of GPUs. These debuted in 2018, with the firm rolling out more advanced hardware since. Meanwhile, the newly-launched PlayStation 5 and Xbox Series X/S consoles are also capable of ray tracing.
But at the moment, both PCs and consoles are only capable of ray tracing certain aspects of a game. There are no ‘fully’ ray traced titles in the wild as of yet for a few reasons and it’s doubtful that we’ll see a game fully rendered using ray tracing any time soon due to the sheer computational demand.
“Gradually, ray tracing will become a standard feature in games,” Kajalin says. “However, it will be optional for the foreseeable future, and most games will look fine even with ray tracing disabled. I would not expect completely ray-traced triple-A titles for a long time. Still, I would not be surprised to see multiple commercially released indie games that use only ray tracing and completely drop rasterisation in the coming years. For many tasks, good old rasterisation works just great and, as technology advances, it will work even better in the future. Because of that, there is no good reason to wait for ‘fully’ ray-traced games.”
Developers also face the problem of having to make a version of the game that uses ray tracing and another that employs more traditional rendering techniques. This introduces a few interesting issues into the process of making games.
“Ray tracing is also still quite slow compared to the years of tricks and techniques developed for games so there is a balance between realism and good performance. Ray tracing can also actively work against some of the optimisations we’ve had in games for years,” Crabtree says.
“There’s a common technique called ‘frustum culling’, which means you don’t draw anything the camera can’t see, so an object behind you might become invisible. Now though, with light reflecting in a window you can now see what’s happening behind you, so that optimisation might be lost.
“Developers need to choose which features work best with their gameplay without having too much of an impact on performance.”
While completely ray traced games are still some way off, Sony and Microsoft’s new consoles have made ray tracing available to more people. Before these machines launched in November 2020, you needed to have a PC with a pretty fancy graphics card. But the PS5 and Xbox Series X/S being capable of ray tracing also makes life much easier for developers.
“Now we have consoles built to handle the task and engines that support ray tracing features, it becomes easier for developers to build on that technology,” Crabtree says.
“Ray tracing can actually improve the development process too. If you know all your players will use ray tracing technology then you can rely on that to calculate your lighting rather than having to use techniques like light baking, which can be a slow process. Unfortunately, most games aren’t ray tracing only, so it’ll take a while till we can actually change the development process fully.”
Not only does ray tracing mean that developers can create games with fancy graphics and realistic lighting, but it can also make it easier for them to make impressive-looking trailers. Of course, you need a lot of other disciplines like scripting and cinematography, but studios will be able to make flashier marketing materials on par with traditional CG trailers.
“Real-time cinematic rendering will make it easier for anyone to produce visually impressive cinematic trailers,” says Ben Lhotavery, studio head at game trailer specialist The Trailer Farm.
“We expect to see more of our customers leverage ray tracing in their engines, and as real-time cinematic rendering is improving exponentially, that quality gap between traditional CG and real-time will get even smaller within the next five years.”
The reason for this is that if you are already rendering your game using ray tracing, you can use the same art assets for a trailer where before someone might have to build them entirely fresh using CG.
“It allows the team working on a trailer to become an extension of the development team,” Lavery explains. “We are working within the engine and have access to the game’s characters, environments and meshes. The trailer stays true to the game’s art style, but we’re able to render with high-fidelity graphics. We will also have access to props, weapons and we won't need to worry about re-linking textures or re-creating shaders – resulting in a lower risk for the customer.”
Overall, ray tracing represents the next great leap in visuals in video games. It might be a long time before we see games that are entirely ray traced, but this tech is going to change how games look, but also how they are made.
“For players, it means more realistic, natural-looking, and more lifelike visuals in games,” Kajalin explains. “For game creators, it simplifies the creation of realistic visuals without having to deal with multiple and often ‘hacky’ rendering techniques.”
Barclays (including its employees, Directors and agents) accepts no responsibility and shall have no liability in contract, tort or otherwise to any person in connection with this content or the use of or reliance on any information or data set out in this content unless it expressly agrees otherwise in writing. It does not constitute an offer to sell or buy any security, investment, financial product or service and does not constitute investment, professional, legal or tax advice, or a recommendation with respect to any securities or financial instruments.
The information, statements and opinions contained in this content are of a general nature only and do not take into account your individual circumstances including any laws, policies, procedures or practices you, or your employer or businesses may have or be subject to. Although the statements of fact on this page have been obtained from and are based upon sources that Barclays believes to be reliable, Barclays does not guarantee their accuracy or completeness.
Barclays (including its employees, Directors and agents) accepts no responsibility and shall have no liability in contract, tort or otherwise to any person in connection with this content or the use of or reliance on any information or data set out in this content unless it expressly agrees otherwise in writing. It does not constitute an offer to sell or buy any security, investment, financial product or service and does not constitute investment, professional, legal or tax advice, or a recommendation with respect to any securities or financial instruments.
The information, statements and opinions contained in this content are of a general nature only and do not take into account your individual circumstances including any laws, policies, procedures or practices you, or your employer or businesses may have or be subject to. Although the statements of fact on this page have been obtained from and are based upon sources that Barclays believes to be reliable, Barclays does not guarantee their accuracy or completeness.