Random Thoughts on Raytracing

Big graphics news at this GDC seem to be DirectX Raytracing. Here’s some incoherent (ha!) thoughts about it.

Raytracing: Yay

“Traditional” rasterized graphics is hard. When entire books are written on how to deal with shadows, and some of the aliasing/efficiency problems are still unsolved, it would be nice to throw something as elegant as a raytracer at it. Or screen-space reflections, another “kinda works, but zomg piles upon piles of special cases, tweaks and fallbacks” area.

There’s a reason why movie industry over the last 10 years has almost exclusively moved into path tracing renderers. Even at Pixar’s Renderman – where their Reyes was in fact an acronym for “Renders Everything You Ever Saw” – they switched to full path tracing in 2013 (for Monsters University), and completely removed Reyes from Renderman in 2016.

DirectX Raytracing

Mixed thoughts about having raytracing in DirectX as it is now.

A quick glance at the API overall seems to make sense. You get different sorts of ray shaders, acceleration structures, tables to index resources, zero-cost interop with “the rest” of graphics or compute, etc etc.

Conceptually it’s not that much different from what Imagination Tech has been trying to do for many years with OpenRL & Wizard chips. Poor Imgtec, either inventing so much or being so ahead of it’s time, and failing to capitalize on that in a fair way. Capitalism is hard, yo :| Fun fact: Wizard GPU pages are under “Legacy GPU Cores” section of their website now…

On the other hand, as Intern Department quipped, DirectX has a long history of “revolutionary” features that turned out to be duds too. DX7 retained mode, DX8 Matrox tessellation, DX9 ATI tessellation, DX10 geometry shaders & removal of FP16, DX11 shader interfaces, deferred contexts etc.

Yes, predicting the future is hard, and once in a while you do a bet on something that turns out to be not that good, or not that needed, or something entirely else happens that forces everyone else to go in another direction. So that’s fair enough, in best case the raytracing abstraction & APIs become an ubiquitous & loved thing, in worst case no one will use it.

I’m not concerned about “ohh vendor lock-in” aspect of DXR; Khronos is apparently working on something there too. So that will cover your “other platforms” part, but whether that will be a conceptually similar API or not remains to be seen.

What I am slightly uneasy about, however, is…

Black Box Raytracing

The API, as it is now, is a bit of a “black box” one.

• What acceleration structure is used, what are the pros/cons of it, the costs to update it, memory consumption etc.? Who knows!
• How is scheduling of work done; what is the balance between lane utilization vs latency vs register pressure vs memory accesses vs (tons of other things)? Who knows!
• What sort of “patterns” the underlying implementation (GPU + driver + DXR runtime) is good or bad at? Raytracing, or path tracing, can get super bad for performance at divergent rays (while staying conceptually elegant); what and how is that mitigated by any sort of ray reordering, bundling, coalescing (insert N other buzzwords here)? Is that done on some parts of the hardware, or some parts of the driver, or DXR runtime? Who knows!
• The “oh we have BVHs of triangles that we can traverse efficiently” part might not be enough. How do you do LOD? As Sebastien and Brian point out, there’s quite some open questions in that area.

There’s been a massive work with modern graphics APIs like Vulkan, D3D12 and partially Metal to move away from black boxes in graphics. DXR seems to be a step against that, with a bunch of “ohh, you never know! might be your GPU, might be your driver, might be your executable name lacking a quake3.exe” in it.

It probably would be better to expose/build whatever “magics” the upcoming GPUs might have to allow people to build efficient tracers themselves. Ability to spawn GPU work from other GPU work; whatever instructions/intrinsics GPUs might have for efficient tracing/traversal/intersection math; whatever fixed function hardware might exist for scheduling, re-scheduling and reordering of work packets for improved coherency & memory accesses, etc. etc.

I have a suspicion that the above is probably not done “because patents”. Maybe Imagination has an imperial ton of patents in the area of ray reordering, and Nvidia has a metric ton of patents in all the raytracing research they’ve been doing for decades by now, and so on. And if that’s true, then indeed “just expose these bits to everyone” is next to impossible, and DXR type approach is “best we can do given the situation”… Sad!

I’ll get back to my own devices :)

So, yeah. Will be interesting to see where this all goes. It’s exciting, but also a bit worrying, and a whole bunch of open questions. Here’s to having it all unfold in a good way, good luck everyone!

And I just realized I’ve never written even a toy path tracer myself; and the only raytracer I’ve done was for an OCaml course in the university, some 17 years ago. So I got myself Peter Shirley’s Ray Tracing in One Weekend and two other minibooks, and will play around with it. Maybe as a test case for Unity’s new Job System, ECS & Burst compiler, or as an excuse to learn Rust, or whatever.