Interactive rendering: the real-time GPU pipeline and its budget

Why this subject matters now

Real-time rendering is the discipline of producing a correct-enough image inside a hard deadline, over and over, without ever missing. The deadline is what separates it from offline rendering, where a path tracer treated on the physically-based-rendering page can spend minutes on a frame and converge to the true solution of the rendering equation. A real-time renderer has milliseconds and must approximate. The whole field is a catalogue of approximations chosen for how much error they hide per millisecond spent, and it only makes sense when read against the budget.

Three things changed the field in the last several years. First, dedicated ray-tracing hardware arrived around 2018: GPUs now contain fixed-function units that intersect a ray with a bounding box or a triangle and that traverse a bounding-volume hierarchy, so a frame can mix rasterized primary visibility with ray-traced shadows, reflections, and global illumination. Second, temporal techniques took over. Antialiasing, ambient occlusion, reflections, and increasingly upscaling all now amortize their cost across frames by reusing the previous frame's result, which is why temporal antialiasing and neural upscalers such as NVIDIA DLSS and AMD FidelityFX Super Resolution became the default rather than the exception. Third, submission moved onto the GPU: instead of the CPU issuing one draw call per object, the GPU culls and builds its own draw lists, a style Epic's Nanite pushed to virtualized geometry. A practitioner today is expected to reason quantitatively about all three, and to know when to spend the budget on more rays versus more temporal reuse versus fewer draw calls.

The pipeline itself, the coordinate transforms, edge-function rasterization, the depth buffer, texture filtering, and the Cook-Torrance BRDF, is derived on the rendering-foundations page and is assumed here. This page owns the part that the frame budget forces: how the pipeline is scheduled, where its bandwidth goes, and which approximations buy the most image per millisecond.

The frame budget is the whole constraint

A display refreshes at a fixed rate, and a renderer that wants to present one new image per refresh without tearing has exactly one refresh interval to produce it. At sixty hertz that interval is

Missing the deadline is not graceful. If the frame is not ready when the display scans out, the renderer either shows the old frame again, which under a fixed refresh means the effective rate halves to thirty, or it tears. Variable-refresh displays soften this by letting the refresh follow the renderer, but the budget remains: to hold a rate, every frame must finish in its interval. This is a real-time constraint in the technical sense, a deadline that must be met every period, not on average. A renderer that averages twelve milliseconds but spikes to twenty every second stutters visibly, which is why the field cares about the worst frame, the 99th-percentile frame time, more than the mean.

The budget is spent on two kinds of work that overlap: CPU work preparing the frame (visibility, animation, submitting commands) and GPU work executing it (transforming vertices, shading pixels, moving bytes). Modern engines pipeline these so the CPU is preparing frame \(n+1\) while the GPU renders frame \(n\); the throughput is then set by the slower of the two, but the latency the player feels is longer. The single most useful habit in this field is to convert every proposed feature into milliseconds and subtract it from the interval before deciding whether it fits.

The real-time pipeline, stage by stage, and where the bottleneck lives

The programmable rasterization pipeline is a fixed sequence of stages, some programmable and some fixed-function, that a triangle passes through. The transforms and the edge-function coverage test are derived on the rendering-foundations page; here the concern is the schedule, the data volume between stages, and which stage stalls.

  vertices        indices
      │              │
      ▼              ▼
 ┌─────────────────────────┐
 │  input assembly (IA)    │  fetch + assemble primitives, fixed-function
 └───────────┬─────────────┘
             ▼
 ┌─────────────────────────┐
 │  vertex shader (VS)      │  per-vertex, programmable: object → clip
 └───────────┬─────────────┘
             ▼
 ┌─────────────────────────┐
 │  tessellation / geometry │  optional: subdivide / amplify primitives
 └───────────┬─────────────┘
             ▼
 ┌─────────────────────────┐
 │  rasterizer (fixed)      │  clip, /w, viewport, coverage, early-Z
 └───────────┬─────────────┘
             ▼
 ┌─────────────────────────┐
 │  fragment shader (FS)    │  per-covered-sample, programmable: shading
 └───────────┬─────────────┘
             ▼
 ┌─────────────────────────┐
 │  output merger (OM/ROP)  │  depth/stencil test, blend, write to targets
 └─────────────────────────┘

Input assembly reads the vertex and index buffers and forms primitives. Its cost is memory: pulling attribute streams through the cache. The vertex shader runs once per vertex and applies the model-view-projection chain plus any skinning; a scene with a million vertices runs it a million times, cheap next to the fragment work unless the mesh is dense relative to its screen area. Tessellation and geometry shaders amplify or generate primitives, and are the one place the pipeline can create data, which makes them easy to misuse: a geometry shader that emits many primitives per input serializes badly on most hardware, which is part of why modern engines replaced them with compute-driven mesh shaders.

The rasterizer is fixed-function. It clips against the frustum, performs the perspective divide, maps to the viewport, and finds covered samples with the edge functions. Crucially it runs the early depth test here when it can: if a fragment's depth fails the depth buffer before the fragment shader runs, the shader is skipped entirely. Early-Z is why sorting geometry front-to-back, or running a cheap depth prepass, saves fragment work, and why any fragment shader that writes its own depth or discards pixels forfeits early-Z and pays for every covered sample.

The fragment shader is where real-time renderers spend most of their budget, because it runs per covered sample and there are millions of them, each evaluating a BRDF against several lights and sampling several textures. The output merger, also called the raster operations unit, does the depth and stencil tests and alpha blending and writes the results; its cost is bandwidth to the render targets, and it is the stage that transparency (which cannot use early-Z and must blend in order) hammers.

The practical fact is that the bottleneck moves. A pipeline is only as fast as its busiest stage, and different scenes stress different stages: a dense mesh far away is vertex-bound, a screen full of overlapping foliage is fragment- and blend-bound, a fullscreen post-process is bandwidth-bound at the output merger, a scene with thousands of small objects is bound on the CPU submitting draws. Profiling a real-time renderer is largely the work of finding the current bottleneck, because optimizing anything else buys nothing. This is the roofline idea applied per stage: a stage is either compute-bound or bandwidth-bound, and the arithmetic intensity, operations per byte, decides which.

Forward, deferred, and clustered shading: the light-count cost model

The central scaling question in real-time shading is how cost grows with the number of lights. The three dominant architectures answer it differently, and the difference is a derivation, not a matter of taste.

Forward shading: cost couples geometry to lights

In classic forward shading, each object is drawn and its fragment shader loops over every light that affects it. Let \(P\) be the number of screen pixels, \(d\) the average depth complexity (how many fragments are produced per pixel before the depth test resolves them, so \(P d\) fragments are shaded), \(L\) the number of lights, and \(c\) the cost of evaluating one light at one fragment. The shading cost is

Every fragment, including those later overwritten by a nearer surface, pays for all lights. The coupling of \(d\) and \(L\) in a product is the weakness: overdraw multiplies the lighting cost. A depth prepass removes the overdraw for opaque geometry by establishing the nearest depth first so that early-Z kills hidden fragments, turning \(P d\) into \(P\), but forward still evaluates all \(L\) lights at every surviving fragment unless it culls lights per object, which is coarse.

Deferred shading: decouple geometry from lights

Deferred shading, popularized for real-time use by the CryEngine work Mittring presented in 2007, splits shading into two passes. The geometry pass rasterizes all opaque objects and writes their surface parameters, base color, world-space normal, roughness, metalness, depth, into a set of screen-sized textures called the G-buffer. No lighting happens yet. The lighting pass then reads the G-buffer per screen pixel and evaluates the lights, once, at the single nearest surface. The cost separates into a term that scales with geometry and a term that scales with lights:

where \(g\) is the per-fragment cost of writing the G-buffer. Lighting now costs \(P L c\), independent of depth complexity, because the G-buffer already resolved visibility: only the nearest surface survives, so lighting is evaluated \(P\) times, not \(P d\) times. The product \(dL\) that hurt forward shading is gone; deferred pays \(d\) for geometry and \(L\) for lighting, added, not multiplied.

The crossover

Setting \(C_{\text{fwd}} = C_{\text{def}}\) and solving for the light count where they tie:

Write the G-buffer write cost as a multiple of a light evaluation, \(g = k c\). Then \(L^\star = \dfrac{d\,k}{d-1}\). For depth complexity \(d = 4\) and a G-buffer that costs about \(k = 3\) light-evaluations to fill, \(L^\star = 4\cdot 3 /(4-1) = 4\) lights: beyond four overlapping lights, deferred wins, and the margin grows linearly with \(L\) because deferred adds one \(Pc\) per light while forward adds \(P d c\). This is the reason deferred shading dominated the era of many-small-light scenes. Its costs are equally derivable: the G-buffer eats bandwidth and memory (next section), it cannot express hardware multisample antialiasing cheaply because every sample would need its own G-buffer, and it cannot handle transparency, which has no single nearest surface, so transparent objects are always drawn forward in a separate pass.

Forward-plus and clustered: the light list

Forward-plus (tiled forward) and clustered shading recover the best of both. They keep forward's single pass and antialiasing- and transparency-friendliness but cull lights spatially first. The screen is divided into tiles (say 16×16 pixels) or, in the clustered variant, into a 3D grid of view-frustum cells called froxels; a compute pass intersects each light's bounding volume with each cell and writes, per cell, the short list of lights that touch it. The fragment shader then loops only over its cell's list, of average length \(\ell \ll L\), so the cost becomes

where \(s\approx \ell c\) is the shading cost against the local list, \(T\) is the number of cells, and \(u\) is the cost of testing one light against one cell. The culling term \(L T u\) is small because it is per cell, not per pixel, and the shading term scales with the local light density, which is what physically matters: a pixel only pays for lights that actually reach it. Clustered forward is the default in Google's Filament, in the Doom-era id Tech renderers, and in Godot's Vulkan backend, precisely because it scales with local light count and stays compatible with multisampling and transparency.

The G-buffer and its memory cost

The G-buffer is the intermediate that makes deferred shading possible, and its size is a direct budget line. A minimal physically based G-buffer stores, per pixel, base color, surface normal, the roughness and metalness that parameterize the Cook-Torrance BRDF, and depth; a motion vector is added for temporal antialiasing. Packing tightly matters because every byte is written once in the geometry pass and read once in the lighting pass, so the round trip is charged twice against bandwidth. A representative layout:

At 1920×1080 the footprint is \(20\ \text{B}\times 2{,}073{,}600 = 41.5\) MB resident, and 41.5 MB written per frame; the lighting pass reads those 20 bytes and writes an 8-byte HDR result, a further 58 MB moved. At 1440p the footprint scales to \(20\times 3{,}686{,}400 = 73.7\) MB and at 4K to \(20\times 8{,}294{,}400 = 166\) MB, so the G-buffer alone can be a large fraction of the framebuffer memory budget at high resolution, and its bandwidth grows linearly with pixel count. This linear-in-\(P\) growth is the reason 4K deferred renderers work hard to shrink the G-buffer, packing normals into two channels via octahedral encoding, storing depth once and reconstructing position from it rather than storing world position outright (which would cost 12 more bytes), and reusing channels. It is also part of the argument for clustered forward at 4K, which needs no G-buffer at all.

Shadow mapping: projection, acne, peter-panning, cascades, and PCF

A shadow map answers, for each shaded point, whether the light can see it. The technique, due to Williams in 1978, renders the scene once from the light's point of view and stores the nearest depth per light-space texel; then, when shading a surface point, the renderer transforms the point into light space and compares its depth to the stored value. If the point is farther than what the light saw, something nearer blocked the light and the point is in shadow.

Formally, let \(M_{\text{light}} = P_{\text{light}} V_{\text{light}}\) be the light's view-projection. A world point \(x\) maps to light clip space \(M_{\text{light}}\, \tilde{x}\), and after the perspective divide and the \([-1,1]\to[0,1]\) remap gives a texture coordinate \((u,v)\) and a reference depth \(z_{\text{ref}}\). The shadow test is

where \(T_{\text{shadow}}\) is the stored depth and \(b\) is a bias discussed next. A directional light uses an orthographic \(P_{\text{light}}\); a spotlight uses a perspective one; a point light needs six faces (a cube map) or a dual-paraboloid map.

Acne and peter-panning: the bias tradeoff

The stored depth is a point sample of a surface that a whole texel-sized patch of the world projects onto, and the shaded point almost never lands exactly where the sample was taken. On a surface tilted relative to the light, the true depth varies across the texel, so comparing a shaded point's exact depth against a neighbor's stored depth produces false self-shadowing: a striped or stippled pattern called shadow acne. The cure is a depth bias \(b\) that pushes the stored surface slightly away from the light before the comparison, so that a surface does not shadow itself.

The needed bias depends on the surface slope. For a surface whose normal makes angle \(\theta\) with the light direction, the depth changes by \(\tan\theta\) per unit of texel-space movement, so the bias must scale as

which is the constant-plus-slope-scaled bias every graphics API exposes. Too little bias leaves acne; too much bias detaches the shadow from the object's contact point, so the object appears to float and its shadow starts a few texels away. That artifact is called peter-panning. The bias must be large enough to clear the depth variation across one texel and no larger, which is why slope-scaled bias, proportional to \(\tan\theta\), is the right functional form: it grows exactly as fast as the error it must hide. A common refinement, normal-offset bias, moves the sample position along the surface normal in world space rather than biasing depth, which reduces peter-panning at grazing angles.

Cascaded shadow maps

A single shadow map covering a large outdoor scene wastes resolution: the same texel budget stretches over near and far geometry equally, but near geometry occupies far more screen pixels and needs finer shadows. Cascaded shadow maps, described by Engel and by Zhang and colleagues in the mid-2000s, split the view frustum into a few depth slices and give each slice its own shadow map, so the near slice packs its full resolution into a small world region. The split distances are chosen to keep shadow texel size roughly proportional to screen pixel size across the whole range, which a logarithmic split achieves. Zhang's practical scheme blends a logarithmic and a uniform split,

for cascade boundary \(i\) of \(N\), near \(n\), far \(f\), and a blend \(\lambda\in[0,1]\). The logarithmic term alone would over-concentrate resolution near the camera; the uniform term spreads it out; \(\lambda\) trades between them.

Percentage-closer filtering

The shadow test returns a hard binary, so a magnified shadow edge is a staircase of lit and shadowed texels. Percentage-closer filtering, introduced by Reeves, Salesin, and Cook in 1987, softens the edge by taking several shadow samples in a neighborhood, performing the depth comparison at each, and averaging the binary results. Note the order: one cannot average the stored depths and then compare, because depth is nonlinear and averaging it is meaningless; PCF compares first, then averages the comparisons, yielding a fractional visibility, say 0.6, that is the estimated fraction of the neighborhood that is lit. A \(3\times 3\) PCF kernel does nine comparisons per shaded pixel; hardware exposes a two-by-two bilinear PCF fetch that does four comparisons in one instruction, and larger soft shadows use wider kernels or precomputed variance-based approximations.

Ambient occlusion and real-time global illumination

Direct lighting is only part of the picture; surfaces are also lit by light bouncing off other surfaces, the indirect term of the rendering equation solved exactly by the path tracer on the physically-based-rendering page. Real-time renderers cannot afford that integral per frame, so they approximate it, and the approximations form a ladder from cheap and crude to expensive and accurate.

Ambient occlusion in screen space

Ambient occlusion approximates one visible consequence of indirect light: creases, contact points, and cavities receive less ambient light because nearby geometry blocks it. The true occlusion at a point \(x\) is the fraction of the hemisphere that is unblocked,

with \(V\) the visibility of direction \(\omega\). Screen-space ambient occlusion (SSAO), introduced by Mittring for Crytek in 2007, estimates this from the depth buffer alone: around each pixel it samples nearby points in a hemisphere, projects them back to screen space, and checks whether the stored depth there is nearer than the sample, which would mean the sample is occluded. Horizon-based ambient occlusion (HBAO), from Bavoil and colleagues at NVIDIA in 2008, refines this by marching along a few directions in screen space and finding the horizon angle above which geometry blocks the sky, integrating the occlusion analytically between the horizon and the surface tangent. Both are approximations with known failures: they only see what is in the depth buffer, so occlusion from off-screen or behind geometry is missed, and the sampling radius is a fixed screen-space size that does not correspond to a fixed world size. They are used everywhere regardless because they cost a millisecond or two and add exactly the contact darkening the eye expects.

Global illumination approximations

Full indirect lighting is harder. The real-time ladder, from oldest to newest, runs roughly as follows. Light probes precompute the incoming radiance at a grid of points, usually as spherical-harmonic coefficients, and interpolate between them at shading time; they capture low-frequency bounce lighting cheaply but cannot represent sharp indirect shadows and go stale when geometry moves. Voxel cone tracing, from Crassin and colleagues at INRIA and NVIDIA in 2011, voxelizes the scene into a sparse octree that stores directional radiance, then approximates the indirect integral by tracing a few wide cones through the voxels, trading rays for prefiltered volume lookups; it produces plausible one-bounce diffuse and glossy indirect light in real time at the cost of voxelization and memory. The modern path replaces both with hardware ray tracing: trace actual rays for indirect light, accept one or a few samples per pixel, and denoise heavily. Reservoir-based spatiotemporal importance resampling (ReSTIR), from Bitterli and colleagues in 2020, makes many-light and indirect sampling tractable by reusing samples across neighboring pixels and across frames, discussed with hardware ray tracing below. The trend is unmistakable: precomputed approximations are giving way to sampled-and-denoised real ray tracing as the hardware matures.

Antialiasing: MSAA, the shift to temporal AA, and neural upscaling

Rasterization samples the image at pixel centers, so a triangle edge that crosses a pixel is either fully in or fully out, producing the jagged staircase of aliasing. The signal being sampled has frequencies above the pixel Nyquist rate, and the fix is either to sample more densely or to reconstruct across time.

MSAA and why deferred shading strained it

Multisample antialiasing (MSAA) stores several coverage and depth samples per pixel but runs the fragment shader only once per covered pixel per triangle, sharing that shaded color across the samples the triangle covers. This is the key economy: geometry edges get supersampled coverage while shading stays at pixel rate, so MSAA antialiases silhouettes cheaply in compute but not in memory. The framebuffer grows linearly with the sample count: at 1080p with an 8-byte HDR color and 4-byte depth per sample, one sample is \((8+4)\times 2{,}073{,}600 = 24.9\) MB, and 4× MSAA is four times that, 99.5 MB, plus a resolve pass that reads all samples and averages them.

MSAA fits forward and clustered-forward renderers, which shade at the sample's surface, but it fits deferred shading badly: the G-buffer would need to store every sample's surface parameters, quadrupling an already large buffer, and the lighting pass would have to detect edge pixels and shade their samples separately. That mismatch, together with MSAA's blindness to shader aliasing (specular sparkle from high-frequency normal and roughness detail, which coverage sampling does nothing for), pushed the field toward temporal methods.

Temporal antialiasing: reprojection and its failure

Temporal antialiasing (TAA) turns time into extra samples. Each frame the camera projection is jittered by a sub-pixel offset drawn from a low-discrepancy sequence, so successive frames sample the scene at slightly different positions within each pixel. The current frame is blended with an accumulated history using an exponential moving average,

where \(x_n\) is the current jittered sample and \(c_{n-1}^{\text{reproj}}\) is last frame's result reprojected to the current pixel. Reprojection is necessary because the camera and objects moved: the surface at this pixel today was at a different pixel yesterday. The renderer writes a per-pixel motion vector \(\mathbf{m}\) (the RT3 channel of the G-buffer), and last frame's color is sampled at \((u,v) - \mathbf{m}\).

The blend accumulates samples like a running average. Treating each frame as an independent sample of variance \(\sigma^2\), the steady-state variance of the exponential moving average is

so the effective sample count is \(N_{\text{eff}} = \sigma^2/\Var = (2-\alpha)/\alpha\). With the common \(\alpha = 0.1\), \(N_{\text{eff}} = 1.9/0.1 = 19\): a single sample per frame, accumulated, gives the smoothness of nineteen samples once converged, at the cost of one sample's shading. That amortization is why TAA is nearly free per frame and why it became the default antialiaser.

The failure is reprojection validity. When a surface becomes newly visible (disocclusion), or moves off screen, or the shading changes faster than motion vectors predict (a moving specular highlight, a shadow edge, a transparent surface with no motion vector), the history is wrong, and blending it in produces ghosting, trails behind moving objects. TAA must therefore reject stale history. The standard mechanism, from Karis's 2014 notes on the Unreal Engine 4 implementation, clamps the history color to the bounding box of the current pixel's neighborhood in color space before blending: if the reprojected history falls outside the range of colors currently around this pixel, it is clamped back in, which suppresses ghosting at the cost of throwing away accumulated samples exactly where they are most needed, at edges. History rejection is thus a bias-variance trade in disguise: accept stale history and get smooth but smeared images, reject aggressively and get sharp but noisier, flickering ones. Every TAA is a tuning of this rejection.

Temporal upscaling: DLSS and FSR

Once a renderer accumulates jittered samples across frames, it can render at a lower resolution and let the temporal accumulation reconstruct a higher-resolution image, which is temporal upscaling. NVIDIA's Deep Learning Super Sampling (DLSS) replaces the hand-tuned history-rejection heuristics with a neural network trained to reconstruct the high-resolution frame from the jittered low-resolution samples, motion vectors, and depth; AMD's FidelityFX Super Resolution (FSR), in its temporal 2.x form, does the same reconstruction with an analytical, non-neural pipeline. Both let a renderer shade, say, 1080p worth of pixels and present a 4K image, cutting the fragment-shading cost by the resolution ratio while relying on temporal reuse to recover detail. Their remaining artifacts, ghosting, disocclusion fizzle, thin-feature instability, are exactly the reprojection failures TAA has, because they are TAA with a smarter reconstruction. Upscaling is the current answer to the fragment budget: rather than shade fewer lights, shade fewer pixels and reconstruct.

Level of detail, culling, instancing, and the draw-call problem

Everything above assumed the geometry reaching the rasterizer was already the right geometry. Getting there, deciding what to draw at all and at what detail, is its own budget battle, and it is usually fought on the CPU.

Culling: don't shade what isn't seen

The cheapest fragment is the one never rasterized. Three culling stages remove work in order of cost. Frustum culling tests each object's bounding volume against the six planes of the view frustum and discards what lies entirely outside; it is cheap and removes most of an open-world scene, which extends far beyond the camera's cone. Occlusion culling removes objects that are inside the frustum but hidden behind nearer geometry; the modern GPU-friendly form uses a hierarchical depth buffer (hi-Z), a mip pyramid of the depth buffer whose coarser levels store the farthest depth in each region, so a bounding box can be rejected by comparing its nearest depth against one coarse hi-Z texel that already bounds everything behind it. Backface culling, fixed-function in the rasterizer, drops triangles facing away from the camera by the sign of their screen-space winding, halving triangle work on closed meshes.

Level of detail

An object far from the camera covers few pixels and does not need its full triangle count; drawing a million-triangle mesh that lands on twenty pixels wastes vertex work and, worse, aliases, because sub-pixel triangles thrash the rasterizer and never fill a quad. Level of detail (LOD) swaps in simplified meshes as an object recedes, chosen by projected screen size. The classic scheme keeps a few discrete LODs and pops between them, which is visible; continuous and virtualized approaches, culminating in Epic's Nanite (Karis and colleagues, 2021), store a hierarchy of mesh clusters and select, per cluster, the detail level whose triangles are about pixel-sized, so the rendered triangle count tracks screen resolution rather than scene complexity. That decouples the vertex budget from how much geometry the artist authored, the same decoupling deferred shading gave the lighting budget.

The draw-call problem and GPU-driven rendering

Each draw call carries CPU overhead: validating state, binding resources, and recording a command into the buffer the GPU will execute. Historically this cost a few microseconds per call, small until multiplied by object count. The arithmetic is unforgiving. At 5 µs per draw, one thousand objects cost 5.0 ms of CPU, already a third of a 60 fps frame; ten thousand objects cost 50 ms, three full frames, and the GPU sits idle waiting for commands. The draw call, not the triangle, is the scaling wall, which is why reducing call count is the first optimization in a CPU-bound frame.

Two techniques attack it. Instancing draws many copies of the same mesh (trees, crowd characters, debris) in a single call, with per-instance data (transforms, colors) read from a buffer, turning ten thousand draws into one. GPU-driven rendering, described in work by Sander, Riccio, Persson, and others through the GPU Pro series and NVIDIA and AMD presentations, goes further: the object list, culling, and LOD selection all run in a compute shader on the GPU, which then emits a single multi-draw-indirect command whose parameters it wrote itself, so the CPU issues essentially one call for the whole scene. This depends on bindless resources, letting a shader index into a global table of textures and buffers rather than binding each per draw, so the GPU can draw arbitrary objects without CPU state changes between them. GPU-driven, bindless submission is how Nanite and modern engines render hundreds of thousands of objects; it moves the draw-call arithmetic off the CPU entirely.

Compute shaders and async compute

Not all GPU work is triangles. A compute shader runs a general parallel program over a grid of thread groups, with no fixed-function rasterization, reading and writing buffers and images directly. Real-time renderers use compute for everything that is not primary visibility: light culling for clustered shading, the hi-Z pyramid build, SSAO, the TAA resolve, bloom and tone-mapping, particle simulation, and the BVH-adjacent work of GPU-driven culling. Compute exposes fast on-chip shared memory within a thread group, which lets a pass stage data once and reuse it, the same locality argument that makes tiled algorithms fast on any parallel machine.

Modern GPUs can execute graphics and compute work concurrently, called async compute. The point is to fill idle units: a shadow pass that is bound on the fixed-function rasterizer and depth units leaves the arithmetic units underused, and a bandwidth-bound post-process leaves the same units idle, so scheduling an ALU-heavy compute pass (light culling, SSAO) to overlap a rasterization-bound graphics pass raises overall utilization without extending the frame. The overlap is a scheduling win, not free work: it only helps when two passes stress complementary parts of the chip, and it costs synchronization care to avoid one pass reading another's unfinished output. Async compute is why a naive sum of individual pass timings overestimates a well-scheduled frame: passes that overlap on the timeline are charged once against wall-clock, not twice.

Hardware ray tracing: BVH, the two-level split, denoising, and ReSTIR

Ray tracing computes visibility by shooting a ray and finding its nearest intersection, the dual of rasterization's per-triangle scan. Its acceleration structure, the bounding-volume hierarchy, and the ray-triangle test are derived on the rendering-foundations page; the offline use of rays to solve the rendering equation is on the physically-based-rendering page. What is new in real time is that the traversal and intersection are now fixed-function hardware, and that the budget allows only about one ray per pixel, which forces a reconstruction problem.

The two-level acceleration structure

Real-time ray tracing splits the BVH into two levels to handle animation and instancing efficiently. Each unique mesh gets a bottom-level acceleration structure (BLAS), a BVH over its triangles in the mesh's local space, built once and reused. A top-level acceleration structure (TLAS) is a BVH over instances, where each instance references a BLAS and carries a transform placing it in the world. A ray traverses the TLAS to find candidate instances, transforms into each instance's local space, and traverses that instance's BLAS. The split matters for the frame budget: when an object moves, only the TLAS must be rebuilt (cheap, a few thousand instances) while the BLAS geometry is untouched; only deforming meshes need a BLAS refit. This mirrors the instancing argument in rasterization, build the expensive structure once, reuse it under many transforms, and it is why the construction and traversal literature, Wald's work on fast BVH construction and Reshetov and colleagues on traversal, is central to making ray tracing fit a frame.

The hardware intersects rays with boxes and triangles and traverses these hierarchies in fixed-function units, because, as the rendering-foundations page argues, BVH traversal is incoherent pointer-chasing that maps poorly onto the streaming execution the rest of the GPU is built for. Dedicated units hide that latency and reorder rays for coherence.

One sample per pixel forces a denoiser

The budget affords roughly one ray per pixel for a given effect, sometimes fewer. One sample is a Monte Carlo estimate of an integral, so it is unbiased but extremely noisy: a single shadow ray gives a binary, a single indirect-lighting ray gives one random bounce, and the raw image is a snowstorm. Real-time ray tracing is therefore inseparable from denoising, which reconstructs a smooth image from the sparse noisy samples using spatial filtering (edge-aware blurs guided by the G-buffer's normals and depth so the blur does not cross real edges) and, crucially, temporal accumulation, the same reprojection and history mechanism as TAA. A one-sample-per-pixel ray-traced image accumulated over many frames and spatially filtered approaches the converged result; without the denoiser it is unusable. NVIDIA's real-time denoisers and the spatiotemporal filters in the research literature exist for exactly this reason, and their failure modes, ghosting, over-blurring, lag on fast motion, are once again the reprojection failures of temporal reuse.

ReSTIR: reuse samples across space and time

When a scene has thousands of lights or requires indirect illumination, even choosing which light to sample at a pixel is expensive, and one sample is a poor estimate. Reservoir-based spatiotemporal importance resampling, from Bitterli, Wyman, Pharr, and colleagues in 2020, attacks this by having each pixel keep a small reservoir that holds one probabilistically chosen sample, selected by weighted reservoir sampling so that a good light or path is retained with probability proportional to its contribution. The key move is reuse: a pixel combines its reservoir with those of its spatial neighbors and with its own reservoir from the previous frame, so that a well-chosen sample found anywhere nearby, or a moment ago, propagates to where it is needed. Because a neighbor's chosen light is likely a good light here too (nearby surfaces see similar illumination), this resampling dramatically lowers the variance of the one-sample-per-pixel estimate without tracing more rays, at the cost of the bias that spatial reuse introduces when neighbors are not actually similar, which the method bounds by reweighting. ReSTIR made many-light direct lighting and, in later work, full path-traced global illumination feasible in real time, and it is the reason a single-sample ray budget can produce a converged-looking image: the samples are reused, in space and time, exactly as TAA reuses shaded pixels and as the denoiser reuses accumulated frames. The three are the same idea, amortize a sparse budget across neighbors and across time, applied at three points in the pipeline.

Implementation

The first block is the budget arithmetic of Problems 1, 2, and 6 in NumPy: it reproduces the frame intervals, the forward-versus-deferred crossover table, the G-buffer bandwidth, and the draw-call collapse under instancing. Running it prints every number used above.

import numpy as np

# --- frame budgets: ms per frame at each refresh rate ---
for fps in (30, 60, 90, 120, 144, 240):
    print(f"{fps:3d} fps -> {1000.0 / fps:6.3f} ms")

# --- forward vs deferred shading cost (Problem 2) ---
P = 1920 * 1080          # 2,073,600 screen pixels at 1080p
d = 4                    # average depth complexity (fragments per pixel)
c = 50                   # ALU ops per light per fragment
g = 150                  # ALU ops to write the G-buffer per fragment (k = g/c = 3)
rate = 5e12              # sustained ops/second on this shader

def forward_ms(L):       # every shaded fragment loops over all L lights
    return P * d * L * c / rate * 1e3
def deferred_ms(L):      # geometry pass writes G-buffer, lighting pass is O(P*L)
    return (P * d * g + P * L * c) / rate * 1e3

for L in (2, 4, 8, 16, 32):
    print(f"L={L:2d}: forward {forward_ms(L):.3f} ms   deferred {deferred_ms(L):.3f} ms")

L_star = d * g / (c * (d - 1))                    # crossover light count
print("crossover L* =", L_star)                   # 4.0

# --- G-buffer bandwidth round trip (Problem 3) ---
def gbuffer_mb(bytes_per_px, w, h):
    return 2 * bytes_per_px * w * h / 1e6          # write + read
print("thin 20B @4K:", round(gbuffer_mb(20, 3840, 2160), 1), "MB")   # 331.8
print("fat  32B @4K:", round(gbuffer_mb(32, 3840, 2160), 1), "MB")   # 530.8

# --- draw-call collapse under instancing (Problem 6) ---
objs, us = 40000, 4.0
print("CPU-driven:", objs * us / 1e3, "ms")        # 160.0 ms
print("instanced :", 300 * us / 1e3, "ms")         # 1.2 ms

# --- TAA effective samples (Problem 5) ---
alpha = 0.1
N_eff = (2 - alpha) / alpha                         # 19.0
frames_95 = np.log(0.05) / np.log(1 - alpha)        # 28.4
print("TAA N_eff:", N_eff, " frames to 95%:", round(frames_95, 1))

import jax.numpy as jnp

# The budget arithmetic is pure array math; JAX vectorizes it over light counts.
P = 1920 * 1080
d, c, g, rate = 4, 50.0, 150.0, 5e12

L = jnp.array([2, 4, 8, 16, 32])
forward_ms = P * d * L * c / rate * 1e3
deferred_ms = (P * d * g + P * L * c) / rate * 1e3
crossover = d * g / (c * (d - 1))                   # 4.0

print("L         :", L)
print("forward ms:", jnp.round(forward_ms, 3))
print("deferred  :", jnp.round(deferred_ms, 3))
print("crossover :", crossover)

# G-buffer round-trip bandwidth at 4K for a sweep of packings
bytes_px = jnp.array([16, 20, 24, 32])
mb = 2 * bytes_px * 3840 * 2160 / 1e6
print("gbuffer MB:", jnp.round(mb, 1))

# TAA converged effective sample count as a function of blend weight
alpha = jnp.array([0.05, 0.1, 0.2])
N_eff = (2 - alpha) / alpha
print("TAA N_eff :", jnp.round(N_eff, 1))           # [39. 19.  9.]

The second block is a deferred lighting pass in GLSL: a fullscreen fragment shader that reconstructs world position from the depth buffer, unpacks the G-buffer, and accumulates a Cook-Torrance response over a small light list, the compute-bound loop whose cost Problem 1 and Problem 2 model. It is written in the reader's own words to illustrate structure, not copied from any engine. The angle brackets in the code are the shader's own operators.

#version 450
// Fullscreen deferred lighting pass. One invocation per screen pixel.
// The G-buffer resolved visibility, so we shade the single nearest surface.

layout(binding = 0) uniform sampler2D gAlbedoAO;   // rgb = base color, a = AO
layout(binding = 1) uniform sampler2D gNormal;     // rgb = packed world normal
layout(binding = 2) uniform sampler2D gMaterial;   // r = roughness, g = metalness
layout(binding = 3) uniform sampler2D gDepth;      // hardware depth

layout(std140, binding = 4) uniform Frame {
    mat4  invViewProj;      // clip -> world, to reconstruct position from depth
    vec3  camPos;
    int   lightCount;       // length of the local light list (clustered cull)
    vec4  lightPosRadius[64];
    vec4  lightColor[64];
};

in  vec2 uv;
out vec4 outColor;

const float PI = 3.14159265;

// GGX normal distribution: concentrates the specular lobe by roughness.
float D_ggx(float NdH, float a) {
    float a2 = a * a;
    float d  = NdH * NdH * (a2 - 1.0) + 1.0;
    return a2 / (PI * d * d);
}
// Smith height-correlated visibility term (geometry masking-shadowing).
float V_smith(float NdV, float NdL, float a) {
    float a2 = a * a;
    float gv = NdL * sqrt(NdV * NdV * (1.0 - a2) + a2);
    float gl = NdV * sqrt(NdL * NdL * (1.0 - a2) + a2);
    return 0.5 / max(gv + gl, 1e-4);
}
// Schlick Fresnel: reflectance rises toward 1 at grazing angles.
vec3 F_schlick(float VdH, vec3 f0) {
    return f0 + (1.0 - f0) * pow(1.0 - VdH, 5.0);
}

vec3 reconstructWorldPos(vec2 uv, float depth) {
    vec4 ndc = vec4(uv * 2.0 - 1.0, depth, 1.0);   // NDC with z in [0,1] or [-1,1]
    vec4 w   = invViewProj * ndc;
    return w.xyz / w.w;                            // undo the perspective divide
}

void main() {
    float depth = texture(gDepth, uv).r;
    if (depth >= 1.0) { outColor = vec4(0.0); return; }  // untouched background

    vec3  albedo    = texture(gAlbedoAO, uv).rgb;
    float ao        = texture(gAlbedoAO, uv).a;
    vec3  N         = normalize(texture(gNormal, uv).rgb * 2.0 - 1.0);
    float roughness = texture(gMaterial, uv).r;
    float metalness = texture(gMaterial, uv).g;

    vec3  P = reconstructWorldPos(uv, depth);
    vec3  V = normalize(camPos - P);
    float NdV = max(dot(N, V), 1e-4);

    // metals take their f0 from albedo; dielectrics use the ~4% base reflectance.
    vec3 f0 = mix(vec3(0.04), albedo, metalness);
    vec3 diffuseAlbedo = albedo * (1.0 - metalness);
    float a = roughness * roughness;

    vec3 color = vec3(0.0);
    for (int i = 0; i < lightCount; ++i) {          // only the clustered local list
        vec3  Lvec = lightPosRadius[i].xyz - P;
        float dist = length(Lvec);
        if (dist > lightPosRadius[i].w) continue;    // outside light radius
        vec3  L   = Lvec / dist;
        vec3  H   = normalize(V + L);
        float NdL = max(dot(N, L), 0.0);
        if (NdL <= 0.0) continue;

        float atten = 1.0 / (dist * dist);           // inverse-square falloff
        vec3  radiance = lightColor[i].rgb * atten;

        float D = D_ggx(max(dot(N, H), 0.0), a);
        float Vt = V_smith(NdV, NdL, a);
        vec3  Fr = F_schlick(max(dot(V, H), 0.0), f0);

        vec3 specular = D * Vt * Fr;                  // Cook-Torrance numerator
        vec3 kd = (1.0 - Fr);                         // energy left for diffuse
        vec3 diffuse = kd * diffuseAlbedo / PI;

        color += (diffuse + specular) * radiance * NdL;
    }
    color += diffuseAlbedo * 0.03 * ao;              // crude ambient term
    outColor = vec4(color, 1.0);
}

#version 450
// Percentage-closer filtering with slope-scaled bias and a cascade select.
// Compares depth first, then averages the binary results (never the depths).

layout(binding = 0) uniform sampler2DArrayShadow shadowCascades; // hw PCF compare
layout(std140, binding = 1) uniform ShadowData {
    mat4  cascadeVP[4];      // light view-projection per cascade
    float cascadeSplit[4];   // view-space far distance of each cascade
    vec2  texelSize;         // 1.0 / shadow map resolution
    int   cascadeCount;
};

int selectCascade(float viewZ) {
    for (int i = 0; i < cascadeCount; ++i)
        if (viewZ < cascadeSplit[i]) return i;       // nearest slice that contains it
    return cascadeCount - 1;
}

// visibility in [0,1]: fraction of the 3x3 neighborhood that is lit.
float pcfShadow(vec3 worldPos, vec3 N, vec3 L, float viewZ) {
    int c = selectCascade(viewZ);
    vec4 lc = cascadeVP[c] * vec4(worldPos, 1.0);
    vec3 proj = lc.xyz / lc.w;                        // light NDC
    proj.xy = proj.xy * 0.5 + 0.5;                    // -> [0,1] texture coords
    if (proj.z > 1.0) return 1.0;                     // beyond far plane: lit

    // slope-scaled bias: needed bias grows as tan(theta) = the depth slope.
    float cosT = clamp(dot(N, L), 0.0, 1.0);
    float tanT = sqrt(max(1.0 - cosT * cosT, 0.0)) / max(cosT, 1e-3);
    float bias = clamp(0.0008 * tanT, 0.0, 0.004) + 0.0004;

    float lit = 0.0;
    for (int dy = -1; dy <= 1; ++dy)
    for (int dx = -1; dx <= 1; ++dx) {
        vec2 off = vec2(dx, dy) * texelSize;
        // hardware compare-sample: returns the fraction passing (proj.z - bias)
        lit += texture(shadowCascades, vec4(proj.xy + off, float(c), proj.z - bias));
    }
    return lit / 9.0;                                 // average the comparisons
}

The third block is a WebGPU compute shader (WGSL) for the clustered light-culling pass: one thread group per screen tile builds the short per-tile light list that the forward-plus shader loops over, the \(LTu\) culling term in the clustered cost model. It is illustrative and simplified (a screen-tile 2D cull rather than a full froxel grid).

// WGSL compute: cull lights against a 16x16 screen tile.
// Dispatch one workgroup per tile; each writes a bounded light list for that tile.

struct Light { posRadius: vec4<f32>, color: vec4<f32> };

@group(0) @binding(0) var<storage, read>        lights   : array<Light>;
@group(0) @binding(1) var<storage, read_write>  tileList : array<u32>; // flat lists
@group(0) @binding(2) var<uniform>              cfg      : Config;

struct Config {
    invViewProj : mat4x4<f32>,
    screen      : vec2<u32>,   // pixels
    tiles       : vec2<u32>,   // tile grid dims
    lightCount  : u32,
    maxPerTile  : u32,
};

// AABB of a tile's frustum in world space, from its screen-space corners at
// near and far, reconstructed through the inverse view-projection.
fn tileBounds(tile: vec2<u32>) -> array<vec3<f32>, 2> {
    let px0 = vec2<f32>(tile) * 16.0;
    let px1 = px0 + vec2<f32>(16.0, 16.0);
    let s   = vec2<f32>(cfg.screen);
    var lo  = vec3<f32>( 1e30,  1e30,  1e30);
    var hi  = vec3<f32>(-1e30, -1e30, -1e30);
    for (var zi = 0u; zi < 2u; zi = zi + 1u) {
        let z = f32(zi);                          // near (0) and far (1) in NDC z
        for (var c = 0u; c < 4u; c = c + 1u) {
            let cx = select(px0.x, px1.x, (c & 1u) == 1u);
            let cy = select(px0.y, px1.y, (c & 2u) == 2u);
            let ndc = vec4<f32>((vec2<f32>(cx, cy) / s) * 2.0 - 1.0, z, 1.0);
            let w = cfg.invViewProj * ndc;
            let p = w.xyz / w.w;
            lo = min(lo, p);
            hi = max(hi, p);
        }
    }
    return array<vec3<f32>, 2>(lo, hi);
}

// squared distance from a point to an AABB; zero if the point is inside.
fn sqDistToAabb(p: vec3<f32>, lo: vec3<f32>, hi: vec3<f32>) -> f32 {
    let d = max(max(lo - p, p - hi), vec3<f32>(0.0));
    return dot(d, d);
}

@compute @workgroup_size(1)
fn main(@builtin(workgroup_id) wg: vec3<u32>) {
    let tile = wg.xy;
    let b = tileBounds(tile);
    let base = (tile.y * cfg.tiles.x + tile.x) * cfg.maxPerTile;
    var count = 0u;
    for (var i = 0u; i < cfg.lightCount && count < cfg.maxPerTile; i = i + 1u) {
        let L = lights[i];
        let r = L.posRadius.w;
        if (sqDistToAabb(L.posRadius.xyz, b[0], b[1]) <= r * r) {
            tileList[base + count] = i;           // light i touches this tile
            count = count + 1u;
        }
    }
    // sentinel terminates the per-tile list for the shading pass.
    if (count < cfg.maxPerTile) { tileList[base + count] = 0xffffffffu; }
}

How it is done in practice

A shipping renderer is a schedule of passes packed into the interval, each with a measured cost, and the engineering is in the packing. A representative modern frame runs, roughly in order: a compute pass to cull and build draw lists (GPU-driven), a depth prepass to establish visibility for early-Z, the G-buffer or the forward geometry pass, a shadow pass per cascade, ray-traced or screen-space passes for shadows, ambient occlusion, reflections, and indirect light (each denoised), the lighting resolve, transparency drawn forward, then a chain of post-processing: temporal antialiasing or upscaling, bloom, exposure and tone-mapping, and UI. Passes that stress different units are overlapped with async compute. The whole thing is instrumented with GPU timestamps so that the team knows the millisecond cost of every pass and can defend the budget.

The numbers this page uses for consumer targets, 600 GB/s of bandwidth, a few teraflops of effective shader throughput, are deliberately mid-range and are the kind measured on a desktop gaming GPU, not a datacenter accelerator. It is worth stating the gap concretely. The NVIDIA H100 80GB HBM3 in this repository sustains about 2,992 GB/s of memory bandwidth on a streaming copy and hundreds of teraflops of tensor throughput (measured: 744.6 bf16 TFLOP/s on a 4096-cube matmul), which is five times the bandwidth and far more compute than a consumer part. That does not make it a game GPU: it lacks the display pipeline and the fixed-function raster and ray units tuned for real-time frames are provisioned differently, and above all a frame budget is about ratios, how many pixels and lights fit in an interval, not absolute throughput. The H100 numbers are useful here only to anchor the bandwidth-to-compute ratio that decides whether a pass is bandwidth- or compute-bound; the frame arithmetic holds on any GPU once its own bandwidth and throughput are substituted.

The recurring practical lesson is that the bottleneck is rarely where intuition puts it. Teams profile obsessively because the binding constraint moves between the CPU (draw submission), the fixed-function units (rasterization, depth, ray traversal), the arithmetic units (shading), and memory bandwidth (G-buffer, post-processing, MSAA), and optimizing a non-bottleneck pass buys nothing. The habit of converting every feature to milliseconds, worked throughout this page, is the core professional skill.

The current research frontier

The last several years have been dominated by the maturation of real-time ray tracing and temporal reuse. ReSTIR, from Bitterli and colleagues (NVIDIA and Dartmouth) in 2020, reframed real-time many-light and path-traced lighting as a resampling problem and has been extended to full global illumination and to volumetrics in follow-on work; it is the most influential idea in the area since deferred shading. Neural denoising and neural upscaling, NVIDIA's DLSS line and the analytical AMD FidelityFX Super Resolution, moved reconstruction from hand-tuned heuristics toward learned or carefully engineered filters, and the frontier now includes neural components inside the shading itself, learned radiance caches and neural materials that evaluate a small network per pixel. Epic's Nanite (Karis and colleagues, 2021) virtualized geometry so that triangle count tracks screen resolution, and its companion Lumen combined screen-space, voxel, and ray-traced signals into a unified real-time global illumination system, a pragmatic layering of every approximation on this page. Work on GPU-driven and bindless rendering, across the GPU Pro and GPU Zen volumes and vendor presentations, continues to move submission and culling onto the GPU. The competing lines are visible in the open engines: id Tech and Filament favor clustered forward and careful hand-tuned passes, Unreal pushes virtualized geometry and layered GI, and the research renderers such as NVIDIA's Falcor exist to prototype the sampling and denoising ideas before they ship. The through-line is amortization: spend a sparse budget of rays and shaded pixels, and reuse each sample across neighbors and across frames as aggressively as the artifacts allow.

Open source to read

• google/filament: a compact, well-documented physically based renderer with a clustered-forward path; read filament/src/details/View.cpp and the material-system docs to see clustered light culling and the PBR shading model in production form.
• KhronosGroup/Vulkan-Samples: focused samples for deferred shading, hi-Z occlusion culling, and subpasses; open the samples/performance directory to see each technique isolated with timings.
• ConfettiFX/The-Forge: a cross-platform rendering framework with GPU-driven and visibility-buffer examples; a good read for bindless resource layouts and multi-draw-indirect submission.
• godotengine/godot: the Vulkan clustered renderer in servers/rendering/renderer_rd/ is a complete, approachable clustered-forward implementation with shadows, SSAO, and SDFGI.
• o3de/o3de: an open engine (Atom renderer) with a fully data-driven, pass-based frame graph; useful for seeing how passes are wired and scheduled.
• bevyengine/bevy: a Rust engine whose renderer (crates/bevy_pbr) implements clustered forward, cascaded shadow maps, and TAA in readable, modern code.
• NVIDIAGameWorks/Falcor: the research renderer behind many ray-tracing and ReSTIR papers; the sample render passes are the reference for real-time path tracing and denoising.
• gpuweb/webgpu-samples: minimal WebGPU samples including a deferred renderer and a compute-based particle system, the quickest way to run the pipeline stages in a browser.

Common misconceptions

"Higher frame rate just means more of the same work." It means a smaller interval: 120 fps halves the budget to 8.3 ms, so every pass must finish in half the time, which usually forces lower resolution, fewer lights, or more upscaling, not merely running faster.

"Deferred shading is always faster than forward." Only past the crossover. Below a handful of overlapping lights, forward avoids the G-buffer's fixed geometry and bandwidth cost and wins; deferred also forfeits cheap MSAA and cannot shade transparency, which is why clustered forward, not deferred, is the modern default.

"MSAA supersamples shading." It supersamples coverage and depth but runs the fragment shader once per covered pixel per triangle, so it antialiases silhouettes but does nothing for specular sparkle or shader aliasing, which is one reason temporal methods replaced it.

"TAA is free smoothing." Its amortization is real, nineteen effective samples at \(\alpha=0.1\), but only where reprojection is valid. At disocclusions and on fast specular motion the history is wrong and must be rejected, trading ghosting for noise; TAA is a bias-variance knob, not a free lunch.

"Fewer triangles is the main way to speed up a scene." Usually the wall is draw calls, not triangles: ten thousand objects at a few microseconds each blow the frame on the CPU before the GPU touches a triangle, which is why instancing and GPU-driven submission matter more than polygon reduction.

"Shadow acne is fixed by cranking up the depth bias." Too much bias causes peter-panning, the shadow detaching from the object's contact point. The correct bias is slope-scaled, proportional to \(\tan\theta\), just large enough to clear the depth variation across one texel and no larger.

"Hardware ray tracing gives you a clean image." At the one-sample-per-pixel the budget allows, the raw result is a snowstorm; real-time ray tracing is inseparable from spatiotemporal denoising and sample reuse (ReSTIR), and its artifacts are the reprojection failures of temporal accumulation.

Self-check

References

1. Akenine-Moller, T., Haines, E., Hoffman, N., Pesce, A., Iwanicki, M., and Hillaire, S. Real-Time Rendering, 4th ed. CRC Press, 2018. The central reference for this page: the pipeline, deferred and clustered shading, shadows, antialiasing, and acceleration structures. realtimerendering.com
2. Pharr, M., Jakob, W., and Humphreys, G. Physically Based Rendering: From Theory to Implementation, 4th ed. MIT Press, 2023. The reference for the ray-tracing, sampling, and reconstruction material. pbr-book.org
3. Marschner, S. and Shirley, P. Fundamentals of Computer Graphics, 4th ed. CRC Press, 2015. Background for the transforms and rasterization the companion foundations page derives.
4. Williams, L. "Casting Curved Shadows on Curved Surfaces." SIGGRAPH, 1978. The original shadow-mapping paper. doi:10.1145/800248.807402
5. Reeves, W., Salesin, D., and Cook, R. "Rendering Antialiased Shadows with Depth Maps." SIGGRAPH, 1987. Percentage-closer filtering. doi:10.1145/37401.37435
6. Zhang, F., Sun, H., Xu, L., and Lun, L. K. "Parallel-Split Shadow Maps for Large-Scale Virtual Environments." VRCIA, 2006. The logarithmic/uniform cascade split. doi:10.1145/1128923.1128975
7. Engel, W. "Cascaded Shadow Maps." In ShaderX5, Charles River Media, 2006. The practical cascaded-shadow-map formulation.
8. Mittring, M. "Finding Next Gen: CryEngine 2." SIGGRAPH Courses (Advanced Real-Time Rendering in 3D Graphics and Games), 2007. Deferred shading and the original screen-space ambient occlusion. doi:10.1145/1281500.1281671
9. Bavoil, L., Sainz, M., and Dimitrov, R. "Image-Space Horizon-Based Ambient Occlusion." ACM SIGGRAPH Talks, 2008. HBAO. doi:10.1145/1401032.1401061
10. Crassin, C., Neyret, F., Sainz, M., Green, S., and Eisemann, E. "Interactive Indirect Illumination Using Voxel Cone Tracing." Computer Graphics Forum (Pacific Graphics), 2011. Voxel cone tracing, from INRIA and NVIDIA. doi:10.1111/j.1467-8659.2011.02063.x
11. Karis, B. "High-Quality Temporal Supersampling." SIGGRAPH Courses (Advances in Real-Time Rendering in Games), 2014. The Unreal Engine 4 TAA with neighborhood clamping, and PBR shading notes. advances.realtimerendering.com
12. Lottes, T. "FXAA." NVIDIA white paper, 2009. The morphological, single-frame antialiasing baseline temporal methods improved on. developer.nvidia.com/fxaa
13. Bitterli, B., Wyman, C., Pharr, M., Shirley, P., Lefohn, A., and Jarosz, W. "Spatiotemporal Reservoir Resampling for Real-Time Ray Tracing with Dynamic Direct Lighting." ACM Transactions on Graphics (SIGGRAPH), 2020. ReSTIR, from NVIDIA and Dartmouth. doi:10.1145/3386569.3392481
14. Wald, I. "On Fast Construction of SAH-Based Bounding Volume Hierarchies." IEEE Symposium on Interactive Ray Tracing, 2007. Fast BVH build. doi:10.1109/RT.2007.4342588
15. Reshetov, A., Soupikov, A., and Hurley, J. "Multi-Level Ray Tracing Algorithm." ACM Transactions on Graphics (SIGGRAPH), 2005. Coherent BVH traversal. doi:10.1145/1073204.1073329
16. Karis, B., Stubbe, R., and Wihlidal, G. "A Deep Dive into Nanite Virtualized Geometry." SIGGRAPH Courses (Advances in Real-Time Rendering), Epic Games, 2021. Virtualized geometry and GPU-driven, cluster-based LOD. advances.realtimerendering.com
17. Nvidia. "NVIDIA DLSS: Deep Learning Super Sampling, Technical Overview." NVIDIA developer documentation, accessed 2026. Temporal neural upscaling. developer.nvidia.com/rtx/dlss
18. AMD. "FidelityFX Super Resolution (FSR): Technical Documentation." AMD GPUOpen, accessed 2026. Analytical temporal upscaling. gpuopen.com/fidelityfx-superresolution
19. Pharr, M. and Fernando, R. (eds.). GPU Gems 2 and Nguyen, H. (ed.), GPU Gems 3. Addison-Wesley, 2005 and 2007. Deferred shading, shadow, and GPU-technique chapters. developer.nvidia.com/gpugems
20. Engel, W. (ed.). GPU Pro and GPU Zen series. CRC Press / Black Cat, 2010-2019. Clustered shading, GPU-driven rendering, and tiled-light techniques by many authors including Persson and Riccio.
21. Olsson, O., Billeter, M., and Assarsson, U. "Clustered Deferred and Forward Shading." High-Performance Graphics, 2012. The clustered (froxel) light-culling formulation, from Chalmers. doi:10.2312/EGGH/HPG12/087-096
22. Harada, T., McKee, J., and Yang, J. "Forward+: Bringing Deferred Lighting to the Next Level." Eurographics Short Papers, 2012. Tiled forward light culling. doi:10.2312/conf/EG2012/short/005-008
23. Karis, B. "Real Shading in Unreal Engine 4." SIGGRAPH Courses (Physically Based Shading in Theory and Practice), Epic Games, 2013. The real-time Cook-Torrance parameterization used in the shader above. blog.selfshadow.com