REALTIME GRAPHICS
// 60fps at 4K — the GPU computes the visible world frame by frame
// Render Pipeline
The GPU render pipeline transforms 3D geometry into 2D pixels through a sequence of programmable and fixed-function stages. Every frame: vertices → triangles → fragments → pixels. The programmer controls the vertex shader and fragment shader; the hardware handles rasterization, depth test, and blending.
Vertex Shader
Transforms each vertex position from object space through world → view → clip space. Runs once per vertex on the GPU. Outputs gl_Position and varyings.
Rasterization
Fixed-function hardware converts triangles into fragments (potential pixels). Interpolates vertex attributes (UVs, normals) across the triangle surface.
Fragment Shader
Computes final pixel color. Texture sampling, lighting calculations, procedural effects. Runs once per fragment — the bottleneck for complex scenes.
Output Merge
Depth test rejects occluded fragments. Alpha blending composites transparency. Stencil operations mask regions. Final pixel written to framebuffer.
// GPU Architecture
Modern GPUs are massively parallel processors with thousands of shader cores executing the same program on different data (SIMD). NVIDIA Ampere/Ada, AMD RDNA3, and Apple M-series GPUs differ in microarchitecture but share the same shader execution model.
Shader Cores (ALUs)
Each SM (NVIDIA) or CU (AMD) contains 64–128 shader processors. RTX 4090: 16,384 CUDA cores across 128 SMs. Peak: 82.6 TFLOPS FP32.
VRAM & Bandwidth
GPU memory bandwidth is the primary bottleneck for texture-heavy scenes. RTX 4090: 1008 GB/s. Texture caches (L1/L2) reduce bandwidth pressure.
Warp / Wavefront
32 threads (NVIDIA warp) or 64 threads (AMD wavefront) execute in lockstep. Divergent branching serializes execution — minimize branching in shaders.
Ray Tracing Cores
Dedicated BVH traversal hardware (RTX). Handles triangle-ray intersection at hardware speed. Enables real-time reflections, shadows, and GI.
// Frame Timing
Real-time graphics demands consistent frame delivery. At 60fps, each frame has 16.67ms budget. CPU submits draw calls; GPU executes them in parallel. GPU timing tools (NVIDIA NSight, RenderDoc) reveal which passes consume the most time.
CPU Frame Time
Game logic, animation, physics, draw call submission. CPU sends command buffer to GPU. Goal: ≤8ms for 60fps split between CPU and GPU.
GPU Frame Time
Shadow map rendering, G-buffer pass, lighting pass, post-processing. GPU stalls occur when VRAM bandwidth or shader ALU are saturated.
V-Sync & Tearing
V-Sync locks frame presentation to display refresh, preventing tearing at cost of latency. NVIDIA G-Sync / AMD FreeSync enable variable refresh rate.
GPU-Driven Rendering
Multi-Draw Indirect (MDI) lets GPU fetch draw calls from VRAM buffer, reducing CPU-GPU synchronization. Enables 10–100× draw call count increase.
// Lighting Systems
Real-time lighting has evolved from per-vertex Gouraud shading to physically-based deferred rendering with ray-traced global illumination. Each technique trades accuracy for performance within the frame budget.
Deferred Shading
Geometry pass writes material data to G-buffer (albedo, normal, roughness, depth). Lighting pass reads G-buffer — decouples light count from geometry complexity.
PBR / Cook-Torrance
Physically-based BRDF. Microfacet model: GGX distribution, Smith geometry function, Fresnel-Schlick approximation. Energy-conserving, artistically predictable.
Lumen (UE5)
Software and hardware ray-traced global illumination. Radiance cache for diffuse GI. Screen space for short-range reflections. Lumen proxy meshes for SDF tracing.
SSAO / GTAO
Screen Space Ambient Occlusion darkens corners and crevices. Ground Truth AO (UE5) uses temporal accumulation for higher quality at similar cost.
// Render Engines
Real-time render engines embed decades of rendering research into accessible pipelines. The gap between real-time and offline rendering has nearly closed — UE5 Nanite + Lumen at 60fps produces imagery indistinguishable from path-traced renders to most viewers.
Unreal Engine 5
Nanite virtualized geometry, Lumen GI, Chaos physics. Industry standard for virtual production, games, and architectural visualization.
Unity HDRP
High Definition Render Pipeline. Deferred rendering, volumetric fog, TAA. Excellent multi-platform targeting. C# scripting with Burst compiler.
Godot 4
Open source. Vulkan backend, Global Illumination via SDFGI and Voxel GI. GDScript + C# + C++. Lightweight, no licensing fees.
TouchDesigner
Real-time node-based. GLSL shaders, TOP/SOP pipelines, Vulkan backend (TD2023+). The live performance and installation standard.
// Procedural Rendering
Procedural techniques generate content algorithmically rather than storing it explicitly. Noise-based terrain, signed distance fields for geometry, procedural textures, and L-system vegetation reduce storage requirements while enabling infinite variation.
SDF Rendering
Signed Distance Fields represent geometry as distance functions. Ray marching through SDF enables complex implicit surfaces with zero triangles. Used in Unreal Nanite, font rendering, and AO.
Compute Shaders
General-purpose GPU computation. Particle systems, fluid simulation, physics, and procedural mesh generation all happen on GPU via compute passes.
Noise Functions
Perlin, Simplex, Worley, FBM. Basis for terrain, clouds, organic textures. GPU-optimized implementations run millions of noise samples per frame.
Mesh Shaders
DX12 Ultimate / Vulkan feature replacing vertex + geometry shaders. Amplification shader culls meshlets; mesh shader generates geometry. Powers Nanite.
// Practitioners
These engineers, researchers, and artists define the field — advancing the science of real-time rendering while making it accessible to creative practitioners.
John Carmack
BSP trees, portal rendering, lightmapping, MegaTexture. Invented much of what modern engines treat as given. Quake's renderer remains a study in ingenuity.
Tim Sweeney
Designed Unreal Engine's architecture. Pushed virtual production. Nanite and Lumen represent his vision of closing the gap with offline rendering.
Brian Karis
Authored UE4's PBR shading model (Real Shading in Unreal Engine 4, SIGGRAPH 2013). Lumen GI architecture. TAA temporal anti-aliasing systems.
Inigo Quilez
SDF ray marching, smooth minimum operators, procedural noise. Co-founded Shadertoy. His GLSL work defined creative coding aesthetics for a generation.
Graham Sellers
OpenGL SuperBible author. GPU-driven rendering advocacy. Persistent buffers, bindless textures — modern OpenGL techniques for reducing CPU overhead.
Naty Hoffman
Physics-based rendering theory. Co-wrote "Physically Based Rendering" with Matt Pharr and Greg Humphreys. Consultant to major game studios.
Alex Evans (Mmalex)
SDF-based rendering in Dreams (PS4). Traced a path from GPU voxel research to a consumer game with fully sculpted real-time geometry. SIGGRAPH 2015.
Patrick Cozzi
3D Tiles specification, glTF format. Real-time geospatial rendering at planetary scale. Making GPU-accelerated earth visualization accessible via WebGL/WebGPU.
// Glossary
- Vertex Shader
- Programmable GPU stage that transforms vertex positions from object space to clip space. Runs once per vertex. Outputs interpolated varyings to fragment stage.
- Fragment Shader
- Computes per-pixel color. Also called pixel shader. The primary tool for visual effects, texturing, and lighting in real-time rendering.
- Rasterization
- Conversion of vector geometry (triangles) to discrete pixels. Determines which screen pixels a triangle covers and interpolates vertex attributes.
- Deferred Shading
- Multi-pass technique storing geometry data in a G-buffer then computing lighting in a full-screen pass. Allows many lights at low cost.
- PBR
- Physically Based Rendering. Microfacet BRDF model that conserves energy and produces consistent results under all lighting conditions.
- G-Buffer
- Geometry Buffer. Set of textures storing albedo, normal, roughness, metalness, and depth for deferred shading. Typically 4–6 render targets.
- Draw Call
- CPU command to GPU to render a set of geometry. Minimizing draw calls reduces CPU overhead. GPU-driven rendering submits draws from GPU memory.
- TAA
- Temporal Anti-Aliasing. Accumulates sub-pixel samples across multiple frames to achieve smooth edges. Industry standard. Causes ghosting on fast motion.
- Nanite
- UE5 virtualized geometry system. Streams and renders as many triangles as there are pixels — no LOD transitions or polygon budgets for static meshes.
- Lumen
- UE5 dynamic global illumination. Combines ray-traced surface cache, radiance cache, and screen-space reflections for real-time GI at 60fps.
- Compute Shader
- GPU shader stage for general-purpose computation. No fixed-function pipeline. Used for physics, particles, post-processing, and procedural generation.
- SDF
- Signed Distance Field. Stores the nearest surface distance per voxel. Used for font rendering, soft shadows, AO, and implicit surface ray marching.
- VRAM
- Video RAM on the GPU. Stores textures, meshes, framebuffers, and shader data. Limited resource — typically 8–24GB. VRAM overflow causes stuttering.
- Warp
- NVIDIA term for 32 threads executing in lockstep on a SM. AMD equivalent: wavefront (64 threads). Divergent code paths serialize warp execution.
- BVH
- Bounding Volume Hierarchy. Tree structure for accelerating ray-geometry intersection. Required for ray tracing. Built on GPU for dynamic scenes.
- DLSS / FSR
- NVIDIA DLSS and AMD FSR use AI/spatial upscaling to render at lower resolution and reconstruct higher-resolution output. 2× or 4× performance gain.
- Mesh Shader
- Next-gen graphics API feature replacing vertex + geometry shaders. Amplification shader selects meshlets; mesh shader computes final geometry.
- Overdraw
- Rendering the same pixel multiple times due to overlapping transparent geometry. Major performance issue. Sort back-to-front or use OIT techniques.
- Occlusion Culling
- Skipping draw calls for geometry hidden behind other objects. Hardware occlusion queries, hierarchical Z-buffer, and HZB culling reduce wasted GPU work.
- WebGPU
- Modern successor to WebGL. Direct access to Vulkan/Metal/D3D12 from browser. Enables compute shaders and GPU-driven rendering on the web.