Guide

Game replay systems explained

A replay is a time machine for gameplay: watch your best lap, share a clutch team fight, let spectators follow a tournament match, or let QA reproduce a bug that only appears on a specific frame. Under the hood, replay systems are one of two things — a recording of player inputs replayed through a deterministic simulation, or a sequence of world snapshots played back like a film. Input-based replays are tiny and editable (camera cuts, slow motion) but demand strict determinism. Snapshot replays are forgiving but balloon in size and hide simulation bugs. Fighting games, RTS titles, and rollback fighters push input recording to its limit; open-world action games often hybridize. This guide covers architecture choices, determinism requirements, ties to multiplayer netcode and anti-cheat verification, file formats, and what to ship before players trust your replay button.

Two architectures: inputs vs snapshots

Input replay stores a header (game version, map ID, RNG seed, character loadouts) plus a per-frame or per-tick log of controller state: buttons pressed, analog stick vectors, mouse deltas. Playback feeds those inputs into the same simulation loop. If the sim is bit-identical across machines, the replay reconstructs the original outcome — including RNG rolls derived from the seed. Storage cost scales with match length, not world complexity: a 20-minute fighting-game set might be tens of kilobytes.

Snapshot replay periodically serializes entity transforms, velocities, animation states, and AI blackboard values. Playback interpolates between keyframes. Snapshots tolerate non-deterministic physics and threaded job systems, but a 60 Hz full-world dump can reach megabytes per second. Compression (delta encoding, quantization) helps; so does recording only relevant entities. Many shipped games use a hybrid: input replay for players, sparse snapshots for cosmetic particles that do not affect outcome.

Ghost replays (racing, time trials) are input replays played back alongside the live player — translucent opponents following recorded lines. Ghosts require the same map version and physics constants; a patch that changes tire grip invalidates old ghosts unless you version and migrate files.

Determinism: the hard requirement for input replays

Determinism means: given identical initial state and input stream, the simulation produces identical state on every run. Break determinism once and replays desync — characters teleport, hits whiff, RNG diverges. Common culprits:

Floating-point order — parallel reductions on different CPU counts yield different rounding; fix with fixed-point math or deterministic summation order.
Unseeded or time-based RNG — rand() without a per-match seed, or physics engines with internal entropy.
Frame-rate dependent logic — movement tied to deltaTime without fixed timestep accumulators.
Non-deterministic container iteration — hashing over unordered maps whose iteration order varies by platform.
External async results — raycast completion order from multithreaded physics.

Fighting games and RTS titles often run a fixed simulation tick (e.g. 60 Hz logic, variable render) so network packets and replay frames align. Lockstep netcode — where every peer waits for every input before advancing — is essentially live multiplayer replay: each machine applies the same input log each frame. When latency hurts, rollback netcode speculates ahead, then rewinds and re-simulates when late inputs arrive; the same rewind machinery powers frame-step debugging in development.

Replay file structure and versioning

A portable replay file typically contains:

Magic bytes and schema version — reject or migrate stale files on load.
Build fingerprint — content hash or semantic version; replays from build 1.4.2 may desync on 1.4.3 if balance changed.
Initial state blob — map, mode, roster, loadout, starting RNG seed.
Input stream — frame index, player ID, bit-packed button mask, analog values; optional checksum every N frames.
Metadata — player names, match type, timestamp, platform for UI and leaderboards.

Versioning policy matters for live games. Hard invalidate deletes incompatible replays when physics patches ship — honest but frustrates ghost communities. Soft migrate replays old balance constants when possible. Sandbox playback runs old replays in an embedded legacy sim — expensive but preserves esports archives. Document which policy you chose; players blame "broken replays" on silent desyncs.

Compression: input streams compress well with LZ4 or zstd; bit-pack digital buttons; quantize analog sticks to 8–16 bits if full float precision is unnecessary. Sign replay files with HMAC if they feed ranked leaderboards — otherwise clients can forge world-record inputs.

Spectator modes and broadcast delay

Esports spectators need a camera that is not tied to a single player pawn. Architectures:

Delayed live stream — server relays state snapshots to observers 30–120 seconds behind to reduce stream sniping in competitive modes.
Director camera AI — heuristics follow action clusters, kill feeds, or objective nodes; human observers override in premium broadcasts.
Free camera replay — after match end, input replay drives sim while camera rig is decoupled — dolly paths, orbit, picture-in-picture.

Spectator bandwidth differs from player bandwidth: you may stream aggregated low-frequency poses for distant characters and full detail near the focal point. Tie spectator delay to matchmaking policy — ranked modes need longer delay than casual custom lobbies.

Replays for debugging, QA and anti-cheat

Internal replay tooling pays off faster than player-facing features. QA files a bug with a replay token; engineers step frame-by-frame, inspect RNG rolls, and bisect which input introduced desync. Automated nightly runs replay a corpus of golden matches after each build; any checksum mismatch blocks release.

Anti-cheat teams use server-stored input logs to verify leaderboard submissions: re-simulate on trusted hardware, compare final hash to client claim. Mismatches flag speedhacks, aimbots that inject impossible inputs, or memory edits. Replays are not sufficient alone — see our anti-cheat guide for server authority patterns — but they turn "trust me" disputes into reproducible evidence. Privacy: voice chat and text must be stripped or opt-in before uploading replays to cloud storage.

Playback UX: controls players expect

Player-facing replay viewers should expose:

Play/pause, frame step forward and back, scrub bar with tick marks for rounds or goals.
Speed control (0.25x–4x) without breaking audio pitch if commentary matters.
Camera modes: first-person, third-person, free fly, cinematic presets.
Hide UI option for content creators; watermark and match ID for clip attribution.
Export clip range to video — GPU capture of replay sim, not re-encoding compressed network video.
Share link or file with size limits; mobile clients may refuse multi-megabyte snapshot replays.

Fail gracefully when a replay is incompatible: show build version required, not a silent crash mid-round. Offer "download legacy viewer" for archived tournament sets if your live client moved on.

Genre and architecture decision table

Genre / need	Recommended approach	Main risk
Fighting game, RTS, turn-based tactics	Input replay + fixed tick + checksums	Desync on any non-deterministic patch
Racing time trial ghosts	Input replay + track version gate	Physics patch invalidates records
Fast FPS / battle royale highlight	Server snapshot + kill-cam clip	Storage cost; stream sniping if live
Open-world action RPG	Hybrid: input for player, snapshots for crowd	Partial desync on cosmetics only
Physics sandbox / destruction	Keyframe snapshots + compression	Non-deterministic engine limits input replay
Mobile casual / short sessions	Lightweight input log + cloud backup	OS kills app mid-record — autosave chunks
Esports broadcast	Server authoritative stream + delayed relay	Infra cost; director tooling complexity

Rollback netcode and replay: shared machinery

Rollback fighters (GGPO-style) keep a ring buffer of past states. When packet loss heals, the client loads state from frame n, reapplies inputs from n to now, and catches up before the next render. That is replay in reverse: rewind, re-simulate forward. If you build rollback for netcode, reuse the same state serialization for save states, replay seek, and QA frame step. One bug in state save/load breaks online play and offline replays simultaneously — which concentrates fix priority correctly.

Not every game needs rollback. Co-op PvE with generous input latency can use server authority with snapshot interpolation. Invest in deterministic input replay when replay fidelity, ghosts, or frame-perfect competition are product requirements — not because rollback is trendy.

Common mistakes

Recording render frames — huge files, no camera freedom, useless for debugging sim logic.
No checksum during playback — desync at frame 4,000 with no hint where divergence started.
Implicit RNG — replays break when any system calls non-seeded random without logging the seed in the header.
Patch without replay policy — leaderboard ghosts silently wrong after balance update.
Client-only replay validation — forged files reach ranked boards.
Ignoring pause and menu inputs — replay stalls or skips when pause not recorded.
Threaded physics in input replay — Heisenbugs that pass QA once and fail on player machines.
Unbounded replay storage on console — OS quota errors; cap count and size per user.

Production checklist

Architecture chosen (input, snapshot, hybrid) with documented tradeoffs for your genre.
Fixed simulation tick or deterministic constraints listed in engineering wiki.
Replay file schema versioned; build fingerprint in header.
RNG seed and initial state serialized; per-frame checksum optional but recommended for competitive modes.
Playback tested on min-spec hardware at 2x speed without desync.
Incompatible replay shows clear UI error with required client version.
Server-side re-simulation path for ranked submissions if applicable.
Spectator delay configured per mode; stream-sniping reviewed.
QA can attach replay ID to bug reports; frame step in dev build.
Storage quotas, cloud sync, and PII stripping policy documented.

Key takeaways

Input replays are small and flexible but require deterministic simulation; snapshots are easier but heavy.
Rollback netcode and replay share state save/load and re-simulation — build both together when either is needed.
Versioning and checksums prevent silent desync and forged leaderboard files.
Ghost racers and esports need explicit policies when patches change physics or balance.
Replays are forensic tools for QA and anti-cheat, not just a player vanity feature.