Guide

LLM knowledge editing explained

Harbor Support shipped a new refund window — 30 days instead of 14 — on a Tuesday morning. The policy PDF was in the RAG index within an hour, but the fine-tuned 7B assistant still answered “14-day refund” on 41% of paraphrased billing prompts because the old rule was baked into weights from last quarter's SFT corpus. Re-running full fine-tuning would take a GPU week and risk regressions on unrelated tone and safety behaviors. Engineers applied knowledge editing: a batch of 37 rank-one weight updates (MEMIT-style) targeting the factual association “Harbor refund period → 30 days,” validated on 200 held-out phrasings. Post-edit efficacy on the target fact hit 94%; unrelated MMLU slices moved less than 0.3 points. Combined with refreshed retrieval, wrong-window answers on billing traffic fell from 41% to 6% within one deploy cycle.

Knowledge editing (also called model editing) updates specific factual or behavioral associations inside a frozen language model without retraining on the full dataset. It sits between LoRA adapters (which learn broad skill deltas) and RAG (which adds external context at inference but does not change what the model “believes” when context is thin). This guide covers the locate-then-edit paradigm, ROME and MEMIT rank-one updates, hypernetwork and meta-learning editors, evaluation metrics (efficacy, specificity, generalization, locality), the Harbor Support policy refactor, a technique decision table against fine-tuning and retrieval, pitfalls, and a production checklist alongside hallucination mitigation patterns.

What knowledge editing solves

Large language models store facts implicitly in billions of parameters. When a fact changes — CEO succession, API pricing, medical guideline revision, product policy — teams face three naive options:

RAG only — inject the new fact at query time. Works when retrieval always fires and the model follows context. Fails on paraphrased questions, multi-hop reasoning without the chunk, or adversarial “ignore documents” jailbreaks.
Full or LoRA fine-tune — relearn from a dataset containing the update. Scales poorly to hundreds of isolated fact changes; each run can shift unrelated behaviors (alignment drift, style change).
Prompt engineering — system message says “refund is 30 days.” Cheap but brittle across template variants, tool calls, and long contexts where instructions get diluted.

Knowledge editing targets a fourth path: surgically rewrite the internal representation of one fact while leaving neighboring knowledge intact. The research goal is high efficacy (the model outputs the new value), specificity (unrelated facts stay correct), generalization (paraphrases and entailed questions work), and locality (downstream layers and unrelated prompts are unchanged).

Locate-then-edit: finding where facts live

Most editing methods assume facts are localized in mid-layer MLP modules (feed-forward blocks) of transformer decoders. The pipeline:

Causal tracing — corrupt the subject token embedding in a factual prompt (“The CEO of Harbor is ___”) and measure which layer restores the correct logit when uncorrupted. Peaks indicate storage sites.
Rank-one update hypothesis — a fact may correspond to a low-rank change in a weight matrix W. Methods like ROME solve for vectors u, v so W' = W + u v^T shifts the model's answer for prompts about subject s to new object o'.
Key-value formulation — treat the subject's hidden state as a key and the desired factual association as a value to inject at the located layer.

Location is model- and fact-dependent. Editing layer 5 may fix “capital of France” but fail on “France's largest city.” Production stacks run layer sweeps on a validation set before batch edits.

ROME: single-fact rank-one editing

ROME (Rank-One Model Editing) edits one relational fact at a time, e.g. (s, r, o) → (s, r, o') where s is subject, r relation, o old object, o' new object.

Identify the critical MLP layer via causal tracing on prompts like “s r”.
Compute a rank-one delta to the MLP's W_out matrix that moves the predicted object token distribution toward o'.
Apply the closed-form update; no gradient steps on the full model.

ROME is fast (seconds per fact on 7B models) and interpretable, but sequential single-fact edits can interfere: editing fact B may partially undo fact A. That motivates mass-editing methods.

MEMIT and batch mass editing

MEMIT (Mass-Editing Memory in a Transformer) extends ROME to hundreds or thousands of facts in one pass by solving a joint optimization over multiple rank-one updates across layers. Key ideas:

Shared layer selection — group edits that trace to the same MLP blocks to reduce cross-talk.
Concurrent update solve — stack constraints from all (s, r, o') tuples and solve for deltas that satisfy each with minimal Frobenius norm perturbation.
Covariance regularization — penalize updates that move activations on unrelated prompts (improves locality).

Harbor Support's 37 policy edits used a MEMIT variant: refund window, shipping cutoff times, and tier-specific SLA strings. Batch editing took 12 minutes on one A100 versus an estimated 18 GPU-hours for a targeted LoRA retrain on the same fact set.

Other editor families

MEND / SERAC — hypernetworks or small auxiliary networks predict weight deltas from edit examples; amortize cost when edits stream in over time.
FT-L — fine-tune only specific layers on a micro-dataset for one fact; simple baseline that often overfits and hurts locality.
In-context editing — prepend demonstrations of the new fact; not weight editing but a strong comparator for efficacy benchmarks.

Evaluation metrics that matter in production

Academic benchmarks (CounterFact, zsRE) use templated prompts. Production needs custom suites:

Metric	Definition	Harbor threshold
Efficacy	Model outputs `o'` on direct queries about `s, r`	≥ 90% on 50+ paraphrases
Generalization	Entailed and indirect questions (“Can I return after 3 weeks?”)	≥ 85%
Specificity	Unrelated facts from a frozen regression set stay correct	≤ 2% drop on 500-prompt suite
Locality	Perplexity and task accuracy on general corpora unchanged	< 0.5% perplexity drift
Portability	Multi-hop chains using the edited fact	Tracked; not a launch gate for v1

Run edits against a canary model shard before promoting weights to the serving fleet. Keep the pre-edit checkpoint for one-click rollback.

Harbor Support policy refactor (worked example)

Edit specification — 37 tuples: subject “Harbor Support refund,” relation “eligible period,” new object “30 calendar days,” plus 12 regional shipping facts.
Tracing — causal trace on Mistral-7B SFT; primary edit layers 4–6 MLP W_out.
MEMIT batch solve — joint rank-one updates with covariance penalty; 12 minutes on one A100.
Validation — 200 paraphrased billing prompts (efficacy 94%), 500-prompt regression (specificity 98.1% unchanged), human spot-check on 50 live tickets.
Deploy — merged edited weights into serving image; RAG index refreshed in parallel; system prompt updated as belt-and-suspenders.
Monitor — weekly audit on billing intent; alert if “14-day” string reappears in model outputs.

They did not edit safety-refusal behaviors or tone — only verifiable policy constants. Subjective alignment still flows through preference training, not MEMIT.

Technique decision table

Approach	Best when	Trade-off
RAG / retrieval	Large doc corpus, facts change often, citations required	Model may ignore context; latency and index ops
LoRA / SFT fine-tune	Broad skill or style change, thousands of examples	Expensive; risks catastrophic forgetting and alignment drift
ROME (single edit)	One urgent factual correction, research or hotfix	Sequential edits interfere; manual layer tuning
MEMIT (batch edit)	Dozens–thousands of structured fact updates on same checkpoint	Still research-grade tooling; portability limits
Hypernetwork editor (MEND)	Continuous stream of small edits, amortized predictor	Training the editor itself; generalization to new model sizes
System prompt only	Low stakes, fast iteration, human-in-the-loop chat	Weakest paraphrase and long-context robustness

Common pitfalls

RAG without weight edit — retrieval fixes doc-grounded answers but parametric memory wins on paraphrase; use both for policy constants.
Editing subjective facts — “best camera” is not a crisp tuple; editing fails or overfits. Reserve for verifiable relations.
Layer guesswork — skipping causal trace causes 30–50% efficacy drops. Automate sweeps.
Edit accumulation — 500 sequential MEMIT rounds degrade MMLU; periodic full retrain or checkpoint refresh still required.
No rollback artifact — edited weights are hard to diff; always version checkpoints and keep pre-edit binary.
Quantized models — INT4 GPTQ weights break rank-one assumptions; edit in FP16, then re-quantize and re-validate.
Multilingual gaps — edit trained on English prompts may not generalize to Spanish billing questions; localize validation sets.
Conflicting facts — editing A→B while corpus still teaches B→A causes oscillation; sync training data and edits.

Production checklist

Express each change as a structured (subject, relation, new_object) tuple with provenance.
Run causal tracing or automated layer search before any weight write.
Build paraphrase and entailment eval sets (50+ prompts per fact minimum).
Hold a 500+ prompt regression suite for specificity and locality.
Apply edits on a canary shard; compare against frozen baseline metrics.
Version pre-edit and post-edit weight checkpoints with immutable storage.
Refresh RAG index and system prompts in the same release train.
Re-quantize and run perplexity + task benchmarks after FP16 edits.
Monitor target strings and entailed questions in production logs weekly.
Plan full retrain when edit count or regression drift exceeds policy thresholds.

Key takeaways

Knowledge editing patches specific facts in model weights — RAG patches context at inference; production policy changes often need both.
ROME solves rank-one MLP updates for single facts; MEMIT batches hundreds of constraints with less cross-interference than sequential ROME.
Efficacy without specificity is useless — always run unrelated-fact regression before shipping edited weights.
Harbor Support cut wrong refund-window answers from 41% to 6% by MEMIT-editing 37 policy tuples plus retrieval refresh, avoiding a full GPU-week retrain.
Edited checkpoints accumulate drift — treat editing as a bridge between full retrains, not a permanent substitute.