Idempotency explained: safe retries, API keys, and distributed systems

What idempotency means

Formally: for operation f and state S, applying f(S) once and applying f(f(S)) yield the same final state. In software terms, the second identical request should not create a second charge, a second shipment, or a second row where one was intended.

Idempotency is not the same as “read-only.” Deleting the same resource twice is idempotent — the first delete removes it, the second finds nothing and still succeeds (often with 404 or 204). Creating a user with the same email twice is not idempotent unless you define semantics that collapse duplicates into a single outcome.

Idempotency vs deduplication

Deduplication is the mechanism: storing a fingerprint of processed work and skipping repeats. Idempotency is the guarantee you want callers to rely on. You implement deduplication (unique keys, caches, conditional writes) to achieve idempotent behavior. The terms overlap in conversation but the distinction matters when designing APIs: publish an idempotency contract, implement deduplication behind it.

Why networks force you to care

TCP guarantees delivery attempts, not application-level success. A client sends POST /payments, your server processes the charge, then the response times out before the client reads it. The client retries. Without idempotency, you have two charges and one confused customer.

The same failure modes appear everywhere:

• Mobile and browser clients — users double-tap submit; service workers replay requests.
• Load balancers and gateways — upstream timeouts trigger automatic retries on “safe” methods, sometimes incorrectly including POST.
• Webhooks — providers retry until they receive 2xx; your handler must tolerate duplicates.
• Message queues — at-least-once delivery is the default; exactly-once end-to-end is a distributed-systems myth without careful design.
• Batch jobs and cron — overlapping runs or crash recovery re-process the same file slice.

The fix is not “never retry.” Retries are how reliable systems recover. The fix is making the operation safe to repeat.

HTTP methods and natural idempotency

The HTTP specification assigns idempotency expectations to verbs. Clients, proxies, and libraries use these rules when deciding whether a failed request can be replayed automatically.

PUT idempotency is powerful: PUT /users/42 with the same body twice leaves user 42 in the same final shape. Contrast POST /users, which typically allocates a new ID on every call. That is why payment and order APIs almost always add explicit idempotency on top of POST — see our REST API design guide for how keys fit into error shapes and status codes.

Safe vs unsafe retries in practice

A safe method (GET) should not change server state. An idempotent method may change state, but repeating it is harmless. POST is neither safe nor idempotent by default. Document which endpoints accept retries and which require an idempotency key header.

Idempotency keys: the Stripe pattern

Payment processors popularized the idempotency key: a unique string the client generates once per logical operation and sends on every attempt (usually header Idempotency-Key or X-Idempotency-Key). The server stores the key with the result of the first successful processing and returns the cached response for duplicates within a TTL window.

Server-side flow

1. Receive request with key K and payload P.
2. Look up K in an idempotency store (Redis, Postgres table).
3. If found and completed, return stored response (same status + body).
4. If found and in-flight, return 409 or block until the first attempt finishes.
5. If not found, insert K as processing, execute business logic inside a transaction, persist result, mark completed.

Keys should be opaque UUIDs generated by the client — not derived from amount or SKU alone (two different purchases could collide). Scope keys per merchant or user so one tenant cannot probe another's keys. Expire records after 24–72 hours; after expiry, a repeat with the same key may execute again, which is acceptable if the client only retries within minutes.

What to store

Cache enough to reconstruct the HTTP response: status code, headers that matter, and body. For long-running operations, return 202 on first accept and let duplicates poll the same job ID. Hash the request body and reject keys reused with different payloads (400) — otherwise a buggy client could charge $10 then retry the same key with $100.

Database patterns for idempotent writes

Keys at the API edge are not enough if your worker or queue consumer writes directly. Push idempotency to the layer that mutates state.

Unique constraints

Add a unique index on a business-natural key: (order_id, event_type), external_payment_id, or webhook_delivery_id. The second insert fails with a constraint violation — catch it and treat as success. This is the simplest durable deduplication for SQL backends.

Upserts and conditional writes

INSERT ... ON CONFLICT DO NOTHING (Postgres) or equivalent upsert semantics let you record “we processed message X” exactly once. Conditional updates — “set status = shipped only where status = paid” — make state transitions idempotent: repeating the ship command on an already-shipped order changes zero rows and exits cleanly.

Outbox and inbox tables

In event-driven systems, an inbox table stores incoming event IDs before processing. Workers insert the ID first; duplicate deliveries hit the unique constraint and skip. Pair with transactional outbox on the publisher side so you never emit an event you did not commit.

Message queues and at-least-once delivery

Kafka, RabbitMQ, and SQS typically guarantee at-least-once: messages may arrive more than once, especially after consumer crashes between processing and acknowledgment. “Exactly-once” marketing usually means exactly-once processing semantics achieved by idempotent consumers plus transactional offsets — not magic transport.

Design consumers as: parse message, derive idempotency key, attempt side effect inside idempotent boundary, ack. Side effects outside the database (send email, charge card) need the same key stored before the external call. If the process dies after charging but before ack, the redelivery must see the stored key and skip the charge.

Ordering vs idempotency

Partitioning by user_id preserves order per user but not global order. Idempotency handles duplicates; it does not fix consuming event B before event A. For strict ordering, combine single-partition keys with version numbers or state machines that reject stale transitions — topics covered in distributed systems consistency.

Webhooks and external callbacks

Webhook senders include a delivery ID or event ID in the payload. Your handler should:

• Verify signatures first (see webhook guide) — reject forgeries before dedup.
• Persist the event ID before heavy work.
• Return 200 quickly once durably recorded; process async if needed.
• On duplicate ID, return 200 immediately without re-running side effects.

Returning 200 on duplicates is critical: if you return 500 because “already processed,” the sender retries forever. Idempotent success looks identical to first-time success from the sender's perspective.

Blockchain and payment verification

On-chain payments add another retry surface: clients poll “did my tx land?” and servers scan mempools. Credit an order exactly once per transaction signature, not per HTTP request. Store tx_signature as a unique key before fulfilling digital goods. The same pattern applies when verifying Solana Pay or wallet transfers — signature is the natural idempotency key. See verify Solana payment for end-to-end flow.

Common mistakes

• Idempotency only in memory — restarts lose the map; duplicates slip through after deploy.
• Same key, different body — must return 400, not silent wrong results.
• Non-atomic check-then-act — two workers both pass the lookup before either inserts; use unique constraints or transactions.
• Caching error responses — if the first attempt failed transiently, some designs delete the key; others cache 5xx and block forever. Prefer short in-flight locks and retryable errors without marking completed.
• Assuming GET is always safe — poorly designed GET endpoints that trigger exports or emails break cache and retry assumptions.
• Ignoring PATCH and custom POST actions — document and implement keys for any mutating endpoint clients retry.

Production checklist

• List all mutating endpoints and webhook handlers; mark which are naturally idempotent.
• Require idempotency keys on POST that create money movement, inventory holds, or irreversible external calls.
• Store keys with response snapshots in a durable store; scope per tenant; TTL aligned with client retry policy.
• Add unique indexes on external IDs at the database layer as a backstop.
• Make queue consumers insert a processed-message row before side effects.
• Test duplicate delivery explicitly — unit tests and chaos-style double-submit integration tests.
• Log and metric duplicate suppressions so you can detect buggy clients without treating them as errors.

Idempotency is invisible when it works and headline news when it does not. Investing once in keys, constraints, and inbox tables pays off across every integration you ship afterward.

Related reading

• REST API design explained — verbs, status codes, and where idempotency keys fit in API contracts
• Webhooks explained — signature verification, retries, and duplicate delivery handling
• Message queues explained — delivery guarantees and idempotent consumer design
• Distributed systems consistency explained — ordering, CAP, and why exactly-once is hard