A request can reach the server even when the client never receives the response. Perhaps the connection timed out after a payment was recorded or an order was created. If the client retries blindly, the system could repeat the operation. Idempotency in APIs helps prevent this outcome: it lets the server recognize that multiple requests belong to the same business attempt and limit duplicate effects.
But adding a header with a key is not enough. You need to decide who generates it, how long it remains valid, which response to repeat, and how to coordinate local writes with external services. The goal is not to hide errors or promise exactly-once execution; it is to make retries safe under explicit conditions.
What idempotency means and what problem it solves
An operation is idempotent when repeating it with the same data does not change the final result after the first execution. For example, setting an order's shipping address to a specific value can be idempotent: applying the same update twice leaves the resource in the same state as applying it once.
By contrast, an operation such as “add one item to the cart” is not idempotent on its own: repeating it may increase the quantity twice. Creating an order or recording a charge can also produce additional effects if each request is treated as a new intention. For these operations, an idempotency key can associate retries with the same logical attempt.
Idempotency does not mean that every response is identical or that a request can never fail. It means controlling the effects of repeating a request according to a contract. A client may receive an error and, on retry, get the result already recorded; or it may receive the same definitive error again. What matters is that the server does not inadvertently execute a second effect.
When to add idempotency and when it is unnecessary
The risk arises when an operation has significant effects and the client cannot know for certain whether the server completed it. A lost response, timeout, disconnection, or automatic retry can make the outcome ambiguous. This is especially important when creating orders, initiating payments, reserving inventory, or processing requests.
Before adding keys, review the existing contract. An update that sets a specific state can be idempotent by design. A read query usually does not create this problem. By contrast, an endpoint that generates a new resource or transaction each time needs a clear strategy if its consumers may retry.
Idempotency has a cost: storing keys and results, defining expiration rules, managing concurrency, and testing more cases. Do not add it indiscriminately to every endpoint. Prioritize it where these three signals coincide:
- The operation can produce a duplicate effect that is difficult or costly to reverse.
- The client or infrastructure retries after transient failures.
- A lost response makes it impossible to tell whether the operation completed.
Also define what “same attempt” means for the business. Two intentional purchases that happen to have the same amount and contents must not be confused. Deduplication should rely on an attempt key, not an assumption that similar requests represent the same intention.
Designing a key: origin, uniqueness, scope, and duration
Typically, the client generates a unique key for each logical operation and retains it for all retries. It can send the key in an agreed header or in the request body, provided the contract is explicit. If it generates a new key on every retry, the server cannot link the requests. If it reuses a key for a new purchase, the server could block a legitimate intention.
The key identifies the attempt, but it does not replace authentication or authorization. Associate it with a scope, such as the authenticated account or merchant and the type of operation. This prevents an accidental match between two clients from mixing their results. The server should check these boundaries on every request.
Also store a fingerprint of the normalized request: the fields that determine the effect, with stable rules for defaults and representation. If a key already exists and a request arrives with incompatible content, reject it as a conflict; do not treat it as a valid retry or execute the new content. The fingerprint should exclude irrelevant data, but not details that change the intention, such as the amount or currency.
Duration depends on the behavior of clients, queues, and recovery processes. A window that is too short allows a late retry to repeat the effect; one that is too long accumulates records and may prevent legitimate reuse. Set a period that matches the longest reasonable retry window and explain what happens afterward. For financial or high-impact operations, you may need to retain a durable reference to the result beyond the operational window.
Responses to repeated keys and requests in progress
The contract should distinguish several cases. If the key has already finished and the fingerprint matches, the server can return the stored result of the original operation. This typically includes the relevant status code and body, though not necessarily every transport header. The response should let the client identify the created resource or the state reached.
If the key is still in progress, do not start a second execution. You can return a status indicating that processing is ongoing, or a temporary conflict that invites the client to check or retry later. The client needs a clear rule: how long to wait, whether to keep the same key, and how to obtain the final result. Do not report success for an operation that has not yet been confirmed.
If the key exists with a different fingerprint, return an explicit error and do not alter the original record. If the first execution failed, define which failures are saved as terminal results and which allow processing to resume. For example, a validation error may be definitive for that request, while an interruption before effects are confirmed may allow recovery. There is no universal policy: it must reflect the point at which the system can establish what happened.
Persistence, concurrency, and effects in other systems
When the key reservation and creation of the local effect share a database, coordinate them atomically. A uniqueness constraint on the scope and key helps ensure that two simultaneous requests do not both pass an initial check. The logic should handle the collision and read the state created by the winning request, rather than relying only on a “check, then insert” sequence.
Store understandable states, such as in progress, completed, and recoverable or definitive failure. Add timestamps and a recovery policy for records whose processing was interrupted. A lock that never expires can leave operations stuck; one that expires without safeguards can allow two workers to act at once. Recovery should verify the effect's state before resuming it.
The local transaction does not automatically include a payment provider or another remote service. If the system saves the order and then fails before calling the provider, or the provider processes the payment and the response is lost, you need to reconcile states. Where appropriate, use a transactional outbox to publish work after confirming the local change, and pass a stable reference to the external system if it supports deduplication. Record external identifiers and allow for lookups or reconciliation.
Do not promise end-to-end “exactly once” execution just because a key exists. Ambiguous failures can occur across networks, databases, and providers. The actual guarantee should describe which effects are deduplicated, within what scope, for how long, and which cases require intervention or reconciliation.
Common mistakes and a checklist
- A new key for every retry: The client must persist and reuse the key from the original attempt.
- A global key with no scope: Associate it with the authenticated client and the relevant operation.
- Different content under the same key: Compare a fingerprint and reject incompatible reuse.
- Expiration without considering late retries: Document the window and decide how older operations are recovered.
- An ambiguous response during concurrency: Specify how to check or retry while the first request remains active.
- Relying on the key to cover external systems: Include references, reconciliation, and handling for partial failures.
Before publishing the endpoint, check that the client generates one key per intention and retains it after a timeout; that two simultaneous requests with the same key do not duplicate the effect; and that the same key with different data does not execute a new operation. Also test failures before and after the write, process restarts, expiration, and lost responses.
Finally, monitor metrics for repeated keys, fingerprint conflicts, stuck operations, and discrepancies with external services. These signals help detect integration errors and adjust the retention window. A useful implementation does not eliminate failures: it makes explicit what can be safely repeated without duplicating effects and provides a safe path for resolving what remains uncertain.