← Back to blog

Why Stripe Never Charges You Twice (Idempotency Keys)

·
#distributed systems#payments#stripe

A customer’s card gets charged twice for the same $50 order. Not because of fraud — the payment API received the request, processed it, and the network dropped before the response made it back. The client saw a timeout and did the only reasonable thing: it retried.

I’ve debugged this exact failure enough times to know it’s not a corner case. At any real scale, network timeouts happen constantly, and “just don’t retry” isn’t a real answer — the alternative is a payment that silently disappears and a customer who never gets their order. So the actual question isn’t whether retries happen. It’s how you make every retry safe, even if the same request arrives ten times.

The Problem, Properly Framed

The obvious “fix” is to deduplicate on the server: if a charge comes in with the same amount, same card, and roughly the same timestamp, treat it as a duplicate. This breaks immediately in production — a customer buying two coffees on the same card within the same minute is a completely legitimate case that looks identical to a duplicate retry. You can’t distinguish “the same request, twice” from “two different requests that happen to look similar” using the request’s contents alone.

The real problem is deeper: the server has no way to know “have I already done this exact request?” A timeout, a retry, a double-click from an impatient user — to the server, these are all indistinguishable from a brand-new charge. Fixing this requires giving the server a piece of information the request itself can’t provide: the client’s intent, separate from the network call that carries it.

The Core Mechanics

The Idempotency Key Is a Contract, Not a Header

The client generates a unique key — a UUID — exactly once, before it ever sends the first attempt. Every retry of that same logical request reuses that same key. This is the mental model shift that most explanations skip: the key doesn’t represent the HTTP call. It represents the user’s intent. Generate a new key on every retry attempt and the entire mechanism is void — the server sees each retry as a first-time request.

sequenceDiagram
    participant Client
    participant API as Payment API
    participant DB as Charges DB

    Client->>API: POST /charge (amount, card)
    API->>DB: insert charge
    DB-->>API: success
    API-->>Client: 200 OK
    Note over Client,API: Network timeout, no response received
    Client->>API: POST /charge (retry, same params)
    API->>DB: insert charge (again)
    DB-->>API: success
    API-->>Client: 200 OK
    Note over DB: Two charges recorded for one purchase

This is what happens with no protection at all — the diagram above is the naive flow, and it’s the default behavior of any API that doesn’t explicitly design against it. Every retry is a brand-new charge, because the server has no memory of the first attempt.

The Database Constraint Does the Real Work

The fix isn’t clever retry logic in application code — it’s a unique constraint at the database layer.

-- migrations/003_idempotency_keys.sql
CREATE TABLE idempotency_keys (
    key TEXT PRIMARY KEY,
    request_hash TEXT NOT NULL,
    status TEXT NOT NULL DEFAULT 'processing',
    response JSONB,
    created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);

The primary key on key is doing the actual work here — not the application logic that reads from this table, but the constraint the database enforces on writes to it.

sequenceDiagram
    participant Client
    participant API as Payment API
    participant IK as Idempotency Store
    participant DB as Charges DB

    Client->>API: POST /charge (Idempotency-Key: abc123)
    API->>IK: insert key abc123 (status: processing)
    IK-->>API: inserted (unique constraint OK)
    API->>DB: create charge
    DB-->>API: success
    API->>IK: update abc123 → response + status: complete
    API-->>Client: 200 OK (charge created)

Here’s the handler that implements it:

# app/payments/idempotency.py
import hashlib
import json

def handle_charge(request, db):
    key = request.headers["Idempotency-Key"]
    request_hash = hashlib.sha256(
        json.dumps(request.body, sort_keys=True).encode()
    ).hexdigest()

    try:
        db.execute(
            "INSERT INTO idempotency_keys (key, request_hash, status) "
            "VALUES (%s, %s, 'processing')",
            (key, request_hash),
        )
    except UniqueViolation:
        existing = db.fetch_one(
            "SELECT request_hash, response, status FROM idempotency_keys WHERE key = %s",
            (key,),
        )
        if existing.request_hash != request_hash:
            raise ConflictError("Idempotency key reused with different parameters")
        if existing.status == "processing":
            raise ConflictError("Request still in flight, retry later")
        return existing.response

    charge = process_charge(request.body)
    db.execute(
        "UPDATE idempotency_keys SET response = %s, status = 'complete' WHERE key = %s",
        (json.dumps(charge), key),
    )
    return charge

Notice that the request body is hashed and checked too — not just the key. If the same key shows up with different parameters, that’s not a network retry, it’s a client bug, and it gets rejected rather than silently processed. The insert is the lock. Everything downstream of it — the hash check, the in-flight rejection, the cached response — exists because that one unique constraint is the only thing standing between one charge and two.

The Race Condition Nobody Talks About

Most explanations stop at “check if the key exists, and skip processing if it does.” That has a race condition: if two retries arrive concurrently, both before the first one finishes, both requests can check “does this key exist?”, both get “no,” and both proceed to charge the card.

stateDiagram-v2
    [*] --> NoKey: no request yet
    NoKey --> Processing: first request inserts key (unique constraint wins)
    Processing --> Processing: concurrent retry hits constraint violation while in-flight — must wait or reject, not double-process
    Processing --> Complete: charge succeeds, response cached
    Processing --> Failed: charge fails, response cached
    Processing --> Abandoned: processing TTL (30s) elapses, no resolution — server likely crashed mid-request
    Abandoned --> Processing: reaper clears the stuck lock, next retry is allowed to reprocess
    Complete --> Expired: 24h window elapses
    Failed --> Expired: 24h window elapses
    Expired --> [*]: retry after expiry = brand-new request

The fix is the same constraint, applied to concurrency instead of just sequential retries: the second insert doesn’t quietly check-then-skip, it fails with a constraint violation, and that failure is the signal to fetch and return the cached response instead of processing again. A key sitting in processing state is the race window — any concurrent request hitting that same key during that window has to be rejected or made to wait, not allowed to proceed as if it were new.

There’s a second failure mode hiding in that same processing state, and it’s the one most explanations of this pattern skip entirely: what happens if the server crashes right after inserting the key but before the charge resolves? Nothing in the design above ever moves that key out of processing. Every future retry — including a completely legitimate one from a customer who never got their confirmation — gets rejected as “still in flight,” forever. The exact mechanism built to stop duplicate charges ends up permanently blocking a real customer instead.

The fix is a processing TTL: if a key has been sitting in processing for longer than a short window — 30 seconds is a reasonable starting point for a payment API — a background reaper treats it as abandoned and clears it, so the next retry is allowed to reprocess from scratch. Without this, the failure mode isn’t theoretical; it’s the first incident you’ll have with this design in production.

The Tradeoffs

None of this is free, and pretending otherwise is how you end up debugging this in production instead of designing around it up front.

The idempotency store becomes a single point of failure in the critical path. If that table or its underlying database is unreachable, you can’t safely process any payment — and the correct behavior is to fail closed, refusing the charge, not fail open and process it optimistically. This has to be decided at design time, not improvised during an incident.

Keys can’t live forever. Stripe expires them after 24 hours. That’s a real trade-off — a shorter window means less storage overhead, but it also means a retry that arrives 25 hours later is treated as a brand-new request, with no protection at all.

The single most common implementation bug is generating the idempotency key per HTTP call instead of per logical operation. If the client’s retry logic regenerates a new key on every attempt — which is the natural thing to do if you’re thinking about “the request” instead of “the intent” — the entire mechanism does nothing. This is the failure mode that actually shows up in production, not the theoretical one.

Not every write needs this. For operations that are already naturally idempotent — setting a status to a fixed value, for instance — this machinery is unnecessary complexity. Reserve it for operations where “doing it twice” has a real cost: charging a card, sending a payment, decrementing inventory.

The observability nobody budgets for. You need alerting on the constraint-violation rate and a dashboard tracking keys stuck in processing past the TTL — otherwise a stuck-key bug is invisible right up until customers start complaining that their retries are failing.

Takeaways

  • Retries are inevitable at scale. Treat every write endpoint as if the client will call it twice, because eventually it will.
  • The key represents intent, not the HTTP call. Generate it once, before the first attempt — regenerating it per retry silently defeats the whole mechanism.
  • The database constraint is the actual lock. Application-level check-then-insert has a race window; a unique constraint at the DB layer doesn’t.
  • The failure mode of the idempotency store is a design decision, not an incident-time improvisation. Fail closed, because a duplicate charge is worse than a rejected one.
  • Expiration is a real trade-off, not an afterthought — pick the window based on how long a legitimate retry could plausibly take.
  • A crash mid-request must not permanently lock a key. Without a processing TTL and a reaper, the mechanism meant to stop duplicates ends up blocking a legitimate customer forever — this is the failure mode most explanations skip.
  • Skip this for naturally idempotent operations. The machinery is only worth it where duplication has a real cost.

Idempotency keys look like a small implementation detail — a header, a database table — but they’re really a statement about what a “request” even means in a distributed system: not the network call, but the intent behind it. Get that distinction right, and duplicate charges stop being an incident category.

If you found this useful or want to discuss it further, connect with me on GitHub or LinkedIn.