Idempotency in APIs: Making Retries Safe

Network requests fail. Timeouts happen. Clients retry. If your API is not idempotent, a retry can create duplicate orders, double-charge customers, or send duplicate notifications. Idempotency means that making the same request multiple times produces the same result as making it once. Some HTTP methods are idempotent by definition (GET, PUT, DELETE). POST is not — and that is where you need to do the work.

The Problem

Consider a payment endpoint:

Client sends POST /api/payments with a payment request.
The server processes the payment, charges the card.
The response is lost due to a network timeout.
The client retries the same POST /api/payments.
The server processes the payment again — the customer is charged twice.

The client did the right thing by retrying. The server failed to recognise the duplicate.

Idempotency Keys

The standard solution is an idempotency key — a unique identifier the client generates and sends with each request. The server stores the key alongside the result. On subsequent requests with the same key, the server returns the stored result instead of processing again.

Example.cs

app.MapPost("/api/payments", async (
    [FromHeader(Name = "Idempotency-Key")] string idempotencyKey,
    CreatePaymentRequest request,
    AppDbContext db,
    IPaymentProcessor processor,
    CancellationToken ct) =>
{
    // Check for existing result
    var existing = await db.IdempotencyRecords
        .FirstOrDefaultAsync(r => r.Key == idempotencyKey, ct);

    if (existing is not null)
    {
        return Results.Json(
            JsonSerializer.Deserialize<object>(existing.ResponseBody),
            statusCode: existing.StatusCode);
    }

    // Process the payment
    var result = await processor.ChargeAsync(request, ct);

    var payment = new Payment
    {
        Amount = request.Amount,
        Currency = request.Currency,
        Status = result.Success ? PaymentStatus.Completed : PaymentStatus.Failed
    };

    db.Payments.Add(payment);

    // Store the idempotency record
    var responseBody = JsonSerializer.Serialize(payment);
    var statusCode = result.Success ? 201 : 422;

    db.IdempotencyRecords.Add(new IdempotencyRecord
    {
        Key = idempotencyKey,
        StatusCode = statusCode,
        ResponseBody = responseBody,
        CreatedAt = DateTime.UtcNow
    });

    await db.SaveChangesAsync(ct);

    return Results.Json(payment, statusCode: statusCode);
});

The IdempotencyRecord entity:

Example.cs

public class IdempotencyRecord
{
    public required string Key { get; set; }
    public int StatusCode { get; set; }
    public required string ResponseBody { get; set; }
    public DateTime CreatedAt { get; set; }
}

Race Conditions

The naive approach above has a race condition. If two identical requests arrive simultaneously, both might pass the "check for existing" step before either writes the record. Use a unique constraint on the idempotency key and handle the conflict:

Example.cs

try
{
    await db.SaveChangesAsync(ct);
}
catch (DbUpdateException ex) when (IsUniqueConstraintViolation(ex))
{
    // Another request with the same key was processed concurrently
    var existing = await db.IdempotencyRecords
        .FirstAsync(r => r.Key == idempotencyKey, ct);

    return Results.Json(
        JsonSerializer.Deserialize<object>(existing.ResponseBody),
        statusCode: existing.StatusCode);
}

The database unique constraint is the source of truth, not the application-level check.

Middleware Approach

For a cleaner separation of concerns, extract idempotency handling into middleware or an endpoint filter:

IdempotencyFilter.cs

public class IdempotencyFilter(AppDbContext db) : IEndpointFilter
{
    public async ValueTask<object?> InvokeAsync(
        EndpointFilterInvocationContext context,
        EndpointFilterDelegate next)
    {
        var httpContext = context.HttpContext;
        var key = httpContext.Request.Headers["Idempotency-Key"].FirstOrDefault();

        if (string.IsNullOrEmpty(key))
        {
            return Results.BadRequest(new ProblemDetails
            {
                Title = "Missing Idempotency-Key",
                Detail = "The Idempotency-Key header is required for this endpoint.",
                Status = 400
            });
        }

        var existing = await db.IdempotencyRecords
            .FirstOrDefaultAsync(r => r.Key == key);

        if (existing is not null)
        {
            httpContext.Response.Headers["X-Idempotent-Replay"] = "true";
            return Results.Json(
                JsonSerializer.Deserialize<object>(existing.ResponseBody),
                statusCode: existing.StatusCode);
        }

        // Proceed with the actual handler
        var result = await next(context);

        // Store the result (simplified — production code would capture the response)
        return result;
    }
}

Apply it to specific endpoints:

Example.cs

app.MapPost("/api/payments", HandleCreatePayment)
    .AddEndpointFilter<IdempotencyFilter>();

Key Design Decisions

Key format: Use UUIDs (GUIDs). They are globally unique without coordination. The client generates them.

Key scope: Scope keys to the authenticated user or API key. Two different users sending the same idempotency key should not conflict.

Expiration: Do not keep idempotency records forever. Expire them after a reasonable period — 24 to 48 hours is typical. Run a background job to clean up:

IdempotencyCleanupJob.cs

public class IdempotencyCleanupJob(AppDbContext db) : BackgroundService
{
    protected override async Task ExecuteAsync(CancellationToken ct)
    {
        while (!ct.IsCancellationRequested)
        {
            var cutoff = DateTime.UtcNow.AddHours(-48);
            await db.IdempotencyRecords
                .Where(r => r.CreatedAt < cutoff)
                .ExecuteDeleteAsync(ct);

            await Task.Delay(TimeSpan.FromHours(1), ct);
        }
    }
}

Response fidelity: Store the complete response body and status code. The replayed response must be identical to the original.

Which Endpoints Need Idempotency?

POST endpoints that create resources or trigger side effects — always.
PUT endpoints — usually idempotent by nature (full replacement), but verify.
PATCH endpoints — depends on whether the patch is absolute or relative. Incrementing a counter with PATCH is not idempotent.
DELETE — idempotent by definition. Deleting something that does not exist returns 404, which is fine.
GET — always idempotent. No action needed.

Idempotency is not optional for production APIs that handle financial transactions, inventory changes, or any operation where duplication has real-world consequences. The implementation cost is modest; the alternative is debugging duplicate charges at 2 AM.