Back to Engineering Notes
Advanced Software EngineeringEngineering Note

Idempotency in Distributed Systems

Idempotency means repeating the same operation does not create extra side effects beyond the first successful result.

Objective

Understand idempotency and why it is essential for retries, background jobs, payment flows, and other distributed operations.


What Idempotency Means

An operation is idempotent if running it multiple times produces the same effective result as running it once.

This matters because distributed systems retry requests when networks fail, time out, or return uncertain results.


Where It Matters

Important use cases include:

Payment or billing requests
Webhook consumers
Queue workers
Form submissions
External API integrations

Without idempotency, retries can create duplicate records, duplicate charges, or repeated side effects.


Common Implementation Techniques

Idempotency Keys

The client or upstream service sends a unique key per logical operation. The server stores the outcome and reuses it if the same key is received again.

Unique Constraints

The database enforces that the same logical action cannot be inserted twice.

State Checks

Before applying a change, the system verifies whether the intended state already exists.


Engineering Considerations

Idempotency is not only a controller concern. It often crosses several layers:

API contract
Database design
Background job semantics
External provider behavior

A retry-safe endpoint can still fail overall if downstream processing is not also retry-safe.


Practical Rule

If an operation can be retried automatically or manually, design it as idempotent from the beginning.