Atomicity in databases and Redis: the bug that never shows up with one user

Most of the data bugs I have dealt with did not come from wrong logic. The logic was right, the tests were green, the review was fine. It only broke when two things ran at once: two workers picking up the same job, a user double-clicking a button, a cron job overlapping a request from the UI.

The pattern is almost always the same — read, compute, write:

budget := repo.GetBudget(ctx, campaignID)   // read
budget = budget - spend                      // compute
repo.SetBudget(ctx, campaignID, budget)      // write

Those three lines are correct for one process. With two, both read 100, both compute 70, both write 70 — and one deduction disappears without a trace. No error, no log line, just a wrong number. When the system writes to a customer's ad budget, a wrong number is real money.

Atomic means the operation either happens completely or not at all, and nobody observes it half-done. How you get there differs between a database and Redis, so I will take them one at a time.

In the database: push the computation into the statement

The cheapest fix for the example above is not a lock. It is not reading the value into the application at all:

UPDATE campaigns
SET    budget = budget - $1
WHERE  id = $2 AND budget >= $1;

A single-row UPDATE is atomic: PostgreSQL locks the row while writing, the second process waits and then sees the new value. The budget >= $1 condition lives inside the statement, so there is no gap between "check" and "write". The application only needs to look at the affected row count:

tag, err := tx.Exec(ctx, q, spend, campaignID)
if err != nil {
    return err
}
if tag.RowsAffected() == 0 {
    return ErrInsufficientBudget
}

Only when the computation is too complex to express in SQL do you need an explicit lock:

BEGIN;
SELECT budget, status FROM campaigns WHERE id = $1 FOR UPDATE;
-- compute in the application
UPDATE campaigns SET budget = $2 WHERE id = $1;
COMMIT;

FOR UPDATE holds the row lock until the transaction ends. In exchange, the transaction has to be short — never call an external API while holding the lock, or you have just turned a correctness bug into an outage.

A common misconception: wrapping code in a transaction does not by itself make read–compute–write safe. Under PostgreSQL's default READ COMMITTED, two transactions can still both SELECT the old value. Transactions give you all-or-nothing on rollback; preventing lost updates takes a conditional write, FOR UPDATE, or a stricter isolation level plus retry logic.

Constraints are the atomic layer you forget about

For "do this exactly once", the answer is usually not a lock but a unique constraint:

INSERT INTO scheduled_runs (schedule_id, run_at)
VALUES ($1, $2)
ON CONFLICT (schedule_id, run_at) DO NOTHING;

If two workers pick up the same schedule, only one manages to insert the row; the other gets 0 rows and stops. The database checks the constraint atomically, no matter how many processes or machines are involved. I trust this more than any application-level locking scheme.

In Redis: each command is atomic, a sequence is not

Redis executes commands one at a time, so every single command is atomic. INCR, DECRBY, HINCRBY and SETNX are never interleaved. Trouble starts when application code stitches several commands together:

// Wrong: two clients both GET 9 and both SET 10
n, _ := rdb.Get(ctx, key).Int()
rdb.Set(ctx, key, n+1, time.Minute)

// Right: one command
rdb.Incr(ctx, key)

Rate limiting is the classic case. INCR followed by a separate EXPIRE means that if the process dies in between, the key never expires and the user is blocked forever. Those two commands have to travel together.

MULTI/EXEC is not a SQL transaction

MULTI ... EXEC guarantees the queued commands run back to back with no other client's commands in between. It has two limits that surprise people:

• No rollback. If one command fails at runtime (say, INCR on a hash key), the remaining commands still execute.
• No reads in the middle. You cannot use the result of the first command to decide the second, because nothing runs until EXEC.

To "read, then decide", use WATCH (optimistic locking: if the key changes, EXEC fails and you retry), or more simply, Lua.

Lua scripts: atomicity for conditional logic

A Lua script runs to completion on the server with nothing interleaved. It is the right tool for any check-then-write on Redis:

-- KEYS[1] = rate-limit key, ARGV[1] = limit, ARGV[2] = TTL (seconds)
local n = redis.call('INCR', KEYS[1])
if n == 1 then
  redis.call('EXPIRE', KEYS[1], ARGV[2])
end
if n > tonumber(ARGV[1]) then
  return 0
end
return 1

The cost: while a script runs, Redis serves nobody else. Keep scripts short and never loop over large data sets.

Distributed locks: releasing is the part people get wrong

Acquiring is well known — a single command, with a random token and a TTL:

SET lock:campaign:123 <token> NX PX 30000

The bug is usually in the release. If the job runs longer than the TTL, the lock has already expired and another worker holds it; a blind DEL now deletes someone else's lock. Release must compare the token before deleting, and those two steps must be atomic:

if redis.call('GET', KEYS[1]) == ARGV[1] then
  return redis.call('DEL', KEYS[1])
end
return 0

One more detail on Redis Cluster: scripts and MULTI are only atomic across keys in the same hash slot. To operate on several keys, give them a shared hash tag, such as {campaign:123}:budget and {campaign:123}:lock.

The rules I kept

1. Read–compute–write in application code is a signal to stop and ask: can this be a single statement?
2. Prefer database constraints over application-level locks. Constraints do not expire and do not depend on clocks.
3. In Redis, one command or one script — do not chain commands in Go and hope.
4. Redis for load, the database for correctness. A Redis lock avoids wasted work; the correctness of money data still lives in Postgres.
5. Test with real concurrency, not by reading code. Run the same operation from a few dozen goroutines and check the total — race bugs rarely surface any other way.

Atomicity is not an advanced technique reserved for big systems. It is the line between code that "works on my machine" and code that stays correct when a second person clicks the button.