Race Conditions in n8n: The Bugs That Only Happen When Two Things Arrive at Once

Every race is the same picture

A race condition needs two things: shared state, and a gap between reading that state and acting on it. In automation work the shared state is almost always a record in a system you do not control, and the gap is almost always the space between a search node and a write node.

Here is the canonical version, and it is in nearly every workflow anyone has ever built.

// Correct code. The bug is the gap between the read and the write.

const found = await notion.dataSources.query({
  data_source_id: dsId,
  filter: { property: "Email", email: { equals: email } }
});

if (found.results.length === 0) {
  await notion.pages.create({ parent: { data_source_id: dsId }, properties });
} else {
  await notion.pages.update({ page_id: found.results[0].id, properties });
}

There is nothing to fix in that code. It checks before it writes, which is the thing you are supposed to do. The problem is the assumption underneath it: that the world does not change between the query and the create. In a single execution that assumption holds. The moment two executions can exist at once, it is a guess, and it is a guess you lose more often the busier you get.

That is why these bugs arrive with success. A client signs more deals, a form gets more submissions, a CRM starts firing webhooks in bursts instead of one an hour, and a workflow that was correct for eight months starts producing duplicates. Nothing changed in the workflow. The gap just started having someone standing in it.

The six races you will actually meet

They all share that one shape, but they arrive through six different doors, and knowing which door you are looking at decides which fix is the right one.

1. Two deliveries about the same entity

A CRM fires record.updated twice in the same second because someone edited two fields, or a form and an integration both report the same signup. Two executions start, both search for the contact, both find nothing, both create it.

This is the one that produces the duplicate rows people actually notice, usually in a report, usually weeks later. It is also the one where the fix is most often free.

2. The same delivery, twice, by design

Webhooks are at least once, not exactly once. If your endpoint is slow to acknowledge, the sender assumes the delivery failed and sends it again, and the sender is behaving correctly when it does. I went into this properly in webhooks explained for production, and it is worth understanding before you reach for a lock, because a lock is the wrong tool for this one.

The tell is that the two executions are identical, same payload, same event id. That identity is the thing that saves you, because it gives you a key to deduplicate on.

3. The schedule that laps itself

A sync runs every five minutes and takes ninety seconds, which is comfortable. Then the dataset grows, or an upstream API slows down, and it takes six minutes. Now run 2 starts while run 1 is still writing.

This one compounds. Each overlapping run makes everything slower, which makes the overlap worse, and the instance ends up thrashing. It is also the easiest of the six to prevent, because you can simply refuse to start.

4. Two workers, one job

This one only exists once you move to queue mode, and it surprises people because queue mode is supposed to be the grown up setup. Multiple workers pull from the same Redis queue. That is the whole point. But a workflow written with the quiet assumption that it is the only one running is now running three times in parallel on three different machines.

5. Read, modify, write

Two runs both need to change a number. Both read 100. One adds 10 and writes 110. The other adds 5 and writes 105. The final value is 105, the expected value is 115, and nothing anywhere records that ten units were lost.

6. The cleanup job reading a half written row

Mature automations grow a repair job: something that sweeps for rows that look wrong and fixes them. A record in the middle of a multi step write always looks wrong, because it is genuinely incomplete for a few hundred milliseconds.

This is drift with an accelerant. The repair job writes over a row that was about to be finished correctly, the sync writes again, and the two of them can fight all afternoon.

What n8n gives you for this, honestly

Very little, and it is better to know that up front than to discover it after shipping.

There is no mutex node. There is no transaction that spans several nodes, so there is no way to say "these six nodes happen together or not at all". Execution order between two separate executions is not defined, and in queue mode they are not even in the same process. Nothing in the editor warns you that a workflow is unsafe to run concurrently, because from n8n's point of view running things concurrently is a feature.

The thing people reach for first is workflow static data, and it does not work.

// Read when the run starts, written back when it ends. Not a lock.

const store = $getWorkflowStaticData('global');

if (store.busy) return [];
store.busy = true;

// ... work ...

store.busy = false;

Static data is read into memory when the execution starts and written back when it ends. Two executions both read busy as false, both set it to true, and whichever finishes last overwrites the other. In queue mode they run in different processes, so there is not even a shared memory to race over.

What you do get is three real levers. Concurrency limits cap how many production executions run at once, through N8N_CONCURRENCY_PRODUCTION_LIMIT on the instance and the concurrency setting on a worker in queue mode. The Remove Duplicates node can keep a history of values it has already seen, backed by the n8n database rather than by memory. And Data Tables give you a small piece of shared, durable state that every worker can see. Those three cover more ground than people expect. The lock is for what is left.

Reach for these in order

A distributed lock is the most powerful fix here and the last one you should reach for, because it is the only one that adds a piece of infrastructure that can itself fail. Work down this list and stop at the first rung that holds.

Rung 1: let the destination refuse the duplicate

If the destination can enforce uniqueness, the race stops being your problem. A unique index has no gap in the middle of it, because the check and the write are the same operation inside one database.

/* The destination refuses the duplicate itself, atomically. */

INSERT INTO contacts (external_id, email, name)
VALUES ($1, $2, $3)
ON CONFLICT (external_id) DO UPDATE
  SET email = EXCLUDED.email,
      name  = EXCLUDED.name
RETURNING id;

Two runs execute that, both succeed, and there is one row. No lock, no waiting, nothing to keep alive. If you are writing to Postgres or Supabase, this is where the conversation should end. The catch is obvious in our world: Notion has no unique constraint, Airtable has none, Sheets certainly does not, and most CRMs will not enforce one on the field you care about. Which is why the rest of this post exists.

Rung 2: deduplicate on the event, not the record

For race 2, the same delivery arriving twice, you do not need mutual exclusion. You need to recognise an event you have already handled. In n8n that is three nodes: read a Data Table row keyed on the event id, an IF on whether it exists, and a write of the key before the work starts.

The key has to come from the sender, not from you. An Idempotency-Key header, an event id, a message id, anything stable across the retry. A key you generate is a new key on every attempt and deduplicates nothing. And then there is the trap that only shows up months later, on the one delivery that arrives without the header.

// A null key matches the previous null row, so every keyless delivery
// after the first would be skipped forever.

const idempotencyKey = $json.headers['idempotency-key'] ?? null;
const recordId       = $json.body.events[0].id.record_id;

const dedupeKey = idempotencyKey ?? `nokey-${recordId}-${Date.now()}`;

return [{ json: { recordId, idempotencyKey, dedupeKey, headerKeyMissing: !idempotencyKey } }];

Without that fallback, a missing header writes a null key, and every future keyless delivery matches that null row and is dropped as a duplicate forever. A unique synthetic key disables deduplication for that one delivery instead, which loses nothing and can never silently drop a real edit.

One more thing about that read then write pair: it is itself a check then act gap. It narrows the window a great deal, which is often enough on its own, but it is only airtight when it runs inside the lock.

Rung 3: refuse to overlap

For race 3, the fix is to not start. Cap the workflow so a second execution cannot begin while the first is running, through the instance concurrency limit or by having the workflow check for a live execution of itself and exit. Slower is fine. Two copies fighting over the same rows is not.

Rung 4: the lock

When the destination cannot enforce uniqueness, the runs are not identical so deduplication does not apply, and you genuinely need two executions to take turns inside one record, you need mutual exclusion.

The lock, node by node

This is the pattern running in production on the Attio, Granola and Notion sync, on self hosted n8n in queue mode with Postgres and Redis. Notion has no unique constraint and the CRM's webhooks arrive in bursts, so rungs 1 and 2 could not carry it alone.

It starts with one Code node that decides the lock identity, because every node after this point needs the same key and none of them can recompute it.

// One node decides the lock identity. Everything after it reads these by node reference.

const NS_LOCK  = 'lock:contact';
const NS_OWNER = 'lockowner:contact';
const NS_EXEC  = 'lockexec';
const NS_STEAL = 'steal:contact';

const recordId = $('Filter Events').first().json?.recordId ?? null;
if (!recordId) throw new Error('No record id, so there is nothing to lock on.');

return [{ json: {
  lockRecordId:  recordId,
  lockKey:       `${NS_LOCK}:${recordId}`,
  lockOwnerKey:  `${NS_OWNER}:${recordId}`,
  lockToken:     String($execution.id),
  lockExecKey:   `${NS_EXEC}:${$execution.id}`,
  lockExecValue: JSON.stringify({ recordId }),
  stealKey:      `${NS_STEAL}:${recordId}`,
}}];

The token is the execution id. It is unique per run, it is already in every log line, and it makes an abandoned lock traceable back to the run that abandoned it. The exec key is the reverse map: given nothing but a failed execution id, an error workflow can find which lock that run was holding.

Then the acquire, which is a single Redis node: operation INCR, the key from Build Lock Key, expire on, TTL 30. INCR is atomic, so the counter is the decision. The run that creates the key gets 1. Every other run gets 2 or higher, because the key already exists. There is no check then set to slip through.

// Acquire Lock is a Redis node: operation INCR, expire on, ttl 30.

// Got Lock?        number  equals  1
{{ Object.values($json)[0] }}

// Retry Budget Left?   number  smaller than  18
{{ $runIndex }}

// Every key after a Redis node comes from the node, never from $json.
{{ $('Build Lock Key').first().json.lockKey }}

That first expression looks strange and it is load bearing. A Redis node replaces $json with its own output, which for an INCR is an object keyed by the Redis key itself, something like { "lock:contact:4412": 3 }. You cannot read $json.lockKey after it, because the webhook payload is gone. Object.values($json)[0] is how you get the counter out without knowing the key name, and every key after a Redis node has to be read back from Build Lock Key by node reference. Getting this wrong is a silent killer: the run goes green, the node reference comes back undefined, and nothing syncs.

The same trap lives in the Data Table node. Its output is the row it just inserted, not the payload that went in, so any node after it that reaches for $json.body finds nothing.

Winning the IF leads to three Redis writes in a row. Stamp Lock TTL sets the key back to 1 with a fresh 30 second TTL, which both clears the counter noise left by rejected retries and starts the clock from the moment you actually hold the lock rather than from whenever the key was first created. Claim Lock Ownership writes the token under a second key on the same 30 seconds. Map Exec To Lock writes the reverse map at 90 seconds, deliberately outliving the lock so a dead run can still be traced after its lock has gone.

Releasing is where most implementations are quietly broken. You do not delete the key. You read the owner key, compare it to your token, and only delete on a match.

// Runs after the owner comparison, and says so when the lock was lost.

const d = $('Build Lock Key').first().json;

let stillOurs = false;
try {
  stillOurs = String($('Read Lock Owner').first().json?.lockOwner ?? '') === String(d.lockToken);
} catch (e) {}

if (!stillOurs) {
  console.log(
    `WARNING the lock on ${d.lockRecordId} was no longer ours at release time. It hit its ` +
    `30s TTL and another execution took over, so the two overlapped. Nothing was deleted, ` +
    `the current holder keeps its lock.`
  );
}

return [{ json: {
  lockRecordId:  d.lockRecordId,
  lockReleased:  stillOurs,
  overlappedRun: !stillOurs,
}}];

Picture a run slower than its own TTL. Its lock expired, a second run legitimately took the key, and now the slow run finishes and deletes it. That deletes somebody else's lock, and two runs are inside the record with no lock at all. Comparing first turns that into a warning line instead of a corruption, and the warning is the thing that tells you your TTL is too short before a customer does.

The timeouts, and what each number buys

A lock is four numbers and a loop. These are the ones that build runs on, and the reasoning matters more than the values.

• TTL 30 seconds on the lock key and the owner key. This is a crash timer, not a work estimate: it is how long a record stays locked by an execution that died.
• Wait 5 seconds between retries, as a Wait node. The retry loop is IF, Wait, Redis INCR, back to the IF.
• A retry budget of 18, tested with {{ $runIndex }} < 18. That is 90 seconds of patient waiting for a lock that is simply busy.
• A hard ceiling of 42. Between 18 and 42 the run stops waiting politely and starts investigating whether the lock is abandoned.
• Exec map at 90 seconds, three times the lock TTL, so the trace outlives the thing it describes.

Between the budget and the ceiling, the run reads the owner key. If the lock key exists but the owner key has expired, the holder is gone and the lock is abandoned. That is not permission to clear it, because six waiting runs all notice the same dead lock at the same moment, and six force clears means six runs proceed.

// The two conditions that decide whether a stuck lock may be cleared.

// Lock Abandoned?   the owner key expired, the lock key did not
{{ String($json.stuckOwner ?? '') }}   is empty

// Won Steal?        INCR on the steal key came back 1
{{ Object.values($json)[0] }}   equals   1

So the right to clear is contested with another INCR, on a steal key with its own short expiry. Exactly one run gets 1 and does the clearing. Everyone else waits. Because the steal key expires on its own, the next stuck lock gets a fresh contest rather than inheriting a stale winner.

Past the hard ceiling, this build stops waiting and proceeds without the lock. That is a deliberate trade for this workload: a sync that drops an edit is worse than a sync that occasionally overlaps, and the release check will report the overlap when it happens. For a ledger, a payment, or a booking that cannot be sold twice, make that branch throw instead. Proceeding without a lock converts a loud failure into a silent one, and that is only acceptable when you have decided, on purpose, that the data can take it.

Two more choices decide how well this behaves. The key names the thing being protected, nothing wider. lock:contact:4412 lets a hundred contacts sync in parallel while making sure no single contact is touched twice. lock:contact-sync serialises your whole integration behind one key and converts a concurrency bug into a throughput bug. The wide key feels safer. It is only slower, and slower means longer queues, which means more runs hitting the budget.

And where Redis actually sits changes the arithmetic. On that build, n8n and Redis are containers on the same Docker network on one host, so an INCR round trip is roughly 3 milliseconds. The critical section is dominated entirely by the Notion and CRM calls inside it, and the lock is effectively free. Point the same workflow at a managed Redis across the internet and every acquire, every stamp, every owner read, and every release costs tens of milliseconds instead. The lock is still correct, but the overhead is now real, the window between your INCR and your first write is wider, and a per record key with a short critical section stops being good practice and starts being the difference between working and not.

The same thing, in code

If you are writing this in a Code node or outside n8n entirely, the canonical form is one command rather than a node chain. Self hosted n8n needs NODE_FUNCTION_ALLOW_EXTERNAL=ioredis before a Code node may require a client.

// The same idea in one command, for code rather than nodes.

const got = await redis.set(key, token, 'PX', 30000, 'NX');
if (got !== 'OK') return [{ json: { locked: false } }];

// Release compares the token first. A bare DEL deletes the next holder's lock.
const RELEASE = `
  if redis.call('GET', KEYS[1]) == ARGV[1] then
    return redis.call('DEL', KEYS[1])
  else
    return 0
  end
`;

await redis.eval(RELEASE, 1, key, token);

NX sets the key only if it does not exist, and PX expires it, so acquire is one atomic round trip. The Lua script runs the compare and the delete together inside Redis, so nothing can happen between them. It is the same design as the node chain, with the retry loop moved into your own code, and it is worth knowing because the node version cannot express a compare and delete in one operation.

One piece of honesty that plenty of posts skip. A lock on a single Redis instance is not a mathematical guarantee. If Redis fails over to a replica that had not caught up, two runs can briefly hold the same key. For syncing CRMs and hotel bookings and invoices, that risk is far smaller than the duplicates you are removing today. For anything where a double write is genuinely unacceptable, the lock is not your safety net. A unique constraint in the destination is.

If you do not have Redis

Most people reading this are on n8n Cloud, where there is no Redis to reach. The honest answer is that your options are worse. They are not nothing.

Postgres advisory locks are the closest real equivalent, and if you already have a Postgres anywhere in the stack, you already have a lock service.

/* Transaction scoped, because n8n pools connections: a session lock taken in
   one node can be held by a connection the next node never sees. */

BEGIN;
SELECT pg_advisory_xact_lock(hashtext('contact:' || $1));
UPDATE contacts SET stage = $2 WHERE external_id = $1;
COMMIT;

The caveat in that comment is the part that catches people. A session level pg_advisory_lock belongs to the connection that took it, and the next node may get a different connection from the pool, leaving the lock held by nobody you can reach. The transaction scoped variant releases on commit or rollback, including the rollback you did not plan.

A lock row in a table with a unique index works anywhere you have real constraints. Insert a row keyed on the record id, let the index reject the second insert, delete it when done, and sweep rows older than the TTL on a schedule. It is a lock built out of rung 1.

An n8n Data Table row as a lock flag is the Cloud friendly version, and it is advisory rather than airtight. Read then write is still two operations, so the window is narrow rather than closed. It genuinely helps against bursts arriving 200 milliseconds apart and it will not save you from two arriving in the same millisecond. Know which one you bought.

A lock page in Notion or a cell in Sheets needs the most care. Both are slow enough that the read to write window is hundreds of milliseconds wide, which is a comfortable size for the exact race you are trying to prevent. Better than nothing, and not a lock.

And the option nobody likes: serialise. Cap the workflow to one execution at a time. It is crude, it costs throughput, and for a workflow handling a few hundred events a day it is completely fine.

When the answer is not a lock

Locks exclude. Sometimes converging is better than excluding, and reaching for a lock first can mean building a queue where you needed a rule.

On that same build, one path was specified with a lock by meeting id during upserts and did not get one. It reconciles instead: it allows the duplicate to exist for a moment and then converges on one record, deterministically, merging by a stable key and stopping to ask a person when a merge is not obviously safe. For records arriving from a transcription tool with fuzzy identity, converging is more honest than pretending you can exclude your way to correctness.

That last part matters as much as any code here. A system that refuses to guess, and escalates, is worth more than one that is confidently wrong at scale.

The audit, in ten questions

Run these against any workflow that writes to a shared record. You are not looking for a bug in the code, you are looking for an assumption that only holds when one execution exists.

• Does any path search for a record and then create it if missing? That is the gap, and it is the default shape of most syncs.
• Could two executions of this workflow ever run at the same time? Bursty webhooks, a fast form, queue mode workers, a manual run while the schedule fires.
• Can the destination enforce uniqueness on the field you match on? If yes, stop reading and go add the constraint.
• Is anything deduplicated on an id the sender provides, rather than on a value you compute?
• What happens to a delivery that arrives with no idempotency header at all?
• Does the scheduled run take longer than its own interval on the worst day of the month, not the average one?
• Does any node read a value, change it, and write it back? Counters, totals, appending to a text field, adding a tag.
• Is anything using workflow static data as a flag across executions?
• If you have a lock, does the release compare an owner token before deleting, or does it just delete the key?
• Is the TTL longer than the p99 of the work it protects, and does the run fail loudly when it never gets in?

If a workflow fails several of these and has not produced a duplicate yet, that is not a clean bill of health. It means the volume has not arrived. These bugs are dormant, not absent, and they wake up on the day the business has a good week.

If you would rather have someone else read your instance with this list in hand, that is the kind of thing I do: a 30 minute call, no obligation, and you will get a straight answer about whether your concurrency is a real risk or a theoretical one.