Preventing Conflict Storms in Offline-First Apps #

A field crew works offline all day, twenty PouchDB clients queue up hundreds of edits each, and the moment they hit Wi-Fi and sync, your CouchDB _conflicts arrays balloon and a review queue you never wanted to staff starts filling. The instinct is to write a smarter resolver, but the durable fix is upstream: most offline-first conflicts are self-inflicted by a data model in which two devices mutate the same document. This page shows how to prevent conflict storms by construction — sharding writes into per-device documents, switching hot records from mutate-in-place to append-only events, and reconciling with a view instead of a shared mutable doc — so reconnection produces zero forks for the common case. It is part of Conflict Generation Models; if your storms come from drifting node clocks rather than offline edits, read Diagnosing Clock-Skew-Induced Conflict Storms first, because no data model can rescue a broken wall clock.

Immediate Triage / Prerequisites #

Before redesigning anything, measure how much of your conflict rate is preventable. Not every conflict is a modeling flaw, so separate the shared-mutable-doc conflicts (fixable by construction) from genuinely concurrent edits to intrinsically shared state. Enumerate documents that currently carry live conflicts using the built-in view:

# _all_docs with a conflicts filter is cheap; use the dedicated view for scale
curl -s "http://localhost:5984/orders/_all_docs?conflicts=true&include_docs=true" | \
  python3 -c "import sys,json;d=json.load(sys.stdin);print(sum(1 for r in d['rows'] if '_conflicts' in r.get('doc',{})),'docs with conflicts')"

Then sample a few of those documents and ask a single diagnostic question of each: did two different devices write to the same _id? If yes, that document is a modeling problem you can design away. Prerequisites for the steps below: Python 3.8+ with requests (pip install requests), a CouchDB 3.x target, and a stable device_id available on each client (a per-install UUID, not a per-user id — a user with two devices is two writers). One principle drives everything here: a document with exactly one writer can never conflict, because a conflict is by definition two branches diverging from a shared ancestor. Design so each mutable document has one and only one writer.

Step-by-Step Implementation #

Follow these steps to reshape an offline-first schema so reconnection stops manufacturing forks. Each step includes a check you can run to confirm the model holds.

1. Namespace every write by device. Give each document an _id that embeds the writing device, e.g. order:5001:<device_id>. Two devices editing “the same order” now write to two distinct documents that can never share an ancestor, so replication has nothing to fork:

   doc_id = f"order:{order_no}:{device_id}"  # one writer owns this _id forever

   Verify by asserting your write path can only ever PUT to an id containing its own device_id — a write to any other device’s id is a bug.

2. Model mutations as append-only events, not in-place edits. Instead of updating a qty field, write an immutable event document ({"type": "qty_set", "value": 9, "hlc": ...}) with a deterministic id. Immutable documents are written once and never updated, so they cannot accumulate a second branch. Deterministic ids also make retries idempotent — re-writing the same event is a no-op, not a duplicate.

3. Batch the offline queue through _bulk_docs. On reconnect, flush the accumulated events in one request rather than one PUT per event. _bulk_docs is atomic per document and dramatically reduces round trips for a device that queued hundreds of edits offline:

   curl -s -X POST http://localhost:5984/orders/_bulk_docs \
     -H "Content-Type: application/json" \
     -d '{"docs":[{"_id":"order:5001:evt:0001","type":"qty_set","value":9}]}'

   Verify each row in the response array has "ok":true; a row with "error":"conflict" means a non-deterministic id collided and must be investigated, not blindly retried.

4. Reconcile with a view, not a shared document. Keep the per-device events as the source of truth and fold them into a single logical record with a MapReduce view or a _find query at read time. The materialized order lives in the view index, so no mutable shared document exists to conflict:

   // map: emit each event under its logical order key for range reads
   function (doc) {
     if (doc.type && doc._id.startsWith("order:")) {
       emit(doc._id.split(":")[1], { type: doc.type, value: doc.value, hlc: doc.hlc });
     }
   }

5. Keep intrinsically shared, hot fields out of shared docs. For a value multiple devices genuinely co-own — a running total, a counter — do not store one number in one document. Either split it into per-device sub-totals summed by the view (step 4), or use a CRDT counter so concurrent increments commute. Reconciling those residual cases is the domain of Manual Review Sync Queues when they cannot be merged automatically.

Complete Working Example #

The script below models an offline-first order as per-device append-only events, flushes a device’s offline queue through _bulk_docs with deterministic ids, and measures the residual conflict rate so you can prove the design is working. It is self-contained and runnable against a CouchDB 3.x target.

import hashlib
import sys
import time

import requests


def event_id(order_no: str, device_id: str, seq: int) -> str:
    """Deterministic, per-device, immutable event id — retries are idempotent."""
    # embedding device_id guarantees a single writer; seq orders one device's events
    raw = f"order:{order_no}:{device_id}:{seq:06d}"
    suffix = hashlib.sha1(raw.encode()).hexdigest()[:8]  # collision-resistant tag
    return f"order:{order_no}:{device_id}:evt:{seq:06d}:{suffix}"


def flush_offline_queue(db_url, order_no, device_id, events, session):
    """Write a device's queued events in one atomic-per-doc _bulk_docs call."""
    docs = []
    for seq, ev in enumerate(events):
        docs.append({
            "_id": event_id(order_no, device_id, seq),
            "type": ev["type"],
            "value": ev["value"],
            "device_id": device_id,       # one writer owns each of these docs
            "hlc": f"{int(time.time() * 1000):013d}.{seq:06d}",
        })
    resp = session.post(f"{db_url}/_bulk_docs", json={"docs": docs}, timeout=60)
    resp.raise_for_status()
    rows = resp.json()
    ok = sum(1 for r in rows if r.get("ok"))
    # an immutable, deterministic id should never legitimately conflict
    conflicts = [r for r in rows if r.get("error") == "conflict"]
    return {"written": ok, "conflicts": conflicts}


def residual_conflict_rate(db_url, session):
    """Fraction of documents that still carry a live conflict — the design KPI."""
    resp = session.get(
        f"{db_url}/_all_docs", params={"conflicts": "true", "include_docs": "true"},
        timeout=60,
    )
    resp.raise_for_status()
    rows = resp.json()["rows"]
    total = len(rows)
    conflicted = sum(1 for r in rows if "_conflicts" in r.get("doc", {}))
    return conflicted / total if total else 0.0


if __name__ == "__main__":
    db = "http://localhost:5984/orders"
    s = requests.Session()

    # two devices edited "the same order" offline; per-device ids keep them apart
    a = flush_offline_queue(db, "5001", "deviceA",
                            [{"type": "qty_set", "value": 6}], s)
    b = flush_offline_queue(db, "5001", "deviceB",
                            [{"type": "qty_set", "value": 9}], s)
    print("deviceA:", a)
    print("deviceB:", b)

    rate = residual_conflict_rate(db, s)
    print(f"residual conflict rate: {rate:.4%}")
    # by construction this stays at zero for the shared-order case above
    assert not a["conflicts"] and not b["conflicts"], "per-device ids must not fork"
    sys.exit(0)

Run it and both devices’ writes land as distinct documents, the _bulk_docs response shows written with an empty conflicts list, and the residual conflict rate reads zero for the case that would previously have forked — the reconciliation happens in the view, never in a contested document.

Gotchas & Edge Cases #

• Some conflicts are irreducible — use a CRDT. A value multiple devices must increment concurrently (a shared inventory count, a like counter) cannot be modeled as single-writer without losing writes. Reach for a CRDT counter whose increments commute; per-device documents summed by a view is the simplest such CRDT and composes cleanly with everything above.
• “One document per user” is not enough. A single user with a phone and a tablet is two writers to that user’s document and will still fork. Namespace by device install, not by user; derive the user’s view by aggregating their devices.
• Attachments conflict independently of the body. Two devices attaching a photo to the same document produce an attachment conflict even if the JSON never diverges. Store binary payloads as their own per-device documents (or content-addressed by hash) rather than as attachments on a shared record.
• Deterministic ids must be genuinely unique per event. If two distinct offline edits hash to the same id, _bulk_docs silently treats the second as an update to the first and you lose data. Seed the id with a monotonic per-device sequence plus a content hash, as the example does, not with a timestamp that can repeat offline.
• Unbounded event logs need compaction. Append-only means the document count grows forever. Periodically snapshot the view’s reduced state into a compacted record and prune superseded events, or the index and _revs history will bloat. Prune with care so replicas that are behind still find a shared ancestor.

Verification & Observability #

Confirm the design works by watching the residual conflict rate rather than resolver throughput — the goal is that the resolver has nothing to do. Track the fraction of documents carrying live conflicts before and after a sync burst:

# residual conflict rate: conflicted docs over total (target ~0 for the modeled cases)
curl -s "http://localhost:5984/orders/_all_docs?conflicts=true&include_docs=true" | \
  python3 -c "import sys,json;d=json.load(sys.stdin);r=d['rows'];c=sum('_conflicts' in x.get('doc',{}) for x in r);print(f'{c}/{len(r)} conflicted')"

# confirm the reconciliation view is built and not lagging behind writes
curl -s "http://localhost:5984/orders/_design/orders/_info" | \
  python3 -c "import sys,json;print(json.load(sys.stdin)['view_index'].get('update_seq'))"

A healthy result is a residual conflict rate that stays flat through reconnection spikes and a view update_seq that tracks the database update_seq closely. If conflicts still appear on documents that should have one writer, the usual cause is a client writing to another device’s namespace or a non-unique deterministic id — both are bugs in the write path, not the resolver. For how these prevented and residual conflicts interact with your overall replication shape, see Sync Topology Models.

FAQ #

Related #

• Diagnosing Clock-Skew-Induced Conflict Storms
• Sync Topology Models
• Manual Review Sync Queues

Part of: Conflict Generation Models