Implementing CRDT Counters in CouchDB #

Two offline devices each incremented the same counter, replication merged their edits, and now CouchDB holds conflicting leaves where one increment silently overwrote the other — your tally is wrong and no timestamp can fix it. The answer is not a smarter Last-Write-Wins; it is a data model that cannot lose an increment in the first place. This guide implements conflict-free replicated counters as CouchDB documents: a G-Counter for increment-only tallies, then a PN-Counter that also supports decrements, both merged by element-wise max across every conflicting leaf so the value converges without loss regardless of replication order. It is the lossless-accumulation choice from algorithm selection for merge, the strategy choosing between LWW, Field-Union, and CRDT merge routes accumulating state toward.

Immediate Triage & Prerequisites #

A CRDT counter is worth the schema change only when losing an increment is unacceptable and a timestamp cannot arbitrate. Confirm that before adopting the model.

• Confirm the loss is real. Read a suspect counter with curl 'http://localhost:5984/appdb/like-post-9?conflicts=true' and GET ?rev= each leaf. If the conflicting leaves each hold a different total and any Last-Write-Wins resolution would drop one, you have unrecoverable increment loss — the exact failure a CRDT prevents.
• Accept the schema commitment. A CRDT counter is not an integer. It stores per-replica sub-counters, and its value is derived, not stored. Every reader must compute sum(p) - sum(n) instead of reading a scalar. This is a permanent data-model decision across all readers, as choosing between LWW, Field-Union, and CRDT merge stresses.
• Assign a stable replica id per writer. Each device, node, or process needs a durable id (its nodeA/nodeB key). Reusing an id across two concurrent writers reintroduces the lost-update problem inside the CRDT; a fresh random id per process leaks keys forever. Pick durable, bounded ids.
• Environment. Python 3.10+ and requests. The counter merge itself is pure arithmetic and runs offline, so you can unit-test convergence before touching a live database.

Step-by-Step Implementation #

Each step includes a check. Build the G-Counter first to understand the invariant, then generalise to the PN-Counter, and always merge across leaves rather than overwriting.

1. Model the G-Counter. An increment-only counter is a map of replica id to that replica’s local count: {"p": {"nodeA": 5, "nodeB": 2}}. The value is sum(p.values()). Each replica writes only its own key. Verify: assert value == sum(state["p"].values()).

2. Increment your own key. To add k, read the current document, add k to your replica’s entry in p, and write back against the current _rev. Never touch another replica’s key. Verify: after the write, your key increased by exactly k and no other key changed.

3. Generalise to a PN-Counter. To support decrements, keep two G-Counters: p for increments and n for decrements. The value is sum(p) - sum(n). A decrement of k adds k to your key in n; you never subtract, so every sub-counter stays monotonically increasing — the property that makes max-merge safe. Verify: assert value == sum(state["p"].values()) - sum(state["n"].values()).

4. Merge conflicting leaves by element-wise max. When replication produces conflicting leaves, take the maximum value for each replica id independently across p, then across n. Because each key only ever grows, the per-key max is that replica’s true count, so no increment is lost. The operation is commutative, associative, and idempotent, so replicas converge regardless of merge order. Verify: merging leaves in any order yields identical p and n maps.

5. Commit the merged winner and tombstone the losers. Write the merged state against the read-winner _rev and add a {"_id": id, "_rev": rev, "_deleted": true} tombstone for every losing leaf in one _bulk_docs batch. On 409, re-read and re-merge — never overwrite with a single leaf, which would discard the others’ counts. Verify: re-read with ?conflicts=true and assert _conflicts is gone and the value is unchanged by the merge.

Complete Working Example #

This self-contained module implements both counters. GCounter is the increment-only base; PNCounter composes two of them for signed values. The resolver fetches every conflicting leaf, merges by element-wise max, and commits the converged winner plus loser tombstones atomically with bounded 409 retries. The merge is pure and order-independent, so the same routine is safe to run from any replica.

import os
import time

import requests


def merge_max(*maps: dict) -> dict:
    """Element-wise maximum across replica maps — the CRDT join for a G-Counter."""
    out: dict = {}
    for m in maps:
        for node, count in m.items():
            out[node] = max(out.get(node, 0), count)  # per-replica count only grows
    return out


class PNCounter:
    """A positive-negative CRDT counter: value = sum(p) - sum(n)."""

    def __init__(self, replica_id: str, p: dict | None = None, n: dict | None = None):
        self.replica_id = replica_id
        self.p = dict(p or {})
        self.n = dict(n or {})

    @classmethod
    def from_doc(cls, replica_id: str, doc: dict) -> "PNCounter":
        return cls(replica_id, doc.get("p"), doc.get("n"))

    @property
    def value(self) -> int:
        return sum(self.p.values()) - sum(self.n.values())

    def increment(self, k: int = 1) -> None:
        if k < 0:
            raise ValueError("use decrement() for negative deltas")
        self.p[self.replica_id] = self.p.get(self.replica_id, 0) + k  # own key only

    def decrement(self, k: int = 1) -> None:
        if k < 0:
            raise ValueError("k must be non-negative")
        self.n[self.replica_id] = self.n.get(self.replica_id, 0) + k  # own key only

    def merged_with(self, *others: "PNCounter") -> "PNCounter":
        p = merge_max(self.p, *(o.p for o in others))
        n = merge_max(self.n, *(o.n for o in others))
        return PNCounter(self.replica_id, p, n)

    def as_doc(self) -> dict:
        return {"p": self.p, "n": self.n, "value": self.value}


class CounterResolver:
    """Merge conflicting counter leaves in CouchDB without losing increments."""

    def __init__(self, db_url: str, replica_id: str, max_retries: int = 5):
        self.db_url = db_url.rstrip("/")
        self.replica_id = replica_id
        self.max_retries = max_retries
        self.session = requests.Session()

    def _fetch_leaves(self, doc_id: str) -> tuple[dict, list[PNCounter]]:
        winner = self.session.get(
            f"{self.db_url}/{doc_id}", params={"conflicts": "true"}
        ).json()
        counters = [PNCounter.from_doc(self.replica_id, winner)]
        for rev in winner.get("_conflicts", []):
            leaf = self.session.get(
                f"{self.db_url}/{doc_id}", params={"rev": rev}
            ).json()
            counters.append(PNCounter.from_doc(self.replica_id, leaf))
        return winner, counters

    def resolve(self, doc_id: str) -> dict:
        for attempt in range(1, self.max_retries + 1):
            winner, counters = self._fetch_leaves(doc_id)
            losers = winner.get("_conflicts", [])
            if not losers:
                return {"ok": True, "value": counters[0].value}

            merged = counters[0].merged_with(*counters[1:])  # order-independent join
            survivor = {**merged.as_doc(), "_id": doc_id, "_rev": winner["_rev"]}
            tombstones = [
                {"_id": doc_id, "_rev": rev, "_deleted": True} for rev in losers
            ]
            resp = self.session.post(
                f"{self.db_url}/_bulk_docs", json={"docs": [survivor, *tombstones]}
            )
            if resp.status_code != 409:
                return {"ok": True, "attempt": attempt, "value": merged.value}
            time.sleep(min(2 ** attempt, 30))  # stale winner _rev; re-read and re-merge
        raise RuntimeError(f"exhausted retries resolving {doc_id}")


if __name__ == "__main__":
    # Offline convergence proof: two replicas increment independently, then merge.
    a = PNCounter("nodeA")
    b = PNCounter("nodeB")
    a.increment(5)
    b.increment(2)
    b.decrement(1)
    assert a.merged_with(b).value == b.merged_with(a).value == 6  # commutative
    print("offline merge value:", a.merged_with(b).value)

    url = os.environ.get("COUCH_URL", "http://localhost:5984/appdb")
    doc = os.environ.get("DOC_ID", "like-post-9")
    print(CounterResolver(url, replica_id="nodeA").resolve(doc))

The offline assertion proves the core property before any network call: merging in either order yields the same value, so every replica converges no matter how replication interleaves.

Gotchas & Edge Cases #

• Unbounded replica-id growth. Every distinct replica id adds a permanent key to p and n. Ephemeral or per-process ids make the document grow without bound. Use durable ids tied to hardware or a stable node name, and retire dead nodes carefully: only remove a key once you can prove no live leaf still carries a higher value for it, otherwise the merge resurrects the stale count on the next sync.
• Never overwrite — always merge across leaves. Resolving a counter conflict by writing one leaf as the winner discards every other leaf’s increments, silently corrupting the total. The resolver must fetch all _conflicts leaves and take the element-wise max. This is the single most common way CRDT counters lose data in practice.
• Integer overflow on hot counters. A per-replica sub-counter is monotonically increasing and never resets, so a very hot counter can grow large over years. Python integers are unbounded, but downstream consumers (a 32/64-bit column, a JSON reader in another language) may not be — size the storage and consider periodic supervised compaction of retired replicas.
• Do not mix a CRDT with Last-Write-Wins on the same document. An LWW resolver that overwrites the whole document will replace the merged p/n maps with a single leaf, erasing other replicas’ counts. If a document holds both a counter and LWW fields, route per field so the LWW logic never touches p or n — the per-field routing shown in writing custom conflict resolver functions in Python.
• A replica must only increment its own key. If two writers share a replica id and both increment concurrently, max-merge keeps only the larger value and the smaller increment is lost — the CRDT guarantee holds per key, so key ownership must be exclusive.

Verification & Observability #

Prove convergence, not just the absence of conflicts. Offline, assert that merging leaves in every order yields the same value — the unit test in the example is the canonical check and belongs in CI. Per document, re-read with ?conflicts=true after resolution and assert both that _conflicts is gone and that the derived value equals the sum you expect from the merged sub-counters; a value that changed across a merge means an increment was dropped and the resolver overwrote instead of joining. Across the pipeline, emit the counter’s derived value and the size of its p/n maps as metrics: a steadily growing map size flags replica-id sprawl before it bloats documents, and a monotonic value that ever decreases (outside a genuine decrement) flags a corrupt merge. Watch GET /_scheduler/jobs to confirm replication stays running, and schedule POST /appdb/_compact off-peak to reclaim storage from tombstoned leaves. When you need to test convergence under realistic interleavings before production, generate conflicting leaves deliberately with testing conflict resolvers with synthetic conflicts.

FAQ #

Related #

• Choosing Between LWW, Field-Union, and CRDT Merge
• Implementing Field-Union Merge in CouchDB
• Writing Custom Conflict Resolver Functions in Python
• Algorithm Selection for Merge

Part of: Algorithm Selection for Merge