CouchDB Sync Conflict Resolution & Automation

Filtering Replicated Writes with validate_doc_update

A rogue edge node is pushing malformed telemetry into your central database, or a mobile client is replicating documents it has no business writing, and the writes land anyway because replication bypasses your application-tier checks entirely. Replication does not call your API handlers — it writes straight into the storage engine — so the only place to reject an unauthorized or malformed document at the target is a validate_doc_update (VDU) function inside a design document. This guide shows how to write a VDU that inspects userCtx.roles and the document body, deploy it, prove it rejects a forbidden write with a 403 and an unauthenticated one with a 401, and confirm that rejected documents are skipped and counted during replication without the whole job failing. It is the enforcement layer of Security Boundaries in CouchDB Replication.

How validate_doc_update filters writes at a CouchDB replication target A direct client PUT and a replicated write both enter the target write path and are passed through the design document's validate_doc_update(newDoc, oldDoc, userCtx, secObj) function. The function runs a userCtx.roles authorization gate, a schema/required-fields check, and an immutable-ownership check. Returning without throwing accepts the write as a new revision with 201. Throwing forbidden yields 403 and throwing unauthorized yields 401; the rejected document is skipped, the replication job's doc_write_failures counter is incremented, the rejection is logged, and the job continues. Inbound writes Client write PUT /db/doc Replicated write pulled from source same write path, no API layer validate_doc_update() (newDoc, oldDoc, userCtx, secObj) 1 · userCtx.roles gate require a writer role 2 · schema / required fields reject malformed bodies 3 · immutable ownership device_id cannot change return throw Accepted → 201 new revision written to the revision tree Rejected throw {forbidden} → 403 throw {unauthorized} → 401 document skipped doc_write_failures++ · logged replication job keeps running
Both direct and replicated writes hit the same target write path and pass through validate_doc_update. Returning accepts the write; throwing {forbidden} or {unauthorized} rejects that one document, which is skipped and counted in doc_write_failures while the replication job continues.

Immediate Triage / Prerequisites

Before writing a VDU, confirm the unwanted writes are actually arriving through replication and not through a leaked admin credential or a direct client call. Grep the target’s log for the two signatures a rejecting VDU emits, and check the replication job’s failure counter:

# VDU rejections surface with the thrown reason in the CouchDB log
grep -E "forbidden|unauthorized" /var/log/couchdb/couch.log | tail -n 20

# a replication job that is dropping documents shows a non-zero doc_write_failures
curl -s http://localhost:5984/_active_tasks | \
  python3 -c "import sys,json;[print(t['doc_id'],t.get('docs_written'),t.get('doc_write_failures')) for t in json.load(sys.stdin) if t['type']=='replication']"

If doc_write_failures is already climbing, a VDU is present and working; if it is zero while junk keeps landing, no validation is enforced yet. Prerequisites for the steps below: an Apache CouchDB 3.x target, an admin credential to deploy the design document, Python 3.8+ with the requests library (pip install requests), and network reach to the target. One rule to internalize first: a VDU runs on the target, on every write, including every replicated one — so a function that throws on valid data will not just block clients, it will stall replication by rejecting every inbound document. Validate carefully, and always let admins through so a broken rule never locks you out of the fix.

Step-by-Step Implementation

The VDU lives in a design document under the validate_doc_update key as a JavaScript function serialized to a string. The signature is function (newDoc, oldDoc, userCtx, secObj), documented in the Apache CouchDB validate_doc_update reference. Each step below includes a command to verify state before moving on.

  1. Write the VDU that checks userCtx.roles and the body. Reject unauthenticated writers with throw({unauthorized: ...}) and malformed or unauthorized content with throw({forbidden: ...}). Always short-circuit for the _admin role so administrative repair writes are never blocked:

    function (newDoc, oldDoc, userCtx, secObj) {
    var isAdmin = userCtx.roles.indexOf('_admin') !== -1;
    // 1. Authorization gate: non-admins must hold the writer role.
    if (!isAdmin && userCtx.roles.indexOf('telemetry_writer') === -1) {
        throw({unauthorized: 'you must hold the telemetry_writer role'});
    }
    // 2. Deletes by an authorized writer are always allowed.
    if (newDoc._deleted) { return; }
    // 3. Schema validation applies to everyone, admins included.
    if (typeof newDoc.device_id !== 'string') {
        throw({forbidden: 'device_id is required and must be a string'});
    }
    if (typeof newDoc.reading !== 'number') {
        throw({forbidden: 'reading is required and must be a number'});
    }
    // 4. Ownership is immutable once the document exists.
    if (oldDoc && oldDoc.device_id !== newDoc.device_id) {
        throw({forbidden: 'device_id is immutable once set'});
    }
}

  2. Deploy the design document. PUT it to the target database under a _design/ id. Because a design document is itself a document, the existing VDU (if any) validates this write — so deploy as admin:

    curl -s -X PUT http://admin:pass@localhost:5984/iot_telemetry/_design/write_guard \
  -H "Content-Type: application/json" -d @write_guard.json
# {"ok":true,"id":"_design/write_guard","rev":"1-..."}

    Confirm it is live by reading it back: curl -s http://localhost:5984/iot_telemetry/_design/write_guard | python3 -m json.tool.

  3. Test an allowed write. As an authorized writer (or admin), PUT a well-formed document and assert a 2xx:

    curl -s -X PUT http://admin:pass@localhost:5984/iot_telemetry/sensor-9 \
  -d '{"device_id":"sensor-9","reading":21.4}'
# {"ok":true,"id":"sensor-9","rev":"1-..."}

  4. Test a forbidden and an unauthorized write. A malformed body must return 403 Forbidden; a write from a session lacking the writer role must return 401 Unauthorized. Verify the status code and the reason echoed back:

    # missing the required numeric reading -> 403 forbidden
curl -s -o /dev/null -w "%{http_code}\n" -X PUT \
  http://admin:pass@localhost:5984/iot_telemetry/sensor-bad -d '{"device_id":"x"}'
# 403

  5. Observe rejects during replication. Start a pull into this target and watch the job skip the documents the VDU rejects. The count lands in doc_write_failures on both _active_tasks and _scheduler/docs, and the reason is logged — but the job itself stays running:

    curl -s http://localhost:5984/_scheduler/docs | \
  python3 -c "import sys,json;[print(d['id'],d['state'],d['info'].get('doc_write_failures')) for d in json.load(sys.stdin)['docs']]"

Complete Working Example

The script below is self-contained and runnable. It deploys the VDU, then proves all three outcomes end to end: an allowed admin write succeeds, a malformed write is rejected with 403 forbidden, and a write from a freshly created role-less user is rejected with 401 unauthorized. It uses only the standard library plus requests.

import sys

import requests

COUCH = "http://localhost:5984"
ADMIN = ("admin", "pass")   # a server/database admin credential
DB = "iot_telemetry"

# The VDU as a JavaScript source string stored under validate_doc_update.
VDU_SOURCE = """
function (newDoc, oldDoc, userCtx, secObj) {
    var isAdmin = userCtx.roles.indexOf('_admin') !== -1;
    if (!isAdmin && userCtx.roles.indexOf('telemetry_writer') === -1) {
        throw({unauthorized: 'you must hold the telemetry_writer role'});
    }
    if (newDoc._deleted) { return; }
    if (typeof newDoc.device_id !== 'string') {
        throw({forbidden: 'device_id is required and must be a string'});
    }
    if (typeof newDoc.reading !== 'number') {
        throw({forbidden: 'reading is required and must be a number'});
    }
    if (oldDoc && oldDoc.device_id !== newDoc.device_id) {
        throw({forbidden: 'device_id is immutable once set'});
    }
}
""".strip()


def deploy_vdu(session: requests.Session) -> None:
    """Create the database (idempotently) and PUT the design document."""
    session.put(f"{COUCH}/{DB}", timeout=30)  # 201 new, 412 if it already exists
    ddoc = {"validate_doc_update": VDU_SOURCE}
    # Overwrite in place if a previous _design/write_guard exists.
    existing = session.get(f"{COUCH}/{DB}/_design/write_guard", timeout=30)
    if existing.status_code == 200:
        ddoc["_rev"] = existing.json()["_rev"]
    resp = session.put(f"{COUCH}/{DB}/_design/write_guard", json=ddoc, timeout=30)
    resp.raise_for_status()
    print("deployed VDU:", resp.json()["rev"])


def make_roleless_user(session: requests.Session, name: str, pw: str) -> None:
    """Create a user with NO roles so its writes trip the authorization gate."""
    user = {"name": name, "password": pw, "roles": [], "type": "user"}
    r = session.put(f"{COUCH}/_users/org.couchdb.user:{name}", json=user, timeout=30)
    if r.status_code not in (201, 202, 409):  # 409 = already exists
        r.raise_for_status()


def try_write(session: requests.Session, doc_id: str, body: dict) -> int:
    """PUT a document and return the HTTP status without raising."""
    resp = session.put(f"{COUCH}/{DB}/{doc_id}", json=body, timeout=30)
    reason = ""
    if resp.status_code >= 400:
        reason = resp.json().get("reason", "")
    print(f"  {doc_id}: {resp.status_code} {reason}")
    return resp.status_code


if __name__ == "__main__":
    admin = requests.Session()
    admin.auth = ADMIN

    deploy_vdu(admin)
    make_roleless_user(admin, "edge-01", "edge-pass")

    print("allowed write (admin, valid body):")
    assert try_write(admin, "sensor-9", {"device_id": "sensor-9", "reading": 21.4}) < 300

    print("forbidden write (admin, malformed body):")
    assert try_write(admin, "sensor-bad", {"device_id": "x"}) == 403

    # Log in as the role-less user; its cookie session lacks telemetry_writer.
    user = requests.Session()
    login = user.post(f"{COUCH}/_session",
                      data={"name": "edge-01", "password": "edge-pass"}, timeout=30)
    login.raise_for_status()

    print("unauthorized write (role-less user, valid body):")
    assert try_write(user, "sensor-42", {"device_id": "sensor-42", "reading": 9.1}) == 401

    print("all three outcomes confirmed")
    sys.exit(0)

Run it against a live target and it prints the deployed design-document revision followed by three lines proving 2xx, 403, and 401 respectively — the exact contract a replication target relies on to keep bad documents out of its revision tree.

Gotchas & Edge Cases

  • A VDU runs on every write, including replicated ones. That is the whole point here, but it also means a rule that is too strict rejects legitimate inbound documents and the replication job’s doc_write_failures climbs silently. Test the VDU against real payloads from the source before deploying it in front of a live pull.
  • A throwing or broken VDU can block all writes. A runtime error in the function — a typo, a reference to an undefined variable, an unguarded property access on oldDoc when it is null on first write — makes every write to that database fail, not just the ones you meant to reject. Always guard oldDoc (it is null for a create) and keep the function total.
  • Admin party bypasses the gate. If the target has no admins configured, CouchDB runs in “admin party” mode and every request carries the _admin role, so your userCtx.roles check passes for everyone and the VDU appears to do nothing. Configure at least one admin before relying on role checks, and deploy the _replicator document schema with a real replication identity rather than an anonymous one.
  • A VDU cannot read other documents. It is a pure function of newDoc, oldDoc, userCtx, and secObj — it has no database handle, so uniqueness constraints, foreign-key checks, or “does this parent exist” rules are impossible here. Enforce cross-document invariants in your resolver or application tier instead.
  • Replicated writes carry the replication user’s identity, not the original author’s. userCtx reflects the credential the replication job authenticates as at the target, so a role gate authorizes the pipeline, not the edge device that first wrote the document. Rotate that credential carefully — see Rotating Replication Credentials Without Downtime.

Verification & Observability

Confirm enforcement at two levels: the individual write and the replication job. For a single write, assert the status code and the echoed reason. For the pipeline, watch doc_write_failures on the job and confirm its state stays running — a healthy VDU rejects documents without ever failing the job:

# per-job: docs_written should rise for good docs, doc_write_failures for rejected ones
curl -s http://localhost:5984/_scheduler/docs | \
  python3 -c "import sys,json;[print(d['id'],d['state'],d['info'].get('docs_written'),d['info'].get('doc_write_failures')) for d in json.load(sys.stdin)['docs']]"

# grep the log to see WHICH documents were rejected and why
grep -E "forbidden|unauthorized" /var/log/couchdb/couch.log | tail -n 20

A correctly filtered pipeline shows a steady docs_written, a doc_write_failures count that matches the number of bad documents at the source, and a state of running throughout. If instead the job flips to crashing or docs_written freezes at zero, the VDU is throwing on valid data — re-read the logged reason, and if it is opaque, the function itself has a bug rather than a policy that is too strict. To limit what even reaches the target, pair the VDU with source-side selective replication: a replication filter function or a Mango selector on the job drops documents before they cross the wire, so the VDU only has to police what genuinely arrives.

FAQ

Does validate_doc_update run on documents that arrive through replication?

Yes. Replication writes into the same storage path as a direct client PUT, so the target database’s VDU is invoked for every replicated document. If it throws, that one document is skipped and counted in the job’s doc_write_failures, but the replication job does not fail and continues with the next change. This is exactly why the target is the right place to enforce write policy — the source cannot be trusted to have done it.

What is the difference between throw({forbidden}) and throw({unauthorized})?

throw({forbidden: 'reason'}) yields HTTP 403 and means “you are authenticated but this write is not allowed” — use it for schema violations and content-level policy. throw({unauthorized: 'reason'}) yields HTTP 401 and means “you are not authenticated as someone permitted to do this” — use it for the userCtx.roles gate. During replication both simply cause the document to be skipped; the distinction matters most for interactive clients reading the status code.

Can I use a VDU to replicate only a subset of documents?

Not directly — a VDU only accepts or rejects writes at the target, it does not decide what the source sends. To limit which documents cross the wire in the first place, attach a replication filter function or a Mango selector to the replication job, which evaluates on the source and skips non-matching documents before transfer. Use selective replication to reduce traffic and the VDU to enforce policy on whatever still arrives; the two are complementary, not alternatives.

Part of: Security Boundaries in CouchDB Replication