Automating collMod Rollback Scripts

Turning a stored validator snapshot into an executable, one-click revert that any on-call engineer can run without reconstructing prior state is the precise operational task this page solves, under rollback automation for schema changes within the broader Zero-Downtime Schema Migration Automation in MongoDB framework. The reader outcome is a generator that reads a captured triple and emits a runnable restore — both a PyMongo function and a mongosh command — that is idempotent, safe to run twice, and guarded against replaying a stale snapshot onto a collection that has moved on. The problem this closes is the gap between having a snapshot and being able to use it under pressure: a captured artifact sitting in a store is only latent safety, and the moment a rejection spike starts hitting users is not the moment to hand-write a collMod. An automated rollback script converts that latent artifact into a single command wired into the deployment pipeline as a first-class revert action.

Operational Mechanics and Write-Path Impact

A rollback script does exactly one thing: it reapplies a previously captured validator, validationLevel, and validationAction to a collection through a single collMod. Because collMod is metadata-only, the restore never walks or rewrites documents — it swaps the enforcement contract the storage engine consults on the write path and nothing else. Its cost is therefore independent of collection size, and its effect is a single oplog entry the primary applies immediately and secondaries replay in order. That is why the restore’s time-to-recover is the replication lag of the slowest secondary, not a function of how much data the collection holds.

Idempotence is the property that makes a generated script safe to hand to an incident responder. Reapplying the same captured triple to a collection already in that state is a harmless no-op at the contract level — the resulting enforcement posture is identical whether the command runs once or three times. A well-built generator makes this explicit by reading the live options first and skipping the collMod when they already equal the target, so a nervous double-click never takes a redundant lock. The restore is convergent: it drives the collection to a known state rather than applying a relative delta, which is exactly the property you want when the current state is uncertain.

The one thing a rollback script must never assume is that the snapshot still describes the change it is meant to undo. Between capture and rollback, another actor may have applied a further change, so restoring an old artifact can revert more than one hop and silently discard intended state. Guarding against that stale-snapshot hazard — not the mechanics of the collMod itself — is where the real engineering lives.

Because the restore reapplies the exact serialized triple, type fidelity in the stored artifact matters as much here as at capture time. If the snapshot recorded a bsonType: "date" bound or a Decimal128 limit through a lossy encoder, the generated collMod reinstates a subtly wrong validator. The generator should deserialize with the same MongoDB Extended JSON codec used to capture the artifact, so a {"$date": ...} or {"$numberDecimal": ...} literal is reconstructed as the native BSON type the original rule carried.

Exact Diagnostic Fingerprints and Fast Resolution

Automated rollbacks fail in a small number of characteristic ways. Match the signature to route the fix fast.

Signature	Where it appears	Root cause	Resolution
Restore succeeds but the collection is stricter/looser than expected	After running the generated script	Snapshot was stale — a later change landed after capture	Compare expected_current against live options before restoring; re-capture and pick the correct artifact
pymongo.errors.OperationFailure code 13 (Unauthorized)	On the restore collMod	The rollback principal lacks the collMod action	Grant dbAdmin on the target database to the rollback runner
OperationFailure code 9 (FailedToParse)	On the restore collMod	Artifact was serialized lossily; a bound is now the wrong type	Regenerate from an Extended-JSON artifact; validate on a throwaway collection
Restore reattaches validator: {} and detaches enforcement	After running the script	Captured triple recorded an empty validator legitimately, or by accident	Confirm the artifact’s captured_at and commit; a true hard rollback keeps {}, an accidental one must be re-captured
Second run of the same script changes nothing	Expected behaviour	The restore is idempotent and already converged	None — this is the designed no-op; confirm with getCollectionInfos

The definitive check before trusting any generated rollback is a live-versus-artifact diff. Read the current options and confirm they match the state the snapshot was meant to revert from; a mismatch is the stale-snapshot signal.

def is_stale(live_options: dict, expected_current: dict) -> bool:
    """A snapshot is stale if the live state no longer matches what it was captured to undo."""
    return any(
        live_options.get(k) != expected_current.get(k)
        for k in ("validator", "validationLevel", "validationAction")
    )

Step-by-Step Playbook

The generator produces a guarded, idempotent restore and offers it to the pipeline as a one-click action. Each step is safe to re-run.

1. Load the snapshot and its expected-current state. Fetch the most recent artifact for the collection, plus the state the change moved the collection to, so the generator can detect drift:

   from pymongo import MongoClient
   
   client = MongoClient("mongodb://localhost:27017/?replicaSet=rs0")
   db = client.billing
   
   def load_snapshot(coll: str) -> dict:
       return db._schema_snapshots.find_one({"coll": coll}, sort=[("captured_at", -1)])
   # load_snapshot("invoices") -> the most recent captured triple for invoices

2. Guard against a stale snapshot. Read the live options and refuse to restore if the collection has drifted from the state this snapshot was built to undo:

   def live_options(coll: str) -> dict:
       info = db.command("listCollections", filter={"name": coll})
       batch = info["cursor"]["firstBatch"]
       opts = batch[0].get("options", {}) if batch else {}
       return {
           "validator": opts.get("validator", {}),
           "validationLevel": opts.get("validationLevel", "off"),
           "validationAction": opts.get("validationAction", "error"),
       }

3. Run the idempotent restore. Reapply the captured triple in one collMod, skipping the command when the collection is already in the target state:

   import logging
   from pymongo.errors import OperationFailure
   
   logger = logging.getLogger("collmod-rollback")
   
   def rollback(coll: str) -> bool:
       """Restore a collection to its snapshotted validator. Returns True if a collMod ran."""
       snap = load_snapshot(coll)
       if snap is None:
           raise RuntimeError(f"no snapshot found for {coll}; cannot roll back safely")
       target = {k: snap[k] for k in ("validator", "validationLevel", "validationAction")}
       if live_options(coll) == target:
           logger.info("%s already at snapshot state; no collMod issued", coll)
           return False
       try:
           db.command("collMod", coll,
                      validator=target["validator"],
                      validationLevel=target["validationLevel"],
                      validationAction=target["validationAction"])
       except OperationFailure as exc:
           logger.error("rollback collMod on %s failed (code %s): %s", coll, exc.code, exc)
           raise
       logger.info("rolled %s back to snapshot %s", coll, snap.get("_id"))
       return True
   
   if __name__ == "__main__":
       rollback("invoices")   # one-click revert, safe to run twice

4. Confirm the restore landed. Read the live options back and assert they equal the captured triple — the restore is only complete when the catalog agrees:

   assert live_options("invoices") == {k: load_snapshot("invoices")[k]
       for k in ("validator", "validationLevel", "validationAction")}
   print("rollback confirmed: live options match the snapshot")

The equivalent one-liner for an operator without the Python toolchain to hand reapplies the captured triple directly in the shell. Substitute the serialized artifact fields for the placeholders:

// mongosh — reapply a captured snapshot in one metadata-only command.
db.runCommand({
  collMod: "invoices",
  validator: { /* snapshot.validator */ },
  validationLevel: "moderate",   // snapshot.validationLevel
  validationAction: "warn"       // snapshot.validationAction
})
// Expected output: { ok: 1 }

Failure Modes & Rollback

Each step of an automated rollback has a distinct failure, and every recovery is itself a metadata-only operation with a replication-lag time-to-recover.

• Stale snapshot restores too far (step 1-2). If two changes landed but you load the older artifact, the restore reverts both and discards the intended intermediate state. The step-2 drift guard is the defence; recovery is to fetch the artifact whose expected_current matches live state, or roll forward to a known-good version rather than back.
• Missing snapshot (step 1). A change applied without a captured artifact leaves nothing to restore. The generator must hard-fail — a rollback with no snapshot is a reconstruction, exactly what the whole discipline exists to avoid. Recovery: rebuild the intended prior validator from schema versioning strategies for NoSQL and treat it as a forward change, not a rollback.
• Live/snapshot type drift (step 3). A lossily-serialized artifact makes the restore collMod fail to parse or reinstate a subtly wrong rule. Recovery: regenerate from an Extended-JSON artifact and dry-run it against a throwaway collection before running it in production.
• Partial propagation (step 4). The primary applies the restore immediately, but a lagging secondary briefly enforces the pre-restore rule. Recovery is simply to wait one replication lag; confirm with a w: "majority" write concern on the collMod if you need the restore acknowledged fleet-wide before proceeding.

The unifying property is that a rollback script never reverses data — it restores the enforcement contract. If documents were written under the intervening rule, undoing them is a separate forward migration, coordinated through the document transformation pipelines for schema migration machinery, not through the rollback script.

Frequently Asked Questions

Related

• Rollback Automation for Schema Changes — the parent workflow this generator plugs into, covering the full snapshot-apply-restore subsystem.
• Zero-Downtime Schema Migration Automation in MongoDB — the section reference for reversing an enforcement change as one choreographed unit with its data migration.
• Snapshotting Validator State Before collMod — the sibling procedure that produces the artifact this script consumes.
• Schema Versioning Strategies for NoSQL — reconstructing an intended prior contract when no snapshot exists, and versioning the artifacts a rollback pins between.