Backward-Compatible Schema Evolution Patterns

Deciding whether a proposed change to a $jsonSchema validator can ship without breaking live traffic is the operational question this page settles, sitting under schema versioning strategies for NoSQL within the broader MongoDB JSON Schema Validation Architecture. The reader outcome is a working taxonomy — which edits are safe to apply immediately, which are breaking until a backfill completes, and which demand a transition window where both the old and new document shapes validate simultaneously — plus the exact anyOf construction and the optional-then-required rollout that turn a would-be breaking change into a series of individually safe steps. The trap is that MongoDB’s permissive write path hides the danger: a tightened rule applies only to future writes, so a change that looks clean at collMod time can detonate later as a code 121 the moment an application updates a grandfathered document.

Operational Mechanics and Write-Path Impact

Compatibility is a property of the direction a constraint moves, not of the field it touches. Because the validator is evaluated on every insert, update, replace, and findAndModify, the only writes at risk are ones that must satisfy a rule stricter than the one under which their data was first produced. That yields a clean rule of thumb: loosening is always safe, tightening is breaking until every document already complies. Adding an optional field, widening an enum, relaxing a minLength, or dropping a required entry can never reject a write that previously passed. Adding a required field, narrowing an enum, tightening a bsonType, or setting additionalProperties: false can reject documents that were perfectly legal a moment earlier.

The subtlety is that “already complies” is asymmetric between inserts and updates, and it interacts with validationLevel. Under strict, a tightened rule is checked against every write regardless of the target’s current state, so a legacy document becomes a latent rejection on its next touch. Under moderate, updates to a currently-invalid document are grandfathered, which buys a remediation window but does not make the change compatible — it only defers the failure. Neither level re-scans stored data, because collMod is metadata-only; compliance of historical documents is always tested lazily, one write at a time. This is why a “backward-compatible” classification is really a claim about the entire installed base of documents, not about the schema diff in isolation.

The technique that spans a genuine breaking transition is anyOf over versioned shapes: the validator accepts a document if it matches either the old contract or the new one. During the window, producers migrate at their own pace, both shapes persist cleanly, and once the backfill guarantees every document matches the new shape, the old branch is dropped in a final tightening collMod. The embedding schema_version fields in MongoDB documents pattern pairs naturally with this: each anyOf branch keys off a specific schema_version, so the accepted shape and the declared version stay in lockstep.

Exact Diagnostic Fingerprints and Fast Resolution

The classification below is the reference you consult before authoring any collMod. It maps each change type to its compatibility class and the migration it demands. “Safe” means the diff can ship in a single metadata-only collMod with no backfill; “breaking” means writes will fail until every stored document already satisfies the new rule.

Change	Compatibility	Required migration
Add optional field (properties only, not in required)	Safe	None — deploy directly
Widen an enum (add allowed values)	Safe	None — deploy directly
Relax minLength / minimum / minItems downward	Safe	None — deploy directly
Remove a required entry	Safe	None — deploy directly
Add a required field	Breaking	Backfill the field on every document first
Narrow an enum (remove allowed values)	Breaking	Remediate documents holding the removed value
Tighten bsonType (e.g. string → int)	Breaking	Coerce mismatched values in place first
Set additionalProperties: false	Breaking	Strip or model every stray field first
Any breaking change under a live window	Transitional	anyOf of old and new shapes until backfill converges

Whether a specific documented base will survive a tightening is a query, not a guess. Because $jsonSchema doubles as a read operator, wrapping the new schema in $nor counts exactly how many stored documents the change would reject:

// How many documents would the tightened rule reject? Dry run, no writes.
db.orders.countDocuments({ $nor: [{ $jsonSchema: {
  bsonType: "object",
  required: ["schema_version", "region", "status"],   // 'region' is the new required field
  properties: { region: { bsonType: "string" } }
}}]})
// Expected output: 0 means the change is safe to ship as-is; any positive integer is your backfill backlog.

A non-zero result is definitive proof the change is breaking for your installed base regardless of what the taxonomy says in the abstract — a technically-safe diff can still be breaking if a prior bug produced non-conforming data. Treat the count as the authoritative gate.

Step-by-Step Playbook

This sequence turns a breaking addition — introducing a new required field — into an additive rollout where no write is ever rejected mid-flight. It is the optional-then-required pattern made concrete.

1. Ship the field as optional. Add the new field to properties but not to required, and keep the action permissive. Existing documents without the field still validate; new documents may carry it:

   db.runCommand({
     collMod: "orders",
     validator: { $jsonSchema: {
       bsonType: "object",
       required: ["schema_version", "status"],
       properties: {
         schema_version: { bsonType: "int", enum: [2, 3] },
         status: { bsonType: "string", enum: ["open", "settled", "void"] },
         region: { bsonType: "string", minLength: 2 }   // optional for now
       }
     }},
     validationLevel: "moderate",
     validationAction: "warn"
   })
   // Expected output: { ok: 1 }

2. Deploy producers that populate the field. Roll out the application change that always sets region on new writes. From this point the backlog stops growing; only pre-existing documents lack the field.

3. Backfill the field on historical documents. Assign a correct value to every document missing it, in unordered batches so one failure never halts the run:

   from pymongo import MongoClient
   
   client = MongoClient("mongodb://localhost:27017")
   orders = client.shop.orders
   
   # Derive a real value where possible; fall back to an explicit sentinel.
   result = orders.update_many(
       {"region": {"$exists": False}},
       [{"$set": {"region": {"$ifNull": ["$legacy_region", "unknown"]}}}]
   )
   print("backfilled", result.modified_count, "documents with region")
   # Expected output: backfilled 4712 documents with region

4. Verify zero gaps, then promote the field to required. Confirm no document lacks the field, then add it to required and tighten the action in one collMod:

   db.orders.countDocuments({ region: { $exists: false } })   // must be 0
   db.runCommand({
     collMod: "orders",
     validator: { $jsonSchema: {
       bsonType: "object",
       required: ["schema_version", "status", "region"],       // now required
       properties: {
         schema_version: { bsonType: "int", enum: [2, 3] },
         status: { bsonType: "string", enum: ["open", "settled", "void"] },
         region: { bsonType: "string", minLength: 2 }
       }
     }},
     validationLevel: "strict",
     validationAction: "error"
   })
   // Expected output: { ok: 1 }

When the breaking change is a bsonType tightening rather than a new field, the same skeleton applies but step 3 becomes an in-place type coercion; the reusable pattern for that is coercing bsonType mismatches in place. Where you cannot convert every document, hold the transition open with an anyOf of both shapes indefinitely rather than forcing a lossy coercion.

Failure Modes & Rollback

• Promoting to required while the backfill is incomplete (step 4). Any document still missing the field becomes a code 121 on its next update under strict. The dry-run count is the gate precisely to prevent this; if the count was skipped, roll back to moderate plus warn and finish the backfill.
• A narrowed enum that a live producer still emits. Removing an allowed value while a service is still writing it produces a steady enum rejection stream. This is a producer-coordination failure, not a data failure — roll the enum back to include the value, coordinate the producer deploy, then narrow again.
• anyOf window left open forever. A transition window is a migration instrument, not a steady state; a permanently dual-shape validator silently permits new writes in the deprecated shape. Alert on how long the old anyOf branch has matched any recent write, and close the window once the backfill converges.

Rollback for any tightening is a single atomic, metadata-only collMod that restores write availability without touching data — time-to-recover is the replication lag of your slowest secondary, typically seconds:

// Soft rollback: revert to the looser previous contract, keep telemetry.
db.runCommand({ collMod: "orders", validationLevel: "moderate", validationAction: "warn" })

// Hard rollback: detach the validator entirely for emergency recovery.
db.runCommand({ collMod: "orders", validator: {}, validationLevel: "off" })

Because a mis-classified change usually surfaces as an unexpected rejection spike, keep the previous validator definition versioned so the soft rollback restores a known-good contract rather than an improvised one. Choosing how strict the terminal contract should be — whether a transition should end at strict plus error or settle at moderate — is the subject of strict vs moderate validation levels.

Frequently Asked Questions

Related

• Schema Versioning Strategies for NoSQL — the parent workflow this taxonomy of safe and breaking changes serves.
• MongoDB JSON Schema Validation Architecture — the overarching enforcement architecture in which every schema change lands.
• Embedding schema_version Fields in MongoDB Documents — the version stamp each anyOf branch keys off during a transition.
• Strict vs Moderate Validation Levels — how the terminal enforcement level shapes where a transition ends.