Building Indexes Before Tightening Validators

The exact operational rule this page answers is when to create an index relative to a collMod that tightens a $jsonSchema: create — and confirm READY — every index the new schema or its queries depend on, then change the validator, never the reverse. It sits under Index Rebuild Ordering During Schema Migration within the broader Zero-Downtime Schema Migration Automation in MongoDB framework. The reader outcome is a sequence that never leaves a supporting index missing at the moment enforcement changes: no query dropping to a collection scan, and no unique index rejecting the very backfill that would have made the collection compliant. The trap is subtle because both halves look independently correct — the index builds fine, the collMod succeeds — yet run in the wrong order they interlock into either a latency regression or a wave of write rejections that reads like an outage.

Operational Mechanics and Write-Path Impact

A validator and an index touch the write path at different moments, and the order they are applied decides whether they cooperate or collide. When collMod tightens a $jsonSchema, the new rule takes effect immediately for subsequent writes, subject to validationLevel. If that schema assumes a value is unique, or if the application queries a newly-required field, the index behind that assumption must already exist. A unique index is the sharpest case: its constraint is evaluated before $jsonSchema, so building it after a backfill has already inserted duplicates means the build itself aborts, and building it before deduplication is impossible. The safe order threads between both: deduplicate, build the unique index to READY, then tighten the validator.

The three states a migration can be in map cleanly to three outcomes.

Ordering	What the write path sees	Result
Index READY → collMod tightens validator	Supporting index present when the new rule and queries go live	Correct: reads use the index, writes satisfy uniqueness
collMod tightens validator → index built after	New required/unique contract live with no index behind it	Queries scan; a unique constraint the schema implies is unenforced until built
Validator tightened during an unfinished backfill	Writes rejected with code 121 before data can comply	Rejection wave; looks like an outage, is an ordering bug

Building the index first is cheap insurance: from MongoDB 4.2 the build is the optimized hybrid build that yields to live traffic, and from 4.4 a single createIndexes covers every data-bearing member through a commit quorum. Because the build does not block the collection for its duration, there is no operational reason to defer it until after the validator — and every reason not to. The validator change itself is metadata-only and instantaneous, exactly as with the level transitions in Strict vs Moderate Validation Levels; it is the dependency that takes time, so the dependency goes first.

Exact Diagnostic Fingerprints and Fast Resolution

Wrong-order symptoms split by which artifact was missing when enforcement changed. Match the signature to the cause before reacting.

Signature	Where it appears	Root cause	Resolution
code: 121, Document failed validation spikes right after a collMod	Driver / client	Validator tightened before the backfill that makes documents compliant finished	Revert validator to warn, finish the backfill, re-promote
E11000 duplicate key, code: 11000 during createIndexes	Index build	A unique index is being built over data that already contains duplicates	Deduplicate first, or add a partialFilterExpression; then rebuild
Query P95 jumps to a COLLSCAN after deploy	explain / slow-query log	The new query pattern shipped, but its index is not yet READY	Confirm the index is in getIndexes; hold the app change until it is
code: 279 (IndexBuildAborted)	Index build	Build killed by stepdown, disk exhaustion, or a concurrent drop	Stabilize, then re-issue createIndexes from scratch — no partial index remains

The single most useful check is whether the index is genuinely finished, because “the createIndexes command returned” and “the index is READY on every member” are not the same claim. This snippet is the gate to run immediately before any collMod:

from pymongo import MongoClient

client = MongoClient("mongodb://primary:27017/?replicaSet=rs0")
db = client.production_db

built = {ix["name"] for ix in db.orders.list_indexes()}
in_progress = list(client.admin.aggregate([
    {"$currentOp": {"allUsers": True}},
    {"$match": {"command.createIndexes": {"$exists": True}}},
]))

ready = "order_ref_unique" in built and not in_progress
print("Safe to tighten validator:", ready)  # must be True before collMod

Step-by-Step Playbook

The sequence guarantees the index exists and is proven before the validator ever tightens. The example adds a unique order_ref to an orders collection whose validator will require it.

1. Deduplicate so a unique index can build. A unique index cannot be created over duplicate keys, so clear them first. This is also part of the backfill discipline in Backfilling New Required Fields in Batches:

db.orders.aggregate([
  { $group: { _id: "$order_ref", ids: { $push: "$_id" }, n: { $sum: 1 } } },
  { $match: { n: { $gt: 1 } } }
]).forEach(g => print(`duplicate order_ref ${g._id}: ${g.n} docs`))
// Expected output: one line per duplicated order_ref — resolve each before step 2

2. Create the index with a commit quorum. Build it simultaneously across every data-bearing member; createIndexes returns once the quorum commits:

db.runCommand({
  createIndexes: "orders",
  indexes: [{ key: { order_ref: 1 }, name: "order_ref_unique", unique: true }],
  commitQuorum: "votingMembers"
})
// Expected output: { numIndexesBefore: 1, numIndexesAfter: 2, ok: 1 }

3. Verify the index is READY. Confirm it appears in listIndexes and that no build for it remains in $currentOp. This is the go/no-go gate — do not proceed while a build is in flight:

built = {ix["name"] for ix in db.orders.list_indexes()}
in_progress = list(db.client.admin.aggregate([
    {"$currentOp": {"allUsers": True}},
    {"$match": {"command.createIndexes": "orders"}},
]))
assert "order_ref_unique" in built and not in_progress, "index not READY — do not collMod"

4. Tighten the validator. With the unique index proven, apply the schema that requires order_ref. Because the constraint is now backed, no compliant write is rejected:

db.runCommand({
  collMod: "orders",
  validator: { $jsonSchema: {
    bsonType: "object",
    required: ["order_ref", "tenant_id"],
    properties: {
      order_ref: { bsonType: "string", pattern: "^ORD-[0-9]{8}$" },
      tenant_id: { bsonType: "string" }
    }
  }},
  validationLevel: "moderate",
  validationAction: "error"
})
// Expected output: { ok: 1 }

Failure Modes & Rollback

Each step fails distinctly, and each has a bounded recovery:

• Duplicate-key abort at step 2 (code 11000). The unique build hit residual duplicates step 1 missed. The build leaves no partial index; re-run the step 1 group query, clear the stragglers, and rebuild. Time-to-recover: the deduplication pass.
• Premature collMod at step 4 (code 121 wave). If the validator was tightened before the backfill completed, compliant-looking writes are rejected. Roll the validator back to warn immediately, finish the data work, then re-promote to error.
• IndexBuildAborted (code 279) at step 2. A stepdown or disk-exhaustion killed the build. No partial index remains; stabilize the primary and disk, then re-issue createIndexes.

Rollback is one metadata command per artifact and loses no data, because the index was added rather than an existing one dropped. Revert the validator first so writes stop being rejected, then drop the index if you are abandoning the change:

// 1. Stop rejecting writes — soft-revert the validator.
db.runCommand({ collMod: "orders", validationLevel: "moderate", validationAction: "warn" })

// 2. Remove the index that was added for the new contract.
db.orders.dropIndex("order_ref_unique")
// Expected output: { nIndexesWas: 2, ok: 1 }

Both take effect on the primary immediately and replicate through the oplog, so time-to-recover is the replication lag of your slowest secondary — typically seconds. Capturing the prior validator so this revert is exact is handled by Snapshotting Validator State Before collMod.

Frequently Asked Questions

Related

• Index Rebuild Ordering During Schema Migration — the parent workflow covering the full order-of-operations for indexes and validators across a migration.
• Zero-Downtime Schema Migration Automation in MongoDB — the section reference this sequencing rule fits inside.
• Backfilling New Required Fields in Batches — the backfill that must finish before a validator requiring the field is promoted to error.
• Snapshotting Validator State Before collMod — capturing the prior validator so the rollback command here is exact rather than reconstructed.