Modeling For Documents, Not Against Them

A large share of complaints about MongoDB are really complaints about a third-normal-form schema that happens to be stored in a document database. Collections named after tables, an id in every document pointing at another collection, and application code doing joins in a loop. That design gives up what documents are good at and keeps none of what relational databases are good at.

The relational instinct

Relational modeling starts from a rule: each fact lives in exactly one place, and you join to reassemble it. That rule exists because storage was expensive and the join was cheap.

Document modeling starts from a different question: what does the application read together? Data read together should be stored together. Duplication is not automatically a defect; it is a deliberate trade of write complexity for read speed.

So the first thing to write down is not the entity diagram. It is the list of queries the application will actually run, and how often. In a relational database you can defer that and normalise; here it is the input to the design.

Embed or reference

Three questions decide it, in this order.

Is it read with the parent, essentially always? An order's line items are never wanted without the order. Embed them — one read returns the whole thing, and it is atomic without a transaction. A user's notification preferences, a post's tags, an address on a customer: all embed.

Is it bounded? Embedding is safe when the array has a ceiling you can name. Line items on an order: a few dozen. Comments on a post: unbounded, and therefore not embeddable — see below.

Does it change independently, or get referenced from elsewhere? A product is referenced by many orders and updated on its own schedule. It gets its own collection.

The pattern that resolves most real cases is neither pure embedding nor pure referencing, but referencing plus a denormalised snapshot:

{
  _id: ObjectId("..."),
  placedAt: ISODate("..."),
  items: [
    {
      productId: ObjectId("..."),   // reference: for looking up current product
      name: "Blue Widget",          // snapshot: what it was called when ordered
      priceMinor: 129900,           // snapshot: what was actually charged
      qty: 2
    }
  ]
}

This is not redundancy to be normalised away — it is correctness. The order must record the price the customer paid, not the price today. A purely referential design that resolves the product at read time will cheerfully reprint last year's receipts at this year's prices. The general rule: anything historical or contractual gets copied at write time; anything current gets referenced.

The unbounded array, and the 16MB ceiling

The most common way a MongoDB schema fails in production is an array that grows without limit. Comments on a post, events on a user, messages in a conversation — embedded because it was convenient at ten and fatal at fifty thousand.

Three things break, roughly in this order:

• Every read of the parent reads the whole array. You fetch a post to show its title and transfer four megabytes of comments.
• Updates rewrite the document. Growing a document past its allocated space means moving it, which is expensive and churns indexes.
• The 16MB document limit is a hard wall. Not a warning — writes simply start failing, in production, for your most active records, which are the ones you least want failing.

The fix depends on the access pattern. If the children are read independently and paginated, give them their own collection with an index on the parent id. If they are read together but there are many, use the bucket pattern — group them into documents of a fixed size, say a hundred per bucket, which keeps read counts low and documents bounded. For time-series-shaped data, bucket by time window.

A useful heuristic: if a user action can add to the array, it is unbounded. Line items are added by one checkout and then frozen. Comments are added forever.

Indexes work the same way, and get ignored the same way

Compound indexes follow the same leftmost-prefix rule as any B-tree. An index on { tenantId: 1, createdAt: -1 } serves a query on tenantId, and on tenantId sorted by createdAt, and does not usefully serve a query on createdAt alone. Equality fields first, then the range or sort field.

What is specific to MongoDB and worth knowing:

• Indexes on array fields are multikey — one index entry per element. Powerful for tag lookups, but an index on an array of a thousand elements creates a thousand entries per document, and you cannot create a compound index across two array fields.
• A sort that cannot use an index has a memory limit. Exceed it and the query fails rather than degrades, which is a surprising way to discover a missing index.
• explain("executionStats") is the equivalent of EXPLAIN ANALYZE. The number to look at is documents examined against documents returned. A ratio near one is healthy; ten thousand examined to return ten is a missing index.

Aggregation: use it, but know what it is

The aggregation pipeline is genuinely good, and the one rule that matters is ordering: filter and reduce early. A $match as the first stage can use an index; the same $match after a $lookup cannot, and you have joined the entire collection to discard most of it.

$lookup deserves a caution. It exists, it works, and it is not a join in the relational sense — it executes per input document and does not have a query planner optimising across the boundary. One $lookup on an indexed field over a filtered set is fine. Three chained lookups over a large collection is a report, and it should run somewhere other than a request handler.

If you find yourself writing multi-stage lookups regularly, that is information: the access pattern is relational, and the model — or the database — is the wrong shape for it.

Transactions exist, and are not the default answer

Multi-document transactions have been available for years and they work. They also carry costs that make them the wrong first instinct: they hold resources across the operation, they have a time limit, they require a replica set, and under contention they abort and must be retried by your code.

The document model's own answer is better where it applies: single document updates are atomic, so a design that keeps a transactional boundary inside one document needs no transaction at all. Decrementing inventory and appending to a reservation list in the same document is atomic for free.

Use transactions for the genuine cross-collection invariants that remain, and treat needing them constantly as a signal about the model.

The short version

Write down the queries before the schema. Store together what is read together, and embed only what is bounded and owned by its parent. Reference the current thing and snapshot the historical thing — an order records what was paid, not what it costs now. Treat any array a user can append to as unbounded and give it a collection or a bucket. Index by the same prefix rules as any B-tree, and check documents examined against returned. Keep transactional boundaries inside single documents where you can.