Why Notifications Need an Integrated Event Source

Every notification system I have worked on started the same way: one line, inside the handler that already had the data.

async createOrder(dto: CreateOrderDto) {
  const order = await this.orders.insert(dto);
  await this.notifications.send(order.userId, `Order ${order.number} confirmed`);
  return order;
}

There is nothing wrong with that line on day one. It is the cheapest thing that works, and shipping it is the correct call. The problem is never this line. The problem is the fortieth line like it, written by four different people, across three services, over two years.

What actually breaks

The failures are predictable enough that you can schedule them. All four come from the same root cause: the trigger is a code path, not a fact.

1. The same fact fires from more than one place

An order gets created by the checkout API. Then by the admin panel. Then by the failed-payment retry job. Then by the partner CSV importer that someone added last quarter. Three of them notify. One forgot. And one of the three sends twice, because it also calls the checkout service internally.

Nobody made a bad decision here. Every new code path that produces the same fact is simply a new place to remember, and remembering does not scale across people or years.

2. Delivery sits inside the request

If the email provider takes four seconds, checkout takes four seconds. If the send throws after the database commit, the order exists and nobody was told. If the send succeeds and the transaction then rolls back, you have notified someone about an order that does not exist. That is the dual-write problem, and it has no local fix — wrapping it in a try/catch just picks which failure mode you prefer, and retrying the handler double-sends.

3. Nobody can answer “why did I get this?”

You have a delivery log: a row saying an email went out at 09:14. What you do not have is the fact that caused it. So when support asks why a customer got a shipping confirmation for a cancelled order, the only available answer is reading code and guessing which of the four paths ran.

4. Every cross-cutting feature becomes a migration

Then the requests arrive, and they all arrive eventually: mute this project, send me a daily digest instead of twelve emails, nothing between 10pm and 8am, and also put it in Slack. Each one is a policy over all notifications. When notifications are forty unrelated call sites, each policy is forty edits and forty chances to miss one.

This is usually the moment a team builds a “notification service.” Worth being precise about what that fixes: if the new service is an HTTP endpoint that the same forty call sites now POST to, you have centralised templating and delivery, and nothing else. The trigger is still a call site. Preferences and digests are still unimplementable, because the service receives send instructions, not facts. Centralising the outbound half is real progress. It is just not the half that was hurting.

The distinction that fixes it

Separate the two vocabularies that the one-liner conflated:

An event is a fact. order.placed happened, at a time, to an entity, caused by an actor. It is true whether or not anyone is ever told. It has no audience, no channel and no copy.
A notification is a decision. Given that fact, somebody should probably see something, on some channel, in some language, possibly collapsed together with eleven similar facts, possibly not at all.

The call site is only qualified to state the fact. It genuinely does not know who is watching this order, what their quiet hours are, or whether this is their twelfth update in a minute. So it should not decide.

Integrated is doing real work in that phrase. It does not mean a feed per feature, and it does not mean a topic that only the notification service is allowed to read. It means one event source that every producer publishes its facts to and any consumer can subscribe to. The second parallel source is the moment you go back to remembering which of them a new code path is supposed to write to — which is the original problem wearing a nicer hat.

One disambiguation, because the words collide: this is not event sourcing. Event sourcing means the log is your state — you rebuild entities by folding events, and there is no row to read. Nothing here asks for that. Your database stays the source of truth exactly as it is; the event stream is a published record of things that already committed. You can run one without the other, and for most products you should.

So what the call site publishes is a fact, and only a fact:

// A fact. Stated once. No audience, no channel, no copy.
{
  id: "01JZ8QK2M7...",          // idempotency key, not a row id
  type: "order.placed",          // versioned contract
  occurredAt: "2026-08-15T09:14:02.113Z",
  tenantId: "acme",
  actor:   { type: "user",  id: "usr_8812" },
  subject: { type: "order", id: "ord_1043" },
  data:    { orderNumber: 1043, total: 8400, currency: "INR", itemCount: 3 },
  traceId: "b7f1c0..."
}

Compare it to the thing that usually gets published instead:

// Not an event. A send() with extra latency.
{ type: "notify", userId: "usr_8812", channel: "email",
  message: "Order #1043 confirmed" }

Audience, channel and wording are frozen at emit time here, which is exactly the coupling you set out to remove. Events that carry rendered copy give you a queue in front of your mailer — genuinely useful for latency, and worth nothing for preferences, digests or audit, because the decisions were already made upstream by code that was not qualified to make them.

Emit once, where the fact becomes true

Two rules make the event source trustworthy enough to build on.

Emit in the domain layer, not the controller. The API, the retry job, the importer and the admin panel all funnel through the same domain operation, so the event has exactly one emit site. If two places emit order.placed, one of them is a bug you have not found yet.

Emit in the same transaction as the state change, using an outbox table that a relay drains into the broker:

await this.db.transaction(async (tx) => {
  const order = await tx.orders.insert(dto);
  await tx.outbox.insert(orderPlaced(order));   // same commit boundary
  return order;
});

Now the event exists if and only if the state change committed. No phantom notifications, no silent misses. The price is at-least-once delivery: the relay can crash between publish and acknowledge, so every consumer has to be idempotent on event id. That is a cheap constraint to design for and an expensive one to retrofit.

What the notification service owns now

One pipeline, in one place, for every notification in the product:

Interest resolution — who cares about order.placed for this tenant? Owners, watchers, role-based subscribers, escalation rules.
Policy — per-user channel opt-ins, mutes, quiet hours, locale, tenant defaults, hard limits.
Aggregation — hold a short window and collapse “12 people commented” into one line. A daily digest is the same code with a longer window.
Rendering — templates per channel and locale, built from the event plus a fresh read of current entity state.
Delivery — provider adapters, backoff, dead-letter queue, per-channel rate limits.
Receipts — delivered, bounced, opened, clicked. These come back as events too, which is how you retire dead addresses and find out that nobody has opened a notification type in six months.

Each of those stages exists because somebody asked for something — mute this, batch that, prove you sent the other — and every one of them needs a stream to operate on. That is the whole argument in one sentence: preferences, batching, digests, deduplication and audit are all functions over a stream of facts, and you cannot write a function over a stream you never materialised.

The upside is that the pipeline collapses into something declarative, and adding the next notification type stops touching business code at all:

onEvent("order.placed", {
  audience: (e) => [ownerOf(e.subject), ...watchersOf(e.subject)],
  channels: ["in_app", "email"],                 // preferences filter this
  collapse: { key: (e) => e.subject.id, window: "5m" },
  render: (e, ctx) => ctx.template("order.placed", {
    order: ctx.load("order", e.subject.id),      // fresh read, not the payload
  }),
});

The part you cannot get any other way

Once the facts are retained rather than consumed and forgotten, a few things stop being incidents:

A provider outage becomes a reprocess from the dead-letter queue, not an evening spent reconstructing who missed what from application logs.
A broken template becomes: fix it, replay that window. Corrected emails go out to exactly the affected recipients.
A new notification type can be backfilled against real history and inspected before a single message is delivered.
“Why did I get this?” becomes one query. The notification references the event, the event references the entity, the actor and the moment.

Replay is only safe because of the idempotency key. Reprocessing an event must never re-send something already delivered, so the send key is (event id, recipient, channel) and it is checked at the delivery boundary. Without that, replay is just a way to spam your users twice as fast.

Notifications are only the first consumer

This is the part that justifies the word integrated, and it usually gets discovered by accident about three months in.

Once the facts exist as a stream that anything can subscribe to, the next four features stop being projects. Customer-facing webhooks are a consumer that POSTs events to a URL. An in-app activity feed is a consumer that writes them to a table. Analytics stops depending on someone remembering to fire a tracking call next to the business logic. Automation rules — when an order over ₹50,000 is placed, alert the account manager — are a consumer with a predicate, and they can be built by someone who has never opened the orders service.

None of those require a change to the producer. That is the actual return on the investment: the cost is paid once, at the emit site, and every subsequent consumer is additive. If you are weighing this decision purely on notifications, you are undercounting.

The bill

This is not free, and posts like this usually stop before admitting it.

A broker and a relay to operate. Consumer lag alerts, poison-message handling, dead-letter dashboards. A new on-call surface, permanently.
Events become a public contract. The moment a second consumer subscribes, the payload shape is an API. Additive changes only; a new required field means order.placed.v2. Without that discipline you trade forty call sites for a distributed schema problem, which is strictly worse.
Eventual consistency reaches the user. The classic: push arrives, the user taps it within 300ms, the read replica has not caught up, and they land on an empty screen. Render from a fresh read, treat the payload as a snapshot of the past, and accept that some notifications have to be delayed slightly on purpose.
Events arrive late and out of order. order.shipped can reach the consumer before order.paid. Never rebuild entity state by folding events inside the notification service — decide with occurredAt, render from the source of truth, and drop anything that is no longer relevant by the time it is processed.
Retention collides with privacy. A retained event log is personal data, and a deletion request now has to reach it too. Keep payloads to identifiers and non-sensitive facts, decide the retention window per event type on day one, and have an answer — redaction, crypto-shredding, or a short window — before legal asks for one.
Debugging goes distributed. Put a trace id on the event and propagate it to the delivery record, or “the email never arrived” becomes a multi-hop scavenger hunt.
Local development gets heavier. Ship an in-process bus implementation so running the app does not require a broker.

When not to build this

Do not build it because it is the better architecture. Build it when at least two of these are true:

you have a second delivery channel, or you know one is coming
users can configure what they receive
someone has asked for digests, batching or quiet hours
the same fact can be produced by more than one code path or service
you have to prove what was sent and why — support volume or compliance
you are past roughly a dozen notification types

With one channel, five notification types and a single deployable, a background job and a notifications table is the right answer. A broker buys you nothing there except operational work and the feeling of having done architecture.

Getting there without a rewrite

Nobody gets to stop shipping features for a quarter, so this runs as a dual-run migration.

Name the facts first. Write the event catalogue before any code: the type, precisely when it is true, its subject, its actor, its payload. Most of the value is in this document. If two people cannot agree on what order.placed means, no broker is going to rescue that.
Emit alongside the existing sends. Outbox and publish, with no consumers doing anything yet. Zero user-visible risk, and history starts accumulating the day it ships.
Shadow the consumer. Build the full pipeline, render notifications into a table, deliver nothing. Diff it against what the old call sites actually sent. This is the step that finds the two code paths which never notified anyone, and it is worth the whole exercise on its own.
Cut over one type at a time, highest volume first, and delete the inline send in the same pull request. Leaving both live past a single deploy is how a migration becomes a permanent double-send.

Rules of thumb

Events are facts, notifications are decisions. Never put copy in an event.
Emit at the source of truth, inside the transaction, from exactly one place.
Consumers idempotent on event id; sends idempotent on (event, recipient, channel).
Payloads carry identifiers and immutable facts. Anything that can change gets read at render time.
Version in the type name, additive changes only.
The event is not the notification and the notification is not the delivery. Three records, three lifecycles, three retention policies.

The short version

The reason to want an integrated event source is not elegance. It is that every notification feature users actually ask for turns out to be a function over a stream of facts, and the shape of your system decides whether those features are an afternoon or a quarter. Choose the shape when the second channel appears — not after the fourth.