Real-Time That Survives a Second Instance

Real-time features have an unusual failure pattern: they work flawlessly in development, they work flawlessly in production on one instance, and they break the day someone adds a second one — not completely, which would be easier, but intermittently, for some users, in a way that looks like a client bug.

What actually breaks at two instances

An HTTP request is stateless. Any instance can serve it, which is why load balancing is boring. A WebSocket is a connection held open on one specific process, and that process is the only one that can write to it.

Two consequences, both immediate:

The handshake gets load-balanced away. Socket.io and similar libraries begin with HTTP polling and upgrade to WebSocket, which takes several requests. Round-robin those across two instances and the upgrade fails, because instance B has never heard of the session instance A started. The symptom is a connection that flaps, or one that silently stays on polling forever and generates far more load than anyone expects.

Emits reach only local connections. Half your users are connected to instance A and half to B. An event emitted on A reaches A's half. The other half sees nothing, with no error anywhere — the emit succeeded, it just had a smaller audience than intended. This is the bug that gets reported as “notifications are unreliable” and reproduces about half the time.

Sticky sessions, or skip the handshake problem

The first problem has two fixes and it is worth understanding both.

Sticky sessions pin a client to an instance, usually by hashing the client IP or issuing a cookie at the balancer. Every request in the handshake reaches the same process. This is the standard answer and it works, at the cost of uneven load distribution — instances do not drain evenly, and a deploy disconnects everyone pinned to the instance going down at once.

Or force a real WebSocket from the start. If you skip the polling transport entirely, the connection is a single upgrade request and there is no multi-request handshake to keep together. This removes the need for stickiness but gives up the fallback for networks where WebSockets are blocked — rarer than it used to be, but not zero on corporate networks.

Whichever you choose, the balancer must be configured for it: connection timeouts long enough that an idle socket is not killed, and upgrade headers passed through. A proxy that closes idle connections after sixty seconds will produce mysterious disconnections that look like client network problems, and the fix is a heartbeat interval shorter than that timeout.

Cross-instance fan-out

The second problem needs a shared bus. Every instance subscribes, and an emit is published to all of them so each can deliver to its own local sockets. With Socket.io this is one line via the Redis adapter; without it the same pattern is a Redis Pub/Sub channel per room and a handler that re-emits locally.

The important shift is architectural rather than mechanical: emitting becomes a message to the cluster, not a write to a socket. Which means the parts of your system that produce events do not need to know where anybody is connected — a background worker or a different service can publish, and whichever instance holds the connection delivers it. That decoupling is worth more than the fan-out itself, and it is why the pattern generalises well past chat.

Two things to get right. Rooms should be identifiers you already have — a user id, a conversation id, a tenant id — so that publishing does not require knowing the connection topology. And Pub/Sub delivers to currently-connected subscribers only, which is fine here because a message for a socket that does not exist has nowhere to go anyway; that is a different problem, below.

The reconnection gap

This is the part most implementations skip, and it is the one users actually notice.

A client's connection drops for eight seconds — a tunnel, a network switch, a laptop lid. Three events are emitted during that window. The client reconnects successfully and has no idea it missed anything. The UI is now silently wrong, and stays wrong until something else forces a refresh.

A real-time transport delivers to the connected. It does not deliver to the absent. If your feature needs the absent to catch up, you need somewhere durable to catch up from:

• Sequence your events per room or per user. The client stores the last sequence number it processed.
• On reconnect, the client sends that number and the server replays what came after — from a database table or a Redis stream, not from memory, because the instance it reconnects to is probably not the one it left.
• Above a threshold, tell the client to resynchronise instead of replaying thousands of events. A full refetch is cheaper than a long replay and simpler to reason about.

The design rule that follows: treat the socket as an accelerator over a source of truth, never as the source of truth. The socket says “something changed, and here is a hint”; the durable store is what the client can always fall back to. Features built the other way round — where the only copy of an event was the one pushed over the wire — are the ones that lose data permanently on a flaky connection.

Presence is harder than it looks

“Who is online” seems like a set of connected socket ids. Then: one user has three tabs, so presence is per-user and needs reference counting. A connection dies without a close frame, so the server believes someone is online for as long as the heartbeat timeout — and if the process crashed, forever.

The workable approach is to treat presence as a lease rather than a state. Each connection writes a key with a short TTL and refreshes it on every heartbeat. Online means the key exists. Nothing has to clean up after a crash, because the lease simply expires — the same reason this pattern shows up in service discovery.

And accept the resolution you are buying: presence with a thirty-second lease means someone can appear online for thirty seconds after they are not. Shorter leases cost more writes. There is no configuration that makes it exact.

When you do not need any of this

The infrastructure above is justified by bidirectional, low-latency, high-frequency communication. A lot of features labelled real-time are none of those.

• Server-Sent Events handle server-to-client streams over plain HTTP: no upgrade, no special proxy configuration, automatic reconnection built into the browser, and a Last-Event-ID header that gives you the replay mechanism described above for free. For notifications, live counters and progress updates — anything where the client only listens — this is the better default and it is consistently overlooked.
• Polling is not embarrassing. A request every ten seconds is trivial to reason about, works through every proxy, needs no sticky sessions and no adapter, and is genuinely correct for a dashboard that updates a few times a minute.
• A hosted service is worth pricing honestly. Sticky sessions, adapters, presence leases, replay and reconnection are a real amount of infrastructure to own for a feature that is often peripheral to the product.

The short version

A connection is state pinned to one process, so scaling out needs sticky sessions for the handshake and a shared bus for fan-out. Emit to the cluster rather than to sockets, so producers stop needing to know the topology. Sequence your events and replay on reconnect, because the transport will not — and keep the socket as an accelerator over a durable store rather than the only copy. Then check whether SSE or polling would have done the job.