What Serverless Changes About Your Backend
Serverless does not remove operational concerns. It moves them somewhere your instincts do not look — connections, cold starts, and nowhere to put background work.
Serverless is sold as not having to think about servers, and the useful correction is that you stop thinking about some things and start thinking about others. The others are unfamiliar, they are not in your monitoring, and several of them only appear under load.
The execution model, and what follows from it
An instance is created to handle a request, may handle a few more, and is then frozen and discarded. There is no long-lived process. Every assumption a traditional Node service makes — that startup happens once, that a connection pool is shared, that an in-memory cache persists, that a scheduled job can run in-process — is now false.
The important nuance is that instances are neither per-request nor permanent. Module-level state does survive between requests on the same warm instance, which is why some things appear to work in development and behave strangely in production: you are relying on reuse that is real but not guaranteed and not shared.
Connections, which is the one that takes you down
A traditional server opens a pool of perhaps ten connections and holds them for its lifetime. One process, ten connections, regardless of traffic.
Serverless has no single process. Under load you have two hundred concurrent instances, each opening its own connection, and Postgres allocates a backend process per connection with real memory cost. Default limits are in the low hundreds. You do not degrade gracefully — you hit the ceiling and every subsequent connection is refused, which surfaces as your entire application erroring at once while CPU sits idle.
The fix is a connection pooler in transaction mode. Instances connect to the pooler, which multiplexes many client connections onto few database ones, handing a connection back after each transaction rather than holding it for the client's lifetime.
Two consequences of transaction mode worth knowing before you are debugging them. Prepared statements and session-level state do not survive across transactions, so ORMs that rely on them need configuring for it. And anything requiring session continuity — advisory locks, LISTEN/NOTIFY, temporary tables — will not work, because the next statement may be on a different underlying connection.
Also: set the client pool size to one or two per instance. The instinct to configure a pool of twenty is exactly backwards here — you have hundreds of instances, each needing almost nothing.
Cold starts, honestly
A request arriving with no warm instance pays for creating one. The magnitude depends on things you control more than people assume.
- Bundle size. The code must be fetched and initialised. Importing a large SDK at module scope to use one function is paid on every cold start.
- Work at module scope. Anything at the top level runs during initialisation. Reading configuration, building clients and compiling schemas are all cold-start cost — worth doing lazily if a given route may not need them.
- Runtime. A lighter runtime starts faster but has a restricted API surface, which usually rules out most database drivers.
Keep it in proportion. If your median response is 300ms and cold starts add 400ms to a small percentage of requests, that is a p99 problem worth a look rather than an architectural crisis. Measure the actual rate before optimising for it — under steady traffic, most requests are warm.
There is nowhere to put background work
This is the change that most often forces a redesign, and it is structural rather than a matter of tuning.
In a normal server you can respond to the client and continue working — send the email, generate the thumbnail, update the search index. In a serverless function, the instance is frozen after the response. Work started and not awaited may complete, or may be suspended mid-flight and resumed much later, or may never finish. The failure is silent and intermittent, which is the worst combination.
Everything that used to be “after the response” now needs a destination:
- A queue with a separate consumer, which is the general answer
- The platform's explicit after-response primitive, where one exists — the point being that it is explicit, so the runtime knows to wait
- A scheduled invocation for anything periodic, since there is no process to hold an interval timer
The execution ceiling forces the same conclusion from another direction. Functions have a maximum duration — tens of seconds typically. A report that takes two minutes cannot be a request; it has to become a job with a status the client can poll or subscribe to.
Statelessness has consequences beyond the obvious
- In-memory caching is per-instance and unpredictable. A cache in module scope gives some instances a hit rate and others nothing, and it multiplies memory by instance count. Shared cache or no cache.
- Rate limiting must be external. An in-memory counter limits per instance, so a hundred instances enforce a hundred times your intended limit.
- The filesystem is ephemeral and mostly read-only. Temporary space exists for scratch work within one invocation; anything that must persist goes to object storage.
- WebSockets do not fit the model at all — a long-lived connection is the opposite of a short-lived instance. That is a managed service or a separate long-running process.
Cost shape, which is the actual decision
Serverless bills per invocation and per unit of execution time. That is excellent for spiky or low traffic — you genuinely pay nothing at night — and it inverts at steady high volume, where a continuously busy function costs considerably more than an always-on instance doing the same work.
The reasonable summary: serverless wins on variable load, on operational simplicity for small teams, and on anything where scaling to zero matters. It stops winning when traffic is steady and high, when workloads are long-running, or when you need session-level database features.
And a hybrid is normal rather than a failure. Serverless for the request path, one always-on worker for queues, schedules and websockets. That arrangement gets the deployment simplicity where it helps and keeps a process where you genuinely need one.
The short version
Put a transaction-mode pooler in front of the database and set client pools to one or two, or you will hit the connection ceiling as a total outage. Keep module scope cheap and measure your real cold-start rate before optimising. Assume work not awaited before the response does not happen — move it to a queue. Nothing in memory is shared, so caching and rate limiting move out. Then check whether your traffic shape is the one this pricing model is good at.