Probes, Limits, and Whether You Need Kubernetes

Most Kubernetes incidents I have seen were not caused by Kubernetes being complicated. They were caused by two settings that everyone configures early, usually by copying an example, and that both fail in ways indistinguishable from an application problem: restarting pods that were healthy, and killing processes that were not leaking.

Probes: three kinds, and only one restarts you

The distinction is the single most important thing to get right, because the consequences differ enormously.

Readiness answers “should traffic come here?” Failing removes the pod from the load balancer. Nothing is killed, and it recovers on its own when the probe passes again. This is the safe probe.

Liveness answers “is this process broken beyond recovery?” Failing kills the container. It exists for genuine deadlock — a process that will never recover on its own.

Startup answers “has it finished booting?” and suspends the other two until it passes. It exists so a slow-starting application is not killed by a liveness probe before it has served anything.

The catastrophic configuration is pointing the liveness probe at an endpoint that checks dependencies:

livenessProbe:
  httpGet:
    path: /health      # and /health queries the database
    port: 3000

Now the database gets slow. Every pod's health check fails at once. Kubernetes concludes every pod is broken and restarts all of them simultaneously. They come up, reconnect to the still-slow database, fail again, and restart again. A recoverable dependency slowdown has become a total outage with a restart loop, and the application logs will show nothing wrong because nothing was wrong.

The rules that avoid this:

Liveness checks only the process itself. A handler that returns 200 without touching anything external. It answers “is the event loop responsive,” nothing more.
Readiness may check dependencies, because failing it only removes traffic — which is the correct response to a pod that cannot serve.
Use a startup probe for anything with a slow boot, rather than inflating the liveness delay.
When in doubt, omit the liveness probe. A pod with only a readiness probe is strictly safer than one with a bad liveness probe. Not restarting a wedged process is a smaller problem than restarting a fleet of healthy ones.

Requests and limits are different things

Requests are what the scheduler reserves — they decide placement. Limits are the ceiling the kernel enforces. They behave completely differently for the two resources, and this is where the second class of mystery originates.

Memory is incompressible. Exceed the limit and the process is killed immediately — OOMKilled, no warning, no chance to clean up. In the logs it looks like a crash with no stack trace, which sends people hunting for an application bug that does not exist.

CPU is compressible. Exceed the limit and you are not killed, you are throttled. The process is simply paused for part of each scheduling period. Nothing errors; latency just gets worse, in a way that does not correlate with anything in your code. A CPU limit set too low produces p99 latency spikes that survive every profiling session, because the process really was running fast — it just was not running.

Which leads to advice that surprises people:

Always set memory requests and limits, equal to each other. This gives the pod a guaranteed allocation and predictable behaviour.
Set CPU requests. Consider not setting CPU limits. A request guarantees a share; a limit prevents using idle capacity that nobody else wants. For latency-sensitive services, throttling a pod while the node sits idle is a pure loss.
For Node, tell the runtime about the limit. The V8 heap does not read cgroup limits by default, so it will happily grow toward a limit it does not know exists and get OOMKilled instead of running a garbage collection. Set --max-old-space-size to somewhat below the container limit.

Graceful shutdown, which is where deploys drop requests

On termination, Kubernetes does two things simultaneously: it sends SIGTERM to the container and it begins removing the pod from endpoints. Those propagate at different speeds, so for a short window after SIGTERM the pod is still receiving new traffic.

An application that exits immediately on SIGTERM therefore drops requests on every single deploy — a small number, intermittently, which is exactly the kind of error that gets written off as flaky clients.

The correct sequence:

On SIGTERM, fail readiness first and keep serving. This actively signals “stop sending me traffic.”
Wait a few seconds for the endpoint removal to propagate — a preStop sleep is the standard way to buy this time.
Stop accepting new connections, finish in-flight requests, then exit.
Ensure terminationGracePeriodSeconds exceeds your longest request, or the kernel kills you mid-request anyway.

Autoscaling on a metric that means something

The default horizontal autoscaler scales on CPU, which is a poor proxy for load in a Node service. A service that is mostly waiting on I/O sits at 20% CPU while its latency climbs, and never scales.

Scale on what actually indicates saturation for your workload: requests per second per pod, request queue depth, or consumer lag for a worker. These require a metrics adapter, which is a genuine cost, and it is the difference between autoscaling that responds to load and autoscaling that responds to coincidence.

Also set a PodDisruptionBudget. Without one, a node drain can evict every replica of a service at once — voluntarily, during a routine cluster upgrade, which is a self-inflicted outage that is entirely preventable with three lines of configuration.

Whether you need Kubernetes at all

Everything above is a standing cost. You need someone who understands probes, resource behaviour, ingress, storage classes, RBAC and upgrade cycles — and that is before your application does anything.

Kubernetes earns its cost when several of these are true:

Enough services that manual placement is genuinely a problem
Real need for bin-packing across a fleet, where utilisation savings pay for the complexity
Multiple teams needing isolated, self-service deploys
Workloads that genuinely benefit from the ecosystem — operators, service mesh, autoscaling on custom metrics
Someone whose job includes the cluster

For a handful of services and a small team, a managed container platform gives you rolling deploys, health checks, autoscaling and TLS for a fraction of the operational surface. That is not a lesser choice; for most products it is the correct one, and adopting Kubernetes because it is what serious companies use is how a two-person team acquires a full-time infrastructure job.

The honest version of the argument: Kubernetes solves problems of scale and organisational independence. If you have neither, you have bought the solution and kept the cost.

The short version

Liveness restarts you, readiness only removes traffic — so liveness must never check a dependency, and omitting it is safer than getting it wrong. Set memory request and limit equal, be cautious with CPU limits, and tell your runtime what the memory ceiling is. Fail readiness before exiting so deploys stop dropping requests. Scale on a metric that reflects your saturation. And be honest about whether the cluster is solving a problem you actually have.