A Pipeline That Fails for the Right Reasons

Two properties determine whether a pipeline helps or is merely endured: how long it takes, and whether people believe its failures. Everything else — which runner, which YAML dialect — is detail.

Duration is a people cost, not a compute cost

A twenty-minute run does not cost twenty minutes of machine time. It costs the author's attention: long enough to switch to something else, short enough that the switch back is disruptive. Reviews get batched, merges queue up behind each other, and the fix for a broken main branch is twenty minutes away from being verified.

The threshold worth targeting is roughly ten minutes for the checks that gate a merge — short enough to wait through. Everything slower belongs somewhere that does not block: nightly, post-merge, or on a schedule.

The second property is trust. A pipeline that fails intermittently for reasons unrelated to the change teaches everyone to re-run rather than read, and once that habit forms a real failure gets re-run too. A flaky check is worse than a missing one, because it consumes attention and provides no signal.

Caching, which is where most of the time goes

Most pipelines spend the majority of their time reinstalling things that did not change. Two levels are worth having.

Dependencies, keyed on the lockfile so the cache invalidates exactly when dependencies change and not otherwise:

- uses: actions/setup-node@v4
  with:
    node-version: 22
    cache: npm            # keyed on package-lock.json automatically

- run: npm ci             # now restores from cache in seconds

Build output, keyed on the source files that produce it. This is the one people skip, and it is often the bigger win — a framework build cache can turn a three-minute compile into twenty seconds when only a handful of files changed.

The rule for cache keys is to include everything that affects the output and nothing that does not. Too narrow and you get stale results; too broad and you never hit the cache. And always provide a restore-key prefix so a near-miss falls back to a recent cache and does a partial rebuild rather than starting cold.

One structural note: splitting into many small jobs looks parallel and often is not, because each job starts on a fresh machine and repeats checkout, setup and dependency restore. Three jobs of two minutes each with ninety seconds of setup is worse than one job of five minutes. Parallelise when the work is genuinely long; otherwise share the setup.

Running only what changed

In a monorepo, running everything on every commit is the default and it does not survive growth. A documentation change should not run the backend integration suite.

The cheap version is path filters: trigger a workflow only when files under a given directory change. Fine for a repository with a few clearly separated areas, and it breaks down once packages depend on each other — a change to a shared library must test everything downstream of it, and a path filter does not know that.

The real version is a dependency graph, which is what monorepo tooling provides. Given the merge base, it computes which projects are affected by the diff, including transitively, and runs tasks only for those. That is the difference between a monorepo that gets slower with every package and one that does not.

A caution worth stating: affected-only computation depends on a correct base commit. Shallow clones frequently break it, producing either everything or nothing. If your affected detection suddenly runs the whole world, check the fetch depth before you check the configuration.

What should actually block a merge

Not everything you run should prevent merging. The question for each check is whether a failure means the change is wrong, or merely means something is worth knowing.

Blocking:

• Type checking — a failure is definitionally broken code
• Unit and integration tests
• Lint errors, though not warnings
• The build itself
• Dependency vulnerability scanning at high severity

Not blocking:

• Coverage thresholds. A percentage target changes behaviour rather than quality — people write tests that execute lines without asserting anything. Report the delta in the pull request and let a human judge.
• Bundle size limits, unless the budget is genuinely a product requirement. Report the change; a legitimate increase should not need someone to override a gate.
• Slow end-to-end suites. Run a small critical-path subset on the pull request and the full suite post-merge or nightly.

And separate “failed” from “could not run.” A job that dies because a registry was unreachable is not a test failure, and treating it as one is how the re-run habit begins. Retry infrastructure steps automatically; never retry a failing test to make it pass.

Preview environments earn their keep

A deployed URL per pull request changes review from reading a diff to using the change. Designers and product people can look without a local setup, and bugs that only appear against real infrastructure are caught before merge rather than after.

The part that determines whether it is useful is data. A preview against an empty database demonstrates nothing. Seed it with realistic — and anonymised — data, or point it at a shared staging database while being explicit that migrations there are shared and therefore need care.

Have a teardown story from the start. Preview environments that accumulate are a cost line and, if they hold data, a compliance question. Destroy them when the pull request closes.

Secrets, and the fork problem

The most common serious mistake in a public repository's CI is running a workflow with access to secrets against code from a fork. A pull request can modify the workflow file, or a build script it calls, and print your deployment credentials.

• Do not expose secrets to workflows triggered by external pull requests. The default trigger runs in the fork's context without them, which is correct — resist the temptation to use the elevated trigger to make an integration test work.
• Scope every credential. A deploy token that can only deploy, a registry token that can only push one package. Assume each will eventually be exposed and limit the consequence.
• Prefer short-lived federated credentials over long-lived stored keys where your cloud supports it. A token that exists for the duration of one job cannot be leaked usefully later.
• Pin third-party actions to a commit SHA, not a tag. A tag can be moved to point at new code, which means a supply-chain compromise arrives without you changing anything.

The short version

Target ten minutes for the gating checks and move everything slower off the critical path. Cache dependencies keyed on the lockfile and build output keyed on source. Use a dependency graph rather than path filters once packages depend on each other. Block on correctness, report on judgement, and never make a flaky check required. Give every pull request a preview with real-shaped data. And assume any secret reachable from a fork is already public.