Skip to content
AhmadKhidir

POST

Slow tests are a debt trap

Aug 20266 MIN READ

#testing#ci#developer-experience#devops

I have seen the exact moment a test suite dies. It is never announced. There is no meeting about it. One day a developer is waiting eleven minutes for the tests to run, and they start skipping the suite locally and pushing straight to CI, because CI is faster. Then CI starts taking half an hour and blocking merges, and the team invents a rule about when you are allowed to run it. Then the suite gets split and only the fast slice runs on every commit, and the slow slice runs nightly, and the nightly slice goes red and stays red for a week while everyone ignores it.

The tests did not fail. The suite died of slow. And the team never consciously decided to stop testing. They made a hundred small decisions, each one reasonable, and the outcome was a suite that nobody runs, which is a suite that might as well not exist.

A suite that is not run is not a suite

This is the first principle, and everything else follows from it. Tests only provide value when they run, and they only run when the cost of running them is low enough. A developer will run a suite that takes ninety seconds. They will not run a suite that takes forty minutes. This is not a discipline problem. It is a human behavior problem, and the fix is to make the fast path the good path, not to lecture people about responsibility.

Treat test runtime like you treat compile time, because it is the same cost. Every second the suite takes is a second every developer on the team pays, on every commit, forever. A ten second test is cheap. A forty minute suite is a tax on every change, and the tax compounds with the number of commits.

The usual suspects

When a suite gets slow, the causes are remarkably consistent. The biggest one is integration tests that spin up the real database, the real cache, the real file system, for every case. These are the tests that take seconds each and hundreds of them take an hour. The second is the missing distinction between unit tests and integration tests, so everything runs everywhere, and CI runs the slow stuff on every commit even when nothing touched it. The third is test data setup that recreates the entire world for each test, when a couple of rows would do.

There is also the slow drift problem, which is the sneaky one. No single commit makes the suite slow. Every commit adds a few milliseconds, or a test that sleeps, or a fixture that loads more data. Nobody notices, because the suite slows down so gradually that each week feels normal. Then someone measures it for the first time in a year and discovers it has quadrupled.

Speed is a design property, not a side effect

The fastest way to have a fast test suite is to design for it from the start, and the design decision that matters most is the split between unit and integration tests. Unit tests, which run against mocked boundaries and in process, should be the majority, and they should run in seconds. Integration tests, which touch real infrastructure, should be the minority, and they should be gated so they only run when the changes actually touch the integration surface.

The tests you write when you are testing a pure function are fast by nature. The tests you write when you are testing "the whole thing" are slow by nature. Neither is wrong, but they serve different purposes, and if you run them together on every commit, you pay the slow cost for the fast work. The move is to make the default loop, the thing every developer runs before they push, the fast suite. The slow suite runs in CI, but it runs in CI the way you would run a weekly backup: often enough to catch real problems, cheaply enough that nobody is tempted to skip it.

The flaky test is a time bomb

There is a special circle of test hell reserved for flaky tests, and it is adjacent to the slow test problem in an important way. A test that passes sometimes and fails sometimes teaches developers to distrust the suite. The suite goes red, and the first response is no longer "what did I break" but "is this the flaky one". That distrust is poison. Once the team stops trusting the red light, the suite has lost its power, regardless of how fast it runs.

The rule that works: a flaky test is a bug in the test, and it gets fixed or removed immediately, not added to the ignore list. Every team that has ever kept a flaky test "just for now" has eventually paid for it with a production incident that slipped through a suite nobody believed.

What you can do this week

If you have a slow suite, you do not need a grand project to fix it. You need a measurement and a small set of wins. Measure the suite once, honestly, and find the ten slowest tests. Ten tests are almost always the majority of the runtime. Look at each one and ask the same question: does this need the real infrastructure, and does it need to run on every commit?

Most of the time, the answer to one of those is no. The test that loads the full fixture can use a smaller fixture. The test that hits the real API can mock the client and keep one slow smoke test for the real thing. The test that sleeps for three seconds can wait on a condition instead. These are not clever moves. They are the obvious ones, and they take a weekend, and they take the suite from forty minutes to eight.

The ceiling is worth protecting

Once the suite is fast, guard the speed. Add a CI check that fails if the suite grows beyond a budget, or if a new test takes more than a few seconds. A team that treats test runtime as a feature to protect will keep its suite fast. A team that treats it as an emergent property will watch it rot.

Here is the honest summary. Tests are not a report you file. They are a safety net you wear every single time you touch the code, and a net that is too heavy to wear is no net at all. The team with the fast suite gets to be confident. The team with the slow suite gets to be hopeful. Confident beats hopeful every time.