3205b346

Created Started Cancelled

By: Tom Sydney Kerckhove <syd@cs-syd.eu>

Declare one sqitch test per change instead of one for the whole plan

sydtest's per-test timeout is a fixed wall-clock budget, so a single test
that walks the whole plan has to fit every change's sqitch invocations
into one 60 second budget. Each migration added eats into what is left,
and machine load does the rest: under `nix flake check` on nix-ci this
showed up as a <<timeout>> naming a different change every time, which is
the giveaway that it is the clock running out and not a slow migration.

So `sqitchPostgresqlSpec` reads the plan at spec-definition time and
declares one test per change. Each one runs in a second or two against
its own budget, the budget stops shrinking as the plan grows, and the
failing change is now a test name rather than whatever was in flight.

The tests still walk the plan in order against a single database, each
deploying on top of what the previous one left behind, because starting
every change from a clean database would hide interactions between
migrations (and would be quadratic). That shared state is why the group
is `sequential` and `doNotRandomiseExecutionOrder`.

`runSqitchPerChangeChecks` keeps doing the whole walk in one IO action,
for the negative tests that wrap it in `expectFailing`; it and the new
per-change tests share `checkStep`.
GitHub