3205b346

By: Tom Sydney Kerckhove <syd@cs-syd.eu>

Declare one sqitch test per change instead of one for the whole plan

sydtest's per-test timeout is a fixed wall-clock budget, so a single test
that walks the whole plan has to fit every change's sqitch invocations
into one 60 second budget. Each migration added eats into what is left,
and machine load does the rest: under `nix flake check` on nix-ci this
showed up as a <<timeout>> naming a different change every time, which is
the giveaway that it is the clock running out and not a slow migration.

So `sqitchPostgresqlSpec` reads the plan at spec-definition time and
declares one test per change. Each one runs in a second or two against
its own budget, the budget stops shrinking as the plan grows, and the
failing change is now a test name rather than whatever was in flight.

The tests still walk the plan in order against a single database, each
deploying on top of what the previous one left behind, because starting
every change from a clean database would hide interactions between
migrations (and would be quadratic). That shared state is why the group
is `sequential` and `doNotRandomiseExecutionOrder`.

`runSqitchPerChangeChecks` keeps doing the whole walk in one IO action,
for the negative tests that wrap it in `expectFailing`; it and the new
per-change tests share `checkStep`.
GitHub
Time to Start Worker time Duration Time to finish Idle
Config 22m34s 1s 1s 22m35s 22m34s
Eval 22m36s 2m42s 2m42s 25m19s 1s
Build 25m19s 0s - - 0s
Suite 22m34s 2m43s - - 40m55s