By: Tom Sydney Kerckhove <syd@cs-syd.eu>
Scale the sqitch tests' timeout with the number of changes The sqitch sanity checks shell out to sqitch once or more per change, so the work they do grows with the plan, but sydtest's per-test timeout is one fixed wall-clock budget. That budget therefore covers less of the job with every migration added, and on a loaded machine it is blown: nix-ci's `nix flake check` failed intermittently with a <<timeout>> on nix-ci-leader-data-gen naming a different change each time, which is the giveaway that the clock ran out rather than that a migration is slow. With 28 changes, roughly a hundred process spawns had to fit in 60 seconds, and process spawns degrade worst under exactly the oversubscription `nix flake check` creates. `withPlanScaledTimeout` multiplies the configured timeout by the number of changes, so the headroom per change stays constant however long the plan gets. It scales what the caller configured rather than replacing it, so --timeout still means something and no-timeout stays no-timeout. It is exported because the schema-equality check in sydtest-sqitch-postgres-persistent deploys the whole plan and has the same problem, and `readSqitchPlanSteps` with it so a caller can read the plan once and hand the steps to both the sizing and the tests. The alternative was to declare one test per change, which gets its headroom the same way: a budget proportional to the work. It is not worth what it costs. Each test is its own mutation coverage unit, so 27 per-change tests become 27 units each deploying their plan prefix from a clean database, turning that phase from linear into quadratic. Sizing the timeout reads the plan during spec definition, so an unreadable sqitch.plan now fails there rather than in one test. The message from readSqitchPlan is unchanged and says which it was. The test inspects the declared forest and applies the scaling function to both Timeout constructors, so it needs no postgres. It matches both tests inside the timeout node: checking only that the node exists passes when the node and the tests are siblings, which would leave both unscaled.