af4083ea

By: Tom Sydney Kerckhove <syd@cs-syd.eu>

Mutation: order covering tests cheapest-first in the mutation child

When a mutation child runs the tests that cover a mutation, it runs them
synchronously under fail-fast, so the first covering test that fails kills the
mutant.  Ordering the covering tests cheapest-first, using the per-test baseline
the coverage phase already measures (and previously discarded into a single
timeout), reaches a killing test sooner and cuts the mutation phase's wall-clock,
since killed mutants are the overwhelming majority of a run.  On two sample
projects the mutation phase ran ~6% and ~24% faster, with identical verdicts.

- reorderTestForestByTiming: a pure, tree-aware, stable reorder.  It sorts
  siblings by ascending baseline (missing = first) and orders each subtree by the
  minimum cost of any leaf it contains, so a shared-setup group reorders as a
  unit and setup is never split.  It mirrors randomiseTestForest: inside a
  DoNotRandomiseExecutionOrder scope it leaves the subtree untouched at every
  depth.  So it is a drop-in alternative to execution-order randomisation, legal
  under exactly the same conditions.  Only sibling order changes, so which tests
  run (and every verdict) is unaffected.
- reorderForMutationChild: the pure gate the child uses -- reorder only when the
  suite randomises execution order and a baseline is available.  Keeping it out
  of the child's IO makes the gate unit-testable.
- The coverage phase persists the merged per-test baselines as baseline.json next
  to the manifest; the run and diff paths union them into the augmented dir the
  children read.
- The child reads that baseline and reorders the filtered forest after filtering,
  so the coverage-order TestIds the filter needs are untouched.

Tests: ReorderSpec covers ascending order, stable ties + missing-as-first, that a
sequential (parallelism) subtree is reordered but a doNotRandomiseExecutionOrder
one is not (incl. no descent into a nested randomise block), that a beforeAll_
group reorders as a unit without splitting setup, that the test-id set is
unchanged, and the gate (on/off/no-baseline).

Bump sydtest to 0.27.2.0 for the new functions exposed from
Test.Syd.Mutation.Forest, and release sydtest-mutation, sydtest-mutation-runtime,
and sydtest-mutation-driver as 0.1.0.0 (first released versions, with CHANGELOGs).

Suite timing

Time to Start Worker time Duration Time to finish Idle
Config 4s 2s 2s 7s 4s
Eval 7s 23s 23s 31s 0s
Build 1h03m53s 1m58s 3m45s 1h07m38s 1h05m08s
Suite 4s 2m24s 1h07m34s 1h07m38s 1h05m14s

Timeline

0s10s20s30s1h4m1h4m1h4m1h4m1h4m1h4m1h5m1h5m1h5m1h5m1h5m1h6m