Created Started Cancelled
By: Tom Sydney Kerckhove <syd@cs-syd.eu>
Make controls sparser and cheaper to run A control is a no-op mutation that is expected to survive, so unlike a real mutation its child cannot stop at the test that kills it: it runs every covering test it is given, to the end. That makes the cheapest kind of mutation to produce the most expensive kind to run, and on a large project the controls of a run came to well over a third of its mutation phase while being a fraction of its mutations. What a control is for is noticing a suite that cannot be relied on, which the number of controls answers rather than the number of tests each one runs. So there is now one every hundred mutations rather than every twelve, and each is checked against ten of its covering tests rather than all of them. Not the ten cheapest: the fastest tests are the least likely to be the ones a suite cannot be relied on for, so taking those every time would aim the canary away from what it is watching for. Which ten follows from the mutation's own id, so different controls take different tests while a rerun takes the same ones. The cap lives in the coverage phase so that the record says what the child will run and the budget is derived from it. Every module still gets a control at its first mutation site, which it did not need before: at one per twelve mutations every module of any size got one anyway, and at one per hundred a project of small modules would get none. That also keeps this repository's own example, whose modules are far smaller than that, exercising controls at all.