d116a081

By: Tom Sydney Kerckhove <syd@cs-syd.eu>

Count a timed-out mutation as killed again

A mutation easily turns a loop into one that never ends, and a suite that
never finishes is a suite that noticed the mutation. That is what a kill
means, so the previous commit was wrong to take it away: it marked caught
mutations uncaught, and then failed the run over them. On nix-ci's
leader-background it turned 21 real kills in the worker's retry and announce
paths into a failing check, several of which had sat out a full 134 or 161
second budget three times over, which is a hang and nothing else.

The argument for taking it away was that a loaded machine times out too, so
crediting the kill flatters the score. The retry answers that, and is what
the previous commit got right: an overrun is retried three times, so one that
survives that is the mutation's doing rather than the machine's. With that in
place, counting it as a kill says something true.

So 'tallyGroups' credits the kill again, 'isMutationFailure' leaves timeouts
alone, and 'assert-score' does not fail on them. 'runFoundFailure' goes with
them: it existed only because a timeout tripping fail-fast could abort a run
the driver then reported as finished, and a timeout no longer trips it.

What stays is the part that was right: an overrun is retried, and a run with
a failed control exits non-zero so its verdict is never stored for a later
attempt to read back instead of re-running.

sydtest itself is unchanged by all this once more, so its version goes back
to where it was.
GitHub

Total

1m30s

Created Started Finished

Time to Start Worker time Duration Time to finish Idle
Config 0s 1s 1s 1s 0s
Eval 1s 1m33s 1m28s 1m30s 0s
Build - - - - -
Suite 0s 1m34s 1m30s 1m30s 0s