34cc1d86

Created Started Cancelled

By: Tom Sydney Kerckhove <syd@cs-syd.eu>

Add a differential against iets and record what it found

iets (https://github.com/numtide/iets) is a whole Nix client rather than an
evaluator, and one of its subcommands walks a jobset the way this tool does.
So the question it raises is not whether another evaluator fits under this
C++ wrapper but whether the wrapper is needed, which is a different question
from the one docs/fix-evaluator.md answered.

bench/jobs-root.nix is this tool's discovery root as a function, so
`iets eval-jobs --select` can be asked to walk exactly what this tool walks.
bench/iets-differential.sh compares the two tools' attribute-to-drvPath maps
on real flakes. bench/package-set.nix asks a much larger version of the same
question of a package set, where throughput shows.

They agree on every drvPath over three flakes and 57,927 nixpkgs attributes,
and iets is 4x to 5x faster at less memory on the flake-shaped workload, so
the measurement points the opposite way from fix's. docs/iets-evaluator.md
records that, and the four things a replacement still cannot do.