cf733769

By: Tom Sydney Kerckhove <syd@cs-syd.eu>

Let NixCI work out the CI job graph

Three things this file was doing by hand, in 290 lines, that NixCI can
do or no longer needs:

Reading nix/ci-workspace-graph.json. NixCI reads this file straight out
of the forge, without cloning the repository, so a configuration that
reads another file is rejected instead of read and every suite fails at
the config step before anything is built.

Restating the crate graph as dependency edges. Discovery derives the
job graph from the derivations themselves and merges what it finds with
what is configured here, so all that is left to configure are the ten
orderings that are not derivation dependencies at all: the impure test
runners, which are small scripts that run `nix build` themselves and so
mention none of the archives, the python module or the cargo artifacts
they will ask for, and fmt, which gates the gammaloop package rather
than feeding it. I measured which edges those are rather than judging
by eye: for each of the 80, is the dependency's derivation in the
dependent's input closure. Seventy are.

Keeping every per-crate artifact out of CI. Those exclusions existed
because the artifacts were roots with hand-written edges; ordered by
discovery they can be jobs like any other, built once and reused, so
doNotBuild is back to the aggregates and duplicates that must not be
built at all.

Discovery is synchronous, so no job starts before the graph is known.
An edge missing from a hand-written graph lets two jobs build the same
derivation at once and we pay for both, which is the shape of the
duplicate work in the suites we looked at.
Time to Start Worker time Duration Time to finish Idle
Config 47s 1s 1s 49s 47s
Eval 2m24s 1m20s 1m20s 3m44s 1m34s
Build 3m39s 7m18s 3h51m55s 3h55m34s 3h48m37s
Test 3h54m25s 0s 0s 3h54m26s 0s
Deploy - - - - 0s
Suite 47s 8m39s - - 3h51m00s

Timeline

Config Eval Build Test Failed Compressed gap
0s2m40s3m3m20s3m40s3h52m3h53m3h53m3h53m3h54m3h54m3h54m3h55m3h55m