1e816296

By: Tom Sydney Kerckhove <syd@cs-syd.eu>

Force what the outputs share before forking anything

Forcing it only once an attribute had been reported meant paying for it
twice: the first worker forced it while evaluating that attribute, and the
coordinator forced it again afterwards so that later workers would not have
to. Worse, the second time blocked the run, because the coordinator is also
the process servicing every worker. Measured against master that made the
whole thing slower than doing nothing: nx-experiment 3.6s against 5.9s,
sydtest 20.5s against 25.3s.

So the first derivation the discovery root can reach is forced before the
first worker is forked, where there is nothing else to be doing: no worker
exists yet, and the one about to be forked would have spent that time
forcing the same thing for itself. Forcing per output type stays, for the
case the walk reaches a cheaper corner of the flake than the run goes on to
work in, and now usually finds its work already done.

The fixture now reports one substrate evaluation rather than two, at both
pool sizes.
GitHub
Time to Start Worker time Duration Time to finish Idle
Config 3m22s 1s 1s 3m23s 3m22s
Eval 3m25s 12s 13s 3m38s 1s
Build 3m31s 6m24s - - 3h09m16s
Suite 3m22s 6m39s - - 3h12m40s

Timeline

Config Eval Build Failed Retried Compressed gap
0s3h13m3h13m3h13m3h14m3h14m3h14m3h15m3h15m3h15m3h16m3h16m