By: Tom Sydney Kerckhove <syd@cs-syd.eu>
Fork every worker from one warm evaluator A pool of N workers evaluated the flake's shared substrate N times: each one built its own evaluator, so it loaded the flake and forced nixpkgs, the overlays and the package set before it could answer anything. Measured with a warm store, that is 195s per worker on nix-ci against 262s of per-attribute work for the whole run, and 19s per worker on sydtest against 9s. The run now forks a substrate host before anything else. It fetches and loads the flake once, forces what the outputs share, and from then on evaluates nothing: it only forks workers on request, and those inherit the forced evaluator copy-on-write. The coordinator still creates each worker's pipes and speaks to it exactly as before; only the fork happens elsewhere, which is what passing descriptors is for. Three things a fork must not carry across, and what is done about each: - The store. A daemon connection is a socket two processes cannot both speak on, and a local store holds an SQLite connection that SQLite documents as unusable in a forked child. Each worker reopens the store and points the evaluator it inherited at the new one. - A background thread. Downloading starts nix's file transfer, whose thread does not survive a fork, so a worker that inherited one would hang on the first thing it asked of it. The host fetches the flake in a child that then exits, and refuses to fork workers at all if it ever finds itself with more than one thread. - The point of the exercise. Forcing one attribute shares most of the derivation graph underneath it, but a flake's checks and its packages can come from different package sets, so the host is also asked to force one attribute per output type, named from a derivation the run has reported. Nothing here fails a run: a host that cannot start, cannot fork, or dies leaves the coordinator forking its own workers, each loading the flake for itself, which is what it did before. The per-worker memory ceiling now counts what a worker does not share, since what it inherited is held once however many workers are looking at it. NIX_CI_EVAL_TRACE reports where a run's time went, which is how the numbers above were measured.