From 82ae0e688c21794bea583f9b48bb3639f7e2601a Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Mon, 30 Apr 2012 16:49:00 -0400 Subject: [PATCH] Update the documentation of build-remote.pl --- doc/manual/build-farm.xml | 149 ++++++++++--------------- doc/manual/env-common.xml | 7 +- doc/manual/writing-nix-expressions.xml | 2 +- 3 files changed, 62 insertions(+), 96 deletions(-) diff --git a/doc/manual/build-farm.xml b/doc/manual/build-farm.xml index 3b973188d..f2d4a477e 100644 --- a/doc/manual/build-farm.xml +++ b/doc/manual/build-farm.xml @@ -1,67 +1,17 @@ + xml:id='chap-distributed-builds'> -Setting up a Build Farm +Setting Up Distributed Builds - -This chapter provides some sketchy information on how to set up -a Nix-based build farm. Nix is particularly suited as a basis for a -build farm, since: - - - - Nix supports distributed builds: a local Nix - installation can forward Nix builds to other machines over the - network. This allows multiple builds to be performed in parallel - (thus improving performance), but more in importantly, it allows Nix - to perform multi-platform builds in a semi-transparent way. For - instance, if you perform a build for a - powerpc-darwin on an - i686-linux machine, Nix can automatically forward - the build to a powerpc-darwin machine, if - available. - - The Nix expression language is ideal for describing - build jobs, plus all their dependencies. For instance, if your - package has some dependency, you don't have to manually install it - on all the machines in the build farm; they will be built - automatically. - - Proper release management requires that builds (if - deployed) are traceable: it should be possible to figure out from - exactly what sources they were built, in what configuration, etc.; - and it should be possible to reproduce the build, if necessary. Nix - makes this possible since Nix's hashing scheme uniquely identifies - builds, and Nix expressions are self-contained. - - Nix will only rebuild things that have actually - changed. For instance, if the sources of a package haven't changed - between runs of the build farm, the package won't be rebuilt (unless - it was garbage-collected). Also, dependencies typically don't - change very often, so they only need to be built - once. - - The results of a Nix build farm can be made - available through a channel, so successful builds can be deployed to - users immediately. - - - - - - -
Overview - -TODO - -The sources of the Nix build farm are at . - -
- - -
Setting up distributed builds +Nix supports distributed builds: a local Nix installation can +forward Nix builds to other machines over the network. This allows +multiple builds to be performed in parallel (thus improving +performance) and allows Nix to perform multi-platform builds in a +semi-transparent way. For instance, if you perform a build for a +powerpc-darwin on an i686-linux +machine, Nix can automatically forward the build to a +powerpc-darwin machine, if available. You can enable distributed builds by setting the environment variable NIX_BUILD_HOOK to point to a program that Nix @@ -79,22 +29,22 @@ variable. remote-systems.conf nix@mcflurry.labs.cs.uu.nl powerpc-darwin /home/nix/.ssh/id_quarterpounder_auto 2 -nix@scratchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 1 +nix@scratchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 1 kvm +nix@itchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 2 -An example build hook can be found in the Nix build farm -sources: . It should be suitable for most purposes, with maybe some minor -adjustments. It uses ssh and -rsync to copy the build inputs and outputs and -perform the remote build. You should define a list of available build -machines and set the environment variable -REMOTE_SYSTEMS to point to it. An example -configuration is shown in . Each -line in the file specifies a machine, with the following bits of -information: +Nix ships with a build hook that should be suitable for most +purposes. It uses ssh and +nix-copy-closure to copy the build inputs and +outputs and perform the remote build. To use it, you should set +NIX_BUILD_HOOK to +prefix/libexec/nix/build-remote.pl. +You should also define a list of available build machines and point +the environment variable NIX_REMOTE_SYSTEMS to it. An +example configuration is shown in . Each line in the file specifies a machine, with the following +bits of information: @@ -104,34 +54,49 @@ information: be an alias defined in your ~/.ssh/config. - The Nix platform type identifier, such as - powerpc-darwin. + A comma-separated list of Nix platform type + identifiers, such as powerpc-darwin. It is + possible for a machine to support multiple platform types, e.g., + i686-linux,x86_64-linux. The SSH private key to be used to log in to the remote machine. Since builds should be non-interactive, this key should not have a passphrase! - The maximum load of the remote - machine. This is just the maximum number of jobs that + The maximum number of builds that build-remote.pl will execute in parallel on the - machine. Typically this should be equal to the number of - CPUs. + machine. Typically this should be equal to the number of CPU cores. + For instance, the machine itchy in the example + will execute up to 8 builds in parallel. + + The “speed factor”, indicating the relative speed of + the machine. If there are multiple machines of the right type, Nix + will prefer the fastest, taking load into account. + + A comma-separated list of + features. If a derivation has the + requiredSystemFeatures attribute, then + build-remote.pl will only perform the + derivation on a machine that has the specified features. For instance, the attribute + + +requiredSystemFeatures = [ "kvm" ]; + + + will cause the build to be performed on a machine that has the + kvm feature (i.e., scratchy in + the example above). You should also set up the environment variable -CURRENT_LOAD to point at a file that -build-remote.pl uses to remember how many jobs it -is currently executing remotely. It doesn't look at the actual load -on the remote machine, so if you have multiple instances of Nix -running, they should use the same CURRENT_LOAD -fileAlthough there are probably some race conditions -in the script right now.. Maybe in the future -build-remote.pl will look at the actual remote -load. The load file should exist, so you should just create it as an -empty file initially. +NIX_CURRENT_LOAD to point at a directory (e.g., +/var/run/nix/current-load) that +build-remote.pl uses to remember how many builds +it is currently executing remotely. It doesn't look at the actual +load on the remote machine, so if you have multiple instances of Nix +running, they should use the same NIX_CURRENT_LOAD +file. Maybe in the future build-remote.pl will +look at the actual remote load. -
- - diff --git a/doc/manual/env-common.xml b/doc/manual/env-common.xml index f52e2633d..edfded7fc 100644 --- a/doc/manual/env-common.xml +++ b/doc/manual/env-common.xml @@ -119,9 +119,10 @@ $ mount -o bind /mnt/otherdisk/nix /nix Specifies the location of the build hook, which is a program (typically some script) that Nix will call whenever it wants to build a derivation. This is used to implement - distributed builds (see ). The protocol by which the calling Nix process and the build - hook communicate is as follows. + distributed builds (see ). The protocol by + which the calling Nix process and the build hook communicate is as + follows. The build hook is called with the following command-line arguments: diff --git a/doc/manual/writing-nix-expressions.xml b/doc/manual/writing-nix-expressions.xml index e16225433..310dd5ae0 100644 --- a/doc/manual/writing-nix-expressions.xml +++ b/doc/manual/writing-nix-expressions.xml @@ -1293,7 +1293,7 @@ set, the attributes of which specify the inputs of the build. can only be performed on a machine and operating system matching the platform identifier. (Nix can automatically forward builds for other platforms by forwarding them to other machines; see .) + linkend='chap-distributed-builds' />.) There must be an attribute named name whose value must be a string. This is used