From 5677230de3cfe2fa315fd37e6e7f09e384e60eee Mon Sep 17 00:00:00 2001 From: DS Date: Fri, 2 Feb 2024 05:03:20 -0800 Subject: [PATCH] doc: update buildNixShellImage docs, add streamNixShellImage docs --- .../images/dockertools.section.md | 359 ++++++++++++------ doc/manpage-urls.json | 3 +- 2 files changed, 255 insertions(+), 107 deletions(-) diff --git a/doc/build-helpers/images/dockertools.section.md b/doc/build-helpers/images/dockertools.section.md index 79c7d3947dc6..9317146b8f94 100644 --- a/doc/build-helpers/images/dockertools.section.md +++ b/doc/build-helpers/images/dockertools.section.md @@ -1282,141 +1282,288 @@ dockerTools.buildLayeredImage { ``` ::: +[]{#ssec-pkgs-dockerTools-buildNixShellImage-arguments} ## buildNixShellImage {#ssec-pkgs-dockerTools-buildNixShellImage} -Create a Docker image that sets up an environment similar to that of running `nix-shell` on a derivation. -When run in Docker, this environment somewhat resembles the Nix sandbox typically used by `nix-build`, with a major difference being that access to the internet is allowed. -It additionally also behaves like an interactive `nix-shell`, running things like `shellHook` and setting an interactive prompt. -If the derivation is fully buildable (i.e. `nix-build` can be used on it), running `buildDerivation` inside such a Docker image will build the derivation, with all its outputs being available in the correct `/nix/store` paths, pointed to by the respective environment variables like `$out`, etc. +`buildNixShellImage` uses [`streamNixShellImage`](#ssec-pkgs-dockerTools-streamNixShellImage) underneath to build a compressed Docker-compatible repository tarball of an image that sets up an environment similar to that of running `nix-shell` on a derivation. +Basically, `buildNixShellImage` runs the script created by `streamNixShellImage` to save the compressed image in the Nix store. -::: {.warning} -The behavior doesn't match `nix-shell` or `nix-build` exactly and this function is known not to work correctly for e.g. fixed-output derivations, content-addressed derivations, impure derivations and other special types of derivations. -::: +`buildNixShellImage` supports the same options as `streamNixShellImage`, see [`streamNixShellImage`](#ssec-pkgs-dockerTools-streamNixShellImage) for details. -### Arguments {#ssec-pkgs-dockerTools-buildNixShellImage-arguments} +[]{#ssec-pkgs-dockerTools-buildNixShellImage-example} +### Examples {#ssec-pkgs-dockerTools-buildNixShellImage-examples} -`drv` +:::{.example #ex-dockerTools-buildNixShellImage-hello} +# Building a Docker image with `buildNixShellImage` with the build environment for the `hello` package -: The derivation on which to base the Docker image. - - Adding packages to the Docker image is possible by e.g. extending the list of `nativeBuildInputs` of this derivation like - - ```nix - buildNixShellImage { - drv = someDrv.overrideAttrs (old: { - nativeBuildInputs = old.nativeBuildInputs or [] ++ [ - somethingExtra - ]; - }); - # ... - } - ``` - - Similarly, you can extend the image initialization script by extending `shellHook` - -`name` _optional_ - -: The name of the resulting image. - - *Default:* `drv.name + "-env"` - -`tag` _optional_ - -: Tag of the generated image. - - *Default:* the resulting image derivation output path's hash - -`uid`/`gid` _optional_ - -: The user/group ID to run the container as. This is like a `nixbld` build user. - - *Default:* 1000/1000 - -`homeDirectory` _optional_ - -: The home directory of the user the container is running as - - *Default:* `/build` - -`shell` _optional_ - -: The path to the `bash` binary to use as the shell. This shell is started when running the image. - - *Default:* `pkgs.bashInteractive + "/bin/bash"` - -`command` _optional_ - -: Run this command in the environment of the derivation, in an interactive shell. See the `--command` option in the [`nix-shell` documentation](https://nixos.org/manual/nix/stable/command-ref/nix-shell.html?highlight=nix-shell#options). - - *Default:* (none) - -`run` _optional_ - -: Same as `command`, but runs the command in a non-interactive shell instead. See the `--run` option in the [`nix-shell` documentation](https://nixos.org/manual/nix/stable/command-ref/nix-shell.html?highlight=nix-shell#options). - - *Default:* (none) - -### Example {#ssec-pkgs-dockerTools-buildNixShellImage-example} - -The following shows how to build the `pkgs.hello` package inside a Docker container built with `buildNixShellImage`. +This example shows how to build the `hello` package inside a Docker container built with `buildNixShellImage`. +The Docker image generated will have a name like `hello--env` and tag `latest`. +This example is the `buildNixShellImage` equivalent of [](#ex-dockerTools-streamNixShellImage-hello). ```nix -with import {}; +{ dockerTools, hello }: dockerTools.buildNixShellImage { drv = hello; + tag = "latest"; } ``` -Build the derivation: +The result of building this package is a `.tar.gz` file that can be loaded into Docker: -```console -nix-build hello.nix +```shell +$ nix-build +(some output removed for clarity) +/nix/store/pkj1sgzaz31wl0pbvbg3yp5b3kxndqms-hello-2.12.1-env.tar.gz + +$ docker load -i /nix/store/pkj1sgzaz31wl0pbvbg3yp5b3kxndqms-hello-2.12.1-env.tar.gz +(some output removed for clarity) +Loaded image: hello-2.12.1-env:latest ``` - these 8 derivations will be built: - /nix/store/xmw3a5ln29rdalavcxk1w3m4zb2n7kk6-nix-shell-rc.drv - ... - Creating layer 56 from paths: ['/nix/store/crpnj8ssz0va2q0p5ibv9i6k6n52gcya-stdenv-linux'] - Creating layer 57 with customisation... - Adding manifests... - Done. - /nix/store/cpyn1lc897ghx0rhr2xy49jvyn52bazv-hello-2.12-env.tar.gz +After starting an interactive container, the derivation can be built by running `buildDerivation`, and the output can be executed as expected: -Load the image: +```shell +$ docker run -it hello-2.12.1-env:latest +[nix-shell:~]$ buildDerivation +Running phase: unpackPhase +unpacking source archive /nix/store/pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz +source root is hello-2.12.1 +(some output removed for clarity) +Running phase: fixupPhase +shrinking RPATHs of ELF executables and libraries in /nix/store/f2vs29jibd7lwxyj35r9h87h6brgdysz-hello-2.12.1 +shrinking /nix/store/f2vs29jibd7lwxyj35r9h87h6brgdysz-hello-2.12.1/bin/hello +checking for references to /build/ in /nix/store/f2vs29jibd7lwxyj35r9h87h6brgdysz-hello-2.12.1... +gzipping man pages under /nix/store/f2vs29jibd7lwxyj35r9h87h6brgdysz-hello-2.12.1/share/man/ +patching script interpreter paths in /nix/store/f2vs29jibd7lwxyj35r9h87h6brgdysz-hello-2.12.1 +stripping (with command strip and flags -S -p) in /nix/store/f2vs29jibd7lwxyj35r9h87h6brgdysz-hello-2.12.1/bin -```console -docker load -i result +[nix-shell:~]$ $out/bin/hello +Hello, world! +``` +::: + +## streamNixShellImage {#ssec-pkgs-dockerTools-streamNixShellImage} + +`streamNixShellImage` builds a **script** which, when run, will stream to stdout a Docker-compatible repository tarball of an image that sets up an environment similar to that of running `nix-shell` on a derivation. +This means that `streamNixShellImage` does not output an image into the Nix store, but only a script that builds the image, saving on IO and disk/cache space, particularly with large images. +See [](#ex-dockerTools-streamNixShellImage-hello) to understand how to load in Docker the image generated by this script. + +The environment set up by `streamNixShellImage` somewhat resembles the Nix sandbox typically used by `nix-build`, with a major difference being that access to the internet is allowed. +It also behaves like an interactive `nix-shell`, running things like `shellHook` (see [](#ex-dockerTools-streamNixShellImage-addingShellHook)) and setting an interactive prompt. +If the derivation is buildable (i.e. `nix-build` can be used on it), running `buildDerivation` in the container will build the derivation, with all its outputs being available in the correct `/nix/store` paths, pointed to by the respective environment variables (e.g. `$out`). + +::: {.caution} +The environment in the image doesn't match `nix-shell` or `nix-build` exactly, and this function is known not to work correctly for fixed-output derivations, content-addressed derivations, impure derivations and other special types of derivations. +::: + +### Inputs {#ssec-pkgs-dockerTools-streamNixShellImage-inputs} + +`streamNixShellImage` expects one argument with the following attributes: + +`drv` (Attribute Set) + +: The derivation for which the environment in the image will be set up. + Adding packages to the Docker image is possible by extending the list of `nativeBuildInputs` of this derivation. + See [](#ex-dockerTools-streamNixShellImage-extendingBuildInputs) for how to do that. + Similarly, you can extend the image initialization script by extending `shellHook`. + [](#ex-dockerTools-streamNixShellImage-addingShellHook) shows how to do that. + +`name` (String; _optional_) + +: The name of the generated image. + + _Default value:_ the value of `drv.name + "-env"`. + +`tag` (String or Null; _optional_) + +: Tag of the generated image. + If `null`, the hash of the nix derivation that builds the Docker image will be used as the tag. + + _Default value:_ `null`. + +`uid` (Number; _optional_) + +: The user ID to run the container as. + This can be seen as a `nixbld` build user. + + _Default value:_ 1000. + +`gid` (Number; _optional_) + +: The group ID to run the container as. + This can be seen as a `nixbld` build group. + + _Default value:_ 1000. + +`homeDirectory` (String; _optional_) + +: The home directory of the user the container is running as. + + _Default value:_ `/build`. + +`shell` (String; _optional_) + +: The path to the `bash` binary to use as the shell. + This shell is started when running the image. + This can be seen as an equivalent of the `NIX_BUILD_SHELL` [environment variable](https://nixos.org/manual/nix/stable/command-ref/nix-shell.html#environment-variables) for {manpage}`nix-shell(1)`. + + _Default value:_ the `bash` binary from the `bashInteractive` package. + +`command` (String or Null; _optional_) + +: If specified, this command will be run in the environment of the derivation in an interactive shell. + A call to `exit` will be added after the command if it is specified, so the shell will exit after it's finished running. + This can be seen as an equivalent of the `--command` option in {manpage}`nix-shell(1)`. + + _Default value:_ `null`. + +`run` (String or Null; _optional_) + +: Similar to the `command` attribute, but runs the command in a non-interactive shell instead. + A call to `exit` will be added after the command if it is specified, so the shell will exit after it's finished running. + This can be seen as an equivalent of the `--run` option in {manpage}`nix-shell(1)`. + + _Default value:_ `null`. + +### Examples {#ssec-pkgs-dockerTools-streamNixShellImage-examples} + +:::{.example #ex-dockerTools-streamNixShellImage-hello} +# Building a Docker image with `streamNixShellImage` with the build environment for the `hello` package + +This example shows how to build the `hello` package inside a Docker container built with `streamNixShellImage`. +The Docker image generated will have a name like `hello--env` and tag `latest`. +This example is the `streamNixShellImage` equivalent of [](#ex-dockerTools-buildNixShellImage-hello). + +```nix +{ dockerTools, hello }: +dockerTools.streamNixShellImage { + drv = hello; + tag = "latest"; +} ``` - 0d9f4c4cd109: Loading layer [==================================================>] 2.56MB/2.56MB - ... - ab1d897c0697: Loading layer [==================================================>] 10.24kB/10.24kB - Loaded image: hello-2.12-env:pgj9h98nal555415faa43vsydg161bdz +The result of building this package is a script. +Running this script and piping it into `docker load` gives you the same image that was built in [](#ex-dockerTools-buildNixShellImage-hello). -Run the container: +```shell +$ nix-build +(some output removed for clarity) +/nix/store/8vhznpz2frqazxnd8pgdvf38jscdypax-stream-hello-2.12.1-env -```console -docker run -it hello-2.12-env:pgj9h98nal555415faa43vsydg161bdz +$ /nix/store/8vhznpz2frqazxnd8pgdvf38jscdypax-stream-hello-2.12.1-env | docker load +(some output removed for clarity) +Loaded image: hello-2.12.1-env:latest ``` - [nix-shell:/build]$ +After starting an interactive container, the derivation can be built by running `buildDerivation`, and the output can be executed as expected: -In the running container, run the build: +```shell +$ docker run -it hello-2.12.1-env:latest +[nix-shell:~]$ buildDerivation +Running phase: unpackPhase +unpacking source archive /nix/store/pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz +source root is hello-2.12.1 +(some output removed for clarity) +Running phase: fixupPhase +shrinking RPATHs of ELF executables and libraries in /nix/store/f2vs29jibd7lwxyj35r9h87h6brgdysz-hello-2.12.1 +shrinking /nix/store/f2vs29jibd7lwxyj35r9h87h6brgdysz-hello-2.12.1/bin/hello +checking for references to /build/ in /nix/store/f2vs29jibd7lwxyj35r9h87h6brgdysz-hello-2.12.1... +gzipping man pages under /nix/store/f2vs29jibd7lwxyj35r9h87h6brgdysz-hello-2.12.1/share/man/ +patching script interpreter paths in /nix/store/f2vs29jibd7lwxyj35r9h87h6brgdysz-hello-2.12.1 +stripping (with command strip and flags -S -p) in /nix/store/f2vs29jibd7lwxyj35r9h87h6brgdysz-hello-2.12.1/bin -```console -buildDerivation +[nix-shell:~]$ $out/bin/hello +Hello, world! +``` +::: + +:::{.example #ex-dockerTools-streamNixShellImage-extendingBuildInputs} +# Adding extra packages to a Docker image built with `streamNixShellImage` + +This example shows how to add extra packages to an image built with `streamNixShellImage`. +In this case, we'll add the `cowsay` package. +The Docker image generated will have a name like `hello--env` and tag `latest`. +This example uses [](#ex-dockerTools-streamNixShellImage-hello) as a starting point. + +```nix +{ dockerTools, cowsay, hello }: +dockerTools.streamNixShellImage { + tag = "latest"; + drv = hello.overrideAttrs (old: { + nativeBuildInputs = old.nativeBuildInputs or [] ++ [ + cowsay + ]; + }); +} ``` - unpacking sources - unpacking source archive /nix/store/8nqv6kshb3vs5q5bs2k600xpj5bkavkc-hello-2.12.tar.gz - ... - patching script interpreter paths in /nix/store/z5wwy5nagzy15gag42vv61c2agdpz2f2-hello-2.12 - checking for references to /build/ in /nix/store/z5wwy5nagzy15gag42vv61c2agdpz2f2-hello-2.12... +The result of building this package is a script which can be run and piped into `docker load` to load the generated image. -Check the build result: +```shell +$ nix-build +(some output removed for clarity) +/nix/store/h5abh0vljgzg381lna922gqknx6yc0v7-stream-hello-2.12.1-env -```console -$out/bin/hello +$ /nix/store/h5abh0vljgzg381lna922gqknx6yc0v7-stream-hello-2.12.1-env | docker load +(some output removed for clarity) +Loaded image: hello-2.12.1-env:latest ``` - Hello, world! +After starting an interactive container, we can verify the extra package is available by running `cowsay`: + +```shell +$ docker run -it hello-2.12.1-env:latest +[nix-shell:~]$ cowsay "Hello, world!" + _______________ +< Hello, world! > + --------------- + \ ^__^ + \ (oo)\_______ + (__)\ )\/\ + ||----w | + || || +``` +::: + +:::{.example #ex-dockerTools-streamNixShellImage-addingShellHook} +# Adding a `shellHook` to a Docker image built with `streamNixShellImage` + +This example shows how to add a `shellHook` command to an image built with `streamNixShellImage`. +In this case, we'll simply output the string `Hello, world!`. +The Docker image generated will have a name like `hello--env` and tag `latest`. +This example uses [](#ex-dockerTools-streamNixShellImage-hello) as a starting point. + +```nix +{ dockerTools, hello }: +dockerTools.streamNixShellImage { + tag = "latest"; + drv = hello.overrideAttrs (old: { + shellHook = '' + ${old.shellHook or ""} + echo "Hello, world!" + ''; + }); +} +``` + +The result of building this package is a script which can be run and piped into `docker load` to load the generated image. + +```shell +$ nix-build +(some output removed for clarity) +/nix/store/iz4dhdvgzazl5vrgyz719iwjzjy6xlx1-stream-hello-2.12.1-env + +$ /nix/store/iz4dhdvgzazl5vrgyz719iwjzjy6xlx1-stream-hello-2.12.1-env | docker load +(some output removed for clarity) +Loaded image: hello-2.12.1-env:latest +``` + +After starting an interactive container, we can see the result of the `shellHook`: + +```shell +$ docker run -it hello-2.12.1-env:latest +Hello, world! + +[nix-shell:~]$ +``` +::: diff --git a/doc/manpage-urls.json b/doc/manpage-urls.json index 2f3fac52c6b7..5739a59d9420 100644 --- a/doc/manpage-urls.json +++ b/doc/manpage-urls.json @@ -317,5 +317,6 @@ "udevadm(8)": "https://www.freedesktop.org/software/systemd/man/udevadm.html", "passwd(5)": "https://man.archlinux.org/man/passwd.5", "group(5)": "https://man.archlinux.org/man/group.5", - "login.defs(5)": "https://man.archlinux.org/man/login.defs.5" + "login.defs(5)": "https://man.archlinux.org/man/login.defs.5", + "nix-shell(1)": "https://nixos.org/manual/nix/stable/command-ref/nix-shell.html" }