Commit graph

171 commits

Author SHA1 Message Date
Vincent Ambo
af9f248926 streamLayeredImage: pass through passthru attributes
This is useful for a use-case we have with a Nix-based CI system that
specifies things like deploy steps as passthru attributes[0].

Previously the only way to do this would have been to concatenate
attributes onto the resulting derivation, but passing them in and
actually treating them as proper passthru attributes is cleaner.

[0]: https://cs.tvl.fyi/depot@f7d7da6aceb407b719cf4683a75878fd3aca319e/-/blob/nix/buildkite/default.nix?L222-226
2022-01-31 10:10:10 +03:00
Andrew Brooks
69ffb0004a dockerTools.buildImage: unpack base image layers in correct order 2021-12-17 18:28:10 -06:00
Robert Hensing
ddda5f28e1 dockerTools: Keep fakechroot disabled by default
Avoid risk of breaking existing images by making it opt-in.
2021-12-04 13:49:10 +00:00
Robert Hensing
0e9bc9ffd1 dockerTools: Add fakechroot to fakeRootCommands 2021-12-04 13:49:10 +00:00
Robert Hensing
66c19d856b
Merge pull request #148341 from hercules-ci/add-dockerTools-customization-layer-dependencies
dockerTools: Add store dependencies of the customization layer
2021-12-03 00:20:44 +01:00
Ryan Burns
aed6bb336e build-support/docker: remove toplevel system attr 2021-10-22 14:04:05 -07:00
Robert Hensing
3b9d05e114 dockerTools: Fix and test #118722 path in contents 2021-10-18 12:41:51 +02:00
Robert Hensing
48cfdc8ca5 dockerTools: Add store dependencies of the customization layer 2021-10-06 22:45:04 +02:00
Robert Hensing
1a0edf135a dockerTools.exportImage: Make $out a tarball again 2021-09-29 09:34:45 +02:00
Robert Hensing
63bf4539b9 dockerTools.runWithOverlay: Avoid cluttering $out and copying 2021-09-29 09:34:06 +02:00
Justin Bedo
fa0cc611ff
dockerTools: fix export 2021-09-29 15:35:25 +10:00
divanorama
b012fe5e7e
use --verbatim-files-from in dockerTools
https://www.gnu.org/software/tar/manual/html_node/files.html
files starting with - can be treated as command line options, which isn't desirable here
2021-09-10 19:12:32 +02:00
Florian Klink
14b61efa87 dockerTools.usrBinEnv: add
This provides a /usr/bin/env, for shell scripts using the
"/usr/bin/env executable" shebang.
2021-08-25 16:08:39 +02:00
Florian Klink
f8031c60b7
dockerTools.fakeNss: add /etc/nsswitch.conf (#134958)
Apparently, a non-existent nsswitch.conf causes a very misleading host
resolution, differing from the defaults people are used to.

According to
https://github.com/golang/go/issues/22846#issuecomment-346377144, glibc
says the default is "dns [!UNAVAIL=return] files".

This means, `/etc/hosts` isn't really honored, causing all sorts of
unexpected behaviour.

Let's prevent this, and first ask `/etc/hosts` before querying DNS, like
we do on NixOS too.
2021-08-20 19:44:17 +02:00
Luke Granger-Brown
dbe9bf9848 dockerTools.pullImage: fix for skopeo 1.4.x
skopeo 1.4.x doesn't accept --src-tls-verify as a flag to the *program*,
only as a flag to copy; we must pass it after the "copy" verb, or it
will fail with:

> FATA[0000] unknown flag: --src-tls-verify
2021-08-14 21:21:26 +00:00
Sandro Jäckel
ceb417aaf1
docker: format 2021-08-10 10:32:46 +02:00
Robert Hensing
1c4b1018a3
Merge pull request #132626 from zowoq/tarsum
tarsum: init
2021-08-05 12:14:35 +02:00
zowoq
5dde5ea73e tarsum: init
- move from dockerTools.tarsum
- remove go from runtime closure
2021-08-05 08:54:41 +10:00
Robert Hensing
641bac80e5
dockerTools.buildImage: Allow build on darwin (#132640)
Indeed Docker can not run darwin exes, but darwin can build
Docker images, as some users already do with buildLayeredImage.
2021-08-04 06:06:06 -04:00
zowoq
05fe220746 dockerTools.tarsum: use current go 2021-07-08 16:31:33 +10:00
Luke Granger-Brown
48c23a086b dockerTools.pullImage: force disable skopeo's progress bar
skopeo will disable the progress bar if it detects that stdout isn't a
TTY - in order to make it think that stdout _isn't_ a TTY and therefore
avoid it printing a lot of "…" on separate lines, we pipe the output
through cat.

This changes the output from:

…
…
…
…
…
…

to the eminently more useful and less spammy:

Getting image source signatures
Copying blob sha256:[snip]
Copying blob sha256:[snip]
Copying blob sha256:[snip]
Copying config sha256:[snip]
Writing manifest to image destination
Storing signatures
2021-07-03 01:40:38 +00:00
Robert Hensing
1834bc8711
Merge pull request #125216 from hercules-ci/follow-up-115491
dockerTools: Fix passthru image tag
2021-06-02 16:58:49 +02:00
Robert Hensing
7f6a395070
Merge pull request #125211 from edwtjo/docker-fetch-wotls
build-support/docker: pass tlsVerify to support http registries
2021-06-01 15:07:48 +02:00
Robert Hensing
ff55c41fac dockerTools: Fix passthru image tag
It should match the actual image tag.
This fixes the problem introduced in 00996b5e03
https://github.com/NixOS/nixpkgs/pull/115491#pullrequestreview-672789901
2021-06-01 14:42:21 +02:00
Edward Tjörnhammar
ab4649e9d1
build-support/docker: pass tlsVerify to support http registries 2021-06-01 14:34:24 +02:00
Robert Hensing
5259d66b74 dockerTools: Allow omitting all store paths
Adds includeStorePaths, allowing the omission of the store paths.
You generally want to leave it on, but tooling may disable this
to insert the store paths more efficiently via other means, such
as bind mounting the host store.
2021-05-26 15:11:42 +02:00
Robert Hensing
69de7cc12a dockerTools: Format 2021-05-26 15:11:42 +02:00
Robert Hensing
578acc7a42
Merge pull request #118018 from considerate/master
dockerTools: Implement merging of image tarballs
2021-04-16 09:17:44 +02:00
Jörg Thalheim
65a40ca547
Merge pull request #102725 from thefloweringash/dockertools-proc
dockerTools: fix absent /proc during runAsRoot
2021-04-12 14:14:24 +01:00
Viktor Kronvall
b2aa1f9d7a dockerTools: preserve order of images in manifest 2021-04-07 23:20:14 +09:00
Viktor Kronvall
dcc9aef015 dockerTools: take a list of images in mergeImages 2021-04-07 22:49:44 +09:00
Viktor Kronvall
ecc293ff7a dockerTools: Implement merging of image tarballs
The `docker load` command supports loading tarballs that contain
multiple docker images with their respective image names and tags. This
enables distributing these images as a single file which simplifies the
release of software when an application requires multiple services to
run.

However, pkgs.dockerTools only create tarballs with a single docker
image and there exists is no mechanism in nixpkgs to combine the created
tarballs. This commit implements merging of tarballs in a way that is
compatible with `docker load`.
2021-04-07 22:49:44 +09:00
Robert Hensing
58b21dea78
Merge pull request #116749 from vroad/docker-layered-image-fakeroot
dockerTools.streamLayeredImage: add fakeRootCommands option
2021-04-07 15:02:24 +02:00
Louis Blin
aae8588182 dockerTools.buildLayeredImage: support fromImage
It is now possible to pass a `fromImage` to `buildLayeredImage` and
`streamLayeredImage`, similar to what `buildImage` currently supports.

This will prepend the layers of the given base image to the resulting
image, while ensuring that at most `maxLayers` are used. It will also
ensure that environment variables from the base image are propagated
to the final image.
2021-03-23 14:50:42 +00:00
vroad
057c7a2d58 dockerTools.streamLayeredImage: source $stdenv/setup before running fakeRootCommands 2021-03-19 11:01:10 +09:00
vroad
5199c7e6da
dockerTools.streamLayeredImage: simplify inherit statements
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2021-03-19 10:56:00 +09:00
vroad
b4d92811be dockerTools.streamLayeredImage: add fakeRootCommands option 2021-03-19 10:25:26 +09:00
Benjamin Hipple
17a9f368e3
Merge pull request #115793 from lbpdt/feature/docker-tools-layered-image-name-slashes
dockerTools.buildLayeredImage: image names with registry/ prefix
2021-03-11 14:58:26 -05:00
Louis Blin
419a4fa596 dockerTools.buildLayeredImage: image names with registry/ prefix
When using `buildLayeredImage`, it is not possible to specify an image
name of the form `<registry>/my/image`, although it is a valid name.

This is due to derivations under `buildLayeredImage` using that image
name as their derivation name, but slashes are not permitted in that
context.

A while ago, #13099 fixed that exact same problem in `buildImage` by
using `baseNameOf name` in derivation names instead of `name`. This
change does the same thing for `buildLayeredImage`.
2021-03-10 17:44:24 +00:00
Louis Blin
bf56388c92 dockerTools.buildLayeredImage: configurable store root
`stream_layered_image.py` currently assumes that the store root will be
at `/nix/store`, although the user might have configured this
differently. This makes `buildLayeredImage` unusable with stores having
a different root, as they will fail an assertion in the python script.

This change updates that assertion to use `builtins.storeDir` as the
source of truth about where the store lives, instead of assuming
`/nix/store`.
2021-03-10 16:44:53 +00:00
Chris Roberts
00996b5e03 dockerTools: Do not lowercase image tag in buildImage
Closes #115455
2021-03-09 09:06:04 +00:00
Robert Hensing
f3d006c1d4
Merge pull request #115083 from osener/fix-dockertools-on-darwin
dockerTools: fix build on Darwin
2021-03-04 16:50:26 +01:00
Ozan Sener
78f322f2d0 dockerTools: fix build on Darwin
Fixes #110665

Introduced by #109420
2021-03-04 16:17:48 +01:00
Ben Siraphob
98f26993f2 treewide: remove stdenv where not needed 2021-03-04 19:54:50 +07:00
Johan Thomsen
793e77d4e2 dockertools: sort tar-members by name for reproducibility 2021-02-18 09:51:41 +01:00
Pavol Rusnak
90f7338112
treewide: stdenv.lib -> lib 2021-01-24 01:49:49 +01:00
Robert Hensing
9797139cbb dockerTools: typo
Italian.
2021-01-15 14:23:51 +01:00
Milan
0d724ba2fc
dockerTools: fix build (#109420) 2021-01-15 03:56:38 +01:00
Robert Hensing
a31607f904 dockerTools: Don't apologize
Warning about future breaking changes is wrong.

 - It suggests that the maintainers don't value backwards compatibility.
   They do.
 - It implies that other parts of Nixpkgs won't ever break. They will.
 - It implies that a well-defined "public" interface exists. It doesn't.
 - If the reasons above didn't apply, it should have been in the manual
   instead.

Breaking changes will come, especially to the interface. That can be the
only way we can make progress without breaking the image _contents_.

I don't think dockerTools is any different from most of Nixpkgs in
these regards.
2021-01-06 13:02:19 +01:00
Robert Hensing
5cacf0fcec dockerTools: use go.GOARCH as default arch 2020-12-15 02:15:35 -08:00