Previously this was done in three derivations (one to build the raw
disk image, one to convert to OVA, one to add a hydra-build-products
file). Now it's done in one step to reduce the amount of copying
to/from S3. In particular, not uploading the raw disk image prevents
us from hitting hydra-queue-runner's size limit of 2 GiB.
This splits a few NixOS tests (namely Chromium, VirtualBox and the
networking tests) into several subtests that are exposed via attributes.
The networking tests were already split up but they didn't expose an
attribute set of available tests but used a function attribute to
specify the resulting test instead.
A new function callSubTests in nixos/release.nix is now responsible for
gathering subtests, which is also used for the installer and boot tests.
The latter is now placed in a tests.boot.* namespace rather than
"polluting" the tests attribute set with its subtest.
This makes it easier to test just a specific channel rather than to
force testing all builds down the users/testers throat. Especially this
makes it easier to test NixOS channel upgrades only against the Chromium
stable channel instead of just removing the beta/dev channels from the
tests entirely (as done in 69ec09f38a).
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
Should clean up a lot of these redundant lines for various sub-tests.
Note that the tests.boot* are now called tests.boot.boot*, but otherwise
all the test attribute names should stay the same.
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
Cc: @edolstra
Cc: @wkennington
Cc: @bobvanderlinden
This should de-clutter the various redundant lines of callTest's on
subtests so that every main test file should have only one line with a
callSubTests instead.
Overrides work the same way as callTest, except that if the system
attribute is explicitly specified we do not generate attributes for all
available systems.
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
Cc: @edolstra
Now subtests are separate derivations, because the individual tests do
not depend on state from previous test runs.
This has the advantage that it's easier to run individiual tests and
it's also easier to pinpoint individual tests that randomly fail.
I ran all of these tests locally and they still succeed.
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
- Enforce that an option declaration has a "defaultText" if and only if the
type of the option derives from "package", "packageSet" or "nixpkgsConfig"
and if a "default" attribute is defined.
- Enforce that the value of the "example" attribute is wrapped with "literalExample"
if the type of the option derives from "package", "packageSet" or "nixpkgsConfig".
- Warn if a "defaultText" is defined in an option declaration if the type of
the option does not derive from "package", "packageSet" or "nixpkgsConfig".
- Warn if no "type" is defined in an option declaration.
The manuals are now evaluated with each derivation in `pkgs` (recursively)
replaced by a fake with path "\${pkgs.path.to.the.attribute}".
It isn't perfect, but it seems to cover a vast majority of use cases.
Caveat: even if the package is reached by a different means,
the path above will be shown and not e.g. `${config.services.foo.package}`.
As before, defaults created by `mkDefault` aren't displayed,
but documentation shouldn't (mostly) be a reason to use that anymore.
Note: t wouldn't be enough to just use `lib.mapAttrsRecursive`,
because derivations are also (special) attribute sets.
It serves as a regression test, because right now if you enable
networking.useNetworkd the default loopback interface doesn't get
assigned any IP addresses.
To be sure, I have bisected this and it has been introduced with the
update to systemd 228 in 1da87d4.
Only the "scripted" networking tests have to succeed in order to trigger
a channel update of nixos-unstable, so I'm leaving this test as broken
and we have to figure out next what's the *exact* reason for the
breakage.
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
We already have separate tests for checking whether the ISO boots
correctly, so it's not necessary to do that here. So now
tests/installer.nix just tests nixos-install, from a regular NixOS VM
that uses the host's Nix store. This makes running the tests more
convenient because we don't have to build a new ISO after every
change.
We want to avoid getting broken LUKS systems into the latest channel, so
let's ensure that the channel update won't happen if LUKS support is
broken again.
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
The i686-linux test has never worked and I wrote the VM test only on
x86_64-linux to verify whether hardening mode works. I don't know why it
fails on i686-linux, but that might be because the inner VirtualBox VM
we're starting during the VM test doesn't use hardware virtualization.
Closes#5708.
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
It has been removed by 71a197bc6e.
I'm reintroducing the test mainly because it actually *is* useful,
because right now, machines with mdraid will not boot. In order to
prevent such things from happening in the future, we should *not* remove
this VM test again.
If it really goes back to failing randomly, we should really try to fix
it instead of removing it again.
Of course it fails right now because of the mdraid bump in 7719f7f.
Also, if you want to have additional info about the reasons, have a look
at the commit message of 666cf992f0.
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
It now strictly evaluates all remaining attributes, preventing
unevaluated thunks that cannot be garbage-collected. It's also applied
to all jobs in Nixpkgs' release.nix.
This reduces hydra-eval-jobs' memory consumption on the 14.12
release-combined jobset from 5.1 GB to 2.0 GB.
Currently just makes sure that by default it's possible to open a
terminal.
And exactly this should be the main point that might confuse users of i3
in NixOS, because i3 doesn't print a warning/error if it is unable to
start the terminal emulator.
Thanks to @waaaaargh for reporting this issue.
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
