Building without tests is useful for bootstrapping with a smaller footprint
or running the tests in a separate derivation. Otherwise, we do compile and
run them.
This isn't fine grained as to allow picking `check` but not `installcheck`
or vice versa, but it's good enough for now.
I've tried to use Nixpkgs' `checkInputs`, but those inputs weren't discovered
properly by the configure script. We can emulate its behavior very well though.
For static builds, we need to propagate all the static library
dependencies to the link of the program. E.g. if libstore-tests-exe
depends on libnixstore-tests, and libnixstore-tests depends on
libstore, then libstore-tests-exe needs to link against libstore.
https://hydra.nixos.org/build/209007480
First, logic is consolidated in the shell script instead of being spread
between them and makefiles. That makes understanding what is going on a
little easier.
This would not be super interesting by itself, but it gives us a way to
debug tests more easily. *That* in turn I hope is much more compelling.
See the updated manual for details.
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
gives 2-5% performance improvement across a board of tests.
LTO is broken when using clang; some libs link fine while others crash
the linker with a segfault in the llvm linker plugin. 🙁
Restart the tests (at most once) on `unexpected EOF` errors.
This is truly ugly, but might prevent half of the CI runs to fail
because of https://github.com/NixOS/nix/issues/3605
It's a second attempt to merge the change. Previous attempt
was reverted in b976b34a5b.
Since then underlying failure exposed by original change was
fixed by https://github.com/NixOS/nix/pull/5354.
Below goes description of original change:
The link failure happens on a system with stable nix-2.3.15
installed in /usr/lib64 (it's libutil.so API differs from master):
```
LANG=C make V=1
g++ -o /home/slyfox/dev/git/nix/src/libstore/libnixstore.so \
-shared -L/usr/lib64 -Wl,--no-copy-dt-needed-entries \
src/libstore/binary-cache-store.o ... src/libstore/uds-remote-store.o \
-lsqlite3 -lcurl -lsodium -pthread -ldl -lseccomp -Wl,-z,defs -Wl,-soname=libnixstore.so
-Wl,-rpath,/home/slyfox/dev/git/nix/src/libutil -Lsrc/libutil -lnixutil
ld: src/libstore/binary-cache-store.o: in function `nix::BinaryCacheStore::BinaryCacheStore(
std::map<std::__cxx11::basic_string<char, std::char_traits<char>, ...
nix/src/libstore/binary-cache-store.cc:30: undefined reference to `nix::readFile(
std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)' ...
...
```
This happens due to `-L/usr/lib64 -Lsrc/libutil` search path ordering.
The change turns it into `-Lsrc/libutil -L/usr/lib64`.
Closes: https://github.com/NixOS/nix/issues/3087
Noticed this minor logging deficiency when debugged --disable-shared
build:
LD
AR
LD
CXX src/libstore/local-store.o
After the change build is logged as expected:
LD src/libmain/libnixmain.a
LD src/libfetchers/libnixfetchers.a
AR src/libmain/libnixmain.a
CXX src/libstore/local-store.o
The link failure happens on a system with stable nix-2.3.15
installed in /usr/lib64 (it's libutil.so API differs from master):
```
LANG=C make V=1
g++ -o /home/slyfox/dev/git/nix/src/libstore/libnixstore.so \
-shared -L/usr/lib64 -Wl,--no-copy-dt-needed-entries \
src/libstore/binary-cache-store.o ... src/libstore/uds-remote-store.o \
-lsqlite3 -lcurl -lsodium -pthread -ldl -lseccomp -Wl,-z,defs -Wl,-soname=libnixstore.so
-Wl,-rpath,/home/slyfox/dev/git/nix/src/libutil -Lsrc/libutil -lnixutil
ld: src/libstore/binary-cache-store.o: in function `nix::BinaryCacheStore::BinaryCacheStore(
std::map<std::__cxx11::basic_string<char, std::char_traits<char>, ...
nix/src/libstore/binary-cache-store.cc:30: undefined reference to `nix::readFile(
std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)' ...
...
```
This happens due to `-L/usr/lib64 -Lsrc/libutil` search path ordering.
The change turns it into `-Lsrc/libutil -L/usr/lib64`.
Closes: https://github.com/NixOS/nix/issues/3087
Previously, the build system used uname(1) output when it wanted to
check the operating system it was being built for, which meant that it
didn't take into-account cross-compilation when the build and host
operating systems were different.
To fix this, instead of consulting uname output, we consult the host
triple, specifically the third "kernel" part.
For "kernel"s with stable ABIs, like Linux or Cygwin, we can use a
simple ifeq to test whether we're compiling for that system, but for
other platforms, like Darwin, FreeBSD, or Solaris, we have to use a
more complicated check to take into account the version numbers at the
end of the "kernel"s. I couldn't find a way to just strip these
version numbers in GNU Make without shelling out, which would be even
more ugly IMO. Because these checks differ between kernels, and the
patsubst ones are quite fiddly, I've added variables for each host OS
we might want to check to make them easier to reuse.
Requires a slight update to the test infra to work properly, but
having the possibility to group tests that way makes the whole thing
quite cleaner imho
In cross, CXX will look like aarch64-unknown-linux-gnu-g++. We could
run some command to check what kind of compiler it is, but for now we
can just check if g++ is anywhere in the string. I couldn’t find any
"ends with" for makefile, so it can be anywhere in CXX.
This combines the *.o into a big .o producing one translation unit.
This preserve our unused static initializers, as specified in the C++
standard:
If no variable or function is odr-used from a given translation
unit, the non-local variables defined in that translation unit may
never be initialized (this models the behavior of an on-demand
dynamic library).
Note that this is very similar to how the --whole-archive flag works.
One advantage of this is that users of the final .a library don’t have
to worry about specifying --whole-archive, or that we have unused
static initializers at all!
autotools-based systems usually allow user to
append own LDFLAGS like
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu"
at ./configure stage
This change plumbs LDFLAGS through similar to existing CXXFLAGS variable.
Signed-off-by: Sergei Trofimovich <siarheit@google.com>
In this mode, the following restrictions apply:
* The builtins currentTime, currentSystem and storePath throw an
error.
* $NIX_PATH and -I are ignored.
* fetchGit and fetchMercurial require a revision hash.
* fetchurl and fetchTarball require a sha256 attribute.
* No file system access is allowed outside of the paths returned by
fetch{Git,Mercurial,url,Tarball}. Thus 'nix build -f ./foo.nix' is
not allowed.
Thus, the evaluation result is completely reproducible from the
command line arguments. E.g.
nix build --pure-eval '(
let
nix = fetchGit { url = https://github.com/NixOS/nixpkgs.git; rev = "9c927de4b179a6dd210dd88d34bda8af4b575680"; };
nixpkgs = fetchGit { url = https://github.com/NixOS/nixpkgs.git; ref = "release-17.09"; rev = "66b4de79e3841530e6d9c6baf98702aa1f7124e4"; };
in (import (nix + "/release.nix") { inherit nix nixpkgs; }).build.x86_64-linux
)'
The goal is to enable completely reproducible and traceable
evaluation. For example, a NixOS configuration could be fully
described by a single Git commit hash. 'nixos-rebuild' would do
something like
nix build --pure-eval '(
(import (fetchGit { url = file:///my-nixos-config; rev = "..."; })).system
')
where the Git repository /my-nixos-config would use further fetchGit
calls or Git externals to fetch Nixpkgs and whatever other
dependencies it has. Either way, the commit hash would uniquely
identify the NixOS configuration and allow it to reproduced.
I got sick of trying to find the failures in the sea of debug output, so
we now:
- Hide test output unless it fails
- Sprinkle in some simple color
- Pad results for a more tabular look
If Nix is getting a more friendly user interface, we might as well get
a friendlier developer interface, right? :)
