Make stages explicit and generalize the pattern of having an stdenv and
a pkgs collection for all stages to a common stage generating function
called stageFun.
Rewrite all stage handling with this new function.
This commit doesn't change the outhash (or drvhash) of the stdenv.
Don't use default parameter values, to make the callsites more readable
and for easier debuggability/changability. Also reordered the
callsites' parameter ordering for consistency.
In the final stdenv don't repeat the name of the shell.
This commit doesn't change the outhash (or drvhash) of the stdenv.
All the different stages of stdenv had the fetchurl inherited anyways,
so make this generic in stdenvBootFun.
This commit doesn't change the outhash (or drvhash) of the stdenv.
Commit 986f361946 started to use
<nix/fetchurl.nix> to "download" the bootstrap binaries from the
Nixpkgs tree, using the file:/// scheme. This has really bad
consequences:
* It makes any derivation depend on the path of the Nixpkgs tree. So
evaluating a package will produce a different .drv file when run
from different locations. No wonder Hydra evaluation has been so
slow lately: for every Nixpkgs evaluation, it had to create tens of
thousands of .drv files, even if nothing had changed.
* It requires the builder to have file system access to the Nixpkgs
tree. So if your tree is in your home directory, the stdenv
bootstrap would probably fail.
So now the binaries are downloaded from tarballs.nixos.org.
Also dropped PowerPC "support".
Binutils nowadays contains ld.gold, which depends on libstdc++. So it
needs to be built with the new GCC rather than the one from
bootstrap-tools.
Issue #1469.
GCC provides a number of libraries that are used by programs built by
GCC, in particular libgcc_s.so and libstdc++.so. This caused programs
that used these libraries to have a runtime dependency on all of GCC
(~77 MiB). Now they only depend on the "lib" output of GCC (~1.6
MiB).
With this and previous multiple-output improvements, closure sizes are
reduced a lot:
hello: 41 MiB -> 22 MiB
patchelf: 118 MiB -> 23 MiB
pan: 364 MiB -> 90 MiB
This reduces the size of closures a lot by getting rid of Glibc's
header files, static libraries and utilities. For instance, the
closure of GNU Hello went from 41 MiB to 22 MiB.
TODO: We may want to merge back in parts of Glibc that we previously
split off (such as the info files).
Since nix-1.4, nix's corepkgs contain a fetchurl suitable for
downloading the bootstrap binaries. Doing this will allow us to have a
nixpkgs with no in-tree binaries without breaking the purity of the
bootstrap (though for now, they are fetched in-tree until the binaries
are added to nixos.org somewhere). As an additional small benefit, the
in-tree binaries do not have to be hashed on every instantiation as they
do now.
The fetchurl in nix-1.2 and 1.3 isn't able to make binaries executable,
so it can't be used for this case. In that case, attempting to build the
bootstrap will show a message asking the user to manually download each
file and add it to the store with nix-store --add (but the hash is
ultimately the same, of course).
Signed-off-by: Shea Levy <shea@shealevy.com>
Due to xz being override in the last stdenv and also in the previous, the
nixpkgs xz ended up being built by bootstrap-tools, and thus depending on it
through libgcc_so.so.1. That ends up making 'nix' with a runtime
dependency on bootstrap-tools.
Although patching it made some programs run (configure tests), some others
crashed with segfault. So I don't think there is any win patching it. The
proper way to solve the bootstrap in the raspberry pi is, as far as I've been
testing, use glibc 2.17 libs in bootstrap-tools with the same ld.so name as the
bootstrapped glibc.
This is a problem inherent in our way to bootstrap, that first replaces
the glibc of a given gcc+glibc (bootstrap-tools) with gcc-wrapper tricks, and
then builds a new gcc. A nicer way would be to build a gcc without glibc,
then the glibc, then the final gcc, as we do with cross-tools.
Some comments about this problem in
https://github.com/NixOS/nixpkgs/issues/234#issuecomment-11764352
This solves the problem of the change of name in ld.so between
glibc 2.13 and 2.17 (at least for armhf). Some comments about it in
https://github.com/NixOS/nixpkgs/issues/234#issuecomment-11764352
As there says, without this, gmp (after glibc built) tests crashed
- segfault.
Now that we have a way to alter /bin/sh in chroots on non-NixOS Linux
platforms, it may be useful to have access to stdenv.shell package and to
static bash contained in bootstrap tools. So make them accessible via stdenv
attribute set.
libstdc++ and libmudflapth link to the dynamic loader; if the
bootstrap uses another dynamic loader name, and the rpath of these
libs isn't changed, they will fail to load.
Conflicts:
pkgs/development/compilers/gcc/4.6/default.nix
pkgs/development/compilers/gcc/4.7/default.nix
The 4.7 had some weird parameters added in crossAttrs; I've removed
them, but I don't understand where they come from.
This allows various applications. It allows users to set global
optimisation flags, e.g.
stdenv.userHook = ''NIX_CFLAGS_COMPILE+=" -funroll-loops"'';
But the impetus is as an alternative to issue #229, allowing impure
stdenv setup for people who want to use distcc:
stdenv.userHook = "source /my/impure/setup-script.sh";
This is probably a bad idea, but at least now it's a bad idea in
people's configuration and not in Nixpkgs. :-)
what the new nix thinks the fuloong is.
Anyone having the old nix should use a nixpkgs previous to this change to build
the new nix. And then, with the new nix, he can use any newer nixpkgs revision.
svn path=/nixpkgs/trunk/; revision=31751
bash's pattern replacement feature. "replace-literal" is an
uncommon command so it was a headache during the bootstrap.
svn path=/nixpkgs/branches/stdenv-updates/; revision=31681
some redundant builds (e.g., GMP was built three times).
* Updated GMP to 5.0.2.
* Updated PPL to 0.11.2.
* Remove ad hoc flags to build GCC's dependencies statically.
Instead, use the ‘makeStaticLibraries’ stdenv adapter.
* Build GMP with C++ support by default.
svn path=/nixpkgs/branches/stdenv-updates/; revision=30891
bootstrap-tools, because zlib was there and binutils were not having an
explicit buildInput for zlib. Due to that, we ended up with stdenv
(gcc-wrapper) depending on bootstrap-tools as runtime dependency.
svn path=/nixpkgs/branches/stdenv-updates/; revision=24790
newest I uploaded yesterday. These have gcc 4.5.1, that fulfills the glibc
2.12.1 requirement of having a compiler equal or greater to gcc 4.4 to build
proper x86 32-bit code (http://sourceware.org/bugzilla/show_bug.cgi?id=12123)
svn path=/nixpkgs/branches/stdenv-updates/; revision=24523
had a little svn mess in the working directory that ended up in a not working
stdenvLinux.
This time I even chose better names for the attributes, so they match better
the comments.
svn path=/nixpkgs/branches/stdenv-updates/; revision=23428
I had done an attempt recently, unsuccesful, which ended in a recent revert.
This change works.
I even updated the comments in the file.
svn path=/nixpkgs/branches/stdenv-updates/; revision=23425
What I did results in segfaults in built binaries.
svn merge -c -23370 ^/nixpkgs/branches/stdenv-updates
svn path=/nixpkgs/branches/stdenv-updates/; revision=23376
This should allow keeping the i686-linux boostrap-tools.
I updated some days ago the x86_64-linux bootstrap-tools, but that update is
not needed anymore. We can revert that boostrap-tools update if anyone wants.
Pro:
- new gcc building gcc and glibc.
Contra:
- maybe some old systems (patched red had kernels come to mind) break with that update?
svn path=/nixpkgs/branches/stdenv-updates/; revision=23370
Allowing 'curl' build without some dependencies, if it is told not to build
with them.
Updating the make-bootstrap-tools for the latest gcc/glibc, removing the
dependency on klibc, and updating unpack-boostrap-tools for the latest
boostrap-tools to work.
svn path=/nixpkgs/branches/stdenv-updates/; revision=23305
Instead, use the generic package override mechanism to use packages
from earlier bootstrap phases.
* Don't rely on the existence of attributes such as
`stdenv.coreutils'.
svn path=/nixpkgs/trunk/; revision=22991
we can't build easily with them gcc-4.4 natively, because of a bug in the ld of the
previous bootstrap-tools.
I updated the unpack script to include the new gcc-4.4 libraries *ppl*.so in patchelfing.
svn path=/nixpkgs/branches/stdenv-updates/; revision=18835
(dynamic libraries of ppl and cloogppl) into the package.
We need newer binutils in the bootstrap-tools for the armv5tel in order to
be able to build gcc 4.4 from them.
svn path=/nixpkgs/branches/stdenv-updates/; revision=18747
the cross compilation functionality.
- I renamed some expected stdenv.mkDerivation parameter attributes so we can
keep this branch properly updated from trunk. We agreed with Nicolas Pierron
doing a massive renaming, so all current buildInputs become hostInputs (input
as build for the host machine, in autotools terminology) , and
then buildInputs would mean "input as for the build machine".
By now, the specific "input as for the build machine" is specified through
buildNativeInputs. We should fix this in the merge to trunk.
- I made the generic stdenv understand the buildNativeInputs, otherwise if
we start changing nixpkgs expressions so they distinguish the current
buildInputs into buildInputs and buildNativeInputs, we could break even more
nixpkgs for other platforms.
- I changed the default result of mkDerivation so it becomes the derivation for
to be run in the build machine. This allows, without any special rewriting,
"fetchurl" derivations to be always results for the build machine to use
them.
- The change above implies that, for anyone wanting to cross-compile, has to
build the hostDrv of the wanted derivation. For example, after this commit,
the usual test of "nix-build -A bison.hostDrv arm.nix" works. I described
the contents of this arm.nix in r18398.
svn path=/nixpkgs/branches/stdenv-updates/; revision=18471
It still does not work, but I think I already get glibc cross compiled.
Next: gcc and g++, and set some setup script hooks on stdenvCross.
It took quite enough hours for this commit.
svn path=/nixpkgs/branches/stdenv-updates/; revision=18351
My idea is to provide special stdenv expressions that will contain in the path
additional cross compilers. As most expressions for programs accept a stdenv parameter,
we could substitute this parameter with the special stdenv, which will have a
generic builder that attempts the usual "--target=..." and can additionally
have an env variable like "cross" with the target architecture set.
So, finally we could have additional expressions like this:
bashRealArm = makeOverridable (import ../shells/bash) {
inherit fetchurl bison;
stdenv = stdenvCross "armv5tel-unknown-linux-gnueabi";
};
Meanwhile it does not work - I still cannot get the cross-gcc to build.
I think it does not fill the previous expressions with a lot of noise, so I
think it may be a good path to follow.
I only touched some files of the current stdenv: gcc-4.3, kernel headers
2.6.28, glibc 2.9, ...
I tried to use the gcc-cross-wrapper, that may be very outdated. Maybe I will
update it, or update the gcc-wrapper expression to make it fit the cross tools,
but meanwhile I even cannot build gcc, so I have not tested the wrapper.
This new idea on cross compiling is not similar to that of the
nixpkgs/branches/cross-compilation, which mostly added bare new expressions for
anything to be cross compiled, if I understood it correctly.
I cared not to break anything of the usual stdenv in all this work.
svn path=/nixpkgs/branches/stdenv-updates/; revision=18343
This comes from:
svn diff ^/nixpkgs/trunk/@18255 ^/nixpkgs/branches/stdenv-updates/ > diff
patch -p0 < diff
and then adding into svn all files new from the patch.
trunk@18255 comes from the last time I updated stdenv-updates from trunk.
svn path=/nixpkgs/stdenv-updates2/; revision=18272
I thought I didn't change stdenv, but I did. This will go soon into the stdenv
branch then.
Reverse-merging r16467 through r16465.
svn path=/nixpkgs/trunk/; revision=16468
LD_LIBRARY_PATH since it breaks /bin/sh on non-NixOS platforms (and
reverted the previous "fix" in r15470).
svn path=/nixpkgs/branches/stdenv-updates/; revision=16029
* Use patchelf 0.5 and use the --force-rpath flag. This removes the
need for setting LD_LIBRARY_PATH and so should fix bootstrapping
Nixpkgs on non-NixOS platforms.
svn path=/nixpkgs/trunk/; revision=16022
instead of "gcc-4.3.3". This fixed the long-standing annoyance that
you can't distinguish the two in (say) nix-store -qR.
* On x86_64-linux, put $out/lib64 in the RPATH in addition to
$out/lib, because some packages (in particular GCC) put libraries in
$out/lib64 and ended up linking against the wrong library.
* Strip $out/lib64.
* Removed g77_42 because it's exactly the same as gfortran.
svn path=/nixpkgs/branches/stdenv-updates/; revision=14708
* Updated stdenv-linux:
- The bootstrap tools are no longer statically linked (except for
binaries in the Nixpkgs tree used to download and unpack the
bootstrap tools).
- x86_64 uses the same static binaries as i686. This makes the
Nixpkgs tree a bit smaller.
- Use the Linux 2.6.28 headers.
svn path=/nixpkgs/branches/stdenv-updates/; revision=13946
setting the NIX_INDENT_MAKE variable; disabled by default) so we
don't need a separate gnumakeNix package.
svn path=/nixpkgs/branches/stdenv-updates/; revision=13807
Instead of using a mishmash of statically linked and dietlibc
programs (which often have problems on various platforms), we just
use normal, dynamically linked binaries, and use some patchelf magic
to rewrite them so that they can find Glibc. Also include G++ and
Perl, since there are some builds in the bootstrap that need them.
svn path=/nixpkgs/branches/stdenv-updates/; revision=13788
bootstrap wasn't recycled in the final allPackages, causing an
additional glibc to be built for packages that explicitly depend on
glibc.
svn path=/nixpkgs/trunk/; revision=12200
* Some fetchurl-related refactoring. The `realCurl' attribute is
gone, `curl' is the real thing. To prevent an infinite recursion in
`fetchurl' (because it depends on curl and building curl needs
fetchurl), curl and its dependencies (openssl, zlib, perl) use
`fetchurlBoot', which is the fetchurl used by the previous bootstrap
phase (e.g. the statically linked version of curl for
stdenv-linux). So as a result you can use https:// urls almost
everywhere.
There's also some hackery to prevent a different curl from being
built in every stdenv-linux bootstrap phase (namely the
stdenv.fetchurl attribute which allows fetchurl to be overriden
everywhere).
svn path=/nixpkgs/trunk/; revision=11905
* GCC 4.2: restore the ability to build a statically linked compiler,
needed for the Nixpkgs bootstrap.
* GCC 4.2: use symlinks instead of hard links in $out/bin
(NIXPKGS-62).
svn path=/nixpkgs/branches/stdenv-updates/; revision=9771
* Removed substitute, it's part of the generic builder now.
* stdenv-initial (Linux): use the real generic builder script. This
does require that sed is in the path of the builder of the initial
stdenv.
svn path=/nixpkgs/trunk/; revision=7498
* To prevent this kind of thing, check that all tools are statically
linked.
* Use findutils 4.2.27, 4.2.28 doesn't build with dietlibc.
svn path=/nixpkgs/trunk/; revision=6881
Glibc. This is useful when building GCC.
* gcc-wrapper: the dynamic linker has a different name on x86_64 and
powerpc.
* gcc-wrapper: "glibc" -> "libc", because someday we might support
different C libraries.
* gcc: don't do a multilib build (e.g., 32-bit support on x86_64),
don't need it.
* gcc: merge in support for static builds.
* gcc: various simplifications in the compiler/linker flags, hope they
work.
svn path=/nixpkgs/trunk/; revision=6823
impure x86_64 environment, make sure that the 32-bit GCC / Glibc
libraries are installed, such as /usr/lib/crti.o.)
svn path=/nixpkgs/trunk/; revision=6818
x86_64. Glibc doens't build yet, though (it needs libgcc_eh, which
is strangely missing from the static GCC build).
svn path=/nixpkgs/trunk/; revision=6815
* The statically linked bootstrap tools are now automatically
reproducable, just do:
$ nix-build ./make-bootstrap-tools.nix
The resulting binaries in result/in-nixpkgs go to
stdenv/linux/bootstrap/<platform>/, and the tarballs in
result/on-server go to
https://svn.cs.uu.nl:12443/repos/trace/tarballs/trunk/stdenv-linux/<platform>/<revision>/.
These are checked out on nix.cs.uu.nl under http://.../dist/tarballs.
* The statically linked libraries all use dietlibc now (except
patchelf and glibc), so they are much smaller. This is especially
nice for the tools in the Nixpkgs tree, since it makes Nixpkgs
tarballs smaller.
* Use Binutils 2.17 and GCC 4.1.1 for the bootstrap.
* The stdenv is now based on Glibc 2.5. I hope it works ;-)
svn path=/nixpkgs/trunk/; revision=6803
