cafkafk/nixpkgs
cafkafk/nixpkgs
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2

Merge pull request #57611 from Ericson2314/stage-braid-not-chain

Browse source 
top-level: Create `pkgs{Build,Host,Target}{Build,Host,Target}`
This commit is contained in:
John Ericson 2019-03-25 21:56:59 -04:00 committed by GitHub
commit aa0cf64422
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
17 changed files with 432 additions and 198 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

374
doc/cross-compilation.xml
View file

@ -12,11 +12,12 @@
computing power and memory to compile their own programs. One might think
that cross-compilation is a fairly niche concern. However, there are
significant advantages to rigorously distinguishing between build-time and
run-time environments! This applies even when one is developing and
deploying on the same machine. Nixpkgs is increasingly adopting the opinion
that packages should be written with cross-compilation in mind, and nixpkgs
should evaluate in a similar way (by minimizing cross-compilation-specific
special cases) whether or not one is cross-compiling.
run-time environments! Significant, because the benefits apply even when one
is developing and deploying on the same machine. Nixpkgs is increasingly
adopting the opinion that packages should be written with cross-compilation
in mind, and nixpkgs should evaluate in a similar way (by minimizing
cross-compilation-specific special cases) whether or not one is
cross-compiling.
</para>
<para>
@ -30,7 +31,7 @@
<section xml:id="sec-cross-packaging">
<title>Packaging in a cross-friendly manner</title>
<section xml:id="sec-cross-platform-parameters">
<section xml:id="ssec-cross-platform-parameters">
<title>Platform parameters</title>
<para>
@ -218,8 +219,20 @@
</variablelist>
</section>
<section xml:id="sec-cross-specifying-dependencies">
<title>Specifying Dependencies</title>
<section xml:id="ssec-cross-dependency-categorization">
<title>Theory of dependency categorization</title>
<note>
<para>
This is a rather philosophical description that isn't very
Nixpkgs-specific. For an overview of all the relevant attributes given to
<varname>mkDerivation</varname>, see
<xref
linkend="ssec-stdenv-dependencies"/>. For a description of how
everything is implemented, see
<xref linkend="ssec-cross-dependency-implementation" />.
</para>
</note>
<para>
In this section we explore the relationship between both runtime and
@ -227,84 +240,98 @@
</para>
<para>
A runtime dependency between 2 packages implies that between them both the
host and target platforms match. This is directly implied by the meaning of
"host platform" and "runtime dependency": The package dependency exists
while both packages are running on a single host platform.
A run time dependency between two packages requires that their host
platforms match. This is directly implied by the meaning of "host platform"
and "runtime dependency": The package dependency exists while both packages
are running on a single host platform.
</para>
<para>
A build time dependency, however, implies a shift in platforms between the
depending package and the depended-on package. The meaning of a build time
dependency is that to build the depending package we need to be able to run
the depended-on's package. The depending package's build platform is
therefore equal to the depended-on package's host platform. Analogously,
the depending package's host platform is equal to the depended-on package's
target platform.
A build time dependency, however, has a shift in platforms between the
depending package and the depended-on package. "build time dependency"
means that to build the depending package we need to be able to run the
depended-on's package. The depending package's build platform is therefore
equal to the depended-on package's host platform.
</para>
<para>
In this manner, given the 3 platforms for one package, we can determine the
three platforms for all its transitive dependencies. This is the most
important guiding principle behind cross-compilation with Nixpkgs, and will
be called the <wordasword>sliding window principle</wordasword>.
If both the dependency and depending packages aren't compilers or other
machine-code-producing tools, we're done. And indeed
<varname>buildInputs</varname> and <varname>nativeBuildInputs</varname>
have covered these simpler build-time and run-time (respectively) changes
for many years. But if the dependency does produce machine code, we might
need to worry about its target platform too. In principle, that target
platform might be any of the depending package's build, host, or target
platforms, but we prohibit dependencies from a "later" platform to an
earlier platform to limit confusion because we've never seen a legitimate
use for them.
</para>
<para>
Some examples will make this clearer. If a package is being built with a
<literal>(build, host, target)</literal> platform triple of <literal>(foo,
bar, bar)</literal>, then its build-time dependencies would have a triple
of <literal>(foo, foo, bar)</literal>, and <emphasis>those
packages'</emphasis> build-time dependencies would have a triple of
<literal>(foo, foo, foo)</literal>. In other words, it should take two
"rounds" of following build-time dependency edges before one reaches a
fixed point where, by the sliding window principle, the platform triple no
longer changes. Indeed, this happens with cross-compilation, where only
rounds of native dependencies starting with the second necessarily coincide
with native packages.
Finally, if the depending package is a compiler or other
machine-code-producing tool, it might need dependencies that run at "emit
time". This is for compilers that (regrettably) insist on being built
together with their source langauges' standard libraries. Assuming build !=
host != target, a run-time dependency of the standard library cannot be run
at the compiler's build time or run time, but only at the run time of code
emitted by the compiler.
</para>
<note>
<para>
The depending package's target platform is unconstrained by the sliding
window principle, which makes sense in that one can in principle build
cross compilers targeting arbitrary platforms.
</para>
</note>
<para>
How does this work in practice? Nixpkgs is now structured so that
build-time dependencies are taken from <varname>buildPackages</varname>,
whereas run-time dependencies are taken from the top level attribute set.
For example, <varname>buildPackages.gcc</varname> should be used at
build-time, while <varname>gcc</varname> should be used at run-time. Now,
for most of Nixpkgs's history, there was no
<varname>buildPackages</varname>, and most packages have not been
refactored to use it explicitly. Instead, one can use the six
(<emphasis>gasp</emphasis>) attributes used for specifying dependencies as
documented in <xref linkend="ssec-stdenv-dependencies"/>. We "splice"
together the run-time and build-time package sets with
<varname>callPackage</varname>, and then <varname>mkDerivation</varname>
for each of four attributes pulls the right derivation out. This splicing
can be skipped when not cross-compiling as the package sets are the same,
but is a bit slow for cross-compiling. Because of this, a
best-of-both-worlds solution is in the works with no splicing or explicit
access of <varname>buildPackages</varname> needed. For now, feel free to
use either method.
Putting this all together, that means we have dependencies in the form
"host → target", in at most the following six combinations:
<table>
<caption>Possible dependency types</caption>
<thead>
<tr>
<th>Dependency's host platform</th>
<th>Dependency's target platform</th>
</tr>
</thead>
<tbody>
<tr>
<td>build</td>
<td>build</td>
</tr>
<tr>
<td>build</td>
<td>host</td>
</tr>
<tr>
<td>build</td>
<td>target</td>
</tr>
<tr>
<td>host</td>
<td>host</td>
</tr>
<tr>
<td>host</td>
<td>target</td>
</tr>
<tr>
<td>target</td>
<td>target</td>
</tr>
</tbody>
</table>
</para>
<note>
<para>
There is also a "backlink" <varname>targetPackages</varname>, yielding a
package set whose <varname>buildPackages</varname> is the current package
set. This is a hack, though, to accommodate compilers with lousy build
systems. Please do not use this unless you are absolutely sure you are
packaging such a compiler and there is no other way.
</para>
</note>
<para>
Some examples will make this table clearer. Suppose there's some package
that is being built with a <literal>(build, host, target)</literal>
platform triple of <literal>(foo, bar, baz)</literal>. If it has a
build-time library dependency, that would be a "host → build" dependency
with a triple of <literal>(foo, foo, *)</literal> (the target platform is
irrelevant). If it needs a compiler to be built, that would be a "build →
host" dependency with a triple of <literal>(foo, foo, *)</literal> (the
target platform is irrelevant). That compiler, would be built with another
compiler, also "build → host" dependency, with a triple of <literal>(foo,
foo, foo)</literal>.
</para>
</section>
<section xml:id="sec-cross-cookbook">
<section xml:id="ssec-cross-cookbook">
<title>Cross packaging cookbook</title>
<para>
@ -450,21 +477,202 @@ nix-build &lt;nixpkgs&gt; --arg crossSystem '{ config = "&lt;arch&gt;-&lt;os&gt;
<section xml:id="sec-cross-infra">
<title>Cross-compilation infrastructure</title>
<para>
To be written.
</para>
<section xml:id="ssec-cross-dependency-implementation">
<title>Implementation of dependencies</title>
<note>
<para>
If one explores Nixpkgs, they will see derivations with names like
<literal>gccCross</literal>. Such <literal>*Cross</literal> derivations is
a holdover from before we properly distinguished between the host and
target platforms—the derivation with "Cross" in the name covered the
<literal>build = host != target</literal> case, while the other covered the
<literal>host = target</literal>, with build platform the same or not based
on whether one was using its <literal>.nativeDrv</literal> or
<literal>.crossDrv</literal>. This ugliness will disappear soon.
The categorizes of dependencies developed in
<xref
linkend="ssec-cross-dependency-categorization"/> are specified as
lists of derivations given to <varname>mkDerivation</varname>, as
documented in <xref linkend="ssec-stdenv-dependencies"/>. In short,
each list of dependencies for "host → target" of "foo → bar" is called
<varname>depsFooBar</varname>, with exceptions for backwards
compatibility that <varname>depsBuildHost</varname> is instead called
<varname>nativeBuildInputs</varname> and <varname>depsHostTarget</varname>
is instead called <varname>buildInputs</varname>. Nixpkgs is now structured
so that each <varname>depsFooBar</varname> is automatically taken from
<varname>pkgsFooBar</varname>. (These <varname>pkgsFooBar</varname>s are
quite new, so there is no special case for
<varname>nativeBuildInputs</varname> and <varname>buildInputs</varname>.)
For example, <varname>pkgsBuildHost.gcc</varname> should be used at
build-time, while <varname>pkgsHostTarget.gcc</varname> should be used at
run-time.
</para>
</note>
<para>
Now, for most of Nixpkgs's history, there were no
<varname>pkgsFooBar</varname> attributes, and most packages have not been
refactored to use it explicitly. Prior to those, there were just
<varname>buildPackages</varname>, <varname>pkgs</varname>, and
<varname>targetPackages</varname>. Those are now redefined as aliases to
<varname>pkgsBuildHost</varname>, <varname>pkgsHostTarget</varname>, and
<varname>pkgsTargetTarget</varname>. It is acceptable, even
recommended, to use them for libraries to show that the host platform is
irrelevant.
</para>
<para>
But before that, there was just <varname>pkgs</varname>, even though both
<varname>buildInputs</varname> and <varname>nativeBuildInputs</varname>
existed. [Cross barely worked, and those were implemented with some hacks
on <varname>mkDerivation</varname> to override dependencies.] What this
means is the vast majority of packages do not use any explicit package set
to populate their dependencies, just using whatever
<varname>callPackage</varname> gives them even if they do correctly sort
their dependencies into the multiple lists described above. And indeed,
asking that users both sort their dependencies, <emphasis>and</emphasis>
take them from the right attribute set, is both too onerous and redundant,
so the recommended approach (for now) is to continue just categorizing by
list and not using an explicit package set.
</para>
<para>
To make this work, we "splice" together the six
<varname>pkgsFooBar</varname> package sets and have
<varname>callPackage</varname> actually take its arguments from that. This
is currently implemented in <filename>pkgs/top-level/splice.nix</filename>.
<varname>mkDerivation</varname> then, for each dependency attribute, pulls
the right derivation out from the splice. This splicing can be skipped when
not cross-compiling as the package sets are the same, but still is a bit
slow for cross-compiling. We'd like to do something better, but haven't
come up with anything yet.
</para>
</section>
<section xml:id="ssec-bootstrapping">
<title>Bootstrapping</title>
<para>
Each of the package sets described above come from a single bootstrapping
stage. While <filename>pkgs/top-level/default.nix</filename>, coordinates
the composition of stages at a high level,
<filename>pkgs/top-level/stage.nix</filename> "ties the knot" (creates the
fixed point) of each stage. The package sets are defined per-stage however,
so they can be thought of as edges between stages (the nodes) in a graph.
Compositions like <literal>pkgsBuildTarget.targetPackages</literal> can be
thought of as paths to this graph.
</para>
<para>
While there are many package sets, and thus many edges, the stages can also
be arranged in a linear chain. In other words, many of the edges are
redundant as far as connectivity is concerned. This hinges on the type of
bootstrapping we do. Currently for cross it is:
<orderedlist>
<listitem>
<para>
<literal>(native, native, native)</literal>
</para>
</listitem>
<listitem>
<para>
<literal>(native, native, foreign)</literal>
</para>
</listitem>
<listitem>
<para>
<literal>(native, foreign, foreign)</literal>
</para>
</listitem>
</orderedlist>
In each stage, <varname>pkgsBuildHost</varname> refers the the previous
stage, <varname>pkgsBuildBuild</varname> refers to the one before that, and
<varname>pkgsHostTarget</varname> refers to the current one, and
<varname>pkgsTargetTarget</varname> refers to the next one. When there is
no previous or next stage, they instead refer to the current stage. Note
how all the invariants regarding the mapping between dependency and depending
packages' build host and target platforms are preserved.
<varname>pkgsBuildTarget</varname> and <varname>pkgsHostHost</varname> are
more complex in that the stage fitting the requirements isn't always a
fixed chain of "prevs" and "nexts" away (modulo the "saturating"
self-references at the ends). We just special case each instead. All the primary
edges are implemented is in <filename>pkgs/stdenv/booter.nix</filename>,
and secondarily aliases in <filename>pkgs/top-level/stage.nix</filename>.
</para>
<note>
<para>
Note the native stages are bootstrapped in legacy ways that predate the
current cross implementation. This is why the the bootstrapping stages
leading up to the final stages are ignored inthe previous paragraph.
</para>
</note>
<para>
If one looks at the 3 platform triples, one can see that they overlap such
that one could put them together into a chain like:
<programlisting>
(native, native, native, foreign, foreign)
</programlisting>
If one imagines the saturating self references at the end being replaced
with infinite stages, and then overlays those platform triples, one ends up
with the infinite tuple:
<programlisting>
(native..., native, native, native, foreign, foreign, foreign...)
</programlisting>
On can then imagine any sequence of platforms such that there are bootstrap
stages with their 3 platforms determined by "sliding a window" that is the
3 tuple through the sequence. This was the original model for
bootstrapping. Without a target platform (assume a better world where all
compilers are multi-target and all standard libraries are built in their
own derivation), this is sufficient. Conversely if one wishes to cross
compile "faster", with a "Canadian Cross" bootstraping stage where
<literal>build != host != target</literal>, more bootstrapping stages are
needed since no sliding window providess the pesky
<varname>pkgsBuildTarget</varname> package set since it skips the Canadian
cross stage's "host".
</para>
<note>
<para>
It is much better to refer to <varname>buildPackages</varname> than
<varname>targetPackages</varname>, or more broadly package sets that do
not mention "target". There are three reasons for this.
</para>
<para>
First, it is because bootstrapping stages do not have a unique
<varname>targetPackages</varname>. For example a <literal>(x86-linux,
x86-linux, arm-linux)</literal> and <literal>(x86-linux, x86-linux,
x86-windows)</literal> package set both have a <literal>(x86-linux,
x86-linux, x86-linux)</literal> package set. Because there is no canonical
<varname>targetPackages</varname> for such a native (<literal>build ==
host == target</literal>) package set, we set their
<varname>targetPackages</varname>
</para>
<para>
Second, it is because this is a frequent source of hard-to-follow
"infinite recursions" / cycles. When only package sets that don't mention
target are used, the package set forms a directed acyclic graph. This
means that all cycles that exist are confined to one stage. This means
they are a lot smaller, and easier to follow in the code or a backtrace. It
also means they are present in native and cross builds alike, and so more
likely to be caught by CI and other users.
</para>
<para>
Thirdly, it is because everything target-mentioning only exists to
accommodate compilers with lousy build systems that insist on the compiler
itself and standard library being built together. Of course that is bad
because bigger derivations means longer rebuilds. It is also problematic because
it tends to make the standard libraries less like other libraries than
they could be, complicating code and build systems alike. Because of the
other problems, and because of these innate disadvantages, compilers ought
to be packaged another way where possible.
</para>
</note>
<note>
<para>
If one explores Nixpkgs, they will see derivations with names like
<literal>gccCross</literal>. Such <literal>*Cross</literal> derivations is
a holdover from before we properly distinguished between the host and
target platforms—the derivation with "Cross" in the name covered the
<literal>build = host != target</literal> case, while the other covered
the <literal>host = target</literal>, with build platform the same or not
based on whether one was using its <literal>.nativeDrv</literal> or
<literal>.crossDrv</literal>. This ugliness will disappear soon.
</para>
</note>
</section>
</section>
</chapter>

3
doc/reviewing-contributions.xml
View file

@ -189,7 +189,8 @@ $ git rebase --onto nixos-unstable BASEBRANCH FETCH_HEAD <co
</listitem>
<listitem>
<para>
The <link xlink:href="https://github.com/Mic92/nix-review">nix-review</link>
The
<link xlink:href="https://github.com/Mic92/nix-review">nix-review</link>
tool can be used to review a pull request content in a single command.
<varname>PRNUMBER</varname> should be replaced by the number at the end
of the pull request title. You can also provide the full github pull

41
doc/stdenv.xml
View file

@ -222,9 +222,10 @@ genericBuild
</footnote>
But even if one is not cross compiling, the platforms imply whether or not
the dependency is needed at run-time or build-time, a concept that makes
perfect sense outside of cross compilation. For now, the run-time/build-time
distinction is just a hint for mental clarity, but in the future it perhaps
could be enforced.
perfect sense outside of cross compilation. By default, the
run-time/build-time distinction is just a hint for mental clarity, but with
<varname>strictDeps</varname> set it is mostly enforced even in the native
case.
</para>
<para>
@ -348,7 +349,10 @@ let f(h, h + 1, i) = i + h
<para>
Overall, the unifying theme here is that propagation shouldn't be
introducing transitive dependencies involving platforms the depending
package is unaware of. The offset bounds checking and definition of
package is unaware of. [One can imagine the dependending package asking for
dependencies with the platforms it knows about; other platforms it doesn't
know how to ask for. The platform description in that scenario is a kind of
unforagable capability.] The offset bounds checking and definition of
<function>mapOffset</function> together ensure that this is the case.
Discovering a new offset is discovering a new platform, and since those
platforms weren't in the derivation "spec" of the needing package, they
@ -2633,21 +2637,20 @@ addEnvHooks "$hostOffset" myBashFunction
happens. It prevents nix from cleaning up the build environment
immediately and allows the user to attach to a build environment using
the <command>cntr</command> command. Upon build error it will print
instructions on how to use <command>cntr</command>, which can be used
to enter the environment for debugging. Installing cntr and
running the command will provide shell access to the build sandbox of
failed build. At <filename>/var/lib/cntr</filename> the sandboxed
filesystem is mounted. All commands and files of the system are still
accessible within the shell. To execute commands from the sandbox use
the cntr exec subcommand. Note that <command>cntr</command> also needs
to be executed on the machine that is doing the build, which might not
be the case when remote builders are enabled. <command>cntr</command> is
only supported on Linux-based platforms. To use it first add
<literal>cntr</literal> to your
<literal>environment.systemPackages</literal> on NixOS or alternatively
to the root user on non-NixOS systems. Then in the package that is
supposed to be inspected, add <literal>breakpointHook</literal> to
<literal>nativeBuildInputs</literal>.
instructions on how to use <command>cntr</command>, which can be used to
enter the environment for debugging. Installing cntr and running the
command will provide shell access to the build sandbox of failed build.
At <filename>/var/lib/cntr</filename> the sandboxed filesystem is
mounted. All commands and files of the system are still accessible
within the shell. To execute commands from the sandbox use the cntr exec
subcommand. Note that <command>cntr</command> also needs to be executed
on the machine that is doing the build, which might not be the case when
remote builders are enabled. <command>cntr</command> is only supported
on Linux-based platforms. To use it first add <literal>cntr</literal> to
your <literal>environment.systemPackages</literal> on NixOS or
alternatively to the root user on non-NixOS systems. Then in the package
that is supposed to be inspected, add <literal>breakpointHook</literal>
to <literal>nativeBuildInputs</literal>.
<programlisting>
nativeBuildInputs = [ breakpointHook ];
</programlisting>

19
doc/submitting-changes.xml
View file

@ -354,23 +354,22 @@ Additional information.
<title>Tested compilation of all pkgs that depend on this change using <command>nix-review</command></title>
<para>
If you are updating a package's version, you can use nix-review to make sure all
packages that depend on the updated package still compile correctly.
The <command>nix-review</command> utility can look for and build all dependencies
either based on uncommited changes with the <literal>wip</literal> option or
specifying a github pull request number.
If you are updating a package's version, you can use nix-review to make
sure all packages that depend on the updated package still compile
correctly. The <command>nix-review</command> utility can look for and build
all dependencies either based on uncommited changes with the
<literal>wip</literal> option or specifying a github pull request number.
</para>
<para>
review changes from pull request number 12345:
<screen>nix-shell -p nix-review --run "nix-review pr 12345"</screen>
review changes from pull request number 12345:
<screen>nix-shell -p nix-review --run "nix-review pr 12345"</screen>
</para>
<para>
review uncommitted changes:
<screen>nix-shell -p nix-review --run "nix-review wip"</screen>
review uncommitted changes:
<screen>nix-shell -p nix-review --run "nix-review wip"</screen>
</para>
</section>
<section xml:id="submitting-changes-tested-execution">

10
pkgs/development/compilers/ghc/8.2.2.nix
View file

@ -1,4 +1,4 @@
{ stdenv, targetPackages
{ stdenv, pkgsBuildTarget, targetPackages
# build-tools
, bootPkgs
@ -70,11 +70,9 @@ let
++ stdenv.lib.optional (!enableIntegerSimple) gmp
++ stdenv.lib.optional (platform.libc != "glibc") libiconv;
toolsForTarget =
if hostPlatform == buildPlatform then
[ targetPackages.stdenv.cc ] ++ stdenv.lib.optional useLLVM llvmPackages.llvm
else assert targetPlatform == hostPlatform; # build != host == target
[ stdenv.cc ] ++ stdenv.lib.optional useLLVM buildLlvmPackages.llvm;
toolsForTarget = [
pkgsBuildTarget.targetPackages.stdenv.cc
] ++ stdenv.lib.optional useLLVM buildLlvmPackages.llvm;
targetCC = builtins.head toolsForTarget;

10
pkgs/development/compilers/ghc/8.4.4.nix
View file

@ -1,4 +1,4 @@
{ stdenv, targetPackages
{ stdenv, pkgsBuildTarget, targetPackages
# build-tools
, bootPkgs
@ -72,11 +72,9 @@ let
++ stdenv.lib.optional (!enableIntegerSimple) gmp
++ stdenv.lib.optional (platform.libc != "glibc" && !targetPlatform.isWindows) libiconv;
toolsForTarget =
if hostPlatform == buildPlatform then
[ targetPackages.stdenv.cc ] ++ stdenv.lib.optional useLLVM llvmPackages.llvm
else assert targetPlatform == hostPlatform; # build != host == target
[ stdenv.cc ] ++ stdenv.lib.optional useLLVM buildLlvmPackages.llvm;
toolsForTarget = [
pkgsBuildTarget.targetPackages.stdenv.cc
] ++ stdenv.lib.optional useLLVM buildLlvmPackages.llvm;
targetCC = builtins.head toolsForTarget;

10
pkgs/development/compilers/ghc/8.6.4.nix
View file

@ -1,4 +1,4 @@
{ stdenv, targetPackages
{ stdenv, pkgsBuildTarget, targetPackages
# build-tools
, bootPkgs
@ -72,11 +72,9 @@ let
++ stdenv.lib.optional (!enableIntegerSimple) gmp
++ stdenv.lib.optional (platform.libc != "glibc" && !targetPlatform.isWindows) libiconv;
toolsForTarget =
if hostPlatform == buildPlatform then
[ targetPackages.stdenv.cc ] ++ stdenv.lib.optional useLLVM llvmPackages.llvm
else assert targetPlatform == hostPlatform; # build != host == target
[ stdenv.cc ] ++ stdenv.lib.optional useLLVM buildLlvmPackages.llvm;
toolsForTarget = [
pkgsBuildTarget.targetPackages.stdenv.cc
] ++ stdenv.lib.optional useLLVM buildLlvmPackages.llvm;
targetCC = builtins.head toolsForTarget;

10
pkgs/development/compilers/ghc/head.nix
View file

@ -1,4 +1,4 @@
{ stdenv, targetPackages
{ stdenv, pkgsBuildTarget, targetPackages
# build-tools
, bootPkgs
@ -69,11 +69,9 @@ let
++ stdenv.lib.optional (!enableIntegerSimple) gmp
++ stdenv.lib.optional (platform.libc != "glibc" && !targetPlatform.isWindows) libiconv;
toolsForTarget =
if hostPlatform == buildPlatform then
[ targetPackages.stdenv.cc ] ++ stdenv.lib.optional useLLVM llvmPackages.llvm
else assert targetPlatform == hostPlatform; # build != host == target
[ stdenv.cc ] ++ stdenv.lib.optional useLLVM buildLlvmPackages.llvm;
toolsForTarget = [
pkgsBuildTarget.targetPackages.stdenv.cc
] ++ stdenv.lib.optional useLLVM buildLlvmPackages.llvm;
targetCC = builtins.head toolsForTarget;

15
pkgs/development/compilers/go/1.11.nix
View file

@ -1,7 +1,8 @@
{ stdenv, fetchFromGitHub, tzdata, iana-etc, go_bootstrap, runCommand, writeScriptBin
, perl, which, pkgconfig, patch, procps, pcre, cacert, llvm, Security, Foundation
, mailcap, runtimeShell
, buildPackages, targetPackages }:
, buildPackages, pkgsTargetTarget
}:
let
@ -152,16 +153,12 @@ stdenv.mkDerivation rec {
# {CC,CXX}_FOR_TARGET must be only set for cross compilation case as go expect those
# to be different from CC/CXX
CC_FOR_TARGET = if (stdenv.hostPlatform != stdenv.targetPlatform) then
"${targetPackages.stdenv.cc}/bin/${targetPackages.stdenv.cc.targetPrefix}cc"
else if (stdenv.buildPlatform != stdenv.targetPlatform) then
"${stdenv.cc.targetPrefix}cc"
CC_FOR_TARGET = if (stdenv.buildPlatform != stdenv.targetPlatform) then
"${pkgsTargetTarget.stdenv.cc}/bin/${pkgsTargetTarget.stdenv.cc.targetPrefix}cc"
else
null;
CXX_FOR_TARGET = if (stdenv.hostPlatform != stdenv.targetPlatform) then
"${targetPackages.stdenv.cc}/bin/${targetPackages.stdenv.cc.targetPrefix}c++"
else if (stdenv.buildPlatform != stdenv.targetPlatform) then
"${stdenv.cc.targetPrefix}c++"
CXX_FOR_TARGET = if (stdenv.buildPlatform != stdenv.targetPlatform) then
"${pkgsTargetTarget.stdenv.cc}/bin/${pkgsTargetTarget.stdenv.cc.targetPrefix}c++"
else
null;

15
pkgs/development/compilers/go/1.12.nix
View file

@ -1,7 +1,8 @@
{ stdenv, fetchurl, tzdata, iana-etc, go_bootstrap, runCommand, writeScriptBin
, perl, which, pkgconfig, patch, procps, pcre, cacert, llvm, Security, Foundation
, mailcap, runtimeShell
, buildPackages, targetPackages }:
, buildPackages, pkgsTargetTarget
}:
let
@ -154,16 +155,12 @@ stdenv.mkDerivation rec {
# {CC,CXX}_FOR_TARGET must be only set for cross compilation case as go expect those
# to be different from CC/CXX
CC_FOR_TARGET = if (stdenv.hostPlatform != stdenv.targetPlatform) then
"${targetPackages.stdenv.cc}/bin/${targetPackages.stdenv.cc.targetPrefix}cc"
else if (stdenv.buildPlatform != stdenv.targetPlatform) then
"${stdenv.cc.targetPrefix}cc"
CC_FOR_TARGET = if (stdenv.buildPlatform != stdenv.targetPlatform) then
"${pkgsTargetTarget.stdenv.cc}/bin/${pkgsTargetTarget.stdenv.cc.targetPrefix}cc"
else
null;
CXX_FOR_TARGET = if (stdenv.hostPlatform != stdenv.targetPlatform) then
"${targetPackages.stdenv.cc}/bin/${targetPackages.stdenv.cc.targetPrefix}c++"
else if (stdenv.buildPlatform != stdenv.targetPlatform) then
"${stdenv.cc.targetPrefix}c++"
CXX_FOR_TARGET = if (stdenv.buildPlatform != stdenv.targetPlatform) then
"${pkgsTargetTarget.stdenv.cc}/bin/${pkgsTargetTarget.stdenv.cc.targetPrefix}c++"
else
null;

4
pkgs/development/interpreters/guile/1.8.nix
View file

@ -1,4 +1,4 @@
{ stdenv, buildPackages
{ stdenv, pkgsBuildBuild, buildPackages
, fetchurl, makeWrapper, gawk, pkgconfig
, libtool, readline, gmp
}:
@ -23,7 +23,7 @@ stdenv.mkDerivation rec {
depsBuildBuild = [ buildPackages.stdenv.cc ]
++ stdenv.lib.optional (stdenv.hostPlatform != stdenv.buildPlatform)
buildPackages.buildPackages.guile_1_8;
pkgsBuildBuild.guile_1_8;
nativeBuildInputs = [ makeWrapper gawk pkgconfig ];
buildInputs = [ readline libtool ];

4
pkgs/development/interpreters/guile/2.0.nix
View file

@ -1,4 +1,4 @@
{ stdenv, buildPackages
{ stdenv, pkgsBuildBuild, buildPackages
, fetchpatch, fetchurl, makeWrapper, gawk, pkgconfig
, libffi, libtool, readline, gmp, boehmgc, libunistring
, coverageAnalysis ? null
@ -22,7 +22,7 @@
depsBuildBuild = [ buildPackages.stdenv.cc ]
++ stdenv.lib.optional (stdenv.hostPlatform != stdenv.buildPlatform)
buildPackages.buildPackages.guile_2_0;
pkgsBuildBuild.guile_2_0;
nativeBuildInputs = [ makeWrapper gawk pkgconfig ];
buildInputs = [ readline libtool libunistring libffi ];

4
pkgs/development/interpreters/guile/default.nix
View file

@ -1,4 +1,4 @@
{ stdenv, buildPackages
{ stdenv, pkgsBuildBuild, buildPackages
, fetchurl, makeWrapper, gawk, pkgconfig
, libffi, libtool, readline, gmp, boehmgc, libunistring
, coverageAnalysis ? null
@ -23,7 +23,7 @@
depsBuildBuild = [ buildPackages.stdenv.cc ]
++ stdenv.lib.optional (stdenv.hostPlatform != stdenv.buildPlatform)
buildPackages.buildPackages.guile;
pkgsBuildBuild.guile;
nativeBuildInputs = [ makeWrapper gawk pkgconfig ];
buildInputs = [ readline libtool libunistring libffi ];

26
pkgs/stdenv/booter.nix
View file

@ -95,13 +95,25 @@ stageFuns: let
__hatPackages = nextStage;
};
};
in
if args.__raw or false
then args'
else allPackages ((builtins.removeAttrs args' ["selfBuild"]) // {
buildPackages = if args.selfBuild or true then null else prevStage;
targetPackages = if args.selfBuild or true then null else nextStage;
});
thisStage =
if args.__raw or false
then args'
else allPackages ((builtins.removeAttrs args' ["selfBuild"]) // {
adjacentPackages = if args.selfBuild or true then null else rec {
pkgsBuildBuild = prevStage.buildPackages;
pkgsBuildHost = prevStage;
pkgsBuildTarget =
if args.stdenv.targetPlatform == args.stdenv.hostPlatform
then pkgsBuildHost
else assert args.stdenv.hostPlatform == args.stdenv.buildPlatform; thisStage;
pkgsHostHost =
if args.stdenv.hostPlatform == args.stdenv.targetPlatform
then thisStage
else assert args.stdenv.buildPlatform == args.stdenv.hostPlatform; pkgsBuildHost;
pkgsTargetTarget = nextStage;
};
});
in thisStage;
# This is a hack for resolving cross-compiled compilers' run-time
# deps. (That is, compilers that are themselves cross-compiled, as

3
pkgs/top-level/all-packages.nix
View file

@ -25,9 +25,6 @@ let
in
{
# Allow callPackage to fill in the pkgs argument
inherit pkgs;
# A stdenv capable of building 32-bit binaries. On x86_64-linux,
# it uses GCC compiled with multilib support; on i686-linux, it's
# just the plain stdenv.

23
pkgs/top-level/splice.nix
View file

@ -96,19 +96,20 @@ let
} @ args:
if actuallySplice then spliceReal args else pkgsHostTarget;
splicedPackages = splicePackages rec {
pkgsBuildBuild = pkgs.buildPackages.buildPackages;
pkgsBuildHost = pkgs.buildPackages;
pkgsBuildTarget =
if pkgs.stdenv.targetPlatform == pkgs.stdenv.hostPlatform
then pkgsBuildHost
else assert pkgs.stdenv.hostPlatform == pkgs.stdenv.buildPlatform; pkgsHostTarget;
pkgsHostHost = {}; # unimplemented
pkgsHostTarget = pkgs;
pkgsTargetTarget = pkgs.targetPackages;
splicedPackages = splicePackages {
inherit (pkgs)
pkgsBuildBuild pkgsBuildHost pkgsBuildTarget
pkgsHostHost pkgsHostTarget
pkgsTargetTarget
;
} // {
# These should never be spliced under any circumstances
inherit (pkgs) pkgs buildPackages targetPackages;
inherit (pkgs)
pkgsBuildBuild pkgsBuildHost pkgsBuildTarget
pkgsHostHost pkgsHostTarget
pkgsTargetTarget
buildPackages pkgs targetPackages
;
inherit (pkgs.stdenv) buildPlatform targetPlatform hostPlatform;
};

59
pkgs/top-level/stage.nix
View file

@ -21,18 +21,23 @@
## Other parameters
##
, # The package set used at build-time. If null, `buildPackages` will
# be defined internally as the final produced package set itself. This allows
# us to avoid expensive splicing.
buildPackages
, # The package set used in the next stage. If null, `targetPackages` will be
# defined internally as the final produced package set itself, just like with
# `buildPackages` and for the same reasons.
, # Either null or an object in the form:
#
# THIS IS A HACK for compilers that don't think critically about cross-
# compilation. Please do *not* use unless you really know what you are doing.
targetPackages
# {
# pkgsBuildBuild = ...;
# pkgsBuildHost = ...;
# pkgsBuildTarget = ...;
# pkgsHostHost = ...;
# # pkgsHostTarget skipped on purpose.
# pkgsTargetTarget ...;
# }
#
# These are references to adjacent bootstrapping stages. The more familiar
# `buildPackages` and `targetPackages` are defined in terms of them. If null,
# they are instead defined internally as the current stage. This allows us to
# avoid expensive splicing. `pkgsHostTarget` is skipped because it is always
# defined as the current stage.
adjacentPackages
, # The standard environment to use for building packages.
stdenv
@ -70,11 +75,33 @@ let
inherit (self) runtimeShell;
};
stdenvBootstappingAndPlatforms = self: super: {
buildPackages = (if buildPackages == null then self else buildPackages)
// { recurseForDerivations = false; };
targetPackages = (if targetPackages == null then self else targetPackages)
stdenvBootstappingAndPlatforms = self: super: let
withFallback = thisPkgs:
(if adjacentPackages == null then self else thisPkgs)
// { recurseForDerivations = false; };
in {
# Here are package sets of from related stages. They are all in the form
# `pkgs{theirHost}{theirTarget}`. For example, `pkgsBuildHost` means their
# host platform is our build platform, and their target platform is our host
# platform. We only care about their host/target platforms, not their build
# platform, because the the former two alone affect the interface of the
# final package; the build platform is just an implementation detail that
# should not leak.
pkgsBuildBuild = withFallback adjacentPackages.pkgsBuildBuild;
pkgsBuildHost = withFallback adjacentPackages.pkgsBuildHost;
pkgsBuildTarget = withFallback adjacentPackages.pkgsBuildTarget;
pkgsHostHost = withFallback adjacentPackages.pkgsHostHost;
pkgsHostTarget = self // { recurseForDerivations = false; }; # always `self`
pkgsTargetTarget = withFallback adjacentPackages.pkgsTargetTarget;
# Older names for package sets. Use these when only the host platform of the
# package set matter (i.e. use `buildPackages` where any of `pkgsBuild*`
# would do, and `targetPackages` when any of `pkgsTarget*` would do (if we
# had more than just `pkgsTargetTarget`).)
buildPackages = self.pkgsBuildHost;
pkgs = self.pkgsHostTarget;
targetPackages = self.pkgsTargetTarget;
inherit stdenv;
};
@ -87,7 +114,7 @@ let
inherit (hostPlatform) system;
};
splice = self: super: import ./splice.nix lib self (buildPackages != null);
splice = self: super: import ./splice.nix lib self (adjacentPackages != null);
allPackages = self: super:
let res = import ./all-packages.nix