The cc and bintools wrapper contained ad hoc bootstrapping logic for
expand-response-params (which was callPackage-ed in a let binding). This
lead to the strange situation that the bootstrapping logic related to
expand-response-params is split between the wrapper derivations (where
it is duplicated) and the actual stdenv bootstrapping.
To clean this up, the wrappers simply should take expand-response-params
as an ordinary input: They need an adjacent expand-response-params (i.e.
one that runs on their host platform), but don't care about the how.
Providing this is only problematic during stdenv bootstrapping where we
have to pull it from the previous stage at times.
We don't need to artificially make sure that we can execute the wrapper
scripts on the build platform by using stdenv's shell (which comes from
buildPackages) since our cross infrastructure will get us the wrapper
from buildPackages. The upside of this change is that cross-compiled
wrappers (e.g. pkgsCross.aarch64-multiplatform.gcc) will actually work
when executed!
For bootstrapping this is also not a problem, since we have a long
build->build platform chain so runtimeShell is just as good as
stdenvNoCC.shell. We do fall back to old ways, though, by explicitly
using the bootstrap-tools shell in stage2, so the adjacent bash is only
used from stage4 onwards. This is unnecessary in principle (I'll try
removing this hack in the future), but ensures this change causes zero
rebuilds.
In delicate code like this, it seems unwise to pass something of as
something it isn't for convenience's (?) sake. It causes a slight
possibility for confusion with `buildPackages.stdenv`. However, it
should be possible to eliminate the need for this in a separate change.
this makes it a lot easier to create a modified stdenv with a
different set of defaultHardeningFlags and as a bonus allows us
to inject the correct defaultHardeningFlags into toolchain wrapper
scripts, reducing repetition.
while most hardening flags are arguably more of a compiler thing,
it works better to put them in bintools-wrapper because cc-wrapper
can easily refer to bintools but not vice-versa.
mkDerivation can still easily refer to either when it is constructed.
this also switches fortran-hook.sh to use the same defaults for
NIX_HARDENING_ENABLE as for C. previously NIX_HARDENING_ENABLE
defaults were apparently used to avoid passing problematic flags
to a fortran compiler, but this falls apart as soon as mkDerivation
sets its own NIX_HARDENING_ENABLE - cc.hardeningUnsupportedFlags
is a more appropriate mechanism for this as it actively filters
out flags from being used by the wrapper, so switch to using that
instead.
this is still an imperfect mechanism because it doesn't handle a
compiler which has both langFortran *and* langC very well - applying
the superset of the two's hardeningUnsupportedFlags to either
compiler's invocation. however this is nothing new - cc-wrapper
already poorly handles a langFortran+langC compiler, applying two
setup hooks that have contradictory options.
Fixed conflict in pkgs/applications/graphics/krita/
krita: 5.1.5 -> 5.2.0
7a40fdc288
, and
treewide: use kde mirror everywhere, don't use pname in download urls
aa15f5066d
-B must be set to the root directory of avrlibc, otherwise gcc cannot
locate crt objects for some attiny devices. -L trains as set by
bintools-wrapper are not necessary with -B set correctly because gcc
takes care of that, and likewise we can drop the -B train from
cc-wrapper because the one spec is enough.
When a response file is in use, "$*" contains the response file and not
the parameters; both the linker and compiler wrappers are updated to use
the response-expanded params.
The compiler driver likes to pass parameters to the linker via a
response file, including -shared.
LLD rejects the combination of (-shared -pie), whereas other linkers
silently ignore the contradiction:
```
ld.lld: error: -shared and -pie may not be used together
```
This breaks certain configurations using LLD as a linker.
Changing `add-hardening.sh` results in a full rebuild. To avoid the
rebuild, here is a quick test case which shows the new hardening script
allows the link to succeed:
```
{ pkgs ? import <nixpkgs> {} }:
let
# gcc silently accepts -shared -pie together, lld does not.
linker = pkgs.wrapBintoolsWith { bintools = pkgs.llvmPackages.lld; };
patchWrapper = prev: prev.overrideAttrs (final: prev: let
prevScript = builtins.match (".*(/nix/store/[a-z0-9]+-add-hardening.sh).*") prev.postFixup;
in {
postFixup = (builtins.replaceStrings prevScript ["${./new-add-hardening.sh}"] prev.postFixup);
});
in
pkgs.stdenv.mkDerivation {
name = "nixpkgs-hardening-bug";
src = pkgs.writeText "src.c" "int main(int argc, char* argv[]) { return 0; }";
NIX_HARDENING_ENABLE = "pie";
unpackPhase = ":";
buildPhase = ''
$CC -c -o src.o $src
bash -x ${patchWrapper linker}/bin/ld.lld -o $out @${pkgs.writeText "responsefile" "-shared"} src.o
'';
}
```
Fixes: #178162
Signed-off-by: Peter Waller <p@pwaller.net>
LLD supports Windows-style linker arguments, but these previously
triggered purity check false positives, because it saw that they
started with a '/' and assumed they were paths.
This tweaks the path detection to allow through certain values that
could be paths, but are much more likely to be LINK.EXE-style flags.
The risk of false negatives here is low — the only things we'd now
fail to catch would be attempts to link with libraries in the root
directory, which doesn't happen in practice.
We also teach the wrapper how to apply its purity checks to library
paths specified with the /LIBPATH: option.
Tested that paths we expect to be rejected (like /lib/libfoo.so) still
are.
NixOS/nixpkgs#146275 has more discussion on this; the abridged version
is that `lld` defaults to using `--build-id=fast` while GNU `ld` defaults
to `--build-id=sha1`. These differ in length and so
`separate-debug-info.sh`, as of this writing, errors on `lld`'s shorter
`--build-id=fast`-generated hashes.
`lld` offers the following `build-id` styles:
- UUID (random; fast but bad for reproducibility)
- fast (xxhash; fast but shorter hashes)
- user provided hexstring
- SHA1
- MD5
GNU `ld` supports the latter three options, `mold` supports all of these
plus SHA256.
UUID is out because it's not reproducible, fast isn't supported by GNU
`ld`
Using a nix provided (sourced from the output base hash) hash as the
`build-id` seems tempting but would require a little extra work
(we have to include some characteristic of the binary being hashed
so that binaries within a derivation still have unique hashes; it
seems easy to get this wrong; i.e. a path based approach would make
two otherwise identical binaries that reside at different paths have
different `build-id` hashes)
That leaves SHA1 and MD5 and GNU `ld` already defaults to the former.
This commit adds `$NIX_BUILD_ID_STYLE` as an escape hatch, in case any
packages have strong opinions about which hash to use.
----
Note that if/when NixOS/nixpkgs#146275 goes through, this change can be
reverted if linker speed is a priority.
The motivation behind this is to alleviate the problem
described in https://github.com/NixOS/nixpkgs/issues/41340.
I'm not sure if this completely fixes the problem, but it
eliminates one more area where we can exceed command line
length limits.
This is essentially the same change as in #112449,
except for `ld-wrapper.sh` instead of `cc-wrapper.sh`.
However, that change alone was not enough; on macOS the
`ld` provided by `darwin.cctools` fails if you use process
substitution to generate the response file, so I put up a
PR to fix that:
https://github.com/tpoechtrager/cctools-port/pull/131
… and I included a patch referencing that fix so that the
new `ld-wrapper` still works on macOS.
not all linkers have a ld binary in bin
also note the '${ld:-}' which allows users to set the ld path with a env
var
> '${foo:-val}' $foo, or val if unset (or null)
For reasons explained in the commit contents, in order to build the
native gnat package for x86_64-darwin, the native gnatboot package for
x86_64-darwin must have access to both the Clang integrated assembler
and the cctools GNU assembler for that platform. This commit creates a
package with both of those assemblers that x86_64-darwin gnatboot can
then be wrapped with.
Co-authored-by: Robin Gloster <mail@glob.in>
stdenv: print message if structuredAttrs is enabled
stdenv: add _append
reduces the chance of a user doing it wrong
fix nix develop issue
output hooks don't work yet in nix develop though
making $outputs be the same on non-structuredAttrs and structuredAttrs
is too much trouble.
lets instead make a function that gets the output names
reading environment file '/nix/store/2x7m69a2sm2kh0r6v0q5s9z1dh41m4xf-xz-5.2.5-env-bin'
nix: src/nix/develop.cc:299: std::string Common::makeRcScript(nix::ref<nix::Store>, const BuildEnvironment&, const Path&): Assertion `outputs != buildEnvironment.vars.end()' failed.
use a function to get all output names instead of using $outputs
copy env functionality from https://github.com/NixOS/nixpkgs/pull/76732/commits
`exit -1` is equivalent to `exit 255`, since Bash does modulo 256 on the
number.
As per ShellCheck:
> SC2242 (error): Can only exit with status 0-255. Other data should be
> written to stdout/stderr.
POSIX sh (and `bash`) impose a restriction on environment variable name
format and disallow hypheps in the names. Normally it's not a problem
as nothing usually tries to refer nyphenated names.
One exception is `nix develop` (https://github.com/NixOS/nix/issues/6848):
$ nix develop -f. gcc -L
gcc-wrapper> ...-get-env.sh: line 70: expand-response-params: bad substitution
Note that bash usually uses explicitly created `expandResponseParams`
variant of the same variable.
To work the problem around let's avoid environment variable export and move
it to `passthru` for `cc` (used ina few places) and remove it completely for
`binutils` (does not seem to be used at all).
Without this change cross-built gcc fails to detect stack protector style:
$ nix log -f pkgs/stdenv/linux/make-bootstrap-tools-cross.nix powerpc64le.bootGCC | fgrep __stack_chk_fail
checking __stack_chk_fail in target C library... no
checking __stack_chk_fail in target C library... no
It happens because gcc treats search paths differently:
https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=gcc/configure.ac;h=446747311a6aec3c810ad6aa4190f7bd383b94f7;hb=HEAD#l2458
if test x$host != x$target || test "x$TARGET_SYSTEM_ROOT" != x ||
test x$build != x$host || test "x$with_build_sysroot" != x; then
...
if test "x$with_build_sysroot" != "x"; then
target_header_dir="${with_build_sysroot}${native_system_header_dir}"
elif test "x$with_sysroot" = x; then
target_header_dir="${test_exec_prefix}/${target_noncanonical}/sys-include"
elif test "x$with_sysroot" = xyes; then
target_header_dir="${test_exec_prefix}/${target_noncanonical}/sys-root${native_system_header_dir}"
else
target_header_dir="${with_sysroot}${native_system_header_dir}"
fi
else
target_header_dir=${native_system_header_dir}
fi
By passing --with-build-sysroot=/ we trick cross-case to use
`target_header_dir="${with_sysroot}${native_system_header_dir}"`
which makes it equivalent to non-cross
`target_header_dir="${with_build_sysroot}${native_system_header_dir}"`
Tested the following setups:
- cross-compiler without libc headers (powerpc64le-static)
- cross-compiler with libc headers (powerpc64le-debug)
- cross-build compiler with libc headers (powerpc64le bootstrapTools)
Before the change only 2 of 3 compilers detected libc headers.
After the change all 3 compilers detected libc headers.
For darwin we silently ignore '-syslibroot //' argument as it does not
introduce impurities.
While at it dropped mingw special case for no-libc build. Before the change
we passed both '--without-headers --with-native-system-headers-dir' for
no-libc gcc-static builds. This tricked darwin builds to find sys/sdt.h
and fail inhibid_libc builds. Now all targets avoid passing native headers
for gcc-static builds.
While at it fixed correct headers passing to
--with-native-system-headers-dir= in host != target case: we were passing
host's headers where intention was to pass target's headers.
Noticed the mismatch as a build failure on pkgsCross.powernv.stdenv.cc
on darwin where `sys/sdt.h` is present in host's headers (libSystem)
but not target's headers (`glibc`).
Co-authored-by: Adam Joseph <54836058+amjoseph-nixpkgs@users.noreply.github.com>
this shouldn't change any binary available in the default build environment
because bintools-unwrapped is already in path ( idk where it comes from but i know because objcopy is in path but not in the wrapper )
this just makes all the binaries available under 'bintools' instead of
having to use 'bintools-unwrapped'
reduces confusion because now 'objcopy' and others will be in 'bintools'
