Without this change $target-cpp is used unwrapped and is missing
standard header search paths among other things).
Example failure:
$ nix build -f. -L pkgsStatic.netbsd.compat
...
> checking how to run the C preprocessor... x86_64-unknown-linux-musl-cpp
> configure: error: in `/build/cvs-export/tools/compat':
> configure: error: C preprocessor "x86_64-unknown-linux-musl-cpp" fails sanity check
> See `config.log' for more details
This is useful to teach `importCargoLock` how to download crates from a
registry other than crates.io. Specifically, we publish our own crates
to an internal registry and this feature lets us pull from it seamlessly.
If RUSTFLAGS is set in the environment, Cargo will ignore rustflags
settings in its TOML configuration. So setting RUSTFLAGS=-g (like
separateDebugInfo does) to generate debug info breaks
dynamically-linked Rust packages on musl. This breakage is visible
for any packages that call into C dynamic libraries. If the binary is
linked directly to a C dynamic library, it will fail to build, and if
it depends on a Rust library which links a C dynamic library, it will
segfault at runtime when it tries to call a function from the C
library. I noticed this because pkgsMusl.crosvm is broken for this
reason, since it sets separateDebugInfo = true.
It shouldn't be possible to end up with broken binaries just by using
RUSTFLAGS to do something innocuous like enable debug info, so I think
that, even though we liked the approach of modiyfing .cargo/config
better at the time, it's become clear that it's too brittle, and we
should bite the bullet and patch the compiler instead when targetting
musl. It does not appear to be necessary to modify the compiler at
all when cross-compiling /from/ dynamically-linked Musl to another
target, so I'm only checking whether the target system is
dynamically-linked Musl when deciding whether to make the modification
to the compiler.
This reverts commit c2eaaae50d
("cargoSetupHook: pass host config flags"), and implements the
compiler patching approach instead.
This commit adds basic support for tree-sitter in the emacs build,
such that (if the user opts into tree-sitter support), tree-sitter
will be enabled and binary library files for tree-sitter can be
included in the `lib` directory of packages passed to
`emacsWithPackages`. The libraries will be aggregated and included in
treesit-extra-load-path.
The previous pattern for this in the community was to add tree-sitter
libaries by patching emacs's `RUNPATH` with `patchelf` in a post-fixup
phase. However, this has the substantial drawback that two different
emacs installations with different lists of available tree-sitter
libraries must be entirely separate builds. By supplying the
tree-sitter libraries in the wrapping layer of `emacsWithpackages`, it
becomes possible to share a single, more-cacheable "core emacs".
This support defaults to "on" only in emacs 29 and up, since previous
versions do not support tree-sitter out of the box.
`fixupPhase` move `$out/man` to `$out/share/man`. So the information of their location in the dune-project file is outdated which breaks dependencies on packages ( `(package foo)`).
Sometimes it's more ergonomic to set up the build environment in
hooks, to add to the default behaviour rather than replacing it. It's
very surprising that the fetcher works fine with a custom unpackPhase,
but not with custom preUnpack or postUnpack.
Packages that use preUnpack or postUnpack and Cargo FODs seem to be
very rare. I searched Nixpkgs for files containing one of
"cargoHash", "cargoDeps", and "cargoSha256", and one of "preUnpack" or
"postUnpack", and only found two such packages:
python3.pkgs.tokenizers and rustdesk. Neither of their Cargo FOD
hashes are affected by this change. So if that's any indication,
we're unlikely to be breaking many out-of-tree hashes with these
changes either.
Currently, separate-debug-info adds a debug output, and the build fail when it is
not created. the output is only created when at least one elf file is
stripped.
As a result, adding separateDebugInfo = true on a lib will break the
static build (unless the lib also contains an executable). In order to
not have to remember to add an exception every time, let's just create
the debug output unconditionally.
(and gitiles)
This allows fetching a patch from servers that return them
base64-encoded, like this:
fetchpatch {
name = "gcc.patch";
url = "f37ae3b1a8^!?format=TEXT";
decode = "base64 -d";
sha256 = "11j1bqz2p8xrfzgfrylgdvmqs45489c4ckl7l0ra1dpfgbqy94a8";
}
Fixes linker errors while building build.rs where it tries to link libiconv but cannot find it.
Rust executable build for Darwin need libiconv, and indeed buildInputs already has this case handled.
So why is another change needed? Suppose we are cross compiling from Darwin (the build platform) to something else, and the package has a build.rs build script.
The build script is built for the build platform (Darwin) and is also a regular Rust executable, needing libiconv, but due to cross compilation (and strict deps) we need an extra nativeBuildInput.
writeShellApplication currently uses the unwrapped (passthru) attribute
which is simply defined as the ShellCheck Haskell package.
Unfortunately the unwrapped version contains everything and the kitchen
sink, while the bin output of the top-level shellcheck package contains
only the static shellcheck executable.
In other words, by using writeShellApplication, currently 3GB of
packages have to be unnecessarily fetched just to run the checkPhase.
$ nix path-info -Sh $(nix build --print-out-paths --no-link nixpkgs#shellcheck.unwrapped)
/nix/store/23x8702b9kqn0r8swah05ky7w5fnh6m2-ShellCheck-0.9.0 3.0G
$ nix path-info -Sh $(nix build --print-out-paths --no-link nixpkgs#shellcheck.bin)
/nix/store/594izb2jz3c57c7hgxfnb6irypnr4575-shellcheck-0.9.0-bin 45.3M
There is no benefit to using shellcheck.unwrapped in this case.
Therefore, replace shellcheck.unwrapped with lib.getExe shellcheck.
This lets you set the version for the derivation produced from
`buildFHSUserEnvChroot` and `buildFHSUserEnvBubblewrap`.
This can help to make it more clear to end-users to see the versions of
the packages they are using.
`cargoDeps` is already passed as `mkDerivation` arguments, and should
not be `passthru`ed again. This fixes the mismatch of `drv.cargoDeps`
and the actual dependency when the original derivation is overriden.
Previously, you had to provide the path to the deps.nix of the package inside
your Nixpkgs checkout as an argument manually. Now it just does that by default
when no argument is passed.
unpackFile doesn't dereference symlinks if cargoDeps is a directory, and
some cargo builds run into permission issues because the files the
symlinks point to are not writable.
v1 lock files (generated by default by Cargo versions 1.40 and below)
use a single table, `metadata`, to store the checksums of packages.
The primary motivation for doing this now is that we're considering
vendoring all Cargo lock files in Nixpkgs, some packages still use it
(e.g. cargo-asm), and adding support for it doesn't increase the
complexity of the function. No matter the outcome of the vendoring
discussion, this is a nice thing to have because Cargo still supports v1
lock files.
NixOS/nixpkgs#146275 has more discussion on this; the abridged version
is that `lld` defaults to using `--build-id=fast` while GNU `ld` defaults
to `--build-id=sha1`. These differ in length and so
`separate-debug-info.sh`, as of this writing, errors on `lld`'s shorter
`--build-id=fast`-generated hashes.
`lld` offers the following `build-id` styles:
- UUID (random; fast but bad for reproducibility)
- fast (xxhash; fast but shorter hashes)
- user provided hexstring
- SHA1
- MD5
GNU `ld` supports the latter three options, `mold` supports all of these
plus SHA256.
UUID is out because it's not reproducible, fast isn't supported by GNU
`ld`
Using a nix provided (sourced from the output base hash) hash as the
`build-id` seems tempting but would require a little extra work
(we have to include some characteristic of the binary being hashed
so that binaries within a derivation still have unique hashes; it
seems easy to get this wrong; i.e. a path based approach would make
two otherwise identical binaries that reside at different paths have
different `build-id` hashes)
That leaves SHA1 and MD5 and GNU `ld` already defaults to the former.
This commit adds `$NIX_BUILD_ID_STYLE` as an escape hatch, in case any
packages have strong opinions about which hash to use.
----
Note that if/when NixOS/nixpkgs#146275 goes through, this change can be
reverted if linker speed is a priority.
The motivation behind this is to alleviate the problem
described in https://github.com/NixOS/nixpkgs/issues/41340.
I'm not sure if this completely fixes the problem, but it
eliminates one more area where we can exceed command line
length limits.
This is essentially the same change as in #112449,
except for `ld-wrapper.sh` instead of `cc-wrapper.sh`.
However, that change alone was not enough; on macOS the
`ld` provided by `darwin.cctools` fails if you use process
substitution to generate the response file, so I put up a
PR to fix that:
https://github.com/tpoechtrager/cctools-port/pull/131
… and I included a patch referencing that fix so that the
new `ld-wrapper` still works on macOS.
with structuredAttrs lists will be bash arrays which cannot be exported
which will be a issue with some patches and some wrappers like cc-wrapper
this makes it clearer that NIX_CFLAGS_COMPILE must be a string as lists
in env cause a eval failure
Before this change, `nix develop` would often result in all the
shell's dependencies being copied to a remote builder, only for the
remote builder to run a trivial derivation build. This change makes
`nix develop` much faster on systems with remote builders configured.
cc-wrapper has essentially two separate codepaths: the `gccForLibs`
codepath, used only by non-gcc (i.e. clang) compilers, and the
"other" codepath.
This PR allows non-clang compilers to opt-in to the `gccForLibs`
codepath (off by default). To allow this, a new parameter
`ccForLibs` is exposed, since it would be extremely confusing for
gcc to be able to use `gccForLibs` but not do so by default.
Fixes "No such file or directory" error when running prefetch-npm-deps
in its default mode (print hashes, for update scripts etc),
in an environment that doesn't have a `nix` binary (for `nix hash`).
otherwise, eval fails when the experimental no-url-literals feature is activated
unquoted urls are discouraged after https://github.com/NixOS/rfcs/pull/45
not all linkers have a ld binary in bin
also note the '${ld:-}' which allows users to set the ld path with a env
var
> '${foo:-val}' $foo, or val if unset (or null)
Used in cases where you need to get the vendor of a target. Such as when
you need to perform dependency resolution outside of Cargo (eg in
Kolloch's crate2nix).
Currently cargo-setup-hook instructs the builder upon cargoSha256 or
cargoHash being out-of-date compared to the Cargo.lock file.
The instructions can be simplified a bit, because nowadays it is fine to
keep a hash empty, instead of filling it with
`0000000000000000000000000000000000000000000000000000`.
Nix nowadays outputs SRI hashes, which should usually be placed in
`cargoHash` instead of `cargoSha256`, but the instructions are still
only referring to `cargoSha256`.
Lastly, the output of Nix doesn't include `got: sha256: ` anymore, as it
now outputs `got: sha256-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX=`.
It would be nice to make it clear that the trailing `=` is important as
well, so the full example SRI hash is mentioned.
The current `//` override to `agda.passthru.tests` is non-recursive so
it destroys everything else under `passthru`, and furthermore does not
go through `mkDerivation` so that we end up with different values for
`agda.tests` and `agda.passthru.tests`.
Fix it by moving the `allPackages` test to the definition of
`withPackages`.
Upstream changes:
singularity 3.8.7 (the legacy) -> apptainer 1.1.3 (the renamed) / singularity 3.10.4 (Sylabs's fork)
Build process:
* Share between different sources
* Fix the sed regexp to make defaultPath patch work
* allowGoReference is now true
* Provied input parameter removeCompat (default to false)
that removes the compatible "*singularity*" symbolic links
and related autocompletion files when projectName != "singularity"
* Change localstatedir to /var/lib
* Format with nixpkgs-fmt
* Fix the defaultPath patching
and use it instead of the `<executable> path` config directive
deprecated in Apptainer
* Provide dependencies for new functionalities such as
squashfuse (unprivileged squashfs mount)
* Provide an attribute `defaultPathInputs` to override
prefix of container runtime default PATH
NixOS module programs.singularity:
* Allow users to specify packages
* Place related directories to /var/lib
* Format with nixpkgs-fmt
singularity-tools:
* Allow users to specify packages
* Place related directories to /var/lib when building images in VM
See docs.
Follow-up work:
- Existing packages should be converted
- `defaultPkgConfigPackages` should assert on `meta.pkgConfigModules`
and let `tests.pkg-config` alone test the build results.
CC @sternenseemann
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
This reverts commit 8c80bd08b7
("build-support/cc-wrapper: pass in non-existent --sysroot= to untangle
from libc").
This change was good in spirit: we caught a few genuine problems with
`scons` based packages (`godot`, `fluxus`) and unexpected `-idirafter`
includes in various boot loadres (`ipxe`, wimboot`):
https://github.com/NixOS/nixpkgs/pull/210004#issuecomment-1407162693
Unfortunately `--sysroot=` also has a negative impact on libary search
order for DT_NEEDED libraries and RUNPATHs of linked libraries. This
unexpectedly broke `dmd`, `d-seams`, `llvmPackages_rocm.compiler-rt`).
An interesting case of unexpected breakage is `usbmuxd2` where the bug
exposed incomplete library move on `libstdc++fs` in `gcc`.
The library breakage is very non-intuitive (on top of already unusual
layout of `cc-wrapper` driver). Let's revert this change for now.
Once it lands we can undo `--sysroot=/` workarounds merged for
`staging-next`.
After https://github.com/NixOS/nixpkgs/pull/210004 `usbmuxd2` started
failing to build as:
usbmuxd2-unstable> .../ld: cannot find -lstdc++fs: No such file or directory
usbmuxd2-unstable> clang-11: error: linker command failed with exit code 1 (use -v to see invocation)
This started happening because #210004 exposed a long-standing bug in
`gcc` derivation: `cc.lib` is missing `libstdc++fs` library:
$ find $(nix-build --no-link -A stdenv.cc.cc.lib) | fgrep libstdc | unnix
/<<NIX>>/gcc-11.3.0-lib/lib/libstdc++fs.la
/<<NIX>>/gcc-11.3.0-lib/lib/libstdc++.la
/<<NIX>>/gcc-11.3.0-lib/lib/libstdc++.so.6.0.29
/<<NIX>>/gcc-11.3.0-lib/lib/libstdc++.so
/<<NIX>>/gcc-11.3.0-lib/lib/libstdc++.so.6
It was not moved from `cc.out` output:
$ find $(nix-build --no-link -A stdenv.cc.cc) | fgrep libstdc | unnix
/<<NIX>>/gcc-11.3.0/lib/libstdc++.a
/<<NIX>>/gcc-11.3.0/lib/libstdc++fs.a
This change adds `cc` library lookup path back to `staging-next` until
`gcc` is fixed.`
FreeBSD doesn't use LLVM's cxxabi implementation, for backwards
compatibility reasons. Software expects the libcxxrt API when
building on FreeBSD. This fixes the build of
pkgsCross.x86_64-freebsd.boost.
Trivial conflict in release notes, except that the xml/docbook parts
are horrible for (semi-)automatic conflict resolution.
Fortunately that's generated anyway.
With all libcs I'm aware of, libdl is now either empty (Glibc, musl,
uclibc, illumos), a symlink to libc or equivalent (Apple), or does not
exist (FreeBSD, NetBSD). So explicitly linking libdl now does nothing
for the former platforms, and breaks the build for the latter
platforms.
With this patch I've removed -ldl from all overridden linker flags for
all free packages in Nixpkgs. Everything still seems to build.
checkInputs used to be added to nativeBuildInputs. Now we have
nativeCheckInputs to do that instead. Doing this treewide change allows
to keep hashes identical to before the introduction of
nativeCheckInputs.
fetchPypi doesn't use python under the hood and doesn't need to be tied
to a specific version of python. Moving it to top level makes it more
consistent with other fetchers and makes code generation easier.
