Commit graph

242 commits

Author SHA1 Message Date
Frederik Rietdijk
8576d24b2a Merge staging-next into staging 2020-06-08 12:08:51 +02:00
hyperfekt
0a8fb89fca rustPlatform: fix cross-compiling by using native diff 2020-06-08 10:05:00 +05:30
Frederik Rietdijk
1c68570ab2 Merge staging-next into staging 2020-06-05 19:42:16 +02:00
Jörg Thalheim
f0396574ab
buildRustPackage: add documentation on how to create cargo.lock patches 2020-06-05 10:41:46 +01:00
Vladimír Čunát
677e3960b5
Merge #82342: rustPlatform: increase build-speed of checkPhase
...for rust-packages (into staging)
2020-06-05 09:12:30 +02:00
Michael Howell
c21cbf22d0
buildRustCrate: Replace hyphen with underscore in env variables (#88054)
* Add test case for include dir
* buildRustCrate: replace hyphen with underscore in env

This fixes a bug that prevents encoding_c from building.
2020-05-26 20:52:18 +02:00
Maximilian Bosch
6b23cfe689
rustPlatform: add buildAndTestSubdir-argument
There are several tarballs (such as the `rust-lang/rust`-source) with a
`Cargo.toml` at root and several sub-packages (with their own Cargo.toml)
without using workspaces[1].

In such a case it's needed to move into a subdir to only build the
specified sub-package (e.g. `rustfmt` or `rsl`), however the artifacts
are at `/target` in the root-dir of the build environment. This breaks
the build since `buildRustPackage` searches for executables in `target`
(which is at the build-env's root) at the end of the `buildPhase`.

With the optional `buildAndTestSubdir`-argument, the builder moves into
the specified subdir using `pushd`/`popd` during `buildPhase` and
`checkPhase`.

Also moved the logic to find executables and libs to the end of the `buildPhase`
from a custom `postBuild`-hook to fix packages with custom `build`/`install`-procedures
such as `uutils-coreutils`.

[1] https://doc.rust-lang.org/book/ch14-03-cargo-workspaces.html
2020-05-13 01:47:17 +02:00
Maximilian Bosch
736462d995
rustPlatform: make it possible to override the profile for cargo test 2020-05-13 01:39:44 +02:00
Vladimír Čunát
5eaabaf089
Merge branch 'staging-next' into PR 82342
Hydra nixpkgs: ?compare=1586582
2020-05-11 08:22:59 +02:00
Maximilian Bosch
f236714d65
rustPlatform: fix log 2020-05-08 22:59:50 +02:00
Maximilian Bosch
e32c005772
rustPlatform: don't install test executables
This is done by gathering all binaries to install before running the
checkPhase.
2020-05-08 22:59:50 +02:00
Benjamin Hipple
7f845578af rustPlatform: fix bug with ambiguous diff tool
If a user provides `nativeBuildInputs = [ llvmPackages.bintools ]` or any other
package containing a `${prefix}/bin/diff`, the builder could use it instead
of the standard unix `diff`, causing a build failure.

This updates the call to specify an abspath to `diff` and avoid reliance on `PATH`.

Resolves #87081
2020-05-06 21:03:41 -04:00
Maximilian Bosch
04248f606f
rustPlatform: increase build-speed of checkPhase for rust-packages
When running `cargo test --release`, the artifacts from `buildPhase`
will be reused here. Previously, most of the stuff had to be recompiled
without optimizations.
2020-05-06 23:47:31 +02:00
Andreas Rammhold
84b91899c3
Merge pull request #85172 from andir/buildRustCrate-proc-macro
buildRustCrate: support proc-macro in default prelude
2020-04-13 23:35:19 +02:00
Andreas Rammhold
a9fdfebc6b
buildRustCrate: support proc-macro in default prelude 2020-04-13 16:01:21 +02:00
Peter Kolloch
bb660fe228 buildRustCrate: Support versioned crate renames 2020-04-10 00:55:44 +02:00
Peter Kolloch
5f9af254a5 buildRustCrate: Document parameters
I know, heretic, but...

I also know that this is not perfect but it is a good start, I think. It
would be nice if this were part of the automatic "nixdoc" function
reference. I'd like guidance if this should be part of the rust section
or something else.
2020-04-10 00:55:05 +02:00
Jörg Thalheim
e8323a0bf9
buildRustPackage: enable strictDeps
This will improve cross-compiling support by forcing
users to specify buildInputs/nativeBuildInputs correctly.
2020-03-29 14:09:32 +01:00
Peter Kolloch
782b304dba buildRustCrate: Add tests for checking files in outputs.
...and remove superfluous dependency files (*.d).
...and copy dSYM directories on Mac OS when in release=false mode.
2020-03-29 13:00:21 +02:00
Andreas Rammhold
c8de31baa6 buildRustCrateTests: Fix link order test on darwin
As it turns out Darwin does most of the things differently then "normal"
systems. They are using a different shared library extension and require
an obscure commandline parameter that has to be added to every build
system out there. That issue seems to be with clang on Darwin as on
Linux that flag isn't required to build the very same tests (when using
clang).

After adjusting these two details the tests are running fine on the
darwin box that I was able to obtain.
2020-03-28 21:13:16 +01:00
Andreas Rammhold
d86bfec309
Merge pull request #83379 from symphorien/rust-link
buildRustCrate: don't sort link flags
2020-03-28 16:02:51 +01:00
Symphorien Gibol
2f7fb1c497 buildRustCrateTests: add regression test for link order 2020-03-28 12:00:00 +00:00
Alyssa Ross
7533876312
buildRustCrate: fewer backslashes
This is a slight readability boost, I think.
2020-03-27 09:56:19 +00:00
Symphorien Gibol
d8b853799d buildRustCrate: don't sort link flags
Linkage order is significant and sorting can result in link errors.
2020-03-25 12:00:00 +00:00
Andreas Rammhold
4e8d0993a9
Merge pull request #82404 from danieldk/build-script-link-fix
buildRustCrate: only link build deps into build script
2020-03-20 17:56:21 +01:00
Benjamin Hipple
05343f6ff1 rust: remove legacy cargo fetcher
We have now migrated every single Rust package in NixPkgs! This deletes the
legacy fetcher, which is now unused.

Resolves #79975
2020-03-18 20:12:32 -07:00
Daniël de Kok
412c72d20f buildRustCrate: sort linker options in-place 2020-03-13 11:21:07 +01:00
Daniël de Kok
ea6e048c37 buildRustCrate: only link build deps into build script
According to the Cargo documentation:

> The build script does not have access to the dependencies listed in
> the dependencies or dev-dependencies section (they’re not built
> yet!). Also, build dependencies are not available to the package
> itself unless also explicitly added in the [dependencies] table.

https://doc.rust-lang.org/cargo/reference/build-scripts.html

This change separates linkage of regular dependencies and build
dependencies.
2020-03-13 11:13:27 +01:00
Andreas Rammhold
e968961d14
Merge pull request #82155 from kolloch/buildRustCrate_findMatchingDir
buildRustCrate: Search for matching Cargo.toml in sub directories
2020-03-10 13:27:26 +01:00
Mario Rodas
cad87836ac
Merge pull request #81974 from bhipple/feature/rust-doc
buildRustPackage: update docstring comment
2020-03-10 06:38:42 -05:00
Peter Kolloch
8a6638daa9 build-support/rust/buildRustCrate: Search for matching Cargo.toml in sub directories
This is what cargo does for git repositories.

See related issues:

* https://github.com/kolloch/crate2nix/issues/53
* https://github.com/kolloch/crate2nix/issues/33
2020-03-09 15:11:50 +01:00
Peter Kolloch
04e7462ee6 buildRustCrate: refactor colored logging
* Make errors include the crate name and make them much more prominent.
* Move more code into lib.sh
* Already source generated logging code and lib.sh in configure
2020-03-09 14:26:28 +01:00
Benjamin Hipple
37fb7a5568 buildRustPackage: update docstring comment
The inlined readme that we were iterating on has been moved to GitHub
issue #79975, and the default is now the new cargo fetcher, so this
doc comment is out of date.
2020-03-07 09:34:29 -05:00
Benjamin Hipple
ad30a30488 rustPlatform.fetchCargo: handle custom Cargo.lock patchfiles with validation
Previously, we would asssert that the lockfiles are consistent during the
unpackPhase, but if the pkg has a patch for the lockfile itself then we must
wait until the patchPhase is complete to check.

This also removes an implicity dependency on the src attribute coming from
`fetchzip` / `fetchFromGitHub`, which happens to name the source directory
"source". Now we glob for it, so different fetchers will work consistently.
2020-02-28 18:54:23 -08:00
Andreas Rammhold
453589696b
Merge pull request #79816 from andir/buildRustCrate-no-override-dep
buildRustCrate: remove superfluous dependency overrides
2020-02-18 15:18:44 +01:00
Benjamin Hipple
6d881472ef rust: Fix for legacy fetch cargo
See inline comment and #79975 for details.
2020-02-15 17:41:35 -08:00
Benjamin Hipple
eb11feaa0b treewide: change fetchCargoTarball default to opt-out
Changes the default fetcher in the Rust Platform to be the newer
`fetchCargoTarball`, and changes every application using the current default to
instead opt out.

This commit does not change any hashes or cause any rebuilds. Once integrated,
we will start deleting the opt-outs and recomputing hashes.

See #79975 for details.
2020-02-13 22:41:37 -08:00
Benjamin Hipple
c1542fc4cd rust: Replace migration README with GitHub issue
The readme was nice to discuss in the implementation PR, but now that this is
merged it's better to have an issue that can be linked against in PRs and
doesn't require further merges to update status.

Ported with a status update in #79975
2020-02-12 22:28:53 -05:00
Frederik Rietdijk
424697d512 Merge master into staging-next 2020-02-12 09:55:31 +01:00
Andreas Rammhold
be5597fc9d
buildRustCrate: remove superfluous dependency overrides
By overriding each dependency on every level of the dependency tree we
are creating a lot of unnecessary instances of the same derivation

Looking at the output size of `nix-instantiate --trace-function-calls
-vvvv …` and the execution time I got about a 10x improvement after
applying this change.

It was probably good intentions that lead to these overrides but in
practice no tooling (that I know of) really needs this. `carnix` and
`crate2nix` are fine without those overrides. Furthermore I believe that
it is the job of the tooling around `buildRustCrate` to provide a
coherent set of overrides. By not enforcing all of the overrides, debug
flags, verbosity, … to be the same throughout the closure we also allow
consumers to override specific aspects of the crates. Some (older?)
crates might need different `crateOverrides` then newer crates with the
same name. Currently such situations can not (easily) be implemented
with the override in-place.
2020-02-11 11:48:45 +01:00
Benjamin Hipple
2115a2037c fetchcargo: use flat tar.gz file for vendored src instead of recursive hash dir
This has several advantages:

1. It takes up less space on disk in-between builds in the nix store.
2. It uses less space in the binary cache for vendor derivation packages.
3. It uses less network traffic downloading from the binary cache.
4. It plays nicely with hashed mirrors like tarballs.nixos.org, which only
   substitute --flat hashes on single files (not recursive directory hashes).
5. It's consistent with how simple `fetchurl` src derivations work.
6. It provides a stronger abstraction between input src-package and output
   package, e.g., it's harder to accidentally depend on the src derivation at
   runtime by referencing something like `${src}/etc/index.html`. Likewise, in
   the store it's harder to get confused with something that is just there as a
   build-time dependency vs. a runtime dependency, since the build-time
   src dependencies are tarred up.

Disadvantages are:
1. It takes slightly longer to untar at the start of a build.

As currently implemented, this attaches the compacted vendor.tar.gz feature as a
rider on `verifyCargoDeps`, since both of them are relatively newly implemented
behavior that change the `cargoSha256`.

If this PR is accepted, I will push forward the remaining rust packages with a
series of treewide PRs to update the `cargoSha256`s.
2020-02-10 10:17:29 -05:00
Andreas Rammhold
56e11bc8df
buildRustCrate: remap the current build dir to / for (more) reproducible builds 2020-02-06 01:18:59 +01:00
Andreas Rammhold
a57d0fe0bb
buildRustCrate: fix #78412
`build.rs` files might create files. Those files are supposed to go into
`OUT_DIR` (envirionment variable) and not be overlayed onto the source
tree.
2020-01-28 14:07:58 +01:00
Andreas Rammhold
19698d15ce
buildRustCrateTests: add regression test for #74071 2020-01-28 14:07:58 +01:00
Andreas Rammhold
78faab1be0 buildRustCrateTests: add test case for rlib linking 2020-01-21 17:46:32 +01:00
Andreas Rammhold
406e0c9d51
buildRustCrateTests: fix some formatting issues 2020-01-21 17:32:48 +01:00
Andreas Rammhold
d6a8b55fb0
buildRustCrate: treat rlib crates just like lib crates
Both version provide `rlib` files to link against. Previously we would
try to find a matching shared library in the `lib` output.
2020-01-21 17:22:59 +01:00
Andreas Rammhold
69c96adc53
buildRustCrateTests: use releaseTools.aggregate
Previously I did use `runCommand` to do the same. Using
releaseTools.aggregate seems a lot saner and we might get nicer hydra
output of the tests that are failing.
2020-01-16 13:24:15 +01:00
Andreas Rammhold
29a8575e3d
buildRustCrate: remove one of the odd library filename cases
It used to be the case (ref missing) that cargo did treat
`src/$libName.rs` as an alternative to `src/lib.rs` when the latter
wasn't present. Recently I failed to reproduce that with vanilla cargo
and it started to cause pain with some crates of the form:

some_crate/
 `- src
   `- main.rs
   `- some_crate.rs

We would build `src/some_crate.rs` and thing it is a library while that
might not be the actual case. This crate is a valid `bin` crate not a
`lib` crate as far as I can tell from the samples I took.

I removed support for the previously required heuristic and commented
out the test cases in case we will need them again. We could crawl in
the Git history but chances are that the next person looking into this
doesn't know about the history.
2020-01-16 13:24:13 +01:00
Benjamin Hipple
6e8c377562 rustPlatform.buildRustPackage: cleaner output on verifyCargoDeps (#77567)
When this fails, the user may want to copy-paste the path to the "bad"
Cargo.lock file to inspect. The trailing `.` on `$cargoDeps.` gets caught in
most terminal copy-pastes. Since half the lines already don't have it, this
removes it from all of them for consistent output.
2020-01-12 17:19:17 +00:00