e42c17ee97
It turns out that `cargo`, with respect to registry dependencies, was ignoring the package versions locked in `Cargo.lock` because we changed the registry index URL. Therefore, every time `rustRegistry` would be updated, we'd always try to use the latest version available for every dependency and as a result the deps' SHA256 hashes would almost always have to be changed. To fix this, now we do a string substitution in `Cargo.lock` of the `crates.io` registry URL with our URL. This should be safe because our registry is just a copy of the `crates.io` registry at a certain point in time. Since now we don't always use the latest version of every dependency, the build of `cargo` actually started to fail because two of the dependencies specified in its `Cargo.lock` file have build failures. To fix the latter problem, I've added a `cargoUpdateHook` variable that gets ran both when fetching dependencies and just before building the program. The purpose of `cargoUpdateHook` is to do any ad-hoc updating of dependencies necessary to get the package to build. The use of the '--precise' flag is needed so that cargo doesn't try to fetch an even newer version whenever `rustRegistry` is updated (and therefore have to change depsSha256 as a consequence).
130 lines
3.6 KiB
Bash
Executable file
130 lines
3.6 KiB
Bash
Executable file
#! /bin/sh
|
|
|
|
source $stdenv/setup
|
|
|
|
set -euo pipefail
|
|
|
|
src=$(realpath $1)
|
|
out=$(realpath $2)
|
|
|
|
echo "Fetching $src to $out"
|
|
|
|
mkdir $out
|
|
|
|
# Configure cargo to fetch from a local copy of the crates.io registry
|
|
#
|
|
# Unfortunately, `cargo fetch` will create an output directory named after a
|
|
# hash of the registry index URL.
|
|
#
|
|
# This makes things difficult for us because we don't want our output to change
|
|
# just because the path to the registry changed, otherwise we'd have to update
|
|
# all deps' SHA256 hashes whenever we simply update the registry to a newer
|
|
# commit.
|
|
#
|
|
# Also, since cargo doesn't seem to support relative URLs in the format
|
|
# file://../path, we use a hack to make sure the registry index path/URL is
|
|
# always the same: we'll create a symlink in the current working directory to
|
|
# the real registry path, while pointing cargo to the following fixed absolute
|
|
# path:
|
|
#
|
|
# file:///proc/self/cwd/symlink-name
|
|
|
|
ln -s $rustRegistry $src/cargo-rust-registry
|
|
|
|
# TODO: replace /proc/self/cwd hack with normal relative path. Probably
|
|
# needs cargo fix.
|
|
cat <<EOF > $out/config
|
|
[registry]
|
|
index = "file:///proc/self/cwd/cargo-rust-registry"
|
|
EOF
|
|
|
|
export CARGO_HOME=$out
|
|
cd $src
|
|
|
|
set +u
|
|
substituteInPlace Cargo.lock \
|
|
--replace "registry+https://github.com/rust-lang/crates.io-index" \
|
|
"registry+file:///proc/self/cwd/cargo-rust-registry"
|
|
set -u
|
|
|
|
eval "$cargoUpdateHook"
|
|
|
|
cargo fetch --verbose
|
|
|
|
# TODO: check that Cargo.lock exists, and hasn't changed
|
|
# TODO: this should be done by cargo itself
|
|
|
|
# Make it deterministic
|
|
|
|
# The registry index changes all the time, so it's not deterministic
|
|
rm -rf $out/registry/index
|
|
|
|
# Make git DBs deterministic
|
|
# TODO: test with git submodules
|
|
[[ ! -d $out/git/checkouts ]] || (cd $out/git/checkouts && for name in *; do
|
|
cd "$out/git/checkouts/$name"
|
|
revs=""
|
|
for branch in *; do
|
|
cd "$branch"
|
|
rev="$(git rev-parse HEAD)"
|
|
revs="$revs $rev"
|
|
cd ..
|
|
done
|
|
|
|
(
|
|
# The following code was adapted from nix-prefetch-git
|
|
|
|
cd "$out/git/db/$name"
|
|
|
|
export GIT_DIR=.
|
|
|
|
# Remove all remote branches
|
|
git branch -r | while read branch; do
|
|
git branch -rD "$branch" >&2
|
|
done
|
|
|
|
# Remove tags that don't point to any HEAD
|
|
git tag | while read tag; do
|
|
rev="$(git rev-parse $tag)"
|
|
if [[ $revs != *" $rev"* ]]; then
|
|
git tag -d "$tag" >&2
|
|
fi
|
|
done
|
|
|
|
# Remove branches that don't point to any HEAD
|
|
branchrefs=()
|
|
eval "$(git for-each-ref --shell --format='branchrefs+=(%(refname))' refs/heads/)"
|
|
|
|
for branchref in "${branchrefs[@]}"; do
|
|
echo "Examining $branchref"
|
|
rev="$(git rev-parse "$branchref")"
|
|
echo "Has rev $rev"
|
|
echo "List of revs: $revs"
|
|
if [[ $revs != *" $rev"* ]]; then
|
|
echo "Deleting $branchref"
|
|
git update-ref -d "$branchref" >&2
|
|
fi
|
|
done
|
|
|
|
echo "$revs" | while read rev; do
|
|
echo "git branch b_$rev $rev"
|
|
git branch b_$rev $rev
|
|
done
|
|
|
|
# Remove files that have timestamps or otherwise have non-deterministic
|
|
# properties.
|
|
rm -rf logs/ hooks/ index FETCH_HEAD ORIG_HEAD refs/remotes/origin/HEAD config
|
|
|
|
# Do a full repack. Must run single-threaded, or else we lose determinism.
|
|
git config pack.threads 1
|
|
git repack -A -d -f
|
|
rm -f config
|
|
|
|
# Garbage collect unreferenced objects.
|
|
git gc --prune=all
|
|
)
|
|
done)
|
|
|
|
# Remove unneeded outputs
|
|
[[ ! -d $out/registry/src ]] || rm -rf $out/registry/src
|
|
[[ ! -d $out/git/checkouts ]] || rm -rf $out/git/checkouts
|