This is currently only used by the Hydra queue runner rework, but like
eff5021eaa it presumably will be useful
for the C++ rewrite of nix-push and
download-from-binary-cache. (@shlevy)
The `set -e` at the top of the script causes the installation to fail to
complete if the shell profile is not writeable. Checking file existence
only is not enough.
The --insecure flag to curl tells curl not to bother checking if the TLS
certificate presented by the server actually matches the hostname
requested, and actually is issued by a trusted CA chain. This almost
entirely negates any benefit from using TLS in the first place.
This removes the --insecure flag to ensure we actually have a secure
connection to the intended hostname before downloading binaries.
Manually tested locally within a dev-shell; was able to download
binaries from https://cache.nixos.org without issue.
[Note: --insecure was only used for fetching NARs, whose integrity is
verified by Nix anyway using the hash from the .narinfo. But if we can
fetch the .narinfo without --insecure, we can also fetch the .nar, so
there is not much point to using --insecure. --Eelco]
Some benchmarking suggested this as a good value. Running
$ benchmark -f ... -t 25 -- sh -c 'rm -f /nix/var/nix/binary-cache*; nix-store -r /nix/store/x5z8a2yvz8h6ccmhwrwrp9igg03575jg-nixos-15.09.git.5fd87e1M.drv --dry-run --option binary-caches-parallel-connections <N>'
gave the following mean elapsed times for these values of N:
N=10: 3.3541
N=20: 2.9320
N=25: 2.6690
N=30: 2.9417
N=50: 3.2021
N=100: 3.5718
N=150: 4.2079
Memory usage is also reduced (N=150 used 186 MB, N=25 only 68 MB).
Closes#708.
This makes that option even more insecure, by also not checking the SSL host.
But without this parameter, one can still get SSL errors even when
"verify-https-binary-caches" is false, which is unexpected IMO.
This reverts commit 76f985b92d. We
shouldn't mess with $MANPATH, because on some "man" implementations
(like NixOS'), the default value on $MANPATH is derived from $PATH. So
if you set $MANPATH, you lose the default locations.
This is not strictly needed for integrity (since we already include
the NAR hash in the fingerprint) but it helps against endless data
attacks [1]. (However, this will also require
download-from-binary-cache.pl to bail out if it receives more than the
specified number of bytes.)
[1] https://isis.poly.edu/~jcappos/papers/cappos_mirror_ccs_08.pdf
In some cases the bash builtin command "cd" can print the variable $CWD
to stdout. This caused the install script to fail while copying files
because the source path was wrong.
Fixes#476.
We only need to sign the store path, NAR hash and references (the
"fingerprint"). Everything else is irrelevant to security. For
instance, the compression algorithm or the hash of the compressed NAR
don't matter as long as the contents of the uncompressed NAR are
correct.
(Maybe we should include derivers in the fingerprint, but they're
broken and nobody cares about them. Also, it might be nice in the
future if .narinfos contained signatures from multiple independent
signers. But that's impossible if the deriver is included in the
fingerprint, since everybody will tend to have a different deriver for
the same store path.)
Also renamed the "Signature" field to "Sig" since the format changed
in an incompatible way.
Sodium's Ed25519 signatures are much shorter than OpenSSL's RSA
signatures. Public keys are also much shorter, so they're now
specified directly in the nix.conf option ‘binary-cache-public-keys’.
The new command ‘nix-store --generate-binary-cache-key’ generates and
prints a public and secret key.
It moves runHook to a later position in the rcfile. After that we are
able to set the PS1 environment-variable for a nix-shell environment
e.g.:
# turn the color of the prompt to blue
shellHook = ''
export PS1="\n\[\033[1;34m\][\u@\h:\w]$\[\033[0m\] ";
'';
‘--run’ is like ‘--command’, except that it runs the command in a
non-interactive shell. This is important if you do things like:
$ nix-shell --command make
Hitting Ctrl-C while make is running drops you into the interactive
Nix shell, which is probably not what you want. So you can now do
$ nix-shell --run make
instead.
So you can have a script like:
#! /usr/bin/env nix-shell
#! nix-shell script.nix -i python
import prettytable
x = prettytable.PrettyTable(["Foo", "Bar"])
for i in range(1, 10): x.add_row([i, i**2])
print x
with a ‘script.nix’ in the same directory:
with import <nixpkgs> {};
runCommand "dummy" { buildInputs = [ python pythonPackages.prettytable ]; } ""
(Of course, in this particular case, using the ‘-p’ flag is more
convenient.)
This allows scripts to fetch their own dependencies via nix-shell. For
instance, here is a Haskell script that, when executed, pulls in GHC
and the HTTP package:
#! /usr/bin/env nix-shell
#! nix-shell -i runghc -p haskellPackages.ghc haskellPackages.HTTP
import Network.HTTP
main = do
resp <- Network.HTTP.simpleHTTP (getRequest "http://nixos.org/")
body <- getResponseBody resp
print (take 100 body)
Or a Perl script that pulls in Perl and some CPAN packages:
#! /usr/bin/env nix-shell
#! nix-shell -i perl -p perl perlPackages.HTMLTokeParserSimple perlPackages.LWP
use HTML::TokeParser::Simple;
my $p = HTML::TokeParser::Simple->new(url => 'http://nixos.org/');
while (my $token = $p->get_tag("a")) {
my $href = $token->get_attr("href");
print "$href\n" if $href;
}
Note that the options to nix-shell must be given on a separate line
that starts with the magic string ‘#! nix-shell’. This is because
‘env’ does not allow passing arguments to an interpreter directly.
Apparently, turning on utf8 encoding on stderr changes its flushing
behaviour, causing sendReply to not send anything.
http://hydra.nixos.org/build/13944384
This makes things more efficient (we don't need to use an SSH master
connection, and we only start a single remote process) and gets rid of
locking issues (the remote nix-store process will keep inputs and
outputs locked as long as they're needed).
It also makes it more or less secure to connect directly to the root
account on the build machine, using a forced command
(e.g. ‘command="nix-store --serve --write"’). This bypasses the Nix
daemon and is therefore more efficient.
Also, don't call nix-store to import the output paths.
