Commit graph

1140 commits

Author SHA1 Message Date
Eelco Dolstra
0dbd4638e0 * Two primops: builtins.intersectAttrs and builtins.functionArgs.
intersectAttrs returns the (right-biased) intersection between two
  attribute sets, e.g. every attribute from the second set that also
  exists in the first.  functionArgs returns the set of attributes
  expected by a function.

  The main goal of these is to allow the elimination of most of
  all-packages.nix.  Most package instantiations in all-packages.nix
  have this form:

    foo = import ./foo.nix {
      inherit a b c;
    };

  With intersectAttrs and functionArgs, this can be written as:

    foo = callPackage (import ./foo.nix) { };

  where

   callPackage = f: args:
     f ((builtins.intersectAttrs (builtins.functionArgs f) pkgs) // args);

  I.e., foo.nix is called with all attributes from "pkgs" that it
  actually needs (e.g., pkgs.a, pkgs.b and pkgs.c).  (callPackage can
  do any other generic package-level stuff we might want, such as
  applying makeOverridable.)  Of course, the automatically supplied
  arguments can be overriden if needed, e.g.

    foo = callPackage (import ./foo.nix) {
      c = c_version_2;
    };

  but for the vast majority of packages, this won't be needed.

  The advantages are to reduce the amount of typing needed to add a
  dependency (from three sites to two), and to reduce the number of
  trivial commits to all-packages.nix.  For the former, there have
  been two previous attempts:

    - Use "args: with args;" in the package's function definition.
      This however obscures the actual expected arguments of a
      function, which is very bad.

    - Use "{ arg1, arg2, ... }:" in the package's function definition
      (i.e. use the ellipis "..." to allow arbitrary additional
      arguments), and then call the function with all of "pkgs" as an
      argument.  But this inhibits error detection if you call it with
      an misspelled (or obsolete) argument.
2009-09-15 13:01:46 +00:00
Michael Raskin
3bca8931e8 Adding tryEval builtin. It allows to catch presence of errors in an expression. 2009-08-25 16:06:46 +00:00
Eelco Dolstra
c6890d6b5c * Replace newlines in table output with spaces. 2009-07-02 08:52:12 +00:00
Eelco Dolstra
749dd97a54 * Support integers and lists of strings in meta fields. This is
useful for fields like meta.maintainers, meta.priority (which can be
  a proper integer now) and even meta.license (if there are multiple
  licenses).
2009-06-30 15:53:39 +00:00
Eelco Dolstra
f2c3fc5191 * Don't show trace information by default (`--show-trace' to enable).
NixOS evaluation errors in particular look intimidating and
  generally aren't very useful.  Ideally the builtins.throw messages
  should be self-contained.
2009-06-30 13:28:29 +00:00
Eelco Dolstra
14bc3ce3d6 * Canonicalise timestamps in the Nix store to 1 (1970-01-01 00:00:01
UTC) rather than 0 (00:00:00).  1 is a better choice because some
  programs use 0 as a special value.  For instance, the Template
  Toolkit uses a timestamp of 0 to denote the non-existence of a file,
  so it barfs on files in the Nix store (see
  template-toolkit-nix-store.patch in Nixpkgs).  Similarly, Maya 2008
  fails to load script directories with a timestamp of 0 and can't be
  patched because it's closed source.

  This will also shut up those "implausibly old time stamp" GNU tar
  warnings.
2009-06-13 16:30:58 +00:00
Eelco Dolstra
a64bbe049e * Change the scoping of "inherit (e) ..." in recs so that the
attributes of the rec are in scope of `e'.  This is useful in
  expressions such as

    rec {
      lib = import ./lib;
      inherit (lib) concatStrings;
    }

  It does change the semantics of expressions such as

    let x = {y = 1;}; in rec { x = {y = 2;}; inherit (x) y; }.y

  This now returns 2 instead of 1.  However, no code in Nixpkgs or
  NixOS seems to rely on the old behaviour.
2009-05-15 13:46:13 +00:00
Eelco Dolstra
d407d572fd * Some syntactic sugar for attribute sets: allow {x.y.z = ...;} as a
shorthand for {x = {y = {z = ...;};};}.  This is especially useful
  for NixOS configuration files, e.g.

    {
      services = {
        sshd = {
          enable = true;
          port = 2022;
        };
      };
    }

  can now be written as

    {
      services.sshd.enable = true;
      services.sshd.port = 2022;
    }

  However, it is currently not permitted to write
  
    {
      services.sshd = {enable = true;};
      services.sshd.port = 2022;
    }

  as this is considered a duplicate definition of `services.sshd'.
2009-05-15 12:35:23 +00:00
Eelco Dolstra
e42975490f * Check for duplicate attributes in fixAttrs, rather than doing a
separate traversal after parsing.  Likewise, check for duplicate
  pattern variables right away.
2009-05-14 14:29:45 +00:00
Eelco Dolstra
50d11b90ca * Allow unsafe (unspecified) comparisons between attrsets unless
NIX_NO_UNSAFE_EQ is set, for now.
2009-05-12 11:06:24 +00:00
Eelco Dolstra
c34e6d71bc * Disallow equality tests between attribute sets. This was always
broken, but now the evaluator checks for it to prevent Nix
  expressions from relying on undefined behaviour.  Equality tests are
  implemented using a shallow pointer equality test between ATerms.
  However, because attribute sets are lazy and contain position
  information, this can give false positives.  For instance,
  previously

    let y = {x = 1;}; in y == y

  evaluated to true, while the equivalent expression

    {x = 1;} == {x = 1;}

  evaluated to false.  So disallow these tests for now.  (Eventually
  we may want to implement deep equality tests for attribute sets,
  like lib.eqStrict.)
  
* Idem: disallow comparisons between functions.

* Implemented deep comparisons of lists.  This had the same problem as
  attribute sets - the elements in the list weren't evaluated.  For
  instance,

    ["xy"] == [("x" + "y")]

  evaluated to false.  Now it works properly.
2009-05-11 15:50:14 +00:00
Eelco Dolstra
7660e2a068 * Remove a right recursion that causes the parser to barf on very long
lists.  The comment about ATreverse requiring unbounded stack space
  was unfounded anyway.
2009-05-07 11:35:52 +00:00
Michael Raskin
098cb9d233 Add an ftruncate call paired with fallocate to play safe with some FSes (namely, BtrFS fallocate sets file size to allocated size, i.e. multiple of block size) 2009-05-04 08:10:24 +00:00
Eelco Dolstra
d5eab2fc82 * Use foreach in a lot of places. 2009-04-21 11:52:16 +00:00
Eelco Dolstra
8f1bf28505 * nix-store --verify: don't bail out if a referenced path is missing.
(It can't fix it though.)
2009-04-21 11:06:27 +00:00
Eelco Dolstra
4e646b0ddb * Fix a few "comparison is always false/true due to limited range of
data type" warnings on 64-bit platforms.  The one in parser.y is
  likely to be a real bug.
2009-04-16 12:03:17 +00:00
Eelco Dolstra
dfb863f333 * Don't cache transient build hook problems. 2009-04-15 06:25:02 +00:00
Eelco Dolstra
351bf658f9 * Do a substitution even if --max-jobs == 0. 2009-03-31 21:14:07 +00:00
Eelco Dolstra
cff2b2a13a * Doh! 2009-03-31 15:50:03 +00:00
Eelco Dolstra
160a60d663 2009-03-30 19:35:55 +00:00
Eelco Dolstra
805144b705 * Make the poll interval configurable. 2009-03-30 11:34:03 +00:00
Eelco Dolstra
7377195297 * With `--max-jobs 0', print a nicer error message than "Assertion
`!awake.empty()' failed."
2009-03-29 18:06:00 +00:00
Eelco Dolstra
737423a89c * Use polling to wait for a remote build slot when using a build hook
(that is, call the build hook with a certain interval until it
  accepts the build).
* build-remote.pl was totally broken: for all system types other than
  the local system type, it would send all builds to the *first*
  machine of the appropriate type.
2009-03-29 18:00:45 +00:00
Eelco Dolstra
6e946c8e72 * Scan for references and compute the SHA-256 hash of the output in
one pass.  This halves the amount of I/O.
2009-03-28 20:51:33 +00:00
Eelco Dolstra
c7152c8f97 * Don't use the non-standard __gnu_cxx::stdio_filebuf class. 2009-03-28 19:41:53 +00:00
Eelco Dolstra
3a2bbe7f8a * Simplify communication with the hook a bit (don't use file
descriptors 3/4, just use stdin/stderr).
2009-03-28 19:29:55 +00:00
Eelco Dolstra
f54e800366 * The `fixedOutput' variable didn't get initialised when using a build
hook, causing negative caching to fail randomly.
2009-03-28 16:12:27 +00:00
Eelco Dolstra
cb85bc396e * Create /nix/store if it doesn't exist. 2009-03-27 14:19:04 +00:00
Eelco Dolstra
5dd8fb2069 * Don't use ULLONG_MAX in maxFreed - use 0 to mean "no limit".
18446744073709551615ULL breaks on GCC 3.3.6 (`integer constant is
  too large for "long" type').
2009-03-26 11:02:07 +00:00
Eelco Dolstra
92f525ecf4 * Negative caching, i.e. caching of build failures. Disabled by
default.  This is mostly useful for Hydra.
2009-03-25 21:05:42 +00:00
Eelco Dolstra
a046858a22 * Doh. 2009-03-24 23:40:24 +00:00
Eelco Dolstra
12c8c64efa 2009-03-24 14:07:37 +00:00
Eelco Dolstra
3a87163b24 * Shut up a GCC warning. 2009-03-23 13:52:52 +00:00
Eelco Dolstra
cacff1be88 * No longer block while waiting for a lock on a store path. Instead
poll for it (i.e. if we can't acquire the lock, then let the main
  select() loop wait for at most a few seconds and then try again).
  This improves parallelism: if two nix-store processes are both
  trying to build a path at the same time, the second one shouldn't
  block; it should first see if it can build other goals.  Also, it
  prevents the deadlocks that have been occuring in Hydra lately,
  where a process waits for a lock held by another process that's
  waiting for a lock held by the first.

  The downside is that polling isn't really elegant, but POSIX doesn't
  provide a way to wait for locks in a select() loop.  The only
  solution would be to spawn a thread for each lock to do a blocking
  fcntl() and then signal the main thread, but that would require
  pthreads.
2009-03-23 01:05:54 +00:00
Eelco Dolstra
58969fa2bf * Refactoring. 2009-03-22 23:53:05 +00:00
Eelco Dolstra
7a57b2920b * Better error message when nix-store --import is applied to garbage
(previously it would likely say "implementation cannot deal with >
  32-bit integers").
2009-03-22 17:51:45 +00:00
Eelco Dolstra
77d272623f * NAR archives: handle files larger than 2^32 bytes. Previously it
would just silently store only (fileSize % 2^32) bytes.
* Use posix_fallocate if available when unpacking archives.
* Provide a better error message when trying to unpack something that
  isn't a NAR archive.
2009-03-22 17:36:43 +00:00
Eelco Dolstra
2897286487 * Unify exportReferencesGraph and exportBuildReferencesGraph, and make
sure that it works as expected when you pass it a derivation.  That
  is, we have to make sure that all build-time dependencies are built,
  and that they are all in the input closure (otherwise remote builds
  might fail, for example).  This is ensured at instantiation time by
  adding all derivations and their sources to inputDrvs and inputSrcs.
2009-03-18 17:36:42 +00:00
Eelco Dolstra
c183ee5c79 * Acquire the locks on the output paths before trying to run the build
hook.  This fixes a problem with log files being partially or
  completely filled with 0's because another nix-store process
  truncates the log file.  It should also be more efficient.
2009-03-18 14:48:42 +00:00
Eelco Dolstra
33ecb42991 * Cleanup. 2009-03-17 11:42:55 +00:00
Eelco Dolstra
d4753c944f 2009-03-06 11:01:45 +00:00
Eelco Dolstra
0e6f604178 * Install some headers in the right location. 2009-03-05 14:57:50 +00:00
Eelco Dolstra
60cb7de336 * Allow options from the Nix config file to be overriden from the
command line (e.g. "--option build-use-chroot true").
2009-02-27 11:04:41 +00:00
Eelco Dolstra
824b154ce8 * Release output locks as soon as possible, not when the destructor of
the DerivationGoal runs.  Otherwise, if a goal is a top-level goal,
  then the lock won't be released until nix-store finishes.  With
  --keep-going and lots of top-level goals, it's possible to run out
  of file descriptors (this happened sometimes in the build farm for
  Nixpkgs).  Also, for failed derivation, it won't be possible to
  build it again  until the lock is released.
  
* Idem for locks on build users: these weren't released in a timely
  manner for failed top-level derivation goals.  So if there were more
  than (say) 10 such failed builds, you would get an error about
  having run out of build users.
2009-02-16 09:24:20 +00:00
Marc Weber
1407a1ec99 added primop functions __isBool, __isString, __isInt 2009-02-05 19:35:40 +00:00
Eelco Dolstra
b682fae9d9 * Build hooks: use nix-store --import. This prevents a redundant
scan for runtime dependencies (i.e. the local machine shouldn't do a
  scan that the remote machine has already done).  Also pipe directly
  into `nix-store --import': don't use a temporary file.
2009-02-02 17:24:10 +00:00
Nicolas Pierron
110606d470 Add the "addErrorContext" builtin to add more information in the stack trace. 2009-01-27 14:36:44 +00:00
Eelco Dolstra
061141e632 * Make it compile on Debian 4.0 (which doesn't define PER_LINUX32_3GB
in sys/personality.h).
2009-01-27 13:36:59 +00:00
Eelco Dolstra
019176137f * When using a build hook, distinguish between transient failures
(e.g. an SSH connection problem) and permanent failures (i.e. the
  builder failed).  This matters to Hydra (it wants to know whether it
  makes sense to retry a build).
2009-01-13 11:39:09 +00:00
Eelco Dolstra
c504d90c11 * Support i686-linux builds directly on x86_64-linux Nix
installations.  This is implemented using the personality() syscall,
  which causes uname to return "i686" in child processes.
2009-01-12 16:30:32 +00:00