eldritch horrors e6cd67591b libexpr: rewrite the parser with pegtl instead of flex/bison ... this gives about 20% performance improvements on pure parsing. obviously it will be less on full eval, but depending on how much parsing is to be done (e.g. including hackage-packages.nix or not) it's more like 4%-10%. this has been tested (with thousands of core hours of fuzzing) to ensure that the ASTs produced by the new parser are exactly the same as the old one would have produced. error messages will change (sometimes by a lot) and are not yet perfect, but we would rather leave this as is for later. test results for running only the parser (excluding the variable binding code) in a tight loop with inputs and parameters as given are promising: - 40% faster on lix's package.nix at 10000 iterations - 1.3% faster on nixpkgs all-packages.nix at 1000 iterations - equivalent on all of nixpkgs concatenated at 100 iterations (excluding invalid files, each file surrounded with parens) more realistic benchmarks are somewhere in between the extremes, parsing once again getting the largest uplift. other realistic workloads improve by a few percentage points as well, notably system builds are 4% faster. Benchmarks summary (from ./bench/summarize.jq bench/bench-*.json) old/bin/nix --extra-experimental-features 'nix-command flakes' eval -f bench/nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix mean: 0.408s ± 0.025s user: 0.355s | system: 0.033s median: 0.389s range: 0.388s ... 0.442s relative: 1 new/bin/nix --extra-experimental-features 'nix-command flakes' eval -f bench/nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix mean: 0.332s ± 0.024s user: 0.279s | system: 0.033s median: 0.314s range: 0.313s ... 0.361s relative: 0.814 --- old/bin/nix --extra-experimental-features 'nix-command flakes' eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' mean: 6.133s ± 0.022s user: 5.395s | system: 0.437s median: 6.128s range: 6.099s ... 6.183s relative: 1 new/bin/nix --extra-experimental-features 'nix-command flakes' eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' mean: 5.925s ± 0.025s user: 5.176s | system: 0.456s median: 5.934s range: 5.861s ... 5.943s relative: 0.966 --- GC_INITIAL_HEAP_SIZE=10g old/bin/nix eval --extra-experimental-features 'nix-command flakes' --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' mean: 4.503s ± 0.027s user: 3.731s | system: 0.547s median: 4.499s range: 4.478s ... 4.541s relative: 1 GC_INITIAL_HEAP_SIZE=10g new/bin/nix eval --extra-experimental-features 'nix-command flakes' --raw --impure --expr 'with import <nixpkgs/nixos> {}; system' mean: 4.285s ± 0.031s user: 3.504s | system: 0.571s median: 4.281s range: 4.221s ... 4.328s relative: 0.951 --- old/bin/nix --extra-experimental-features 'nix-command flakes' search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870 hello mean: 16.475s ± 0.07s user: 14.088s | system: 1.572s median: 16.495s range: 16.351s ... 16.536s relative: 1 new/bin/nix --extra-experimental-features 'nix-command flakes' search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870 hello mean: 15.973s ± 0.013s user: 13.558s | system: 1.615s median: 15.973s range: 15.946s ... 15.99s relative: 0.97 --- Change-Id: Ie66ec2d045dec964632c6541e25f8f0797319ee2		2024-06-25 12:24:58 +00:00
..
ca	chore: rebrand Nix to Lix when it makes sense	2024-06-01 20:31:24 +02:00
common	Add build-dir setting, clean up default TMPDIR handling	2024-06-24 11:30:32 +03:00
config	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
dyn-drv	remove the autoconf+Make buildsystem	2024-05-07 17:04:30 -06:00
flakes	flakes: add --commit-lock-file message test	2024-06-23 17:29:40 +00:00
lang	libexpr: rewrite the parser with pegtl instead of flex/bison	2024-06-25 12:24:58 +00:00
nested-sandboxing	chore: rebrand Nix to Lix when it makes sense	2024-06-01 20:31:24 +02:00
plugins	remove the autoconf+Make buildsystem	2024-05-07 17:04:30 -06:00
repl_characterization	libutil: return Pid from startProcess, not pid_t	2024-06-23 11:52:49 +00:00
test-libstoreconsumer	remove the autoconf+Make buildsystem	2024-05-07 17:04:30 -06:00
add.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
bad.tar.xz	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
bash-profile.sh	Fix failing darwin tests	2024-05-06 18:56:40 +02:00
big-derivation-attr.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
binary-cache-build-remote.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
binary-cache.sh	chore: rebrand Nix to Lix when it makes sense	2024-06-01 20:31:24 +02:00
brotli.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
build-delete.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
build-dry.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
build-hook-ca-fixed.nix	Merge pull request #9032 from Ma27/structured-attrs-env-vars	2024-03-07 10:46:47 +01:00
build-hook-ca-floating.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
build-hook.nix	Merge pull request #9032 from Ma27/structured-attrs-env-vars	2024-03-07 10:46:47 +01:00
build-remote-content-addressed-fixed.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
build-remote-content-addressed-floating.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
build-remote-input-addressed.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
build-remote-trustless-after.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
build-remote-trustless-should-fail-0.sh	chore: rebrand Nix to Lix when it makes sense	2024-06-01 20:31:24 +02:00
build-remote-trustless-should-pass-0.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
build-remote-trustless-should-pass-1.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
build-remote-trustless-should-pass-2.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
build-remote-trustless-should-pass-3.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
build-remote-trustless.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
build-remote.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
build.sh	nix3-build: show all FOD errors with --keep-going	2024-05-15 15:35:18 +02:00
ca-shell.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
case-hack.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
case.nar	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
check-refs.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
check-refs.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
check-reqs.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
check-reqs.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
check.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
check.sh	Add build-dir setting, clean up default TMPDIR handling	2024-06-24 11:30:32 +03:00
common.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
completions.sh	Merge pull request #8817 from iFreilicht/flake-update-lock-overhaul	2024-03-25 17:36:24 -06:00
compression-levels.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
compute-levels.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
config.nix.in	Merge pull request #9032 from Ma27/structured-attrs-env-vars	2024-03-07 10:46:47 +01:00
config.sh	Rename nix show-config to nix config show	2024-05-03 16:26:16 +02:00
db-migration.sh	chore: rebrand Nix to Lix when it makes sense	2024-06-01 20:31:24 +02:00
debugger.sh	Merge pull request #9933 from pennae/debugger-fix	2024-03-04 08:57:44 +01:00
dependencies.builder0.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
dependencies.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
dependencies.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
derivation-json.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
dummy	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
dump-db.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
eval-store.sh	Merge pull request #9589 from obsidiansystems/floating-content-addressing-derivations-eval-store	2024-03-04 07:15:08 +01:00
eval.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
eval.sh	libutil/args: warn on unknown settings after parsing all flags	2024-05-30 03:07:21 +00:00
experimental-features.sh	Rename nix show-config to nix config show	2024-05-03 16:26:16 +02:00
export-graph.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
export-graph.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
export.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
extra-sandbox-profile.nix	libstore: Fix sandbox=relaxed	2024-05-08 19:31:43 +00:00
extra-sandbox-profile.sh	libstore: Fix sandbox=relaxed	2024-05-08 19:31:43 +00:00
failing.nix	Merge pull request #9032 from Ma27/structured-attrs-env-vars	2024-03-07 10:46:47 +01:00
fetchClosure.sh	Add pre-commit checks	2024-03-29 22:57:40 -07:00
fetchGit.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
fetchGitRefs.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
fetchGitSubmodules.sh	libfetchers: represent unfetched submodules consistently	2024-06-18 00:54:51 +00:00
fetchMercurial.sh	tests: actually run mercurial tests	2024-05-04 16:48:07 +02:00
fetchPath.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
fetchTree-file.sh	chore: rebrand Nix to Lix when it makes sense	2024-06-01 20:31:24 +02:00
fetchurl.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
filter-source.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
filter-source.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
fixed.builder1.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
fixed.builder2.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
fixed.nix	Merge pull request #9662 from shlevy/flat-fixed-references-assert	2024-03-04 07:50:02 +01:00
fixed.sh	Merge pull request #9662 from shlevy/flat-fixed-references-assert	2024-03-04 07:50:02 +01:00
fmt.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
fmt.simple.sh	Add pre-commit checks	2024-03-29 22:57:40 -07:00
fod-failing.nix	nix3-build: show all FOD errors with --keep-going	2024-05-15 15:35:18 +02:00
function-trace.sh	Add pre-commit checks	2024-03-29 22:57:40 -07:00
gc-auto.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
gc-concurrent.builder.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
gc-concurrent.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
gc-concurrent.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
gc-concurrent2.builder.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
gc-non-blocking.sh	Merge pull request #8544 from edolstra/handle-missing-gc-socket	2024-03-07 00:43:51 -07:00
gc-runtime.nix	gc: Find roots using libproc on Darwin	2024-04-25 23:24:21 -04:00
gc-runtime.sh	gc: Find roots using libproc on Darwin	2024-04-25 23:24:21 -04:00
gc.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
hash-check.nix	Add pre-commit checks	2024-03-29 22:57:40 -07:00
hash.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
hermetic.nix	Merge pull request #9032 from Ma27/structured-attrs-env-vars	2024-03-07 10:46:47 +01:00
ifd.nix	Merge branch '2.18-maintenance' into ifd-buildStore-2.18	2024-01-11 07:21:16 -05:00
import-derivation.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
import-derivation.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
impure-derivations.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
impure-derivations.sh	chore: rebrand Nix to Lix when it makes sense	2024-06-01 20:31:24 +02:00
impure-eval.sh	Merge pull request #4093 from matthewbauer/eval-system	2024-03-04 07:21:01 +01:00
init.sh	Fix failing darwin tests	2024-05-06 18:56:40 +02:00
install-darwin.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
lang-test-infra.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
lang.sh	Add pre-commit checks	2024-03-29 22:57:40 -07:00
legacy-ssh-store.sh	"but doctor, I AM the untrusted store": nix doctor had wrong trustedness	2024-04-08 15:40:12 -07:00
linux-sandbox-cert-test.nix	Add pre-commit checks	2024-03-29 22:57:40 -07:00
linux-sandbox.sh	libstore/build: copy ca-certificates too	2024-05-31 07:54:18 +00:00
local-store.sh	"but doctor, I AM the untrusted store": nix doctor had wrong trustedness	2024-04-08 15:40:12 -07:00
logging.sh	Merge pull request #9637 from 9999years/fix-logging-test-on-macos	2024-03-04 07:26:34 +01:00
meson.build	Merge "Revert "libfetchers: make attribute / URL query handling consistent"" into main	2024-06-25 10:19:52 +00:00
misc.sh	libutil/args: warn on unknown settings after parsing all flags	2024-05-30 03:07:21 +00:00
multiple-outputs.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
multiple-outputs.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
nar-access.nix	Add pre-commit checks	2024-03-29 22:57:40 -07:00
nar-access.sh	Merge pull request #9288 from edolstra/fix-nar-access-test	2024-03-04 05:27:18 +01:00
nested-sandboxing.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
nix-build-examples.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
nix-build.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
nix-channel.sh	Add pre-commit checks	2024-03-29 22:57:40 -07:00
nix-collect-garbage-d.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
nix-copy-ssh-common.sh	Merge pull request #9604 from obsidiansystems/test-substitute-on-destination	2024-03-07 03:34:00 +01:00
nix-copy-ssh-ng.sh	Merge pull request #9604 from obsidiansystems/test-substitute-on-destination	2024-03-07 03:34:00 +01:00
nix-copy-ssh.sh	Merge pull request #9604 from obsidiansystems/test-substitute-on-destination	2024-03-07 03:34:00 +01:00
nix-daemon-untrusting.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
nix-profile.sh	libfetchers: log fetches by URL just before they happen	2024-05-24 15:15:42 -06:00
nix-shell.sh	Merge pull request #9930 from rvl/print-dev-env-unbound-variables	2024-03-04 08:51:49 +01:00
nix_path.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
optimise-store.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
output-normalization.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
parallel.builder.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
parallel.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
parallel.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
pass-as-file.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
path-from-hash-part.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
path.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
placeholders.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
plugins.sh	tests: don't build test plugin shared libs on static builds	2024-05-12 23:04:21 +02:00
post-hook.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
pure-eval.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
pure-eval.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
push-to-store-old.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
push-to-store.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
read-only-store.sh	Add pre-commit checks	2024-03-29 22:57:40 -07:00
readfile-context.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
readfile-context.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
recursive.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
recursive.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
referrers.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
remote-store.sh	"but doctor, I AM the untrusted store": nix doctor had wrong trustedness	2024-04-08 15:40:12 -07:00
repair.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
repl.sh	Pretty-print values in the REPL	2024-03-09 07:20:23 -07:00
restricted.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
restricted.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
search.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
search.sh	Merge pull request #9481 from iFreilicht/disallow-nix-search-without-search-terms	2024-03-04 08:40:12 +01:00
secure-drv-outputs.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
secure-drv-outputs.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
selfref-gc.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
shell-hello.nix	Merge pull request #9648 from cole-h/nix-shell-ordering	2024-03-04 07:50:55 +01:00
shell.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
shell.sh	Merge pull request #9648 from cole-h/nix-shell-ordering	2024-03-04 07:50:55 +01:00
shell.shebang.nix	Add pre-commit checks	2024-03-29 22:57:40 -07:00
shell.shebang.rb	Add pre-commit checks	2024-03-29 22:57:40 -07:00
shell.shebang.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
signing.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
simple-failing.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
simple.builder.sh	Add pre-commit checks	2024-03-29 22:57:40 -07:00
simple.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
simple.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
ssh-relay.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
store-ping.sh	Merge pull request #9632 from cole-h/nix-daemon-testing	2024-03-04 08:17:32 +01:00
structured-attrs-shell.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
structured-attrs.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
structured-attrs.sh	Merge pull request #9032 from Ma27/structured-attrs-env-vars	2024-03-07 10:46:47 +01:00
substitute-truncated-nar.sh	libutil: remove sinkToSource eof callback	2024-06-23 11:52:49 +00:00
substitute-with-invalid-ca.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
suggestions.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
supplementary-groups.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
symlink-derivation.nix	Merge pull request #10456 from NixOS/fixpermdeniedbind	2024-04-11 15:43:58 +03:00
tarball.sh	Merge pull request #9225 from drupol/nixpkgs-pr-107251/make-gnutar-reproducible	2024-03-04 05:34:19 +01:00
test-infra.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
test-libstoreconsumer.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
test-repl-characterization.sh	Implement a parser for a literate testing system for the repl	2024-03-14 14:30:38 -07:00
timeout.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
timeout.sh	tests: fix functional-timeout	2024-05-28 04:20:00 +00:00
toString-path.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
undefined-variable.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
user-envs-migration.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
user-envs.builder.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
user-envs.nix	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
user-envs.sh	Merge pull request #9257 from Artturin/nixenvjsondrvpath	2024-03-04 05:59:15 +01:00
why-depends.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00
zstd.sh	Put functional tests in tests/functional	2023-12-01 12:06:43 -05:00