a67950f20b added `url` attribute
from `fetchurl` and therefore also from `fetchzip`.
We previously relied on `url` from fetchgit-based fetchers
to find the repo URL but now it will just return tarballs
in the case of `fetchFrom{GitHub,GitLab}`.
Let’s add an attribute to `fetch{git,FromGitHub,FromGitLab}`
to expose a repo URL consistently.
That way gitUpdater could be used to extract most recent published
releases for .nix files that don't use git directly to define builds.
An example is iproute2 package which does not have a single place
to lookup most recent release.
- Add the 'gitUpdater' helper function to update git based packages, using the
'genericUpdater' function.
- Rework argument passing to the `list-git-tags' and 'list-archive-two-level-versions' scripts.
- Replace 'genericUpdater' plus 'list-git-tags' by 'gitUpdater'
It is important to keep the version as parsed by `builtins.parseDrvName`
monotonic for packages that often switch between stable and unstable versions,
for nix-env to be able to update them reliably.
Let’s optionally use the version format described by RFC 107:
https://github.com/NixOS/rfcs/pull/107
The downside is that it requires fetching
more commits to be able to determine the latest tag.
When `fetchFromGitHub` uses `fetchzip` instead of `fetchgit` internally,
which is the most common use case, there is no `src.url` attribute
so the update will fail.
Let’s fix that by falling back to `src.meta.homepage`,
which these fetchers set to the repository URL.
In flakes, packages are not exposed directly but instead they are declared
inside “packages” or “legacyPackages” output under their host platform.
flake-compat reflects this.
Let’s look for an attribute also in these outputs if the direct lookup fails.
When update-source-version is run in a repo with flake-compat,
it would find the files in Nix store.
Let’s replace them with the local path of the repository.
Adds a --rev=<revision> parameter to the script that makes it possible
to explicitly specify a new revision.
Useful to update unstable packages, where the version and revision may
be independent.
This improves on the previous verison of this script.
Previously it only accepted one attr, and required
explicit passing of the "broken" value.
This script is meant to be used to mark failing hydra builds as broken in the meta attrs
To use the script, you should pass the list of failing attrs as arguments to the script.
Example: `cat failing-attrs | xargs ./pkgs/common-update/scripts/mark-broken`
Generating a list of failing attrs: (this should be improved at a later date)
- Go to the most recent hydra evaluation with all builds completed
- Select the "builds still failing" tab
- Highlight and select all packages, should be prefixed with `nixpkgs.`
- Use regex and editor foo to leave only the attr names
- Use the above example command to then execute the script
OTHER NOTES:
- The `denyFileList` and `denyAttrList` will likely need to be updated slightly
to align with the conventions used in nixpkgs at execution time
- Any attrs which failed for any reason will be written to `failed-marks.txt`.
Those attrs will likely need manual attention as disablement will likely be conditional.
- updateScript:
A nix expression that can be used in passThrough to update a package
- list-git-tags:
A shell script to list available tags in a git repository
- list-archive-two-level-versions:
A shell script to list available versions in a web site in two
levels: there is a page listing the available major.minor versions,
and for each of them there is another page listings the patch level
versions major.minor.patch.
It is suitable for Xfce packages for instance.
How the updater works:
1. collect the available versions from the source repository (using a
script given as argument)
2. print the collected versions (for debugging)
3. (optionally) apply some transformation to the collected versions to
make them compatible with the versions used in nixpkgs (for instance,
tags in the Xfce git repository may be prefixed with the package name,
and the prefix need to be removed)
4. sort the available versions in decreasing order
5. choose the first version that pass validation:
- check if the version may be a development version
- if the version IS unstable, skip it and give a warning about
skipping a development version (for debugging)
- if the version COULD BE unstable, take it and give a warning
about taking a potential development version (for debugging)
- if the version IS stable, take it
6. update the package version and checksum in its nix expression
7. print the git commands for adding the modified files and for
committing the changes
Previously, when downloading src failed for other reason than hash mismatch,
the error ended up in newHash. This made evaluation fail since the error message
is not valid hash. Now the failure will make newHash empty.
It is also much cleaner than previously since \K is very cool thing
and we no longer grep for legacy messages.
Fix issues reported by shellcheck and few other style issues.
Though we need to ignore $systemArg complaints because Nix does not support passing --system as a single argument.
Some fetcher functions support SRI-style `hash` attribute in addition to legacy type-specific attributes. When `hash` is used `outputHashAlgo` is null so let’s complain when SRI-style hash value was not detected.
Such attributes match the form ${type}${separator}${hash}: True SRI uses dash as a separator and only supports base64, whereas Nix’s SRI-style format uses a colon and supports all the same encodings like regular hashes (16/32/64).
To keep this program reasonably simple, we will upgrade Nix’s SRI-like format to pure SRI instead of preserving it.
--ignore-same-hash allows ignoring if the same has is used.
This gives the ability to run update-source-version multiple times to
change multiple keys.
Fix version-key, which was not being used to determine the oldVersion
variable.
If $attr.src.drvAttrs.urls is not found, look for
$attr.src.drvAttrs.url since it is being used like that in fetchgit
You can now optionally invoke update-source-versions with:
* --system flag changing the host platform, to be passed dirrectly to Nix commands.
This is useful for binary packages which have different sources for each platform.
* --file flag allowing to change the file to be modified. This is useful for packages
that offer multiple variants, listed in a different file than the derivation itself;
e.g. packages.nix of Sublime Text 3.
* --version-key, which is now a keyword flag instead of a positional argument.
