cafkafk/nixpkgs
cafkafk/nixpkgs
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2

nixos/postgresql: convert manual chapter to MD

Browse source
This commit is contained in:
pennae 2023-01-03 00:58:36 +01:00
parent 5b012f2c55
commit 453b2bed05
3 changed files with 355 additions and 161 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

173
nixos/modules/services/databases/postgresql.md Normal file
View file

@ -0,0 +1,173 @@
# PostgreSQL {#module-postgresql}
<!-- FIXME: render nicely -->
<!-- FIXME: source can be added automatically -->
*Source:* {file}`modules/services/databases/postgresql.nix`
*Upstream documentation:* <http://www.postgresql.org/docs/>
<!-- FIXME: more stuff, like maintainer? -->
PostgreSQL is an advanced, free relational database.
<!-- MORE -->
## Configuring {#module-services-postgres-configuring}
To enable PostgreSQL, add the following to your {file}`configuration.nix`:
```
services.postgresql.enable = true;
services.postgresql.package = pkgs.postgresql_11;
```
Note that you are required to specify the desired version of PostgreSQL (e.g. `pkgs.postgresql_11`). Since upgrading your PostgreSQL version requires a database dump and reload (see below), NixOS cannot provide a default value for [](#opt-services.postgresql.package) such as the most recent release of PostgreSQL.
<!--
After running {command}`nixos-rebuild`, you can verify
whether PostgreSQL works by running {command}`psql`:
```ShellSession
$ psql
psql (9.2.9)
Type "help" for help.
alice=>
```
-->
By default, PostgreSQL stores its databases in {file}`/var/lib/postgresql/$psqlSchema`. You can override this using [](#opt-services.postgresql.dataDir), e.g.
```
services.postgresql.dataDir = "/data/postgresql";
```
## Upgrading {#module-services-postgres-upgrading}
::: {.note}
The steps below demonstrate how to upgrade from an older version to `pkgs.postgresql_13`.
These instructions are also applicable to other versions.
:::
Major PostgreSQL upgrades require a downtime and a few imperative steps to be called. This is the case because
each major version has some internal changes in the databases' state during major releases. Because of that,
NixOS places the state into {file}`/var/lib/postgresql/&lt;version&gt;` where each `version`
can be obtained like this:
```
$ nix-instantiate --eval -A postgresql_13.psqlSchema
"13"
```
For an upgrade, a script like this can be used to simplify the process:
```
{ config, pkgs, ... }:
{
environment.systemPackages = [
(let
# XXX specify the postgresql package you'd like to upgrade to.
# Do not forget to list the extensions you need.
newPostgres = pkgs.postgresql_13.withPackages (pp: [
# pp.plv8
]);
in pkgs.writeScriptBin "upgrade-pg-cluster" ''
set -eux
# XXX it's perhaps advisable to stop all services that depend on postgresql
systemctl stop postgresql
export NEWDATA="/var/lib/postgresql/${newPostgres.psqlSchema}"
export NEWBIN="${newPostgres}/bin"
export OLDDATA="${config.services.postgresql.dataDir}"
export OLDBIN="${config.services.postgresql.package}/bin"
install -d -m 0700 -o postgres -g postgres "$NEWDATA"
cd "$NEWDATA"
sudo -u postgres $NEWBIN/initdb -D "$NEWDATA"
sudo -u postgres $NEWBIN/pg_upgrade \
--old-datadir "$OLDDATA" --new-datadir "$NEWDATA" \
--old-bindir $OLDBIN --new-bindir $NEWBIN \
"$@"
'')
];
}
```
The upgrade process is:
1. Rebuild nixos configuration with the configuration above added to your {file}`configuration.nix`. Alternatively, add that into separate file and reference it in `imports` list.
2. Login as root (`sudo su -`)
3. Run `upgrade-pg-cluster`. It will stop old postgresql, initialize a new one and migrate the old one to the new one. You may supply arguments like `--jobs 4` and `--link` to speedup migration process. See <https://www.postgresql.org/docs/current/pgupgrade.html> for details.
4. Change postgresql package in NixOS configuration to the one you were upgrading to via [](#opt-services.postgresql.package). Rebuild NixOS. This should start new postgres using upgraded data directory and all services you stopped during the upgrade.
5. After the upgrade it's advisable to analyze the new cluster.
- For PostgreSQL ≥ 14, use the `vacuumdb` command printed by the upgrades script.
- For PostgreSQL < 14, run (as `su -l postgres` in the [](#opt-services.postgresql.dataDir), in this example {file}`/var/lib/postgresql/13`):
```
$ ./analyze_new_cluster.sh
```
::: {.warning}
The next step removes the old state-directory!
:::
```
$ ./delete_old_cluster.sh
```
## Options {#module-services-postgres-options}
A complete list of options for the PostgreSQL module may be found [here](#opt-services.postgresql.enable).
## Plugins {#module-services-postgres-plugins}
Plugins collection for each PostgreSQL version can be accessed with `.pkgs`. For example, for `pkgs.postgresql_11` package, its plugin collection is accessed by `pkgs.postgresql_11.pkgs`:
```ShellSession
$ nix repl '<nixpkgs>'
Loading '<nixpkgs>'...
Added 10574 variables.
nix-repl> postgresql_11.pkgs.<TAB><TAB>
postgresql_11.pkgs.cstore_fdw postgresql_11.pkgs.pg_repack
postgresql_11.pkgs.pg_auto_failover postgresql_11.pkgs.pg_safeupdate
postgresql_11.pkgs.pg_bigm postgresql_11.pkgs.pg_similarity
postgresql_11.pkgs.pg_cron postgresql_11.pkgs.pg_topn
postgresql_11.pkgs.pg_hll postgresql_11.pkgs.pgjwt
postgresql_11.pkgs.pg_partman postgresql_11.pkgs.pgroonga
...
```
To add plugins via NixOS configuration, set `services.postgresql.extraPlugins`:
```
services.postgresql.package = pkgs.postgresql_11;
services.postgresql.extraPlugins = with pkgs.postgresql_11.pkgs; [
pg_repack
postgis
];
```
You can build custom PostgreSQL-with-plugins (to be used outside of NixOS) using function `.withPackages`. For example, creating a custom PostgreSQL package in an overlay can look like:
```
self: super: {
postgresql_custom = self.postgresql_11.withPackages (ps: [
ps.pg_repack
ps.postgis
]);
}
```
Here's a recipe on how to override a particular plugin through an overlay:
```
self: super: {
postgresql_11 = super.postgresql_11.override { this = self.postgresql_11; } // {
pkgs = super.postgresql_11.pkgs // {
pg_repack = super.postgresql_11.pkgs.pg_repack.overrideAttrs (_: {
name = "pg_repack-v20181024";
src = self.fetchzip {
url = "https://github.com/reorg/pg_repack/archive/923fa2f3c709a506e111cc963034bf2fd127aa00.tar.gz";
sha256 = "17k6hq9xaax87yz79j773qyigm4fwk8z4zh5cyp6z0sxnwfqxxw5";
};
});
};
};
}
```

2
nixos/modules/services/databases/postgresql.nix
View file

@ -585,6 +585,8 @@ in
}; };
# Don't edit the docbook xml directly, edit the md and generate it:
# `pandoc postgresql.md -t docbook --top-level-division=chapter --extract-media=media -f markdown-smart --lua-filter ../../../../doc/build-aux/pandoc-filters/myst-reader/roles.lua --lua-filter ../../../../doc/build-aux/pandoc-filters/docbook-writer/rst-roles.lua > postgresql.xml`
meta.doc = ./postgresql.xml; meta.doc = ./postgresql.xml;
meta.maintainers = with lib.maintainers; [ thoughtpolice danbst ]; meta.maintainers = with lib.maintainers; [ thoughtpolice danbst ];
} }

341
nixos/modules/services/databases/postgresql.xml
View file

@ -1,74 +1,70 @@
<chapter xmlns="http://docbook.org/ns/docbook" <chapter xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="module-postgresql">
xmlns:xlink="http://www.w3.org/1999/xlink" <title>PostgreSQL</title>
xmlns:xi="http://www.w3.org/2001/XInclude"
version="5.0"
xml:id="module-postgresql">
<title>PostgreSQL</title>
<!-- FIXME: render nicely -->
<!-- FIXME: source can be added automatically -->
<para>
<emphasis>Source:</emphasis> <filename>modules/services/databases/postgresql.nix</filename>
</para>
<para>
<emphasis>Upstream documentation:</emphasis> <link xlink:href="http://www.postgresql.org/docs/"/>
</para>
<!-- FIXME: more stuff, like maintainer? -->
<para>
PostgreSQL is an advanced, free relational database.
<!-- MORE -->
</para>
<section xml:id="module-services-postgres-configuring">
<title>Configuring</title>
<para> <para>
To enable PostgreSQL, add the following to your <filename>configuration.nix</filename>: <emphasis>Source:</emphasis>
<programlisting> <filename>modules/services/databases/postgresql.nix</filename>
</para>
<para>
<emphasis>Upstream documentation:</emphasis>
<link xlink:href="http://www.postgresql.org/docs/" role="uri">http://www.postgresql.org/docs/</link>
</para>
<para>
PostgreSQL is an advanced, free relational database. <!-- MORE -->
</para>
<section xml:id="module-services-postgres-configuring">
<title>Configuring</title>
<para>
To enable PostgreSQL, add the following to your
<filename>configuration.nix</filename>:
</para>
<programlisting>
services.postgresql.enable = true; services.postgresql.enable = true;
services.postgresql.package = pkgs.postgresql_11; services.postgresql.package = pkgs.postgresql_11;
</programlisting> </programlisting>
Note that you are required to specify the desired version of PostgreSQL (e.g. <literal>pkgs.postgresql_11</literal>). Since upgrading your PostgreSQL version requires a database dump and reload (see below), NixOS cannot provide a default value for <xref linkend="opt-services.postgresql.package"/> such as the most recent release of PostgreSQL. <para>
</para> Note that you are required to specify the desired version of
PostgreSQL (e.g. <literal>pkgs.postgresql_11</literal>). Since
<!-- upgrading your PostgreSQL version requires a database dump and
<para>After running <command>nixos-rebuild</command>, you can verify reload (see below), NixOS cannot provide a default value for
whether PostgreSQL works by running <command>psql</command>: <xref linkend="opt-services.postgresql.package"></xref> such as
the most recent release of PostgreSQL.
<screen> </para>
<prompt>$ </prompt>psql <para>
psql (9.2.9) By default, PostgreSQL stores its databases in
Type "help" for help. <filename>/var/lib/postgresql/$psqlSchema</filename>. You can
override this using
<prompt>alice=></prompt> <xref linkend="opt-services.postgresql.dataDir"></xref>, e.g.
</screen> </para>
--> <programlisting>
services.postgresql.dataDir = &quot;/data/postgresql&quot;;
<para>
By default, PostgreSQL stores its databases in <filename>/var/lib/postgresql/$psqlSchema</filename>. You can override this using <xref linkend="opt-services.postgresql.dataDir"/>, e.g.
<programlisting>
services.postgresql.dataDir = "/data/postgresql";
</programlisting> </programlisting>
</para> </section>
</section> <section xml:id="module-services-postgres-upgrading">
<section xml:id="module-services-postgres-upgrading"> <title>Upgrading</title>
<title>Upgrading</title> <note>
<para>
<note> The steps below demonstrate how to upgrade from an older version
<para> to <literal>pkgs.postgresql_13</literal>. These instructions are
The steps below demonstrate how to upgrade from an older version to <literal>pkgs.postgresql_13</literal>. also applicable to other versions.
These instructions are also applicable to other versions. </para>
</para> </note>
</note> <para>
<para> Major PostgreSQL upgrades require a downtime and a few imperative
Major PostgreSQL upgrades require a downtime and a few imperative steps to be called. This is the case because steps to be called. This is the case because each major version
each major version has some internal changes in the databases' state during major releases. Because of that, has some internal changes in the databases' state during major
NixOS places the state into <filename>/var/lib/postgresql/&lt;version&gt;</filename> where each <literal>version</literal> releases. Because of that, NixOS places the state into
can be obtained like this: <filename>/var/lib/postgresql/&lt;version&gt;</filename> where
<programlisting> each <literal>version</literal> can be obtained like this:
<prompt>$ </prompt>nix-instantiate --eval -A postgresql_13.psqlSchema </para>
"13" <programlisting>
$ nix-instantiate --eval -A postgresql_13.psqlSchema
&quot;13&quot;
</programlisting> </programlisting>
For an upgrade, a script like this can be used to simplify the process: <para>
<programlisting> For an upgrade, a script like this can be used to simplify the
process:
</para>
<programlisting>
{ config, pkgs, ... }: { config, pkgs, ... }:
{ {
environment.systemPackages = [ environment.systemPackages = [
@ -78,104 +74,126 @@ services.postgresql.dataDir = "/data/postgresql";
newPostgres = pkgs.postgresql_13.withPackages (pp: [ newPostgres = pkgs.postgresql_13.withPackages (pp: [
# pp.plv8 # pp.plv8
]); ]);
in pkgs.writeScriptBin "upgrade-pg-cluster" '' in pkgs.writeScriptBin &quot;upgrade-pg-cluster&quot; ''
set -eux set -eux
# XXX it's perhaps advisable to stop all services that depend on postgresql # XXX it's perhaps advisable to stop all services that depend on postgresql
systemctl stop postgresql systemctl stop postgresql
export NEWDATA="/var/lib/postgresql/${newPostgres.psqlSchema}" export NEWDATA=&quot;/var/lib/postgresql/${newPostgres.psqlSchema}&quot;
export NEWBIN="${newPostgres}/bin" export NEWBIN=&quot;${newPostgres}/bin&quot;
export OLDDATA="${config.services.postgresql.dataDir}" export OLDDATA=&quot;${config.services.postgresql.dataDir}&quot;
export OLDBIN="${config.services.postgresql.package}/bin" export OLDBIN=&quot;${config.services.postgresql.package}/bin&quot;
install -d -m 0700 -o postgres -g postgres "$NEWDATA" install -d -m 0700 -o postgres -g postgres &quot;$NEWDATA&quot;
cd "$NEWDATA" cd &quot;$NEWDATA&quot;
sudo -u postgres $NEWBIN/initdb -D "$NEWDATA" sudo -u postgres $NEWBIN/initdb -D &quot;$NEWDATA&quot;
sudo -u postgres $NEWBIN/pg_upgrade \ sudo -u postgres $NEWBIN/pg_upgrade \
--old-datadir "$OLDDATA" --new-datadir "$NEWDATA" \ --old-datadir &quot;$OLDDATA&quot; --new-datadir &quot;$NEWDATA&quot; \
--old-bindir $OLDBIN --new-bindir $NEWBIN \ --old-bindir $OLDBIN --new-bindir $NEWBIN \
"$@" &quot;$@&quot;
'') '')
]; ];
} }
</programlisting> </programlisting>
</para>
<para>
The upgrade process is:
</para>
<orderedlist>
<listitem>
<para> <para>
Rebuild nixos configuration with the configuration above added to your <filename>configuration.nix</filename>. Alternatively, add that into separate file and reference it in <literal>imports</literal> list. The upgrade process is:
</para> </para>
</listitem> <orderedlist numeration="arabic">
<listitem> <listitem>
<para> <para>
Login as root (<literal>sudo su -</literal>) Rebuild nixos configuration with the configuration above added
</para> to your <filename>configuration.nix</filename>. Alternatively,
</listitem> add that into separate file and reference it in
<listitem> <literal>imports</literal> list.
<para> </para>
Run <literal>upgrade-pg-cluster</literal>. It will stop old postgresql, initialize a new one and migrate the old one to the new one. You may supply arguments like <literal>--jobs 4</literal> and <literal>--link</literal> to speedup migration process. See <link xlink:href="https://www.postgresql.org/docs/current/pgupgrade.html" /> for details. </listitem>
</para> <listitem>
</listitem> <para>
<listitem> Login as root (<literal>sudo su -</literal>)
<para> </para>
Change postgresql package in NixOS configuration to the one you were upgrading to via <xref linkend="opt-services.postgresql.package" />. Rebuild NixOS. This should start new postgres using upgraded data directory and all services you stopped during the upgrade. </listitem>
</para> <listitem>
</listitem> <para>
<listitem> Run <literal>upgrade-pg-cluster</literal>. It will stop old
<para> postgresql, initialize a new one and migrate the old one to
After the upgrade it's advisable to analyze the new cluster. the new one. You may supply arguments like
</para> <literal>--jobs 4</literal> and <literal>--link</literal> to
<itemizedlist> speedup migration process. See
<listitem> <link xlink:href="https://www.postgresql.org/docs/current/pgupgrade.html" role="uri">https://www.postgresql.org/docs/current/pgupgrade.html</link>
<para> for details.
For PostgreSQL ≥ 14, use the <literal>vacuumdb</literal> command printed by the upgrades script. </para>
</para> </listitem>
</listitem> <listitem>
<listitem> <para>
<para> Change postgresql package in NixOS configuration to the one
For PostgreSQL &lt; 14, run (as <literal>su -l postgres</literal> in the <xref linkend="opt-services.postgresql.dataDir" />, in this example <filename>/var/lib/postgresql/13</filename>): you were upgrading to via
<programlisting> <xref linkend="opt-services.postgresql.package"></xref>.
<prompt>$ </prompt>./analyze_new_cluster.sh Rebuild NixOS. This should start new postgres using upgraded
data directory and all services you stopped during the
upgrade.
</para>
</listitem>
<listitem>
<para>
After the upgrade it's advisable to analyze the new cluster.
</para>
<itemizedlist>
<listitem>
<para>
For PostgreSQL ≥ 14, use the <literal>vacuumdb</literal>
command printed by the upgrades script.
</para>
</listitem>
<listitem>
<para>
For PostgreSQL &lt; 14, run (as
<literal>su -l postgres</literal> in the
<xref linkend="opt-services.postgresql.dataDir"></xref>,
in this example
<filename>/var/lib/postgresql/13</filename>):
</para>
<programlisting>
$ ./analyze_new_cluster.sh
</programlisting> </programlisting>
</para> </listitem>
</listitem> </itemizedlist>
</itemizedlist> <warning>
<para> <para>
<warning><para>The next step removes the old state-directory!</para></warning> The next step removes the old state-directory!
<programlisting> </para>
<prompt>$ </prompt>./delete_old_cluster.sh </warning>
<programlisting>
$ ./delete_old_cluster.sh
</programlisting> </programlisting>
</listitem>
</orderedlist>
</section>
<section xml:id="module-services-postgres-options">
<title>Options</title>
<para>
A complete list of options for the PostgreSQL module may be found
<link linkend="opt-services.postgresql.enable">here</link>.
</para> </para>
</listitem> </section>
</orderedlist> <section xml:id="module-services-postgres-plugins">
</section> <title>Plugins</title>
<section xml:id="module-services-postgres-options"> <para>
<title>Options</title> Plugins collection for each PostgreSQL version can be accessed
with <literal>.pkgs</literal>. For example, for
<para> <literal>pkgs.postgresql_11</literal> package, its plugin
A complete list of options for the PostgreSQL module may be found <link linkend="opt-services.postgresql.enable">here</link>. collection is accessed by
</para> <literal>pkgs.postgresql_11.pkgs</literal>:
</section> </para>
<section xml:id="module-services-postgres-plugins"> <programlisting>
<title>Plugins</title> $ nix repl '&lt;nixpkgs&gt;'
<para>
Plugins collection for each PostgreSQL version can be accessed with <literal>.pkgs</literal>. For example, for <literal>pkgs.postgresql_11</literal> package, its plugin collection is accessed by <literal>pkgs.postgresql_11.pkgs</literal>:
<screen>
<prompt>$ </prompt>nix repl '&lt;nixpkgs&gt;'
Loading '&lt;nixpkgs&gt;'... Loading '&lt;nixpkgs&gt;'...
Added 10574 variables. Added 10574 variables.
<prompt>nix-repl&gt; </prompt>postgresql_11.pkgs.&lt;TAB&gt;&lt;TAB&gt; nix-repl&gt; postgresql_11.pkgs.&lt;TAB&gt;&lt;TAB&gt;
postgresql_11.pkgs.cstore_fdw postgresql_11.pkgs.pg_repack postgresql_11.pkgs.cstore_fdw postgresql_11.pkgs.pg_repack
postgresql_11.pkgs.pg_auto_failover postgresql_11.pkgs.pg_safeupdate postgresql_11.pkgs.pg_auto_failover postgresql_11.pkgs.pg_safeupdate
postgresql_11.pkgs.pg_bigm postgresql_11.pkgs.pg_similarity postgresql_11.pkgs.pg_bigm postgresql_11.pkgs.pg_similarity
@ -183,23 +201,25 @@ postgresql_11.pkgs.pg_cron postgresql_11.pkgs.pg_topn
postgresql_11.pkgs.pg_hll postgresql_11.pkgs.pgjwt postgresql_11.pkgs.pg_hll postgresql_11.pkgs.pgjwt
postgresql_11.pkgs.pg_partman postgresql_11.pkgs.pgroonga postgresql_11.pkgs.pg_partman postgresql_11.pkgs.pgroonga
... ...
</screen> </programlisting>
</para> <para>
To add plugins via NixOS configuration, set
<para> <literal>services.postgresql.extraPlugins</literal>:
To add plugins via NixOS configuration, set <literal>services.postgresql.extraPlugins</literal>: </para>
<programlisting> <programlisting>
services.postgresql.package = pkgs.postgresql_11; services.postgresql.package = pkgs.postgresql_11;
services.postgresql.extraPlugins = with pkgs.postgresql_11.pkgs; [ services.postgresql.extraPlugins = with pkgs.postgresql_11.pkgs; [
pg_repack pg_repack
postgis postgis
]; ];
</programlisting> </programlisting>
</para> <para>
You can build custom PostgreSQL-with-plugins (to be used outside
<para> of NixOS) using function <literal>.withPackages</literal>. For
You can build custom PostgreSQL-with-plugins (to be used outside of NixOS) using function <literal>.withPackages</literal>. For example, creating a custom PostgreSQL package in an overlay can look like: example, creating a custom PostgreSQL package in an overlay can
<programlisting> look like:
</para>
<programlisting>
self: super: { self: super: {
postgresql_custom = self.postgresql_11.withPackages (ps: [ postgresql_custom = self.postgresql_11.withPackages (ps: [
ps.pg_repack ps.pg_repack
@ -207,25 +227,24 @@ self: super: {
]); ]);
} }
</programlisting> </programlisting>
</para> <para>
Here's a recipe on how to override a particular plugin through an
<para> overlay:
Here's a recipe on how to override a particular plugin through an overlay: </para>
<programlisting> <programlisting>
self: super: { self: super: {
postgresql_11 = super.postgresql_11.override { this = self.postgresql_11; } // { postgresql_11 = super.postgresql_11.override { this = self.postgresql_11; } // {
pkgs = super.postgresql_11.pkgs // { pkgs = super.postgresql_11.pkgs // {
pg_repack = super.postgresql_11.pkgs.pg_repack.overrideAttrs (_: { pg_repack = super.postgresql_11.pkgs.pg_repack.overrideAttrs (_: {
name = "pg_repack-v20181024"; name = &quot;pg_repack-v20181024&quot;;
src = self.fetchzip { src = self.fetchzip {
url = "https://github.com/reorg/pg_repack/archive/923fa2f3c709a506e111cc963034bf2fd127aa00.tar.gz"; url = &quot;https://github.com/reorg/pg_repack/archive/923fa2f3c709a506e111cc963034bf2fd127aa00.tar.gz&quot;;
sha256 = "17k6hq9xaax87yz79j773qyigm4fwk8z4zh5cyp6z0sxnwfqxxw5"; sha256 = &quot;17k6hq9xaax87yz79j773qyigm4fwk8z4zh5cyp6z0sxnwfqxxw5&quot;;
}; };
}); });
}; };
}; };
} }
</programlisting> </programlisting>
</para> </section>
</section>
</chapter> </chapter>