Commit graph

18991 commits

Author SHA1 Message Date
Andreas Rammhold
2aa64e5df5
nixos/unbound: add option to configure the local control socket path
This option allows users to specify a local UNIX control socket to
"remote control" the daemon. System users, that should be permitted to
access the daemon, must be in the `unbound` group in order to access the
socket. When a socket path is configured we are also creating the
required group.

Currently this only supports the UNIX socket mode while unbound actually
supports more advanced types. Users are still able to configure more
complex scenarios via the `extraConfig` attribute.

When this option is set to `null` (the default) it doesn't affect the
system configuration at all. The unbound defaults for control sockets
apply and no additional groups are created.
2020-11-03 19:21:25 +01:00
Andreas Rammhold
b67cc6298e
nixos/tests/unbound: add test to verify control sockets work 2020-11-03 19:21:24 +01:00
Andreas Rammhold
a040a8a2e3
nixos/tests/unbound: init 2020-11-03 19:21:24 +01:00
Andreas Rammhold
aadc07618a
nixos/unbound: drop ReadWritePaths from systemd unit configuration
Both of the configured paths should be implicit due to RuntimeDirectory
& StateDirectory.
2020-11-03 19:21:24 +01:00
Andreas Rammhold
72fbf05c17
nixos/unbound: note about the AmbientCapabilities 2020-11-03 19:21:24 +01:00
Andreas Rammhold
5e602f88d1
nixos/modules/services/networking/unbound: update systemd unit
Previously we just applied a very minimal set of restrictions and
trusted unbound to properly drop root privs and capabilities.

With this change I am (for the most part) just using the upstream
example unit file for unbound. The main difference is that we start
unbound was `unbound` user with the required capabilities instead of
letting unbound do the chroot & uid/gid changes.

The upstream unit configuration this is based on is a lot stricter with
all kinds of permissions then our previous variant. It also came with
the default of having the `Type` set to `notify`, therefore we are also
using the `unbound-with-systemd` package here. Unbound will start up,
read the configuration files and start listening on the configured ports
before systemd will declare the unit "running". This will likely help
with startup order and the occasional race condition during system
activation where the DNS service is started but not yet ready to answer
queries.

Aditionally to the much stricter runtime environmet I removed the
`/dev/urandom` mount lines we previously had in the code (that would
randomly fail during `stop`-phase).

The `preStart` script is now only required if we enabled the trust
anchor updates (which are still enabled by default).

Another beneefit of the refactoring is that we can now issue reloads via
either `pkill -HUP unbound` or `systemctl reload unbound` to reload the
running configuration without taking the daemon offline. A prerequisite
of this was that unbound configuration is available on a well known path
on the file system. I went for /etc/unbound/unbound.conf as that is the
default in the CLI tooling which in turn enables us to use
`unbound-control` without passing a custom configuration location.
2020-11-03 19:21:24 +01:00
Silvan Mosberger
8a7ea52173
Merge pull request #99019 from sumnerevans/master
Add ability to configure executable for redshift service
2020-11-03 01:00:40 +01:00
Silvan Mosberger
aeaf78adb8
Merge pull request #102204 from danderson/danderson/transmission-dir
nixos/transmission: point at the settings dir in cfg.home.
2020-11-03 00:45:04 +01:00
Graham Christensen
75a2bc94fa
Merge pull request #101192 from grahamc/nixpkgs-location-basic-auth
nginx: support basic auth in location blocks
2020-11-02 09:44:54 -05:00
Graham Christensen
3361a037b9
nginx: add a warning that nginx's basic auth isn't very good. 2020-11-02 08:16:01 -05:00
Graham Christensen
a4b86b2bf5
nginx: test basic auth 2020-11-02 08:16:01 -05:00
Graham Christensen
c7bf3828f0
nginx: add basic auth support for locations 2020-11-02 08:16:00 -05:00
Graham Christensen
33cf4f0e8e
nginx: factor out the generation of basic auth generation 2020-11-02 08:16:00 -05:00
Andreas Rammhold
2ba1c007f6
Merge pull request #102350 from andir/nixos-test-prometheus
nixos/tests/prometheus: remove invalid thanos config flag
2020-11-01 19:45:00 +01:00
Maximilian Bosch
4f3f06d070
Merge pull request #101553 from Mic92/nextcloud
Nextcloud: fix ldap integration
2020-11-01 16:10:18 +01:00
Andreas Rammhold
e4865130cf
nixos/tests/prometheus: remove invalid thanos config flag
Upstream has apparently changed the configuration format and is now
throwing an error when the `encrypt_sse` option is set. According to the
current version of the documentation encryption moved to the
`sse_config` option that (is optional and) offers all the features we do
not use or care about for this test.
2020-11-01 14:33:11 +01:00
Jörg Thalheim
7b5cebfa71
Merge pull request #102237 from oxzi/tlp-deprecation-note
nixos/tlp: Fix deprecation hint
2020-11-01 11:46:11 +01:00
Frederik Rietdijk
54f7498601
Merge pull request #101369 from doronbehar/pkg/kdeApplications/qt515
kdeApplications: Use latest qt515 by default
2020-11-01 11:05:05 +01:00
Andreas Rammhold
fd0b3839b2
Merge pull request #102249 from rnhmjoj/firefox-audio
nixos/tests/firefox: add audio subtest
2020-10-31 21:23:10 +01:00
zimbatm
7d834eff6c
nixos/manual: make reproducible (#102234) 2020-10-31 21:18:16 +01:00
rnhmjoj
f7904ca45b
nixos/tests/firefox: add audio subtest 2020-10-31 20:53:15 +01:00
Philipp Kern
ec6b0950ef
nixos/prometheus: Support environmentFile (#97933)
For the same reason Alertmanager supports environmentFile to pass
secrets along, it is useful to support the same for Prometheus'
configuration to store bearer tokens outside the Nix store.
2020-10-31 20:52:13 +01:00
WORLDofPEACE
eaaf9254aa
Merge pull request #100520 from hyperfekt/patch-3
nixos-install: add passthrough --keep-going flag
2020-10-31 15:19:51 -04:00
Niklas Hambüchen
441abe9949 release notes: Document deprecation warning for StartLimitInterval in [Service] 2020-10-31 18:11:03 +01:00
hyperfekt
1338647a8c nixos-install: pass through keep-going flag 2020-10-31 17:13:45 +01:00
lf-
644079e707 nixos/modules: deprecation warning for StartLimitInterval in [Service]
This implements
https://github.com/NixOS/nixpkgs/issues/45786#issuecomment-440091879
2020-10-31 16:50:35 +01:00
Niklas Hambüchen
c178fe4bbb nixos/modules: Reformat warnings section 2020-10-31 16:50:25 +01:00
Alvar Penning
0ad1519ad9 nixos/tlp: Fix deprecation hint
The deprecated extraConfig option refers to the config option, which
does not exists. The settings option should be used.
2020-10-31 16:33:45 +01:00
lf-
b37bbca521 nixos/modules: fix systemd start rate-limits
These were broken since 2016:
f0367da7d1
since StartLimitIntervalSec got moved into [Unit] from [Service].
StartLimitBurst has also been moved accordingly, so let's fix that one
too.

NixOS systems have been producing logs such as:
/nix/store/wf98r55aszi1bkmln1lvdbp7znsfr70i-unit-caddy.service/caddy.service:31:
Unknown key name 'StartLimitIntervalSec' in section 'Service', ignoring.

I have also removed some unnecessary duplication in units disabling
rate limiting since setting either interval or burst to zero disables it
(ad16158c10/src/basic/ratelimit.c (L16))
2020-10-31 01:35:56 -07:00
Jade
2df221ec8a
nixos/postgresql: fix inaccurate docs for authentication (#97622)
* nixos/postgresql: fix inaccurate docs for authentication

We actually use peer authentication, then md5 based authentication.
trust is not used.

* Use a link for mkForce docs

Co-authored-by: aszlig <aszlig@redmoonstudios.org>

Co-authored-by: lf- <lf-@users.noreply.github.com>
Co-authored-by: aszlig <aszlig@redmoonstudios.org>
2020-10-31 03:35:19 -04:00
WORLDofPEACE
7b3b82f7af
Merge pull request #100136 from xaverdh/nixos-install-support-impure
nixos-install: pass through impure flag
2020-10-31 01:17:07 -04:00
David Anderson
43effbbc59 nixos/transmission: point at the settings dir in cfg.home.
Without this, transmission starts with an empty config when using
a custom home location.

Signed-off-by: David Anderson <dave@natulte.net>
2020-10-30 19:03:42 -07:00
David Anderson
9a8d6011aa nixos/tailscale: add tailscale to environment.systemPackages.
Use of Tailscale requires using the `tailscale` CLI to talk to the
daemon. If the CLI isn't in systemPackages, the resulting user experience
is confusing as the Tailscale daemon does nothing.

Signed-off-by: David Anderson <dave@natulte.net>
2020-10-30 17:58:14 -07:00
Graham Christensen
38a394bdee
Merge pull request #102174 from grahamc/ami-root-use-gpt
AMI root partition table: use GPT to support >2T partitions
2020-10-30 16:14:37 -04:00
Graham Christensen
860a3a23c6
Merge pull request #102175 from grahamc/ami-random
amazon-image: random.trust_cpu=on to cut 10s from boot
2020-10-30 16:13:41 -04:00
Graham Christensen
c06b97175b
Merge pull request #102173 from grahamc/create-amis
create-amis.sh: fixup shellcheck issues, improve error logging, and add configurable service names
2020-10-30 16:13:18 -04:00
Graham Christensen
82578fc725
Merge pull request #102172 from grahamc/stage-1-datestamps
stage-1: add datestamps to logs
2020-10-30 16:13:02 -04:00
Graham Christensen
b34cf366aa
Merge pull request #102171 from grahamc/faster-ext-resize
stage-1: modprobe ext{2,3,4} before resizing (so resizing takes less than 45 minutes)
2020-10-30 16:12:50 -04:00
WORLDofPEACE
214af51225
Merge pull request #101067 from deviant/remove-caddy-agree
nixos/caddy: remove services.caddy.agree
2020-10-30 16:02:44 -04:00
Graham Christensen
d77ddf2a40
nixos.amazonAmi: use legacy+gpt disk images to support partitions >2T 2020-10-30 15:50:25 -04:00
Graham Christensen
d78aa080f5
make-disk-image: support legacy+gpt 2020-10-30 15:50:24 -04:00
Doron Behar
77e081bb2b nixos/sddm: Use libsForQt514.sddm if needed (for lxqt)
Currently lxqt is a desktop environment that's compiled against qt514.
To avoid possible issues (#101369), we (hopefully) use the same qt
version as the desktop environment at hand. LXQT should move to qt515,
and for the long term the correct qt version should be inherited by the
sddm module.
2020-10-30 20:37:59 +02:00
Doron Behar
e681f442c9 nixos/plasma: Fix attribute path to kinit 2020-10-30 20:37:58 +02:00
Graham Christensen
c851030763
amazon-image: random.trust_cpu=on to cut 10s from boot
Ubuntu and other distros already have this set via kernel config.
2020-10-30 13:45:19 -04:00
Graham Christensen
74a577b293
create-amis: improve wording around the service name's IAM role
Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2020-10-30 12:40:17 -04:00
Graham Christensen
ece5c0f304
stage-1: modprobe ext{2,3,4} before resizing
I noticed booting a system with an ext4 root which expanded to 5T took
quite a long time (12 minutes in some cases, 43(!) in others.)

I changed stage-1 to run `resize2fs -d 62` for extra debug output and
timing information. It revealed the adjust_superblock step taking
almost all of the time:

    [Fri Oct 30 11:10:15 UTC 2020] zero_high_bits_in_metadata: Memory used: 132k/0k (63k/70k), time:  0.00/ 0.00/ 0.00
    [Fri Oct 30 11:21:09 UTC 2020] adjust_superblock: Memory used: 396k/4556k (295k/102k), time: 654.21/ 0.59/ 5.13

but when I ran resize2fs on a disk with the identical content growing
to the identical target size, it would only take about 30 seconds. I
looked at what happened between those two steps in the fast case with
strace and found:

```
   235	getrusage(RUSAGE_SELF, {ru_utime={tv_sec=0, tv_usec=1795}, ru_stime={tv_sec=0, tv_usec=3590}, ...}) = 0
   236	write(1, "zero_high_bits_in_metadata: Memo"..., 84zero_high_bits_in_metadata: Memory used: 132k/0k (72k/61k), time:  0.00/ 0.00/ 0.00
   237	) = 84
   238	gettimeofday({tv_sec=1604061278, tv_usec=480147}, NULL) = 0
   239	getrusage(RUSAGE_SELF, {ru_utime={tv_sec=0, tv_usec=1802}, ru_stime={tv_sec=0, tv_usec=3603}, ...}) = 0
   240	gettimeofday({tv_sec=1604061278, tv_usec=480192}, NULL) = 0
   241	mmap(NULL, 2564096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fa3c7355000
   242	access("/sys/fs/ext4/features/lazy_itable_init", F_OK) = 0
   243	brk(0xf85000)                           = 0xf85000
   244	brk(0xfa6000)                           = 0xfa6000
   245	gettimeofday({tv_sec=1604061278, tv_usec=538828}, NULL) = 0
   246	getrusage(RUSAGE_SELF, {ru_utime={tv_sec=0, tv_usec=58720}, ru_stime={tv_sec=0, tv_usec=3603}, ...}) = 0
   247	write(1, "adjust_superblock: Memory used: "..., 79adjust_superblock: Memory used: 396k/2504k (305k/92k), time:  0.06/ 0.06/ 0.00
   248	) = 79
   249	gettimeofday({tv_sec=1604061278, tv_usec=539119}, NULL) = 0
   250	getrusage(RUSAGE_SELF, {ru_utime={tv_sec=0, tv_usec=58812}, ru_stime={tv_sec=0, tv_usec=3603}, ...}) = 0
   251	gettimeofday({tv_sec=1604061279, tv_usec=939}, NULL) = 0
   252	getrusage(RUSAGE_SELF, {ru_utime={tv_sec=0, tv_usec=520411}, ru_stime={tv_sec=0, tv_usec=3603}, ...}) = 0
   253	write(1, "fix_uninit_block_bitmaps 2: Memo"..., 88fix_uninit_block_bitmaps 2: Memory used: 396k/2504k (305k/92k), time:  0.46/ 0.46/ 0.00
   254	) = 88
```

In particular the access to /sys/fs seemed interesting. Looking
at the source of resize2fs:

```
[root@ip-172-31-22-182:~/e2fsprogs-1.45.5]# rg -B2 -A1 /sys/fs/ext4/features/lazy_itable_init .
./resize/resize2fs.c
923-	if (getenv("RESIZE2FS_FORCE_LAZY_ITABLE_INIT") ||
924-	    (!getenv("RESIZE2FS_FORCE_ITABLE_INIT") &&
925:	     access("/sys/fs/ext4/features/lazy_itable_init", F_OK) == 0))
926-		lazy_itable_init = 1;
```

I confirmed /sys is mounted, and then found a bug suggesting the
ext4 module is maybe not loaded:
https://bugzilla.redhat.com/show_bug.cgi?id=1071909

My home server doesn't have ext4 loaded and had 3T to play with, so
I tried (and succeeded with) replicating the issue locally:

```
[root@kif:/scratch]# lsmod | grep -i ext

[root@kif:/scratch]# zfs create -V 3G rpool/scratch/ext4

[root@kif:/scratch]# time mkfs.ext4 /dev/zvol/rpool/scratch/ext4
mke2fs 1.45.5 (07-Jan-2020)
Discarding device blocks: done
Creating filesystem with 786432 4k blocks and 196608 inodes
Filesystem UUID: 560a4a8f-93dc-40cc-97a5-f10049bf801f
Superblock backups stored on blocks:
	32768, 98304, 163840, 229376, 294912

Allocating group tables: done
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done

real	0m2.261s
user	0m0.000s
sys	0m0.025s

[root@kif:/scratch]# zfs set volsize=3T rpool/scratch/ext4

[root@kif:/scratch]# time resize2fs -d 62 /dev/zvol/rpool/scratch/ext4
resize2fs 1.45.5 (07-Jan-2020)
fs has 11 inodes, 1 groups required.
fs requires 16390 data blocks.
With 1 group(s), we have 22234 blocks available.
Last group's overhead is 10534
Need 16390 data blocks in last group
Final size of last group is 26924
Estimated blocks needed: 26924
Extents safety margin: 49
Resizing the filesystem on /dev/zvol/rpool/scratch/ext4 to 805306368 (4k) blocks.
read_bitmaps: Memory used: 132k/0k (63k/70k), time:  0.00/ 0.00/ 0.00
read_bitmaps: I/O read: 1MB, write: 0MB, rate: 3802.28MB/s
fix_uninit_block_bitmaps 1: Memory used: 132k/0k (63k/70k), time:  0.00/ 0.00/ 0.00
resize_group_descriptors: Memory used: 132k/0k (68k/65k), time:  0.00/ 0.00/ 0.00
move_bg_metadata: Memory used: 132k/0k (68k/65k), time:  0.00/ 0.00/ 0.00
zero_high_bits_in_metadata: Memory used: 132k/0k (68k/65k), time:  0.00/ 0.00/ 0.00
```

here it got stuck for quite some time ... straceing this 20 minutes in revealed this in a tight loop:

```
getuid()                                = 0
geteuid()                               = 0
getgid()                                = 0
getegid()                               = 0
prctl(PR_GET_DUMPABLE)                  = 1 (SUID_DUMP_USER)
fallocate(3, FALLOC_FL_ZERO_RANGE, 2222649901056, 2097152) = 0
fsync(3)                                = 0
```

it finally ended 43(!) minutes later:

```
adjust_superblock: Memory used: 264k/3592k (210k/55k), time: 2554.03/ 0.16/15.07
fix_uninit_block_bitmaps 2: Memory used: 264k/3592k (210k/55k), time:  0.16/ 0.16/ 0.00
blocks_to_move: Memory used: 264k/3592k (211k/54k), time:  0.00/ 0.00/ 0.00
Number of free blocks: 755396/780023556, Needed: 0
block_mover: Memory used: 264k/3592k (216k/49k), time:  0.05/ 0.01/ 0.00
block_mover: I/O read: 1MB, write: 0MB, rate: 18.68MB/s
inode_scan_and_fix: Memory used: 264k/3592k (216k/49k), time:  0.00/ 0.00/ 0.00
inode_ref_fix: Memory used: 264k/3592k (216k/49k), time:  0.00/ 0.00/ 0.00
move_itables: Memory used: 264k/3592k (216k/49k), time:  0.00/ 0.00/ 0.00
calculate_summary_stats: Memory used: 264k/3592k (216k/49k), time: 16.35/16.35/ 0.00
fix_resize_inode: Memory used: 264k/3592k (222k/43k), time:  0.04/ 0.00/ 0.00
fix_resize_inode: I/O read: 1MB, write: 0MB, rate: 22.80MB/s
fix_sb_journal_backup: Memory used: 264k/3592k (222k/43k), time:  0.00/ 0.00/ 0.00
overall resize2fs: Memory used: 264k/3592k (222k/43k), time: 2570.90/16.68/15.07
overall resize2fs: I/O read: 1MB, write: 1MB, rate: 0.00MB/s
The filesystem on /dev/zvol/rpool/scratch/ext4 is now 805306368 (4k) blocks long.

real	43m1.943s
user	0m16.761s
sys	0m15.069s
```

I then cleaned up and recreated the zvol, loaded the ext4 module, created the ext4 fs,
resized the volume, and resize2fs'd and it went quite quickly:

```
[root@kif:/scratch]# zfs destroy rpool/scratch/ext4

[root@kif:/scratch]# zfs create -V 3G rpool/scratch/ext4

[root@kif:/scratch]# modprobe ext4

[root@kif:/scratch]# time resize2fs -d 62 /dev/zvol/rpool/scratch/ext4

[root@kif:/scratch]# time mkfs.ext4 /dev/zvol/rpool/scratch/ext4
mke2fs 1.45.5 (07-Jan-2020)
Discarding device blocks: done
Creating filesystem with 786432 4k blocks and 196608 inodes
Filesystem UUID: 5b415f2f-a8c4-4ba0-ac1d-78860de77610
Superblock backups stored on blocks:
	32768, 98304, 163840, 229376, 294912

Allocating group tables: done
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done

real	0m1.013s
user	0m0.001s
sys	0m0.023s

[root@kif:/scratch]# zfs set volsize=3T rpool/scratch/ext4

[root@kif:/scratch]# time resize2fs -d 62 /dev/zvol/rpool/scratch/ext4
resize2fs 1.45.5 (07-Jan-2020)
fs has 11 inodes, 1 groups required.
fs requires 16390 data blocks.
With 1 group(s), we have 22234 blocks available.
Last group's overhead is 10534
Need 16390 data blocks in last group
Final size of last group is 26924
Estimated blocks needed: 26924
Extents safety margin: 49
Resizing the filesystem on /dev/zvol/rpool/scratch/ext4 to 805306368 (4k) blocks.
read_bitmaps: Memory used: 132k/0k (63k/70k), time:  0.00/ 0.00/ 0.00
read_bitmaps: I/O read: 1MB, write: 0MB, rate: 3389.83MB/s
fix_uninit_block_bitmaps 1: Memory used: 132k/0k (63k/70k), time:  0.00/ 0.00/ 0.00
resize_group_descriptors: Memory used: 132k/0k (68k/65k), time:  0.00/ 0.00/ 0.00
move_bg_metadata: Memory used: 132k/0k (68k/65k), time:  0.00/ 0.00/ 0.00
zero_high_bits_in_metadata: Memory used: 132k/0k (68k/65k), time:  0.00/ 0.00/ 0.00
adjust_superblock: Memory used: 264k/1540k (210k/55k), time:  0.02/ 0.02/ 0.00
fix_uninit_block_bitmaps 2: Memory used: 264k/1540k (210k/55k), time:  0.15/ 0.15/ 0.00
blocks_to_move: Memory used: 264k/1540k (211k/54k), time:  0.00/ 0.00/ 0.00
Number of free blocks: 755396/780023556, Needed: 0
block_mover: Memory used: 264k/3592k (216k/49k), time:  0.01/ 0.01/ 0.00
block_mover: I/O read: 1MB, write: 0MB, rate: 157.11MB/s
inode_scan_and_fix: Memory used: 264k/3592k (216k/49k), time:  0.00/ 0.00/ 0.00
inode_ref_fix: Memory used: 264k/3592k (216k/49k), time:  0.00/ 0.00/ 0.00
move_itables: Memory used: 264k/3592k (216k/49k), time:  0.00/ 0.00/ 0.00

calculate_summary_stats: Memory used: 264k/3592k (216k/49k), time: 16.20/16.20/ 0.00
fix_resize_inode: Memory used: 264k/3592k (222k/43k), time:  0.00/ 0.00/ 0.00
fix_resize_inode: I/O read: 1MB, write: 0MB, rate: 5319.15MB/s
fix_sb_journal_backup: Memory used: 264k/3592k (222k/43k), time:  0.00/ 0.00/ 0.00
overall resize2fs: Memory used: 264k/3592k (222k/43k), time: 16.45/16.38/ 0.00
overall resize2fs: I/O read: 1MB, write: 1MB, rate: 0.06MB/s
The filesystem on /dev/zvol/rpool/scratch/ext4 is now 805306368 (4k) blocks long.

real	0m17.908s
user	0m16.386s
sys	0m0.079s
```

Success!
2020-10-30 12:18:23 -04:00
Graham Christensen
a179781696
stage-1: add datestamps to logs
When the stage-1 logs get imported in to the journal, they all get
loaded with the same timestamp. This makes it difficult to identify
what might be taking a long time in early boot.
2020-10-30 12:16:35 -04:00
Graham Christensen
2bf1fc0345
create-amis: allow customizing the service role name
The complete setup on the AWS end can be configured
with the following Terraform configuration. It generates
a ./credentials.sh which I just copy/pasted in to the
create-amis.sh script near the top. Note: the entire stack
of users and bucket can be destroyed at the end of the
import.

    variable "region" {
      type = string
    }
    variable "availability_zone" {
      type = string
    }

    provider "aws" {
      region = var.region
    }

    resource "aws_s3_bucket" "nixos-amis" {
      bucket_prefix = "nixos-amis-"
      lifecycle_rule {
        enabled = true
        abort_incomplete_multipart_upload_days = 1
        expiration {
          days = 7
        }
      }
    }

    resource "local_file" "credential-file" {
      file_permission = "0700"
      filename = "${path.module}/credentials.sh"
      sensitive_content = <<SCRIPT
    export service_role_name="${aws_iam_role.vmimport.name}"
    export bucket="${aws_s3_bucket.nixos-amis.bucket}"
    export AWS_ACCESS_KEY_ID="${aws_iam_access_key.uploader.id}"
    export AWS_SECRET_ACCESS_KEY="${aws_iam_access_key.uploader.secret}"
    SCRIPT
    }

    # The following resources are for the *uploader*
    resource "aws_iam_user" "uploader" {
      name = "nixos-amis-uploader"
    }

    resource "aws_iam_access_key" "uploader" {
      user = aws_iam_user.uploader.name
    }

    resource "aws_iam_user_policy" "upload-to-nixos-amis" {
      user = aws_iam_user.uploader.name

      policy = data.aws_iam_policy_document.upload-policy-document.json
    }

    data "aws_iam_policy_document" "upload-policy-document" {
      statement {
        effect = "Allow"

        actions = [
          "s3:ListBucket",
          "s3:GetBucketLocation",
        ]

        resources = [
          aws_s3_bucket.nixos-amis.arn
        ]
      }

      statement {
        effect = "Allow"

        actions = [
          "s3:PutObject",
          "s3:GetObject",
          "s3:DeleteObject",
        ]

        resources = [
          "${aws_s3_bucket.nixos-amis.arn}/*"
        ]
      }

      statement {
        effect = "Allow"
        actions = [
          "ec2:ImportSnapshot",
          "ec2:DescribeImportSnapshotTasks",
          "ec2:DescribeImportSnapshotTasks",
          "ec2:RegisterImage",
          "ec2:DescribeImages"
        ]
        resources = [
          "*"
        ]
      }
    }

    # The following resources are for the *vmimport service user*
    # See: https://docs.aws.amazon.com/vm-import/latest/userguide/vmie_prereqs.html#vmimport-role
    resource "aws_iam_role" "vmimport" {
      assume_role_policy = data.aws_iam_policy_document.vmimport-trust.json
    }

    resource "aws_iam_role_policy" "vmimport-access" {
      role = aws_iam_role.vmimport.id
      policy = data.aws_iam_policy_document.vmimport-access.json
    }

    data "aws_iam_policy_document" "vmimport-access" {
      statement {
        effect = "Allow"
        actions = [
          "s3:GetBucketLocation",
          "s3:GetObject",
          "s3:ListBucket",
        ]
        resources = [
          aws_s3_bucket.nixos-amis.arn,
          "${aws_s3_bucket.nixos-amis.arn}/*"
        ]
      }
      statement {
        effect = "Allow"
        actions = [
          "ec2:ModifySnapshotAttribute",
          "ec2:CopySnapshot",
          "ec2:RegisterImage",
          "ec2:Describe*"
        ]
        resources = [
          "*"
        ]
      }
    }

    data "aws_iam_policy_document" "vmimport-trust" {
      statement {
        effect = "Allow"
        principals {
          type = "Service"
          identifiers = [ "vmie.amazonaws.com" ]
        }

        actions = [
          "sts:AssumeRole"
        ]

        condition {
          test = "StringEquals"
          variable = "sts:ExternalId"
          values = [ "vmimport" ]
        }
      }
    }
2020-10-30 12:12:08 -04:00
Graham Christensen
e253de8a77
create-amis.sh: log the full response if describing the import snapshot tasks fails 2020-10-30 12:08:01 -04:00
Graham Christensen
f92a883ddb
nixos ec2/create-amis.sh: shellcheck: $ is not needed in arithmetic 2020-10-30 12:08:01 -04:00