aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorGravatar Tom van der Lee <tom@vanderlee.com>2026-09-24 11:34:23 +0000
committerGravatar Tom van der Lee <tom@vanderlee.com>2026-09-24 11:34:23 +0000
commit096e82ca45a5360b4b2dd084e0fb1a937a227a69 (patch)
tree47e6a3757da156f37606bef55812783a18856847
parentd09bdcd6b9fd6d815d6272a7bc337b410b19b665 (diff)
downloaddesktop-flake-096e82ca45a5360b4b2dd084e0fb1a937a227a69.tar.gz
desktop-flake-096e82ca45a5360b4b2dd084e0fb1a937a227a69.tar.bz2
desktop-flake-096e82ca45a5360b4b2dd084e0fb1a937a227a69.zip
Switch Docker to rootful after the rootless setup broke in practice
Rootless Docker's per-user systemd unit didn't start in a session that was already logged in before a switch, and DOCKER_HOST only reaches freshly-started shells -- both bit Tom immediately (docker run failed against the default rootful socket path, which never existed under rootless). virtualisation.docker.enable = true now runs dockerd as a standard root system service, started at boot; tom joins the docker group for sudo-less access. Trades rootless's isolation guarantee for reliability on a long-lived desktop session. Archives switch-docker-rootful and syncs the container-runtime capability's requirements and Purpose text to match. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HngxrvqEzq8oBEBDtrYJyB
-rw-r--r--modules/nixos/common.nix16
-rw-r--r--openspec/changes/archive/2026-09-24-switch-docker-rootful/.openspec.yaml2
-rw-r--r--openspec/changes/archive/2026-09-24-switch-docker-rootful/design.md27
-rw-r--r--openspec/changes/archive/2026-09-24-switch-docker-rootful/proposal.md22
-rw-r--r--openspec/changes/archive/2026-09-24-switch-docker-rootful/specs/container-runtime/spec.md25
-rw-r--r--openspec/changes/archive/2026-09-24-switch-docker-rootful/tasks.md10
-rw-r--r--openspec/specs/container-runtime/spec.md20
7 files changed, 106 insertions, 16 deletions
diff --git a/modules/nixos/common.nix b/modules/nixos/common.nix
index 10f48c6..9ee164b 100644
--- a/modules/nixos/common.nix
+++ b/modules/nixos/common.nix
@@ -18,7 +18,10 @@
18 18
19 users.users.tom = { 19 users.users.tom = {
20 isNormalUser = true; 20 isNormalUser = true;
21 extraGroups = [ "wheel" ]; 21 extraGroups = [
22 "wheel"
23 "docker"
24 ];
22 shell = pkgs.bash; 25 shell = pkgs.bash;
23 # Fetched once from https://github.com/tomvanderlee.keys and pinned here 26 # Fetched once from https://github.com/tomvanderlee.keys and pinned here
24 # as a static literal (not fetched at build time, which would require 27 # as a static literal (not fetched at build time, which would require
@@ -69,9 +72,10 @@
69 openFirewall = true; 72 openFirewall = true;
70 }; 73 };
71 74
72 # Rootless: runs as the user's own systemd user service, not as root, and 75 # Rootful: standard system service, starts at boot independent of any
73 # has no docker-group privilege-escalation path. setSocketVariable exports 76 # login session (rootless was tried first but its per-user systemd unit
74 # DOCKER_HOST automatically so the docker CLI (dev-tools.nix) just works. 77 # didn't reliably start in already-logged-in sessions after a switch).
75 virtualisation.docker.rootless.enable = true; 78 # tom's docker group membership is root-equivalent access, accepted here
76 virtualisation.docker.rootless.setSocketVariable = true; 79 # for reliability over the rootless isolation guarantee.
80 virtualisation.docker.enable = true;
77} 81}
diff --git a/openspec/changes/archive/2026-09-24-switch-docker-rootful/.openspec.yaml b/openspec/changes/archive/2026-09-24-switch-docker-rootful/.openspec.yaml
new file mode 100644
index 0000000..75289e4
--- /dev/null
+++ b/openspec/changes/archive/2026-09-24-switch-docker-rootful/.openspec.yaml
@@ -0,0 +1,2 @@
1schema: spec-driven
2created: 2026-09-24
diff --git a/openspec/changes/archive/2026-09-24-switch-docker-rootful/design.md b/openspec/changes/archive/2026-09-24-switch-docker-rootful/design.md
new file mode 100644
index 0000000..9ed45ac
--- /dev/null
+++ b/openspec/changes/archive/2026-09-24-switch-docker-rootful/design.md
@@ -0,0 +1,27 @@
1## Context
2
3See proposal.md - Why. `add-uv-and-docker-runtime`'s design.md chose rootless specifically to avoid `docker`-group root-equivalent access, accepting rootless's compatibility gaps as the trade-off. That trade-off didn't hold up: the rootless per-user systemd service (`systemd.user.services.docker`, `wantedBy = [ "default.target" ]` at the *user* manager level) doesn't get started in a session that was already active before the switch — NixOS's `nixos-rebuild switch` reloads and restarts *system* units automatically, but does not reach into already-running per-user systemd managers to link and start newly-enabled user units. Combined with `DOCKER_HOST` only being exported into freshly-started shells (via `environment.extraInit` → `/etc/profile`), a user mid-session sees exactly the failure Tom hit: the `docker` CLI falls back to the default rootful socket path, which doesn't exist because no rootful daemon was ever configured.
4
5## Goals / Non-Goals
6
7**Goals:**
8- `docker` CLI commands work reliably without session-timing caveats — the daemon should be up as soon as the system has booted, regardless of who's logged in or when.
9- No manual per-session environment setup (`DOCKER_HOST`) required.
10
11**Non-Goals:**
12- Re-attempting rootless with a fix for the session-timing issue (e.g. instructing users to always log out/in after a switch) — rejected as more operational friction than it's worth for a single-user desktop/dev machine; the security benefit of rootless is being deliberately traded away here.
13- Any change to the `docker`/`docker-compose` CLI packages themselves (`modules/home/dev-tools.nix`) — unaffected either way.
14- Docker daemon TLS/remote-access configuration — out of scope, same as before.
15
16## Decisions
17
18**Decision: rootful (`virtualisation.docker.enable = true;`) over continuing to debug rootless.**
19- Alternatives considered: (a) keep rootless, document "log out and back in after any switch that touches Docker" — rejected, this is a real, easy-to-forget footgun on a machine where the whole point is long-lived desktop sessions; (b) keep rootless but add a `systemd.user.services.docker` restart hook to `nixos-rebuild switch` itself — not a supported NixOS mechanism for reaching into other users' already-running session managers, would require nonstandard tooling; (c) rootful — accepted, standard system-service semantics (starts at boot, `nixos-rebuild switch` restarts system units automatically like every other service in this repo, e.g. GDM/SSH/dconf), well-trodden path, and matches how every other service in `modules/nixos/common.nix` already behaves.
20
21**Decision: use `tom`'s group membership (`extraGroups = [ "wheel" "docker" ]`) rather than requiring `sudo docker ...`.**
22- Requiring `sudo` for every Docker command would be significant day-to-day friction for a dev machine and isn't how Docker is conventionally used; group membership is the standard, documented NixOS/Docker pattern. This is the security trade-off being explicitly accepted: `docker` group membership is root-equivalent (a container can bind-mount `/` and modify the host), same risk profile most NixOS Docker users accept by default.
23
24## Risks / Trade-offs
25
26- [`docker` group membership is root-equivalent — any process able to run `docker` commands as `tom` can trivially become root on the host] → Explicitly accepted per the user's request; this is the well-known, standard trade-off of rootful Docker, and the alternative (rootless) already proved less reliable in practice on this machine.
27- [Anyone who already had the rootless daemon actually running before this switch loses access to whatever images/containers existed in the rootless-specific storage path] → Not applicable here: the rootless daemon never successfully served a `docker run` in practice (that's the bug being fixed), so there's nothing to migrate.
diff --git a/openspec/changes/archive/2026-09-24-switch-docker-rootful/proposal.md b/openspec/changes/archive/2026-09-24-switch-docker-rootful/proposal.md
new file mode 100644
index 0000000..8b2f4da
--- /dev/null
+++ b/openspec/changes/archive/2026-09-24-switch-docker-rootful/proposal.md
@@ -0,0 +1,22 @@
1## Why
2
3The rootless Docker setup from `add-uv-and-docker-runtime` broke in practice: `docker run` failed with `failed to connect to the docker API at unix:///var/run/docker.sock ... no such file or directory` after a `nixos-rebuild switch`, because a newly-enabled `systemd.user.services.docker` unit doesn't start in an already-logged-in session, and `$DOCKER_HOST` (exported via `environment.extraInit`) only reaches freshly-started shells — both are real, recurring friction for a desktop dev machine where sessions stay open for days. Tom asked to switch to standard rootful Docker instead.
4
5## What Changes
6
7- Replace `virtualisation.docker.rootless.enable`/`setSocketVariable` in `modules/nixos/common.nix` with `virtualisation.docker.enable = true;` (the standard rootful daemon, a system-level service rather than a per-user systemd unit).
8- Add `"docker"` to `tom`'s `extraGroups` in `modules/nixos/common.nix` (alongside the existing `"wheel"`), so `tom` can use the Docker socket without `sudo`. This is the rootful equivalent of the rootless setup's `DOCKER_HOST` auto-export — standard mechanism, well-documented NixOS pattern.
9- **This reverses the explicit rootless decision** made in `design.md` for `add-uv-and-docker-runtime` (chosen specifically to avoid `docker`-group root-equivalent privilege escalation). Trade-off is now accepted deliberately: `docker` group membership is root-equivalent (a container can bind-mount `/` and modify the host as root), but the daemon runs as a system service that starts at boot (not tied to a logged-in session) and needs no `DOCKER_HOST` env-var propagation, avoiding the exact class of bug just hit.
10- **BREAKING**: for anyone who already has a rootless daemon running (e.g. before switching, if a session picked it up), those containers/images live in a different rootless-specific storage location and won't carry over to the rootful daemon's storage — acceptable here since the rootless setup never successfully ran a container in practice.
11
12## Capabilities
13
14### Modified Capabilities
15- `container-runtime`: switch from a per-user rootless Docker daemon to a system-level rootful Docker daemon, on every NixOS host.
16
17## Impact
18
19- **Affected code**: `modules/nixos/common.nix` (swap `virtualisation.docker.rootless.*` for `virtualisation.docker.enable` and `tom`'s group membership).
20- **Not affected**: `modules/home/dev-tools.nix` (the `docker`/`docker-compose` CLI packages are unchanged — same client either way), `lib/unfree-allowlist.nix` (Docker is free-licensed regardless of mode).
21- **Dependencies**: none new — `virtualisation.docker.enable` is the same nixpkgs module family as the rootless option already in use.
22- **Platforms**: NixOS hosts only, same scope as before.
diff --git a/openspec/changes/archive/2026-09-24-switch-docker-rootful/specs/container-runtime/spec.md b/openspec/changes/archive/2026-09-24-switch-docker-rootful/specs/container-runtime/spec.md
new file mode 100644
index 0000000..7edb25d
--- /dev/null
+++ b/openspec/changes/archive/2026-09-24-switch-docker-rootful/specs/container-runtime/spec.md
@@ -0,0 +1,25 @@
1## REMOVED Requirements
2
3### Requirement: A rootless Docker daemon runs on every NixOS host
4**Reason**: Switched to a standard rootful Docker daemon — the rootless per-user systemd service didn't reliably start in already-logged-in sessions, causing real `docker` CLI failures in practice.
5**Migration**: Replaced by "A Docker daemon runs on every NixOS host" below.
6
7### Requirement: The Docker CLI connects to the rootless daemon without manual configuration
8**Reason**: Rootful Docker uses the standard system-wide socket path and `docker`-group membership instead of a per-session `DOCKER_HOST` export, so this requirement no longer applies.
9**Migration**: Replaced by "The user account has access to the Docker socket" below.
10
11## ADDED Requirements
12
13### Requirement: A Docker daemon runs on every NixOS host
14Every NixOS host built from this flake's shared system module SHALL run a Docker daemon as a system-level service, started at boot rather than tied to a user login session, without requiring per-host opt-in.
15
16#### Scenario: Daemon available after boot
17- **WHEN** a NixOS host built from this flake's shared system module finishes booting
18- **THEN** a Docker daemon is running as a system service, independent of whether any user has logged in
19
20### Requirement: The user account has access to the Docker socket
21The user account provisioned on a NixOS host built from this flake's shared system module SHALL be a member of the group that grants access to the Docker daemon's socket, so Docker CLI commands work without `sudo` or manual connection configuration.
22
23#### Scenario: Docker command works without sudo
24- **WHEN** the provisioned user runs a `docker` CLI command on a NixOS host built from this flake's shared system module
25- **THEN** the command reaches the daemon and executes, without requiring `sudo` or any manually exported connection configuration
diff --git a/openspec/changes/archive/2026-09-24-switch-docker-rootful/tasks.md b/openspec/changes/archive/2026-09-24-switch-docker-rootful/tasks.md
new file mode 100644
index 0000000..f73e8d0
--- /dev/null
+++ b/openspec/changes/archive/2026-09-24-switch-docker-rootful/tasks.md
@@ -0,0 +1,10 @@
1## 1. Switch to rootful Docker
2
3- [x] 1.1 In `modules/nixos/common.nix`, replace `virtualisation.docker.rootless.enable`/`setSocketVariable` with `virtualisation.docker.enable = true;`.
4- [x] 1.2 Add `"docker"` to `users.users.tom.extraGroups` in `modules/nixos/common.nix` (alongside `"wheel"`).
5- [x] 1.3 Build a NixOS host's system closure with `hosts/test-vm` visible via the `path:` flake-ref scheme: `nix build 'path:/home/tom/desktop-flake#nixosConfigurations.test-vm.config.system.build.toplevel'`, and verify it succeeds.
6- [x] 1.4 Inspect the built closure's Docker systemd unit (system-level `docker.service`, not a user unit this time) and verify it's a root-run system service, and verify `tom`'s configured groups include `docker`.
7
8## 2. Spec sync
9
10- [x] 2.1 Verify `openspec validate --change switch-docker-rootful --strict` passes, confirming the delta spec matches the shipped behavior.
diff --git a/openspec/specs/container-runtime/spec.md b/openspec/specs/container-runtime/spec.md
index 0b07f92..a49f94f 100644
--- a/openspec/specs/container-runtime/spec.md
+++ b/openspec/specs/container-runtime/spec.md
@@ -2,20 +2,20 @@
2 2
3## Purpose 3## Purpose
4 4
5Provides a working, unprivileged Docker container runtime on every NixOS host built from this flake, so the already-provisioned Docker CLI tooling has a daemon to talk to without a separately-managed installation step. 5Provides a working Docker container runtime on every NixOS host built from this flake, so the already-provisioned Docker CLI tooling has a daemon to talk to without a separately-managed installation step.
6 6
7## Requirements 7## Requirements
8 8
9### Requirement: A rootless Docker daemon runs on every NixOS host 9### Requirement: A Docker daemon runs on every NixOS host
10Every NixOS host built from this flake's shared system module SHALL run a Docker daemon in rootless mode (as the user's own account, not as root), without requiring per-host opt-in. 10Every NixOS host built from this flake's shared system module SHALL run a Docker daemon as a system-level service, started at boot rather than tied to a user login session, without requiring per-host opt-in.
11 11
12#### Scenario: Daemon available after boot 12#### Scenario: Daemon available after boot
13- **WHEN** a NixOS host built from this flake's shared system module finishes booting and the user logs in 13- **WHEN** a NixOS host built from this flake's shared system module finishes booting
14- **THEN** a Docker daemon is running under that user's own account, not as root 14- **THEN** a Docker daemon is running as a system service, independent of whether any user has logged in
15 15
16### Requirement: The Docker CLI connects to the rootless daemon without manual configuration 16### Requirement: The user account has access to the Docker socket
17A user on a NixOS host built from this flake's shared system module SHALL be able to run Docker CLI commands (e.g. `docker run`, `docker build`) against the rootless daemon without manually setting `DOCKER_HOST` or other connection configuration. 17The user account provisioned on a NixOS host built from this flake's shared system module SHALL be a member of the group that grants access to the Docker daemon's socket, so Docker CLI commands work without `sudo` or manual connection configuration.
18 18
19#### Scenario: Docker command works out of the box 19#### Scenario: Docker command works without sudo
20- **WHEN** a user runs a `docker` CLI command on a NixOS host built from this flake's shared system module 20- **WHEN** the provisioned user runs a `docker` CLI command on a NixOS host built from this flake's shared system module
21- **THEN** the command reaches the rootless daemon and executes, without the user having exported `DOCKER_HOST` or written any Docker connection config themselves 21- **THEN** the command reaches the daemon and executes, without requiring `sudo` or any manually exported connection configuration