Identity (id01)¶
id01 runs Authentik, the fleet's identity provider: the one place that holds the fleet's users and decides who is signed in. Once the fleet leaves bootstrap mode, every other stack's web interface asks Authentik for a sign-in first, so id01 is the first host built after the foundation.
Its own stack is authentik-server. Authentik cannot sit behind Authentik, so its route never asks for a sign-in, in either mode. Step 5 says why.
At the end of this page Authentik is running with one admin account, and nothing uses it yet. Its first start is slow, so allow the run up to a quarter of an hour.
Status: written, not yet run.
Prerequisites¶
- The foundation is finished, through The handover.
- The handover cleaned the shell, and describing a host needs the deploy age key. Restore it for steps 1 to 3, as in Running from a shell again.
- Postfix on ci01 accepts mail from the internal subnet. See Core infrastructure (ci01).
- A MaxMind account, for the free GeoLite2 databases. Step 2 says what happens without one.
1. Describe the host¶
In the private repo's hosts.yml, add id01 to the docker_host group. The first three keys are the ones km01's entry explains:
id01:
ansible_host: 172.16.7.131
serverHostname: "id01"
docker_stacks:
- system-agent
- traefik-agent
- authentik-server
komodo_stack_env:
authentik-server:
AUTHENTIK_EMAIL__USERNAME: ""
AUTHENTIK_EMAIL__PASSWORD: ""
The list above is what the host runs once the fleet has left bootstrap mode. While docker_stacks_bootstrap: true is set, the run deploys the host without system-agent and traefik-agent, and with traefik-bootstrap in their place. nixos-sync.yml and komodo-sync.yml make that swap when they write the host's files, so the list itself never changes. See Bootstrap mode.
komodo_stack_env sets keys of one stack's Environment for this host, and here it blanks two. That removes a login Authentik would otherwise try against Postfix, which offers none. See Blanking a reference.
In opentofu/prod.tfvars, add its VM inside vms. The fields are the ones km01's entry explains:
id01 = {
server = "vh01"
vm_id = 7131
cores = 4
memory_mb = 8192
vlan_id = 7
ipv4_address = "172.16.7.131/24"
ipv4_gateway = "172.16.7.1"
dns_servers = ["172.16.7.1"]
tags = ["docker"]
extra_disks = {
persist = { interface = "scsi2", size_gb = 20 }
}
}
Authentik's worker is what the four cores and 8 GB are for, with Postgres beside it. The database holds every user, group, application, and flow, and 20 GB is plenty for it.
id01 is on the internal VLAN. Anything the internet is to reach goes through the tunnel on bh01, never through id01 itself.
Then generate id01's files: its SSH host keys, its NixOS file, and its Komodo file.
From ~/src/fleet-ansible, with the private repo and fleet-nixos checked out next to it, and the deploy key in SOPS_AGE_KEY or SOPS_AGE_KEY_FILE.
A host with no file under secrets/host-keys/ in the private repo gets its SSH host keys first:
The command writes secrets/host-keys/<host>.yaml, adds the host to .sops.yaml, and encrypts secrets/fleet.yaml again so that the host can read it. For a host that has its keys already, it changes nothing. See Host keys.
Then write the NixOS files and the Komodo files:
ansible-playbook -i ../fleet-private/hosts.yml nixos-sync.yml
ansible-playbook -i ../fleet-private/hosts.yml komodo-sync.yml
Add what changed, and read the diff before it is committed:
git -C ../fleet-private add hosts.yml group_vars/ opentofu/ komodo/ nixos/ secrets/ .sops.yaml
git -C ../fleet-private diff --cached
The diff of nixos/hosts/<host>.json shows the host's network, its features, and the folders, seed files and ports its stacks need. The diff of komodo/stacks/<host>.toml shows the Stacks the next run deploys and every value in their Environment. The files under secrets/ are encrypted, so their diff shows that a value changed and not what it changed to.
Then commit and push:
The run builds the host from the files git tracks in the private repo, and Komodo reads the pushed copy. The run stops at a host whose committed files differ from what the inventory gives.
2. Stage the values¶
Create these in Komodo before the run. See Creating one for the clicks.
Authentik's own¶
| Name | Kind | Value | Used for |
|---|---|---|---|
AUTHENTIK_SECRET_KEY |
Secret | 96 alphanumeric characters | Signing sessions and encrypting what Authentik stores |
AUTHENTIK_POSTGRES_USER |
Secret | Your choice | The login of Authentik's database |
AUTHENTIK_POSTGRES_PASSWORD |
Secret | 48 alphanumeric characters | The same login |
Generate the two long values in a shell:
tr -dc 'A-Za-z0-9' < /dev/urandom | head -c 96; echo
tr -dc 'A-Za-z0-9' < /dev/urandom | head -c 48; echo
Warning
AUTHENTIK_SECRET_KEY signs sessions and encrypts what Authentik stores. Changing it later ends every session and makes every stored credential unreadable. Keep a copy in your password manager.
MaxMind¶
The stack's geoipupdate service downloads MaxMind's GeoLite2 databases, which map an address to a country and a city. It signs in to MaxMind with these two.
| Name | Kind | Value |
|---|---|---|
GLOBAL_GEOIPUPDATE_ACCOUNT_ID |
Secret | Your MaxMind account ID |
GLOBAL_GEOIPUPDATE_LICENSE_KEY |
Secret | A licence key created in that account |
Without an account, create both with the value unused, so that the references resolve. geoipupdate then fails to sign in, and nothing else in the stack reads its databases. What that does to the run is not yet confirmed, so the account is the safer choice.
Mail¶
Authentik sends its mail, such as a password reset, through Postfix on ci01. These five say where Postfix is and how to talk to it. They are settings, so leave Is Secret unticked.
| Name | Value |
|---|---|
GLOBAL_EMAIL_HOST |
ci01's address, 172.16.7.121 in these pages |
GLOBAL_EMAIL_PORT |
25 |
GLOBAL_EMAIL_TLS |
false |
GLOBAL_EMAIL_SSL |
false |
GLOBAL_EMAIL_FROM |
The address service mail is sent as, such as authentik@myah-mitchell.com |
The sender's domain has to be one Postfix on ci01 accepts, which is the fleet's own domain unless you changed POSTFIX_ALLOWED_SENDER_DOMAINS. TLS stays off because the connection never leaves the internal network. Postfix applies TLS itself when it hands the mail to your relay.
The names start with GLOBAL_ because they describe Postfix, not Authentik. As fleet-stacks stands, authentik-server is the one stack that reads them. A stack of your own that sends mail can reference the same five.
GLOBAL_AUTHENTIK_HOST exists already, from Setting up Komodo.
3. Run the build¶
In Semaphore, run the site Template with Target set to id01.
From ~/src/fleet-ansible, in a shell prepared for runs after the handover. See Running from a shell again.
The run creates the VM, installs NixOS on it, deploys its configuration, and has Komodo deploy two Stacks: traefik-bootstrap-id01 and authentik-server.
Authentik's first start is slow. It runs its whole database migration before the server answers, and the worker restarts a few times while Postgres comes up. The run waits up to fifteen minutes for a host's Stacks.
Manual steps, instead of site.yml
Create the VM by hand¶
The run's first stage creates the VM through OpenTofu, from the host's entry in opentofu/prod.tfvars. By hand, on the Proxmox host, with the values from that same entry:
qm create <vmid> --name <host> --ostype l26 \
--cpu x86-64-v2-AES --cores <cores> --numa 1 \
--memory <memory> --balloon 0 \
--scsihw virtio-scsi-single \
--scsi0 <vm-storage>:<os-size>,discard=on,ssd=1,iothread=1 \
--scsi1 <vm-storage>:<docker-size>,discard=on,ssd=1,iothread=1 \
--scsi2 <vm-storage>:<persist-size>,discard=on,ssd=1,iothread=1 \
--ide2 local:iso/fleet-nixos-installer.iso,media=cdrom \
--ide0 <vm-storage>:cloudinit \
--ipconfig0 ip=<address>,gw=<gateway> \
--nameserver <dns-server> \
--net0 virtio,bridge=vmbr0,tag=<vlan-id> \
--boot 'order=scsi0;ide2' \
--serial0 socket --vga serial0 \
--agent enabled=1 --onboot 1 \
--tags 'nixos;<tags>'
qm start <vmid>
| Placeholder | Value |
|---|---|
<vmid> |
The entry's vm_id |
<host> |
The entry's name, which is the host's name in the inventory |
<cores>, <memory> |
The entry's cores and memory_mb |
<vm-storage> |
The Proxmox storage that holds VM disks, local-zfs unless the server's entry sets datastore_id |
<os-size>, <docker-size> |
The entry's os_disk_gb and docker_disk_gb in GB, 20 and 40 when it sets neither |
<persist-size> |
The persist disk's size_gb |
<address>, <gateway> |
The entry's ipv4_address, with its prefix length, and ipv4_gateway |
<dns-server> |
The entry's dns_servers, separated by spaces inside one pair of quotes when there are several |
<vlan-id> |
The entry's vlan_id |
<tags> |
The entry's tags, separated by semicolons |
An entry with no dns_servers takes no --nameserver option, and one with no vlan_id takes no ,tag= part.
The three disks are blank. The VM finds nothing to boot on scsi0, so it boots the installer ISO on ide2, and the installer takes its address from the cloud-init drive on ide0.
The ISO has to be on the Proxmox host first. See The installer ISO.
Create all three disks before the first start. The install partitions scsi0 and scsi1 and formats scsi2, and it stops when one is missing.
A VM made this way is not in OpenTofu's state, and it does not carry the tofu tag that marks the VMs OpenTofu manages. Leave its entry out of opentofu/prod.tfvars, or a later run tries to create a second VM under the same name and address.
Install NixOS without the rest of the run¶
The run's second and third stages wait for the VM's SSH port, install NixOS when the VM answers as the installer, and deploy the host's configuration. The same three commands of the flake do it from a shell.
From ~/src/fleet-ansible, with the host's files committed in the private repo, the deploy key in SOPS_AGE_KEY or SOPS_AGE_KEY_FILE, and the fleet's SSH key in ~/.ssh/config. See The fleet's SSH key.
Ask the VM what it runs:
It prints installer for a VM that booted the ISO. Then install:
Warning
The install wipes scsi0 and scsi1. It formats scsi2 only when the disk is blank, and it refuses any machine that is not the installer.
The last line reads install-host: <host> is installed and is rebooting. Repeat the host-state command until it prints installed, then deploy the configuration:
<host> is the host's name in the inventory, and <address> is its ansible_host. The configuration is the whole base build: accounts, SSH, the firewall, Docker, the persistent disk, Periphery, Node Exporter, and the folders, seed files and ports of every stack in the host's file. See The commands.
For traefik-bootstrap:
A host deployed with the stack in its file has these folders already. The commands make them on a host whose file does not list the stack.
sudo install -d -o $USER -g 101000 -m 0750 \
/opt/docker/logs/traefik /opt/docker/volumes/traefik
sudo install -d -o 101000 -g 101000 -m 0755 /opt/docker/logs/traefik/traefik
sudo install -d -o 101000 -g 101000 /opt/docker/volumes/traefik/traefik-certs
sudo install -d -o 101000 -g 101000 /opt/docker/volumes/traefik/traefik-plugins
No command opens the stack's ports, because the firewall changes only through the host's configuration. The ports below are open once the host's file, nixos/hosts/<host>.json, holds them: nixos-sync.yml writes that file in the private repo, and a deploy of the host applies it. See Host layout.
| Port | Allowed from |
|---|---|
80/tcp |
Anywhere |
443/tcp |
Anywhere |
8443/tcp |
Anywhere |
For authentik-server:
A host deployed with the stack in its file has these folders already. The commands make them on a host whose file does not list the stack.
sudo install -d -o $USER -g 101000 -m 0750 \
/opt/docker/logs/authentik /opt/docker/volumes/authentik
sudo install -d -o 101000 -g 101000 /opt/docker/volumes/authentik/authentik-media
sudo install -d -o 101000 -g 101000 /opt/docker/volumes/authentik/authentik-templates
sudo install -d -o 101000 -g 101000 /opt/docker/volumes/authentik/authentik-certs
sudo install -d -o 100000 -g 100000 /opt/docker/volumes/authentik/postgres-data
sudo install -d -o 100000 -g 100000 /opt/docker/volumes/authentik/postgres-backup-data
sudo install -d -o 101000 -g 101000 /opt/docker/volumes/authentik/geoip-data
Deploy the Stacks by hand¶
The run's last stage has Komodo read the host's Stacks from komodo/stacks/<host>.toml in the private repo and deploy them. To create and deploy one in Komodo's UI instead, follow Deploying a stack by hand.
A Stack made by hand under the name the run would give it is taken over by the next run, not duplicated.
4. Verify¶
The run ends with ansible's recap, one line of counts for the host. failed=0 and unreachable=0 mean every stage finished, and the last stage only finishes once Komodo reports every one of the host's Stacks as running.
In Komodo's UI, open Resources > Servers. The host shows as connected.
Open Resources > Stacks and filter by the host's Server. Each Stack shows as Running.
If the run stopped instead, its last message names the stage and what it stopped at. Fix the cause and run it again. Every stage is safe to repeat.
The authentik-server Stack has seven services:
5. Create the admin account¶
Open Authentik's setup flow in a browser:
The name needs a DNS record pointing at id01, or an entry in your own hosts file.
Note
In bootstrap mode the browser warns about the certificate. Traefik serves its own self-signed one until the host leaves bootstrap mode, so accept the warning for now.
Enter an email address and a password for the akadmin account, and store the password in your password manager. The flow runs once. A later visit to the same address goes to the normal sign-in page.
This account can grant itself access to anything Authentik guards, which after bootstrap mode is the whole fleet.
| Hostname | Used for |
|---|---|
authentik.id01.home.myah-mitchell.com |
Reaching Authentik on the internal network |
auth.myah-mitchell.com |
The public name. Nothing publishes it yet. See Not yet confirmed |
Nothing asks Authentik for a sign-in yet. That starts when the fleet leaves bootstrap mode, which is also where Authentik gets its first Provider, the object that says how a sign-in is done. One Provider there covers every web interface under the fleet's domain. See Leaving bootstrap mode.
Background: why Authentik's own route never asks for a sign-in
Outside bootstrap mode, a route in the fleet is guarded by forward auth. Before Traefik passes a request to an application, it asks Authentik whether the browser is signed in. When it is not, the browser is sent to Authentik's sign-in page.
That sign-in page is served by Authentik, on this route. If the route were guarded the same way, a browser sent to the sign-in page would be stopped on the way to it and sent to the sign-in page again, with no end.
So Authentik's route uses the chain with no sign-in in both modes, and Authentik guards itself with its own login. That is why the akadmin password matters as much as it does. The auth chain entry says what a chain is.
What's next¶
Build pk01, the certificate authority. See Certificates (pk01).
Not yet confirmed¶
- The whole page. id01 has not been built by the run.
- The public name. The route publisher sends a route to tf01 only for a container that carries
kop-publiclabels, and Authentik's container has none, soauth.myah-mitchell.comcannot reach the tunnel on bh01 as the repo stands. - geoipupdate with the value
unused. Its health check only asks the program for its version, so a failed download does not show there. If the container exits and restarts instead, the Stack never shows as running, and the run's last wait fails after fifteen minutes with everything else up.