Files
homelab/ansible/roles/semaphore
Hermes Agent service account 80f810fb0c feat(semaphore): rewrite role with rootful Podman Quadlet + PostgreSQL
Complete rewrite of the semaphore role. Supersedes three prior
iterations whose admin-user-creation logic was unreliable across
Semaphore CLI versions.

Architecture:
  - Rootful Podman Quadlet under /etc/containers/systemd/
  - Separate PostgreSQL 16-alpine container on a user-defined
    podman network (semaphore-net)
  - Named volumes for both data stores (semaphore_data,
    semaphore_postgres_data) so container recreation is
    non-destructive
  - Pinned image tags: semaphoreui/semaphore:v2.18.5-ansible2.16.5
    and postgres:16-alpine
  - Post-deploy HTTP health check fails the playbook if Semaphore
    doesn't respond on /api/ping within ~60s

Admin user creation remains intentionally manual after first deploy;
the role README documents the exact podman exec command.

Removes the duplicate deploy_semaphore.yml and the now-unneeded
cleanup_semaphore.yml; day1_deploy_semaphore.yml is the canonical
entry point.
2026-05-29 21:34:22 -05:00
..

semaphore

Deploys SemaphoreUI with a PostgreSQL backend on the mk-labs imagineering VM (figment) via rootful Podman Quadlets.

Architecture

                                 ┌─────────────────────┐
   user ─── HTTPS ─── Traefik ──▶│  figment (10.1.71.37) │
                                 │                       │
                                 │  ┌─────────────────┐  │
                                 │  │ semaphore       │  │
                                 │  │ :3000           │──┼──┐
                                 │  └─────────────────┘  │  │ semaphore-net
                                 │                       │  │ (podman)
                                 │  ┌─────────────────┐  │  │
                                 │  │ semaphore-      │◀─┼──┘
                                 │  │ postgres :5432  │  │
                                 │  └─────────────────┘  │
                                 │                       │
                                 │  /etc/containers/     │
                                 │    systemd/*.container│
                                 └───────────────────────┘
  • Both containers run as rootful Podman services, managed by systemd-generated units from Quadlet files in /etc/containers/systemd/.
  • The two containers share a user-defined Podman network (semaphore-net) so Semaphore can address PostgreSQL by container name.
  • Data persists on two named Podman volumes (semaphore_data, semaphore_postgres_data) so container recreation is non-destructive.
  • Image tags are pinned — no floating :latest.

Required vault variables

Variable Purpose
vault_semaphore_database_password PostgreSQL role password for the semaphore DB user.
vault_semaphore_admin_password Initial admin password (used during manual user-add).
vault_semaphore_access_key_encryption 32-byte key for Semaphore-stored access keys.

Generate the access-key encryption value with:

head -c 32 /dev/urandom | base64

Usage

- name: Deploy SemaphoreUI
  hosts: semaphore_server
  become: true
  roles:
    - semaphore

Or via the dedicated playbook:

ansible-playbook -i inventory.yml playbooks/day1_deploy_semaphore.yml

First-run admin user creation (manual)

Semaphore's CLI user add is intentionally invoked once, by hand, after the first deployment succeeds:

sudo podman exec -it semaphore semaphore user add --admin \
  --login admin \
  --name "Administrator" \
  --email admin@local.mk-labs.cloud \
  --password '<vault_semaphore_admin_password>'

Subsequent password rotations should also be done via the CLI, not by re-running this role.

Pinned versions

  • Semaphore: docker.io/semaphoreui/semaphore:v2.18.5-ansible2.16.5 (bundles Ansible 2.16.5 inside the container, matching our controller)
  • PostgreSQL: docker.io/library/postgres:16-alpine

To bump versions, update semaphore_image / semaphore_postgres_image in defaults/main.yml. Test against figment, snapshot first.

Verification

The role completes with an HTTP health check against http://127.0.0.1:3000/api/ping. If Semaphore doesn't respond within ~60 seconds, the role fails loudly rather than reporting green-but-broken.

Troubleshooting

# Container status
sudo systemctl status semaphore semaphore-postgres
sudo podman ps -a

# Logs
sudo journalctl -u semaphore -f
sudo journalctl -u semaphore-postgres -f
sudo podman logs semaphore --tail 100

# Network
sudo podman network inspect semaphore-net

# Wipe and redeploy (destructive — destroys all Semaphore data)
sudo systemctl stop semaphore semaphore-postgres
sudo podman volume rm semaphore_data semaphore_postgres_data
sudo rm /etc/containers/systemd/semaphore.container /etc/containers/systemd/semaphore-postgres.container
sudo systemctl daemon-reload
# then re-run the playbook