Semaphore rejected the previous become_key=wed-ssh with 'access key
type not supported for ansible become user' — that field is for a
sudo PASSWORD (login_password type), not a reused SSH key. wed has
passwordless sudo on every host (set up by the VM template), so the
correct value is the built-in 'None' key.
Adds wed-ssh (the universal automation account pre-baked in every
VM template) to the declared key set and switches the production
inventory to use it for both ssh_key and become_key. Retains
jarvis-ssh for cases that specifically need admin-level access.
This aligns Semaphore-driven jobs with the established homelab
convention: wed runs the playbooks, jarvis is the higher-privilege
admin account provisioned by linux-baseline.
Operator (Ryan) cleaned out the previous templates + inventory in
Semaphore before this commit so the configure step re-creates them
wired to wed-ssh on its next run.
The 'uri' module with body_format=json was sending integer fields as
strings when they came from quoted Jinja expressions in inline YAML
(e.g. ssh_key_id: "{{ ... | int }}"), because YAML loads the quoted
form as a string and Ansible doesn't coerce back. Semaphore rejects
that with HTTP 400.
Build each body as a Jinja dict literal in a folded scalar so types
survive: integers stay integers, strings stay strings.
Also restores no_log: true on key creation now that we're past the
debug round.
Jinja interpreted semaphore_config.keys as the dict method rather than
the 'keys' field, causing 'Invalid data passed to loop' failure. Bracket
subscript disambiguates.
Adds an idempotent configuration pass that drives a freshly-deployed
Semaphore instance into its desired state via the REST API. Declared
in group_vars/all/semaphore.yml, applied by tasks/configure.yml,
toggled by semaphore_configure feature flag (default off).
Object types managed:
- Project (mk-labs)
- Keys (ansible-vault-pass, gitea-deploy, jarvis-ssh)
- Repositories (homelab on gitea)
- Inventories (production -> ansible/inventory.yml in homelab repo)
- Environments (default with ANSIBLE_HOST_KEY_CHECKING=False)
- Templates (day0_linux_baseline + variants, day1_deploy_semaphore)
with survey vars for runtime parameters
Each object found-or-created by name; existing ones never modified.
no_log on token-bearing calls to keep secrets out of stdout.
Inputs (already in vault):
vault_semaphore_api_token
vault_jarvis_ssh_private_key
vault_gitea_deploy_key
vault_ansible_vault_password
Three new vault entries required for Semaphore config-as-code:
- vault_jarvis_ssh_private_key (matches jarvis_ssh_public_key in
group_vars/all/vars; used by Semaphore to SSH to the fleet)
- vault_gitea_deploy_key (existing deploy key on the homelab
repo; used by Semaphore to clone)
- vault_semaphore_api_token (minted from the Semaphore UI; used
by Ansible to drive Semaphore configuration)
These wire up the inputs the upcoming semaphore configure step
will consume.
Quadlet regenerates the systemd unit on daemon-reload but does not
restart the running container — the process keeps its original
environment until restarted. Add explicit 'state: restarted' tasks
gated on the Quadlet template's changed status so env-var updates
actually take effect on re-runs.
The SPA was rendering <base href="https://semaphore.local.mk-labs.cloud/">
regardless of which hostname served the page, causing the imagineering
alias to load the same UI but rewrite all in-page navigation back to the
semaphore hostname. Confusing for users hitting either Traefik alias.
Leave SEMAPHORE_WEB_ROOT empty so Semaphore emits relative URLs and is
fully host-agnostic. Both DNS names (semaphore + imagineering) now serve
cleanly without preference between them.
The template only emits SEMAPHORE_WEB_ROOT when the variable is set, so
the env-var is absent from the container when not needed.
The v2.18 image's entrypoint runs the setup wizard on first boot. Without
the SEMAPHORE_ADMIN_* variables it prompts on stdin, fails with 'Username
cannot be empty', and the container exits — leading to a crash loop.
Set:
SEMAPHORE_ADMIN=admin
SEMAPHORE_ADMIN_NAME=Administrator
SEMAPHORE_ADMIN_EMAIL=admin@local.mk-labs.cloud
SEMAPHORE_ADMIN_PASSWORD={{ vault_semaphore_admin_password }}
SEMAPHORE_PLAYBOOK_PATH=/var/lib/semaphore/playbooks
The env-var bootstrap path is stable in v2.x; only the legacy
'semaphore user add' CLI invocation was unreliable. Drop the manual
user-add step from the README.
Complete rewrite of the semaphore role. Supersedes three prior
iterations whose admin-user-creation logic was unreliable across
Semaphore CLI versions.
Architecture:
- Rootful Podman Quadlet under /etc/containers/systemd/
- Separate PostgreSQL 16-alpine container on a user-defined
podman network (semaphore-net)
- Named volumes for both data stores (semaphore_data,
semaphore_postgres_data) so container recreation is
non-destructive
- Pinned image tags: semaphoreui/semaphore:v2.18.5-ansible2.16.5
and postgres:16-alpine
- Post-deploy HTTP health check fails the playbook if Semaphore
doesn't respond on /api/ping within ~60s
Admin user creation remains intentionally manual after first deploy;
the role README documents the exact podman exec command.
Removes the duplicate deploy_semaphore.yml and the now-unneeded
cleanup_semaphore.yml; day1_deploy_semaphore.yml is the canonical
entry point.
Previous template used inline arithmetic that miscounted the box
width, leaving the right border misaligned on real hostnames. Switch
to computed labels + a fixed inner_width so any hostname / OS string
pads to the same border position.
Introduces a single, idempotent baseline role to supersede the
overlapping day0-baseline and common roles. Capabilities are
feature-flagged so they can be toggled per-host:
- packages (common + OS-family + per-host extras)
- timezone + locale
- chrony time sync against sundial
- baseline users (jarvis admin account with SSH key + NOPASSWD sudo)
- SSH hardening via /etc/ssh/sshd_config.d/ drop-in
- unattended security upgrades (Debian family)
- sysctl drop-in at /etc/sysctl.d/99-mk-labs.conf
- journald retention caps
- branded MOTD
Ubuntu/Debian is first-class; vars/RedHat.yml provides a placeholder
for future distros via the ansible_os_family pattern.
The legacy day0-baseline and common roles remain in place for now and
will be removed during the playbook cleanup sweep, alongside the
existing playbook naming inconsistencies.
- Explicit playbook for removing old container, systemd services, and quadlets
- Optional semaphore_force_clean variable for data removal
- Safer than tags for destructive operations
- Add quadlet tasks and handlers for modern rootless Podman deployment
- Fix broken Jinja in semaphore.container.j2
- Add proper 0777 permissions handling for rootless
- Support semaphore_use_postgres toggle with fallback to legacy BoltDB path
- Parse Host() rules from router definitions
- Supports multiple hostnames per service file (e.g. semaphore + imagineering)
- More robust and future-proof
- Switch podman_container env to SEMAPHORE_DB_DIALECT
- Remove creates: guard on podman generate systemd task
- Add changed_when so unit is always updated on role run
- Updated container quadlet template to use modern SEMAPHORE_DB_DIALECT variable
- Updated env.j2 template for consistency with current SemaphoreUI expectations
- Resolves BoltDB not being honoured on v2.18.x
- Move all roles from playbooks/roles/ to roles/
- Update roles_path in ansible.cfg
- Add cast user to common role
- Create standalone podman role
- Add semaphore role with Podman + Quadlet support
The previous Hermes deployment configuration has been superseded.
This directory is no longer used and is being removed as part of
the transition to the new command centre on carousel-of-progress.
