Changes:
- host_vars/astro-orbiter/vars.yml: add llm_router_models_max: 4 (overrides
conservative default of 1). Detailed VRAM OOM risk note included inline:
worst-case 3-model co-residency ~31GB > 24GB RTX 3090. LRU eviction
mitigates in single-user operation; flagged for Ryan's review.
- playbooks/day2_bump_router_models_max.yml: new targeted playbook; deploys
updated router unit, restarts the live service, verifies /health 200 and
/v1/models lists all 3 GGUFs post-restart.
- group_vars/all/semaphore.yml: add llm_router_update_unit template pointing
at the new playbook.
- roles/llm-inference-multimodel/defaults/main.yml: update comment to reflect
the var is now overridden in host_vars rather than 'hardcoded to 1'.
- roles/llm-inference-multimodel/templates/llama-server-router.service.j2:
correct stale 'HARDCODED TO 1' comment — value is variable-driven.
Constraints honored:
- --parallel 1 left untouched (not in scope, not modified anywhere)
- No ad-hoc SSH/systemctl/curl state mutation; all execution via Semaphore
- No installed/vendored code patched
Adds idempotent, data-driven GGUF staging for the two new router models on
astro-orbiter alongside the production Qwen3.6-35B-A3B-UD-Q4_K_S. Both files
were already staged live (byte-exact); this commit codifies them in Ansible so
future re-runs and any new model adds are version-controlled and audit-friendly.
Changes:
- roles/llm-inference-multimodel/tasks/stage_model.yml (NEW)
Idempotent per-model task: stat -> exact byte-size guard -> conditional
get_url -> ownership/mode ensure -> notify router restart handler only on
actual download. Loops from models.yml; nothing hardcoded.
- roles/llm-inference-multimodel/tasks/models.yml
Appends the stage_model.yml loop (tagged: models) after the existing Qwen3.6
download tasks. Data driven from host_vars/astro-orbiter/vars.yml.
- roles/llm-inference-multimodel/defaults/main.yml
Adds llm_staged_models: [] default (empty = safe no-op for hosts with no
staged model list defined).
- roles/llm-inference-multimodel/handlers/main.yml
Adds 'restart llama-server-router on new GGUF' handler. Only fires when
stage_model.yml performs an actual download or corrects ownership/mode.
Normal idempotent re-runs (files already correct) do NOT fire this handler.
- host_vars/astro-orbiter/vars.yml
Adds llm_staged_models list with the two new models:
* Phi-3.5-mini-instruct-Q8_0.gguf (4,061,222,688 bytes,
bartowski/Phi-3.5-mini-instruct-GGUF)
* Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf (4,920,739,232 bytes,
bartowski/Meta-Llama-3.1-8B-Instruct-GGUF)
- playbooks/day1_deploy_llm_inference_multimodel.yml
Updates header comment: removes stale 'Semaphore broken' note, documents
the correct execution channel (Semaphore template
llm_inference_multimodel_stage_models, --tags models).
- group_vars/all/semaphore.yml
Adds llm_inference_multimodel_stage_models template entry (config-as-code).
Template is scoped to --tags models explicitly. Phase 4 (verify) is
EXCLUDED: verify.yml starts llama-server-qwen on :8002, which would collide
with the production llama-server-router.service already running on :8002.
Semaphore template created via API: project 1 / template id 19.
Execution: triggered immediately after this commit via Semaphore REST API.
Semaphore runs ansible-playbook from the repo root, so ansible.cfg in
ansible/ is never loaded. The env var is the reliable path.
Also persists the setting in group_vars so semaphore_configure re-runs
don't regress it.
Semaphore runs ansible-playbook from the cloned repo root, not from
the playbook directory. The previous value '../roles' resolved
outside the repo. Correct path is 'ansible/roles'.
Semaphore runs ansible-playbook from the playbook's directory, not
from ansible/ where ansible.cfg lives. Roles aren't found at runtime:
'role linux-baseline was not found'.
Set ANSIBLE_ROLES_PATH=../roles in the default environment so the
search picks up ansible/roles/ relative to the playbook directory.
Semaphore rejected the previous become_key=wed-ssh with 'access key
type not supported for ansible become user' — that field is for a
sudo PASSWORD (login_password type), not a reused SSH key. wed has
passwordless sudo on every host (set up by the VM template), so the
correct value is the built-in 'None' key.
Adds wed-ssh (the universal automation account pre-baked in every
VM template) to the declared key set and switches the production
inventory to use it for both ssh_key and become_key. Retains
jarvis-ssh for cases that specifically need admin-level access.
This aligns Semaphore-driven jobs with the established homelab
convention: wed runs the playbooks, jarvis is the higher-privilege
admin account provisioned by linux-baseline.
Operator (Ryan) cleaned out the previous templates + inventory in
Semaphore before this commit so the configure step re-creates them
wired to wed-ssh on its next run.
Adds an idempotent configuration pass that drives a freshly-deployed
Semaphore instance into its desired state via the REST API. Declared
in group_vars/all/semaphore.yml, applied by tasks/configure.yml,
toggled by semaphore_configure feature flag (default off).
Object types managed:
- Project (mk-labs)
- Keys (ansible-vault-pass, gitea-deploy, jarvis-ssh)
- Repositories (homelab on gitea)
- Inventories (production -> ansible/inventory.yml in homelab repo)
- Environments (default with ANSIBLE_HOST_KEY_CHECKING=False)
- Templates (day0_linux_baseline + variants, day1_deploy_semaphore)
with survey vars for runtime parameters
Each object found-or-created by name; existing ones never modified.
no_log on token-bearing calls to keep secrets out of stdout.
Inputs (already in vault):
vault_semaphore_api_token
vault_jarvis_ssh_private_key
vault_gitea_deploy_key
vault_ansible_vault_password
