feat(llm-inference-multimodel): bump router --models-max 1->4 on astro-orbiter (t_33acbb2e)

Changes:
- host_vars/astro-orbiter/vars.yml: add llm_router_models_max: 4 (overrides
  conservative default of 1). Detailed VRAM OOM risk note included inline:
  worst-case 3-model co-residency ~31GB > 24GB RTX 3090. LRU eviction
  mitigates in single-user operation; flagged for Ryan's review.
- playbooks/day2_bump_router_models_max.yml: new targeted playbook; deploys
  updated router unit, restarts the live service, verifies /health 200 and
  /v1/models lists all 3 GGUFs post-restart.
- group_vars/all/semaphore.yml: add llm_router_update_unit template pointing
  at the new playbook.
- roles/llm-inference-multimodel/defaults/main.yml: update comment to reflect
  the var is now overridden in host_vars rather than 'hardcoded to 1'.
- roles/llm-inference-multimodel/templates/llama-server-router.service.j2:
  correct stale 'HARDCODED TO 1' comment — value is variable-driven.

Constraints honored:
- --parallel 1 left untouched (not in scope, not modified anywhere)
- No ad-hoc SSH/systemctl/curl state mutation; all execution via Semaphore
- No installed/vendored code patched
This commit is contained in:
Hermes Agent service account
2026-08-12 22:26:49 -05:00
parent 081156ecab
commit 9c969f783d
5 changed files with 205 additions and 5 deletions

View File

@@ -177,3 +177,17 @@ semaphore_config:
# llama-server-qwen on :8002, conflicting with the
# production llama-server-router.service. Excluded
# here deliberately. See t_730f9584.)
- name: "llm_router_update_unit"
description: >-
Re-render and reload the llama-server-router systemd unit on astro-orbiter,
then restart the live service so new args (e.g. --models-max) take effect.
Drives playbooks/day2_bump_router_models_max.yml. Added 2026-08-12 (t_33acbb2e):
bump --models-max 1 -> 4 with full VRAM budget note in host_vars.
app: ansible
playbook: ansible/playbooks/day2_bump_router_models_max.yml
inventory: production
repository: homelab
environment: default
vault_password: ansible-vault-pass
arguments: '["--diff"]'