feat(llm-inference-multimodel): codify Phi-3.5-mini + Llama-3.1-8B GGUF staging (t_730f9584)

Adds idempotent, data-driven GGUF staging for the two new router models on
astro-orbiter alongside the production Qwen3.6-35B-A3B-UD-Q4_K_S. Both files
were already staged live (byte-exact); this commit codifies them in Ansible so
future re-runs and any new model adds are version-controlled and audit-friendly.

Changes:
- roles/llm-inference-multimodel/tasks/stage_model.yml (NEW)
  Idempotent per-model task: stat -> exact byte-size guard -> conditional
  get_url -> ownership/mode ensure -> notify router restart handler only on
  actual download. Loops from models.yml; nothing hardcoded.

- roles/llm-inference-multimodel/tasks/models.yml
  Appends the stage_model.yml loop (tagged: models) after the existing Qwen3.6
  download tasks. Data driven from host_vars/astro-orbiter/vars.yml.

- roles/llm-inference-multimodel/defaults/main.yml
  Adds llm_staged_models: [] default (empty = safe no-op for hosts with no
  staged model list defined).

- roles/llm-inference-multimodel/handlers/main.yml
  Adds 'restart llama-server-router on new GGUF' handler. Only fires when
  stage_model.yml performs an actual download or corrects ownership/mode.
  Normal idempotent re-runs (files already correct) do NOT fire this handler.

- host_vars/astro-orbiter/vars.yml
  Adds llm_staged_models list with the two new models:
    * Phi-3.5-mini-instruct-Q8_0.gguf (4,061,222,688 bytes,
      bartowski/Phi-3.5-mini-instruct-GGUF)
    * Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf (4,920,739,232 bytes,
      bartowski/Meta-Llama-3.1-8B-Instruct-GGUF)

- playbooks/day1_deploy_llm_inference_multimodel.yml
  Updates header comment: removes stale 'Semaphore broken' note, documents
  the correct execution channel (Semaphore template
  llm_inference_multimodel_stage_models, --tags models).

- group_vars/all/semaphore.yml
  Adds llm_inference_multimodel_stage_models template entry (config-as-code).
  Template is scoped to --tags models explicitly. Phase 4 (verify) is
  EXCLUDED: verify.yml starts llama-server-qwen on :8002, which would collide
  with the production llama-server-router.service already running on :8002.

Semaphore template created via API: project 1 / template id 19.
Execution: triggered immediately after this commit via Semaphore REST API.
This commit is contained in:
Hermes Agent service account
2026-08-12 22:19:34 -05:00
parent 3783ded62a
commit 081156ecab
7 changed files with 162 additions and 3 deletions

View File

@@ -79,6 +79,21 @@ llm_qwen_expected_vram_gb: 20 # verified 2026-08-07: ~20,390 MiB / 24,576 MiB
# llm_qwen_model_url is intentionally not set — see models.yml WARN task for
# the HuggingFace URL if a re-download is ever needed.
# --- Staged GGUF models (data-driven, idempotent staging) --------------------
# Additional GGUFs to ensure are present in llm_models_dir, alongside the
# production Qwen3.6-35B. Consumed by tasks/models.yml (loop over
# tasks/stage_model.yml). Each entry:
# filename: target filename in llm_models_dir
# url: HuggingFace resolve URL (public repos; no auth needed)
# size_bytes: EXACT expected byte size (HF manifest) — guard: download only
# if the file is missing OR its size != this value (idempotent;
# never re-pulls a correct file, never needlessly restarts).
# source_repo: upstream HF repo (audit/lineage)
# The REAL list is defined per-host in host_vars/astro-orbiter/vars.yml (NOT
# hardcoded here) so the role stays generic and reusable for future model adds.
# Empty default = nothing staged (safe no-op).
llm_staged_models: []
# --- Existing Gemma baseline (rollback target — never modified by this role) -
# Populated by Phase 0 discovery (tasks/discover.yml) if not already known.
# Set here only as a fallback name to search for; discovery is authoritative.