feat(llm-inference-multimodel): codify Phi-3.5-mini + Llama-3.1-8B GGUF staging (t_730f9584)
Adds idempotent, data-driven GGUF staging for the two new router models on
astro-orbiter alongside the production Qwen3.6-35B-A3B-UD-Q4_K_S. Both files
were already staged live (byte-exact); this commit codifies them in Ansible so
future re-runs and any new model adds are version-controlled and audit-friendly.
Changes:
- roles/llm-inference-multimodel/tasks/stage_model.yml (NEW)
Idempotent per-model task: stat -> exact byte-size guard -> conditional
get_url -> ownership/mode ensure -> notify router restart handler only on
actual download. Loops from models.yml; nothing hardcoded.
- roles/llm-inference-multimodel/tasks/models.yml
Appends the stage_model.yml loop (tagged: models) after the existing Qwen3.6
download tasks. Data driven from host_vars/astro-orbiter/vars.yml.
- roles/llm-inference-multimodel/defaults/main.yml
Adds llm_staged_models: [] default (empty = safe no-op for hosts with no
staged model list defined).
- roles/llm-inference-multimodel/handlers/main.yml
Adds 'restart llama-server-router on new GGUF' handler. Only fires when
stage_model.yml performs an actual download or corrects ownership/mode.
Normal idempotent re-runs (files already correct) do NOT fire this handler.
- host_vars/astro-orbiter/vars.yml
Adds llm_staged_models list with the two new models:
* Phi-3.5-mini-instruct-Q8_0.gguf (4,061,222,688 bytes,
bartowski/Phi-3.5-mini-instruct-GGUF)
* Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf (4,920,739,232 bytes,
bartowski/Meta-Llama-3.1-8B-Instruct-GGUF)
- playbooks/day1_deploy_llm_inference_multimodel.yml
Updates header comment: removes stale 'Semaphore broken' note, documents
the correct execution channel (Semaphore template
llm_inference_multimodel_stage_models, --tags models).
- group_vars/all/semaphore.yml
Adds llm_inference_multimodel_stage_models template entry (config-as-code).
Template is scoped to --tags models explicitly. Phase 4 (verify) is
EXCLUDED: verify.yml starts llama-server-qwen on :8002, which would collide
with the production llama-server-router.service already running on :8002.
Semaphore template created via API: project 1 / template id 19.
Execution: triggered immediately after this commit via Semaphore REST API.
This commit is contained in:
@@ -152,3 +152,28 @@ semaphore_config:
|
||||
required: true
|
||||
type: TextVar
|
||||
default_value: "all"
|
||||
|
||||
- name: "llm_inference_multimodel_stage_models"
|
||||
description: >-
|
||||
Stage additional GGUF models into /opt/models on astro-orbiter via the
|
||||
llm-inference-multimodel role (--tags models only). Idempotent: skips
|
||||
files already present at the correct byte size. Notifies the
|
||||
llama-server-router restart handler ONLY when a new GGUF is actually
|
||||
downloaded. Does NOT touch Phase 4 (verify) or the legacy
|
||||
llama-server-qwen service. Safe to run repeatedly.
|
||||
app: ansible
|
||||
playbook: ansible/playbooks/day1_deploy_llm_inference_multimodel.yml
|
||||
inventory: production
|
||||
repository: homelab
|
||||
environment: default
|
||||
vault_password: ansible-vault-pass
|
||||
arguments: '["--tags","models","--diff"]'
|
||||
# Scoped to --tags models:
|
||||
# Phase 0 (discover) -- skipped (no tag)
|
||||
# Phase 1 (models) -- RUN (idempotent GGUF staging via stage_model.yml)
|
||||
# Phase 2 (systemd) -- skipped
|
||||
# Phase 3 (firewall) -- skipped
|
||||
# Phase 4 (verify) -- SKIPPED (collision risk: verify.yml would start
|
||||
# llama-server-qwen on :8002, conflicting with the
|
||||
# production llama-server-router.service. Excluded
|
||||
# here deliberately. See t_730f9584.)
|
||||
|
||||
Reference in New Issue
Block a user