081156ecab
Adds idempotent, data-driven GGUF staging for the two new router models on
astro-orbiter alongside the production Qwen3.6-35B-A3B-UD-Q4_K_S. Both files
were already staged live (byte-exact); this commit codifies them in Ansible so
future re-runs and any new model adds are version-controlled and audit-friendly.
Changes:
- roles/llm-inference-multimodel/tasks/stage_model.yml (NEW)
Idempotent per-model task: stat -> exact byte-size guard -> conditional
get_url -> ownership/mode ensure -> notify router restart handler only on
actual download. Loops from models.yml; nothing hardcoded.
- roles/llm-inference-multimodel/tasks/models.yml
Appends the stage_model.yml loop (tagged: models) after the existing Qwen3.6
download tasks. Data driven from host_vars/astro-orbiter/vars.yml.
- roles/llm-inference-multimodel/defaults/main.yml
Adds llm_staged_models: [] default (empty = safe no-op for hosts with no
staged model list defined).
- roles/llm-inference-multimodel/handlers/main.yml
Adds 'restart llama-server-router on new GGUF' handler. Only fires when
stage_model.yml performs an actual download or corrects ownership/mode.
Normal idempotent re-runs (files already correct) do NOT fire this handler.
- host_vars/astro-orbiter/vars.yml
Adds llm_staged_models list with the two new models:
* Phi-3.5-mini-instruct-Q8_0.gguf (4,061,222,688 bytes,
bartowski/Phi-3.5-mini-instruct-GGUF)
* Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf (4,920,739,232 bytes,
bartowski/Meta-Llama-3.1-8B-Instruct-GGUF)
- playbooks/day1_deploy_llm_inference_multimodel.yml
Updates header comment: removes stale 'Semaphore broken' note, documents
the correct execution channel (Semaphore template
llm_inference_multimodel_stage_models, --tags models).
- group_vars/all/semaphore.yml
Adds llm_inference_multimodel_stage_models template entry (config-as-code).
Template is scoped to --tags models explicitly. Phase 4 (verify) is
EXCLUDED: verify.yml starts llama-server-qwen on :8002, which would collide
with the production llama-server-router.service already running on :8002.
Semaphore template created via API: project 1 / template id 19.
Execution: triggered immediately after this commit via Semaphore REST API.
36 lines
1.8 KiB
YAML
36 lines
1.8 KiB
YAML
---
|
|
# ------------------------------------------------------------------------------
|
|
# FILE: playbooks/day1_deploy_llm_inference_multimodel.yml
|
|
# DESCRIPTION: Day 1 playbook for the dual-model (aux + tool-calling) rollout
|
|
# on astro-orbiter. Builds on roles/llm-inference (CUDA/driver
|
|
# already done) — does not replace it.
|
|
#
|
|
# Usage:
|
|
# cd ~/git/homelab/ansible
|
|
# ansible-playbook -i inventory.yml playbooks/day1_deploy_llm_inference_multimodel.yml
|
|
# # or scope to specific phases:
|
|
# ansible-playbook -i inventory.yml playbooks/day1_deploy_llm_inference_multimodel.yml --tags discover
|
|
#
|
|
# EXECUTION CHANNEL (2026-08-12, War Machine): run via the Semaphore template
|
|
# "llm_inference_multimodel_stage_models" (scoped to --tags models). Do NOT
|
|
# run this via direct ansible-playbook or ad-hoc ssh/curl/systemctl — all
|
|
# homelab inference changes go through Ansible roles executed by Semaphore for
|
|
# audit/visibility. Phase 1 (models) is idempotent: it only downloads/stages a
|
|
# GGUF when missing or size-mismatched, and only restarts the router when a new
|
|
# GGUF is detected (normal re-runs that find the files correct touch nothing).
|
|
#
|
|
# Phases (see roles/llm-inference-multimodel/README.md for detail):
|
|
# 0. discover — read-only; confirm existing Gemma service management
|
|
# 1. models — idempotent GGUF downloads (Phi-4-14B, Mistral-Small-24B)
|
|
# 2. systemd — deploy both unit files, do NOT auto-start
|
|
# 3. firewall — scope ports 8000/8001, non-0.0.0.0 bind
|
|
# 4. verify — start both services, smoke test, VRAM check
|
|
# ------------------------------------------------------------------------------
|
|
|
|
- name: Deploy dual-model LLM inference stack on astro-orbiter
|
|
hosts: astro_orbiter
|
|
gather_facts: true
|
|
|
|
roles:
|
|
- role: llm-inference-multimodel
|