081156ecab
Adds idempotent, data-driven GGUF staging for the two new router models on
astro-orbiter alongside the production Qwen3.6-35B-A3B-UD-Q4_K_S. Both files
were already staged live (byte-exact); this commit codifies them in Ansible so
future re-runs and any new model adds are version-controlled and audit-friendly.
Changes:
- roles/llm-inference-multimodel/tasks/stage_model.yml (NEW)
Idempotent per-model task: stat -> exact byte-size guard -> conditional
get_url -> ownership/mode ensure -> notify router restart handler only on
actual download. Loops from models.yml; nothing hardcoded.
- roles/llm-inference-multimodel/tasks/models.yml
Appends the stage_model.yml loop (tagged: models) after the existing Qwen3.6
download tasks. Data driven from host_vars/astro-orbiter/vars.yml.
- roles/llm-inference-multimodel/defaults/main.yml
Adds llm_staged_models: [] default (empty = safe no-op for hosts with no
staged model list defined).
- roles/llm-inference-multimodel/handlers/main.yml
Adds 'restart llama-server-router on new GGUF' handler. Only fires when
stage_model.yml performs an actual download or corrects ownership/mode.
Normal idempotent re-runs (files already correct) do NOT fire this handler.
- host_vars/astro-orbiter/vars.yml
Adds llm_staged_models list with the two new models:
* Phi-3.5-mini-instruct-Q8_0.gguf (4,061,222,688 bytes,
bartowski/Phi-3.5-mini-instruct-GGUF)
* Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf (4,920,739,232 bytes,
bartowski/Meta-Llama-3.1-8B-Instruct-GGUF)
- playbooks/day1_deploy_llm_inference_multimodel.yml
Updates header comment: removes stale 'Semaphore broken' note, documents
the correct execution channel (Semaphore template
llm_inference_multimodel_stage_models, --tags models).
- group_vars/all/semaphore.yml
Adds llm_inference_multimodel_stage_models template entry (config-as-code).
Template is scoped to --tags models explicitly. Phase 4 (verify) is
EXCLUDED: verify.yml starts llama-server-qwen on :8002, which would collide
with the production llama-server-router.service already running on :8002.
Semaphore template created via API: project 1 / template id 19.
Execution: triggered immediately after this commit via Semaphore REST API.
42 lines
2.1 KiB
YAML
42 lines
2.1 KiB
YAML
---
|
|
# ------------------------------------------------------------------------------
|
|
# FILE: ansible/host_vars/astro_orbiter/vars.yml
|
|
# HOST: astro-orbiter (10.1.71.130)
|
|
# ROLE: llama.cpp LLM inference host — Ryzen 7 5800XT / RTX 3090 (ATX rebuild,
|
|
# 2026-08-04). Superseded the prior AMD RX 5700 / Ollama config below;
|
|
# drive was transplanted into new hardware, not reinstalled.
|
|
# ------------------------------------------------------------------------------
|
|
|
|
ansible_host: 10.1.71.130
|
|
ansible_user: jarvis
|
|
ansible_ssh_private_key_file: ~/.ssh/id_jarvis
|
|
ansible_become: true
|
|
|
|
# LVM root expansion — xlarge template uses sda3 partition, standard VG/LV names
|
|
common_expand_root_lvm: true
|
|
common_root_pv: /dev/sda3
|
|
common_root_vg: ubuntu-vg
|
|
common_root_lv: ubuntu-lv
|
|
|
|
# --- Staged GGUF models for the llama.cpp router (:8002) ---------------------
|
|
# Data-driven list consumed by roles/llm-inference-multimodel tasks/models.yml
|
|
# (loop -> tasks/stage_model.yml). Each entry is idempotently staged into
|
|
# /opt/models: stat + EXACT-size check vs HF manifest; skip (no download, no
|
|
# restart) when present + size matches. Source repos are public bartowski GGUFs
|
|
# on HuggingFace (no auth). A router restart is notified ONLY when a new GGUF
|
|
# is actually downloaded.
|
|
# Added 2026-08-12 (War Machine): codify Phi-3.5-mini-instruct-Q8_0 and
|
|
# Meta-Llama-3.1-8B-Instruct-Q4_K_M as router models alongside the production
|
|
# Qwen3.6-35B-A3B-UD-Q4_K_S. The live files were already present/correct on
|
|
# astro-orbiter; this pass codifies them. Future adds = append to this list.
|
|
llm_staged_models:
|
|
- filename: "Phi-3.5-mini-instruct-Q8_0.gguf"
|
|
url: "https://huggingface.co/bartowski/Phi-3.5-mini-instruct-GGUF/resolve/main/Phi-3.5-mini-instruct-Q8_0.gguf"
|
|
size_bytes: 4061222688
|
|
source_repo: "bartowski/Phi-3.5-mini-instruct-GGUF"
|
|
- filename: "Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf"
|
|
url: "https://huggingface.co/bartowski/Meta-Llama-3.1-8B-Instruct-GGUF/resolve/main/Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf"
|
|
size_bytes: 4920739232
|
|
source_repo: "bartowski/Meta-Llama-3.1-8B-Instruct-GGUF"
|
|
|