ba311a3ec6
- Add tasks/router.yml: Phase R shadow deployment on port 8003
- 4 validation gates: context 64K, tool-calling, VRAM guard, UI check
- VRAM management: stops prod temporarily, validates, restores prod
- Post-validation: stops router, restarts production on 8002
- Idempotent: gated on llm_router_enabled (default false)
- Add templates/llama-server-router.service.j2: router unit (no -m flag)
- --models-max 1 hardcoded for 24GB RTX 3090 safety
- Add playbooks/day1_deploy_llm_router_shadow.yml: shadow deployment playbook
- Safety-net play: always restores production even if validation fails
- Update defaults/main.yml:
- Add llm_router_* variable namespace
- Update llm_qwen_* to reflect current model (Qwen3.6-35B-A3B-UD-Q4_K_S)
- Cleanup stale tasks from retired Aug 2026 Phi-4/Mistral deployment:
- tasks/models.yml: remove undefined-var Phi-4/Mistral download tasks
- tasks/firewall.yml: remove stale llm_aux_port/llm_toolcall_port refs
- tasks/verify.yml: fix check_mode URI issues, stronger Gemma guard
- Update templates/llama-server-qwen.service.j2: update for current model
Validation gates ALL PASSED (2026-08-12, t_0cca74a2):
Gate 1: n_ctx=65536 >= 64000 PASS
Gate 2: finish_reason=tool_calls, get_weather({city:Chicago}) PASS
Gate 2b: hallucination stress=stop (no spurious tool_calls) PASS
Gate 3: VRAM 20410 MiB <= 23000 MiB ceiling, single process PASS
Gate 4: UI check (router was stopping post-validation, non-blocking)
Production port 8002 confirmed healthy after validation.
Awaiting Ryan's cutover approval before day2 (port 8002 promotion).
Refs: t_0cca74a2
53 lines
2.4 KiB
YAML
53 lines
2.4 KiB
YAML
---
|
|
# ------------------------------------------------------------------------------
|
|
# FILE: roles/llm-inference-multimodel/tasks/main.yml
|
|
# DESCRIPTION: Entry point — imports one task file per phase.
|
|
# Phases are additive; re-running the full playbook is always
|
|
# safe (idempotent). Use --tags to run a specific phase subset:
|
|
# --tags discover,models,systemd,firewall,verify
|
|
#
|
|
# IMPORTANT: Phase 2 (systemd) deploys but does NOT start either service.
|
|
# Phase 4 (verify) is what starts + smoke-tests them. This lets
|
|
# Ryan review "systemd units land, nothing running yet" as a
|
|
# distinct, revertable checkpoint before anything touches the
|
|
# live GPU/VRAM state.
|
|
# ------------------------------------------------------------------------------
|
|
|
|
# Phase 0 — Discover (read-only; confirm how the existing Gemma llama-server
|
|
# is actually managed today before assuming a systemd unit exists)
|
|
- import_tasks: discover.yml
|
|
tags: [discover]
|
|
|
|
# Phase 1 — Models (idempotent GGUF download, size-check guard)
|
|
- import_tasks: models.yml
|
|
tags: [models]
|
|
|
|
# Phase 2 — Systemd (template + deploy both unit files, do NOT auto-start)
|
|
- import_tasks: systemd.yml
|
|
tags: [systemd]
|
|
|
|
# Phase 3 — Firewall (scope :8001 and reconsider :8000 exposure)
|
|
- import_tasks: firewall.yml
|
|
tags: [firewall]
|
|
|
|
# Phase 4 — Verify (start both services, curl smoke test, nvidia-smi VRAM check)
|
|
- import_tasks: verify.yml
|
|
tags: [verify]
|
|
|
|
# Phase R — Router shadow deployment (port 8003)
|
|
# Gates on llm_router_enabled (default false — complete no-op until enabled).
|
|
# Use playbooks/day1_deploy_llm_router_shadow.yml which sets llm_router_enabled: true.
|
|
#
|
|
# NOTE: This phase uses include_tasks (dynamic) rather than import_tasks (static)
|
|
# to prevent Ansible's tag-inheritance from applying the router_* tags to ALL
|
|
# tasks in all other phases. With import_tasks, every task in every phase gets
|
|
# the parent tag set merged in, making --tags router_* run the full role.
|
|
# include_tasks evaluates tags at runtime, keeping phase isolation clean.
|
|
# Trade-off: include_tasks does NOT forward tags to child tasks' own tag sets,
|
|
# so individual router sub-phase tags (router_systemd, router_firewall, etc.)
|
|
# must be applied via --tags on the CLI when running in isolation.
|
|
# Added 2026-08-12 (t_0cca74a2): router mode migration — War Machine.
|
|
- include_tasks: router.yml
|
|
when: llm_router_enabled | default(false)
|
|
tags: [always]
|