a47b29d49f
- Add templates/llama-server-router-preset.ini.j2: defines all 3 router GGUFs (Qwen3.6-35B, Phi-3.5-mini-Q8_0, Meta-Llama-3.1-8B) with explicit ctx-size, gpu-layers, cache settings carried over from --models-dir baseline. The Phi entry adds alias=Phi-3.5-mini-instruct-8bit (Ryan's aux.title_generation target). - Update templates/llama-server-router.service.j2: Jinja2 conditional emits --models-preset <path> when llm_router_preset_enabled=true, otherwise --models-dir (backward compat, default unchanged). - Add tasks/preset.yml: deploy preset INI, restart router on change, verify both Phi-3.5-mini-instruct-Q8_0 (primary) and Phi-3.5-mini-instruct-8bit (alias) appear in /v1/models, plus Qwen and Llama IDs unchanged. - Update defaults/main.yml: add llm_router_preset_enabled (default false) and llm_router_preset_path=/opt/llama-server-router-preset.ini. - Update tasks/main.yml: import preset.yml as Phase P (gated, no-op by default). - Update handlers/main.yml: add 'restart router' handler for preset changes. - Add playbooks/day2_add_phi_alias.yml: single-command deployment. GH #22364 note: --models-preset mode creates an extra 'default' entry in /v1/models — cosmetic, does not affect model selection by name.
63 lines
3.0 KiB
YAML
63 lines
3.0 KiB
YAML
---
|
|
# ------------------------------------------------------------------------------
|
|
# FILE: roles/llm-inference-multimodel/tasks/main.yml
|
|
# DESCRIPTION: Entry point — imports one task file per phase.
|
|
# Phases are additive; re-running the full playbook is always
|
|
# safe (idempotent). Use --tags to run a specific phase subset:
|
|
# --tags discover,models,systemd,firewall,verify
|
|
#
|
|
# IMPORTANT: Phase 2 (systemd) deploys but does NOT start either service.
|
|
# Phase 4 (verify) is what starts + smoke-tests them. This lets
|
|
# Ryan review "systemd units land, nothing running yet" as a
|
|
# distinct, revertable checkpoint before anything touches the
|
|
# live GPU/VRAM state.
|
|
# ------------------------------------------------------------------------------
|
|
|
|
# Phase 0 — Discover (read-only; confirm how the existing Gemma llama-server
|
|
# is actually managed today before assuming a systemd unit exists)
|
|
- import_tasks: discover.yml
|
|
tags: [discover]
|
|
|
|
# Phase 1 — Models (idempotent GGUF download, size-check guard)
|
|
- import_tasks: models.yml
|
|
tags: [models]
|
|
|
|
# Phase 2 — Systemd (template + deploy both unit files, do NOT auto-start)
|
|
- import_tasks: systemd.yml
|
|
tags: [systemd]
|
|
|
|
# Phase 3 — Firewall (scope :8001 and reconsider :8000 exposure)
|
|
- import_tasks: firewall.yml
|
|
tags: [firewall]
|
|
|
|
# Phase 4 — Verify (start both services, curl smoke test, nvidia-smi VRAM check)
|
|
- import_tasks: verify.yml
|
|
tags: [verify]
|
|
|
|
# Phase R — Router shadow deployment (port 8003)
|
|
# Gates on llm_router_enabled (default false — complete no-op until enabled).
|
|
# Use playbooks/day1_deploy_llm_router_shadow.yml which sets llm_router_enabled: true.
|
|
#
|
|
# NOTE: This phase uses include_tasks (dynamic) rather than import_tasks (static)
|
|
# to prevent Ansible's tag-inheritance from applying the router_* tags to ALL
|
|
# tasks in all other phases. With import_tasks, every task in every phase gets
|
|
# the parent tag set merged in, making --tags router_* run the full role.
|
|
# include_tasks evaluates tags at runtime, keeping phase isolation clean.
|
|
# Trade-off: include_tasks does NOT forward tags to child tasks' own tag sets,
|
|
# so individual router sub-phase tags (router_systemd, router_firewall, etc.)
|
|
# must be applied via --tags on the CLI when running in isolation.
|
|
# Added 2026-08-12 (t_0cca74a2): router mode migration — War Machine.
|
|
- include_tasks: router.yml
|
|
when: llm_router_enabled | default(false)
|
|
tags: [always]
|
|
|
|
# Phase P — Router preset mode / alias deployment
|
|
# Gates on llm_router_preset_enabled (default false — complete no-op until enabled).
|
|
# Use playbooks/day2_add_phi_alias.yml which sets llm_router_preset_enabled: true.
|
|
# Purpose: switch from --models-dir to --models-preset to enable model aliases.
|
|
# The Phi-3.5-mini-instruct-Q8_0 entry gains alias "Phi-3.5-mini-instruct-8bit".
|
|
# Added 2026-08-12 (t_9adf0889): Phi alias — War Machine.
|
|
- include_tasks: preset.yml
|
|
when: llm_router_preset_enabled | default(false)
|
|
tags: [always]
|