a47b29d49f
- Add templates/llama-server-router-preset.ini.j2: defines all 3 router GGUFs (Qwen3.6-35B, Phi-3.5-mini-Q8_0, Meta-Llama-3.1-8B) with explicit ctx-size, gpu-layers, cache settings carried over from --models-dir baseline. The Phi entry adds alias=Phi-3.5-mini-instruct-8bit (Ryan's aux.title_generation target). - Update templates/llama-server-router.service.j2: Jinja2 conditional emits --models-preset <path> when llm_router_preset_enabled=true, otherwise --models-dir (backward compat, default unchanged). - Add tasks/preset.yml: deploy preset INI, restart router on change, verify both Phi-3.5-mini-instruct-Q8_0 (primary) and Phi-3.5-mini-instruct-8bit (alias) appear in /v1/models, plus Qwen and Llama IDs unchanged. - Update defaults/main.yml: add llm_router_preset_enabled (default false) and llm_router_preset_path=/opt/llama-server-router-preset.ini. - Update tasks/main.yml: import preset.yml as Phase P (gated, no-op by default). - Update handlers/main.yml: add 'restart router' handler for preset changes. - Add playbooks/day2_add_phi_alias.yml: single-command deployment. GH #22364 note: --models-preset mode creates an extra 'default' entry in /v1/models — cosmetic, does not affect model selection by name.
47 lines
2.3 KiB
YAML
47 lines
2.3 KiB
YAML
---
|
|
# ------------------------------------------------------------------------------
|
|
# FILE: roles/llm-inference-multimodel/handlers/main.yml
|
|
# DESCRIPTION: Only a daemon-reload handler lives here now (harmless, no
|
|
# process impact). Per-service restart/start decisions are made
|
|
# explicitly in tasks/verify.yml (Phase 4), keyed off the
|
|
# per-unit `changed` result registered in tasks/systemd.yml
|
|
# (Phase 2) — NEVER combined, so a content change to one unit
|
|
# template still never restarts the other (plan §2/§6
|
|
# requirement: independent restart/rollback).
|
|
#
|
|
# BUGFIX: this file used to also define "restart
|
|
# llama-server-aux" / "restart llama-server-toolcall" handlers,
|
|
# notified from Phase 2's template tasks and fired there via
|
|
# `meta: flush_handlers` — causing both live services to
|
|
# restart during Phase 2, before Phase 3/4 had run. See
|
|
# tasks/systemd.yml for the full writeup. Restart logic moved
|
|
# to tasks/verify.yml so it only ever fires in Phase 4.
|
|
# ------------------------------------------------------------------------------
|
|
|
|
- name: Reload systemd
|
|
ansible.builtin.systemd:
|
|
daemon_reload: true
|
|
become: true
|
|
listen: "reload systemd"
|
|
|
|
# Restart the llama.cpp router so it re-discovers /opt/models after a NEW GGUF
|
|
# is staged. NOTIFIED ONLY from tasks/stage_model.yml when an actual download
|
|
# (or permission correction) occurs — a normal idempotent re-run that finds the
|
|
# files already correct will NOT fire this, so the live router is left
|
|
# untouched. Safe on the idle GPU (router holds no resident model when all
|
|
# entries are "unloaded"; restart is sub-second).
|
|
- name: restart llama-server-router on new GGUF
|
|
ansible.builtin.systemd:
|
|
name: "{{ llm_router_service_name | default('llama-server-router') }}"
|
|
state: restarted
|
|
become: true
|
|
|
|
# Restart handler for preset mode changes (notified by tasks/preset.yml when
|
|
# the preset INI or unit file changes). Distinct from "restart llama-server-router
|
|
# on new GGUF" so each change path can notify independently.
|
|
- name: restart router
|
|
ansible.builtin.systemd:
|
|
name: "{{ llm_router_service_name | default('llama-server-router') }}"
|
|
state: restarted
|
|
become: true
|