feat(llm-router): switch to --models-preset mode; add Phi-3.5-mini-instruct-8bit alias (t_9adf0889)

- Add templates/llama-server-router-preset.ini.j2: defines all 3 router GGUFs
  (Qwen3.6-35B, Phi-3.5-mini-Q8_0, Meta-Llama-3.1-8B) with explicit ctx-size,
  gpu-layers, cache settings carried over from --models-dir baseline. The Phi
  entry adds alias=Phi-3.5-mini-instruct-8bit (Ryan's aux.title_generation target).

- Update templates/llama-server-router.service.j2: Jinja2 conditional emits
  --models-preset <path> when llm_router_preset_enabled=true, otherwise
  --models-dir (backward compat, default unchanged).

- Add tasks/preset.yml: deploy preset INI, restart router on change, verify
  both Phi-3.5-mini-instruct-Q8_0 (primary) and Phi-3.5-mini-instruct-8bit
  (alias) appear in /v1/models, plus Qwen and Llama IDs unchanged.

- Update defaults/main.yml: add llm_router_preset_enabled (default false) and
  llm_router_preset_path=/opt/llama-server-router-preset.ini.

- Update tasks/main.yml: import preset.yml as Phase P (gated, no-op by default).
- Update handlers/main.yml: add 'restart router' handler for preset changes.
- Add playbooks/day2_add_phi_alias.yml: single-command deployment.

GH #22364 note: --models-preset mode creates an extra 'default' entry in
/v1/models — cosmetic, does not affect model selection by name.
This commit is contained in:
Hermes Agent service account
2026-08-12 22:58:12 -05:00
parent 9c969f783d
commit a47b29d49f
7 changed files with 489 additions and 6 deletions

View File

@@ -50,3 +50,13 @@
- include_tasks: router.yml
when: llm_router_enabled | default(false)
tags: [always]
# Phase P — Router preset mode / alias deployment
# Gates on llm_router_preset_enabled (default false — complete no-op until enabled).
# Use playbooks/day2_add_phi_alias.yml which sets llm_router_preset_enabled: true.
# Purpose: switch from --models-dir to --models-preset to enable model aliases.
# The Phi-3.5-mini-instruct-Q8_0 entry gains alias "Phi-3.5-mini-instruct-8bit".
# Added 2026-08-12 (t_9adf0889): Phi alias — War Machine.
- include_tasks: preset.yml
when: llm_router_preset_enabled | default(false)
tags: [always]