---
# ------------------------------------------------------------------------------
# Playbook: day2_cpu_offload_aux_models.yml
# Purpose:  CPU-offload Qwen2.5-Coder-14B and Meta-Llama-3.1-8B on
#           astro-orbiter's production router (port 8002).
#
# What this playbook does:
#   1. Re-renders llama-server-router-preset.ini (Coder + Llama sections now
#      use per-model n-gpu-layers vars = 0 -> full CPU inference).
#   2. Re-renders the router unit (--models-max now 4 via host_vars, global
#      --n-gpu-layers removed per t_72646029 unit template fix) and restarts
#      llama-server-router so both changes take effect.
#   3. Verifies per the role's router_preset phase.
#
# Context (2026-08-17):
#   - RAM/model-swap audit, TIER 1 (Coder-14B CPU offload) + TIER 2
#     (Llama-3.1-8B CPU offload) — Ryan approved 1 & 2 on 2026-08-17.
#     See inbox/ryan/2026-08-17-llm-system-ram-model-swap.md.
#   - Unit template fix (t_72646029): global --n-gpu-layers removed from
#     ExecStart in preset mode. Each INI section now sets n-gpu-layers
#     explicitly (Qwen3.8=99, Phi=99, nomic=99, Coder=0, Llama=0).
#   - Concurrent residency after change: Qwen3.8-27B (20,302 MiB @ 128K ctx)
#     + nomic-embed (558 MiB, pinned) + Coder (CPU, ~1,390 MiB CUDA ctx) +
#     Llama (CPU, ~1,706 MiB CUDA ctx) = ~24,004 MiB. NOTE: llama.cpp 6ea215d
#     allocates CUDA-context VRAM even at n-gpu-layers=0, so CPU models are not
#     0-VRAM; total sits at the 24,576 MiB physical limit (headroom ~572 MiB).
#     Qwen3.8 is never evicted for a CPU aux model; Phi-3.5-mini (GPU, 8.3GB)
#     still evicts as before.
#   - CPU speed (8-core Ryzen 7 5800XT): ~5-10 tok/s (14B), ~10-20 tok/s (8B).
#   - Semaphore SSH gap for astro-orbiter still applies (t_730f9584 /
#     t_33acbb2e); running direct CLI Ansible per standing exception.
#
# Run:
#   cd /home/hermes/git/homelab/ansible
#   env -u ANSIBLE_VAULT_PASSWORD_FILE ansible-playbook \
#     -i inventory.yml \
#     playbooks/day2_cpu_offload_aux_models.yml
#
# Rollback:
#   git checkout -- \
#     roles/llm-inference-multimodel/templates/llama-server-router.service.j2 \
#     roles/llm-inference-multimodel/templates/llama-server-router-preset.ini.j2 \
#     roles/llm-inference-multimodel/defaults/main.yml \
#     host_vars/astro-orbiter/vars.yml
#   (restores n-gpu-layers=99 global flag, models-max=2, all GPU)
#   then re-run this playbook to redeploy rollback state.
#   Note: playbooks/day2_cpu_offload_aux_models.yml is untracked — left on disk.
# ------------------------------------------------------------------------------
- name: CPU-offload Coder-14B and Llama-3.1-8B on astro-orbiter
  hosts: astro-orbiter
  become: true
  vars:
    llm_router_preset_enabled: true
    llm_router_enabled: true
    llm_router_port: 8002

  roles:
    - role: llm-inference-multimodel
      tags: [always]