llm-router: CPU-offload Coder-14B + Llama-3.1-8B (t_72646029)

- Remove global --n-gpu-layers from router unit ExecStart in preset mode
  (llama.cpp CLI arg outranked per-model INI n-gpu-layers=0; root cause from
  War Machine's run 1). Flag now emitted only in --models-dir mode.
- All 5 preset INI sections carry explicit n-gpu-layers:
  Qwen3.8=99, Phi=99, nomic=99, Coder=0, Llama=0.
- host_vars/astro-orbiter: llm_router_models_max 2 -> 4 so CPU-offloaded
  models count as loaded without LRU-evicting Qwen3.8.
- defaults: llm_router_coder_gpu_layers / llm_router_llama_gpu_layers = 0.
- verify.yml: fix pre-existing .meta attribute crash in router mode.
- New playbook day2_cpu_offload_aux_models.yml.

Deployed + verified on astro-orbiter (gates A-E PASS): concurrent residency
achieved, Qwen3.8 stays GPU-resident. Measured CPU throughput Llama 9.0 /
Coder 4.7 tok/s. VRAM note: llama.cpp 6ea215d allocates ~1.4-1.7GB CUDA-context
per CPU model even at n-gpu-layers=0 -> ~24,004 MiB steady-state, below the
24,576 MiB physical limit. Comments corrected to match the measurement.
Report: friday/inbox/ryan/2026-08-17-llm-cpu-offload-coder-llama-deployed.md

ansible/roles/llm-inference-multimodel/templates/llama-server-router-preset.ini.j2

@@ -68,17 +68,12 @@
 ; back to non-fused implementation. Inference works correctly but may be
 ; slower on the GDN layers. An updated llama.cpp may improve throughput.
 ; load-on-startup NOT set (loads on first request, ~30-60s cold load).
-; With models-max=2 in host_vars, nomic-embed occupies slot 1 (pinned),
-; and the generative slot (slot 2) is Qwen3.8 on first request. Auxiliary
-; models (Phi, Llama, Coder) evict Qwen3.8 when requested; Qwen3.8 evicts
-; them in turn. One cold-load (~30-60s) per switch between Qwen3.8 and
-; auxiliary models is expected and acceptable. In practice, once Hermes
-; config.yaml references Qwen3.8 as primary, it stays resident.
-; ctx-size raised to 131072 (128K) per Ryan approval (t_441470b9, 2026-08-16).
+; n-gpu-layers=99: GPU (all layers). Explicit here so global CLI flag removal
+; (t_72646029, 2026-08-17) does not change Qwen3.8 behavior.
 ; Primary model ID: Qwen3.8-27B-Q4_K_M
 [Qwen3.8-27B-Q4_K_M]
 model              = {{ llm_models_dir }}/Qwen3.8-27B-Q4_K_M.gguf
-n-gpu-layers       = {{ llm_router_gpu_layers }}
+n-gpu-layers       = 99
 ctx-size           = {{ llm_router_qwen38_ctx_size }}
 cache-type-k       = {{ llm_router_cache_type_k }}
 cache-type-v       = {{ llm_router_cache_type_v }}
@@ -110,7 +105,7 @@ parallel           = {{ llm_router_parallel }}
 [Phi-3.5-mini-instruct-Q8_0]
 model        = {{ llm_models_dir }}/Phi-3.5-mini-instruct-Q8_0.gguf
 alias        = Phi-3.5-mini-instruct-8bit
-n-gpu-layers = {{ llm_router_gpu_layers }}
+n-gpu-layers = 99
 ctx-size     = {{ llm_router_phi_ctx_size }}
 flash-attn   = {{ llm_router_phi_flash_attn }}
 cache-type-k = {{ llm_router_cache_type_k }}
@@ -124,10 +119,15 @@ parallel     = {{ llm_router_parallel }}
 ; Alias:            Meta-Llama-3.1-8B-Instruct-4bit (NEW — friendlier name)
 ; Both names resolve to this GGUF child process.
 ; ~4.6GB, general-purpose small model. Works with json_schema structured output.
+; n-gpu-layers=0 (CPU offload, t_72646029 2026-08-17): Llama moves to full CPU
+; inference to allow concurrent residency with Qwen3.8-27B (which uses ~20.8GB
+; VRAM including nomic-embed). At models-max=4, Llama and Coder run on CPU —
+; llama.cpp 6ea215d still holds ~1.4-1.7GB CUDA-context VRAM per CPU model, so
+; steady-state is ~24,004 MiB (below the 24,576 MiB physical limit).
 [Meta-Llama-3.1-8B-Instruct-Q4_K_M]
 model        = {{ llm_models_dir }}/Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf
 alias        = Meta-Llama-3.1-8B-Instruct-4bit
-n-gpu-layers = {{ llm_router_gpu_layers }}
+n-gpu-layers = {{ llm_router_llama_gpu_layers }}
 ctx-size     = {{ llm_router_llama_ctx_size }}
 flash-attn   = {{ llm_router_llama_flash_attn }}
 cache-type-k = {{ llm_router_cache_type_k }}
@@ -140,14 +140,19 @@ parallel     = {{ llm_router_parallel }}
 ; Primary model ID: Qwen2.5-Coder-14B-Instruct-Q4_K_M (filename-derived)
 ; Alias:            Qwen2.5-Coder-14B-Instruct-4bit (friendlier name)
 ; Both names resolve to this GGUF child process.
-; ~8.4GB weights + ~0.6GB KV @ 16K ctx = ~9.0GB VRAM.
+; ~8.4GB weights + ~0.6GB KV @ 16K ctx = ~9.0GB VRAM (GPU); ~1,390 MiB CUDA ctx (CPU).
 ; ctx-size=16384, flash-attn=true per task t_55c164f5 / Ryan's request.
 ; Source: bartowski/Qwen2.5-Coder-14B-Instruct-GGUF (public, no auth)
 ; Added 2026-08-13 (t_55c164f5) — War Machine.
+; n-gpu-layers=0 (CPU offload, t_72646029 2026-08-17): Coder moves to full CPU
+; inference to allow concurrent residency with Qwen3.8-27B (which uses ~20.8GB
+; VRAM including nomic-embed). At models-max=4, Coder and Llama run on CPU —
+; llama.cpp 6ea215d still holds ~1.4-1.7GB CUDA-context VRAM per CPU model, so
+; steady-state is ~24,004 MiB (below the 24,576 MiB physical limit).
 [Qwen2.5-Coder-14B-Instruct-Q4_K_M]
 model        = {{ llm_models_dir }}/Qwen2.5-Coder-14B-Instruct-Q4_K_M.gguf
 alias        = Qwen2.5-Coder-14B-Instruct-4bit
-n-gpu-layers = {{ llm_router_gpu_layers }}
+n-gpu-layers = {{ llm_router_coder_gpu_layers }}
 ctx-size     = {{ llm_router_coder_ctx_size }}
 flash-attn   = {{ llm_router_coder_flash_attn }}
 cache-type-k = {{ llm_router_cache_type_k }}
@@ -194,7 +199,7 @@ parallel     = {{ llm_router_parallel }}
 [nomic-embed-text-v1.5]
 model              = {{ llm_models_dir }}/nomic-embed-text-v1.5-Q4_K_M.gguf
 embedding          = true
-n-gpu-layers       = {{ llm_router_gpu_layers }}
+n-gpu-layers       = 99
 ctx-size           = {{ llm_router_nomic_ctx_size }}
 batch-size         = {{ llm_router_nomic_batch_size }}
 ubatch-size        = {{ llm_router_nomic_ubatch_size }}