feat(llm): swap astro-orbiter primary model Qwen3.6 -> Qwen3.8-27B-Q4_K_M

Ryan-directed model swap (kanban t_f5f7e9ad, 2026-08-16).

Changes:
- Replace [Qwen3.6-35B-A3B-UD-Q4_K_S] with [Qwen3.8-27B-Q4_K_M] in
  llama-server-router-preset.ini.j2 (production model slot).
- Qwen3.8-27B: dense 27B VLM, Apache-2.0, Alibaba Aug 2026.
  Unsloth Dynamic V3.0 GGUF quantization.
  Q4_K_M chosen: 17,106,775,008 bytes, 17.1GB. Measured VRAM: 17,068 MiB
  at ctx=32768 (q4_0 KV cache).
- ctx-size set to 32768 (32K) via new variable llm_router_qwen38_ctx_size.
  Native context is 262K; 32K chosen to maintain eviction headroom on 24GB RTX 3090.
- models-max reduced 4 -> 2 in host_vars. Qwen3.8 (17.6GB) + nomic-embed
  (558MB) exhaust the 24GB card; no auxiliary model can co-reside with Qwen3.8.
  LRU eviction handles model switching with ~30-60s cold-load latency.
- llm_router_expected_model_id updated to Qwen3.8-27B-Q4_K_M.
- Qwen3.6 GGUF retained at /opt/models/Qwen3.6-35B-A3B-UD-Q4_K_S.gguf
  (not deleted — pending stable period and explicit cleanup task).
- day2_swap_qwen38.yml playbook added for Ansible idempotent redeployment.

Architecture note: Qwen3.8 uses Gated DeltaNet; llama.cpp 6ea215d logs
'fused Gated Delta Net (chunked) not supported, set to disabled'. Inference
works correctly on the non-fused fallback. A llama.cpp update may improve
throughput on the GDN layers.

Smoke test passed: model responded via router endpoint (http://10.1.71.130:8002).
VRAM: 17,630 MiB (Qwen3.8) + 5,928 MiB (Llama-8B concurrent) = 23,558 MiB.

Also commits accumulated but unpushed changes:
- nomic-embed batch-size/rope-scaling fix (t_openviking_embed_batch)
- per-model ctx-size day2 playbook (day2_per_model_ctx_size.yml)
- llama-server-router.service.j2 minor update

ansible/playbooks/day2_add_nomic_embed.yml

@@ -58,6 +58,18 @@
     llm_router_coder_ctx_size: 16384
     llm_router_coder_flash_attn: "true"
     llm_router_nomic_ctx_size: 8192
+    # NOTE (2026-08-14, t_openviking_embed_batch): per-model batch-size/
+    # ubatch-size lines in the preset INI are NOT honored by llama-server's
+    # router — only ctx-size is applied per-model; batch-size/ubatch-size for
+    # every spawned child come from the router's own global CLI flags
+    # (confirmed via `ps aux` on astro-orbiter: child process launched with
+    # the router's --batch-size/--ubatch-size regardless of the INI values).
+    # Kept below for documentation/future-proofing but the REAL fix is the
+    # global llm_router_batch_size / llm_router_ubatch_size override further
+    # down, which raises the physical batch for ALL models on this router
+    # (Qwen3.6-35B, Phi, Llama, Coder, nomic).
+    llm_router_nomic_batch_size: 4096
+    llm_router_nomic_ubatch_size: 4096
 
     # All other vars inherit from host_vars + defaults/main.yml.
     llm_router_enabled: true
@@ -74,8 +86,19 @@
     llm_router_ctx_size: 65536          # Qwen3.6-35B default; per-model overrides above
     llm_router_parallel: 1
     llm_router_gpu_layers: 99
-    llm_router_batch_size: 2048
-    llm_router_ubatch_size: 512
+    # FIX (2026-08-14, t_openviking_embed_batch): raised from 512 to 4096.
+    # This is a GLOBAL router flag applied to every spawned model process
+    # (per-model INI batch-size/ubatch-size overrides are not honored by
+    # llama-server's router — see note above nomic vars). 512 tokens was too
+    # small for OpenViking's chunked-document embedding inputs (observed
+    # 2000-3400 tokens/chunk), causing hard 500 errors ("input (N tokens) is
+    # too large to process") that tripped OpenViking's circuit breaker into a
+    # permanent fail/re-enqueue loop. 4096 comfortably covers observed chunk
+    # sizes and stays under nomic's ctx-size=8192. VRAM impact of raising
+    # ubatch-size is in compute-buffer scratch space, not KV cache; monitored
+    # post-deploy against the 23000 MiB budget (host_vars/astro-orbiter).
+    llm_router_batch_size: 4096
+    llm_router_ubatch_size: 4096
     llm_router_cache_type_k: q4_0
     llm_router_cache_type_v: q4_0
     llm_router_flash_attn: "auto"