feat(deploy-vllm): swap Qwen2.5-32B for DeepSeek-R1-Distill-Qwen-32B-AWQ (t_r1d32b_swap)

Single-model deployment per Ryan's direction:
- Primary model: casperhansen/deepseek-r1-distill-qwen-32b-awq, max_model_len=32768
- nomic-embed-text-v1.5 and Qwen3-8B-AWQ both disabled (single-model requirement)
- kv_cache_dtype: int4_per_token_head required to fit 32768 ctx on 24GB RTX 3090
  (fp16 KV: 26GB needed, doesn't fit at any utilization; fp8 KV: OOM'd during
  FlashInfer warmup with ~50-150MB margin; int4 KV: clean single-attempt start)
- Added kv_cache_dtype / kv_cache_memory_bytes as new optional per-model
  template fields in vllm.service.j2 (guarded, no effect on other models)

Verified live: /health 200, /v1/models confirms max_model_len=32768,
live /v1/completions smoke test + manual chat completion both passed
(genuine <think> reasoning trace, correct arithmetic). NRestarts=0,
steady-state VRAM 23.2GB/24.576GB. Ansible idempotent re-run confirmed
changed=0.

Known follow-up (not done here): Hindsight's HINDSIGHT_API_LLM_MODEL
cluster config still references the retired Qwen2.5-32B-Instruct-AWQ —
needs separate GitOps update to point at the new model.

ansible/roles/deploy-vllm/README.md

@@ -57,6 +57,11 @@ systemd units on distinct ports, not a single multiplexed server (unlike
 llama-swap's matrix DSL). Today's phase enables only the primary model;
 flip `enabled: true` on the others as VRAM allows (see "Phased Strategy"):
 
+**⚠️ Table below reflects the ORIGINAL Qwen2.5-32B deployment. As of
+2026-09-01 (t_r1d32b_swap) the primary model is
+`DeepSeek-R1-Distill-Qwen-32B-AWQ`, single-model only (nomic-embed also
+disabled) — see the "SUPERSEDED" section further down for current state.**
+
 | id | hf_repo | role | port | quant | enabled |
 |---|---|---|---|---|---|
 | Qwen2.5-32B-Instruct-AWQ | Qwen/Qwen2.5-32B-Instruct-AWQ | primary | 8000 | awq | **true** |
@@ -309,6 +314,95 @@ aux-model migration was authorized or attempted in this task. If those 21
 profiles need a replacement aux-model path, that is separate, new,
 explicitly-scoped follow-up work, not implied by this decision.
 
+## SUPERSEDED (t_r1d32b_swap, 2026-09-01): Qwen2.5-32B-Instruct-AWQ retired, replaced with DeepSeek-R1-Distill-Qwen-32B-AWQ, single-model deployment
+
+Ryan direction: "Swap Qwen2.5-32B for DeepSeek-R1-Distill-Qwen-32B,
+max-model-len 32768. Single model only." Confirmed with Ryan that "single
+model only" includes disabling `nomic-embed-text-v1.5` (:8020) as well —
+nothing in production consumed it (Hindsight uses its own bundled 384-dim
+embedder; OpenViking pointed at the retired llama-swap endpoint). DeepSeek
+gets the entire 24GB card.
+
+**Model choice:** `casperhansen/deepseek-r1-distill-qwen-32b-awq` — same
+AutoAWQ toolchain/quant style as the outgoing Qwen2.5-32B-Instruct-AWQ,
+widely-used community quant, `Qwen2ForCausalLM` architecture (DeepSeek-R1
+reasoning distilled onto a Qwen2.5-32B base) — no new vLLM code path
+required. Native `max_position_embeddings: 131072`; capped at 32768 per
+the task's explicit requirement.
+
+**Executed:**
+1. Stopped + disabled `vllm-nomic-embed-text-v1.5.service` (single-model
+   requirement), freed its ~19GB Qwen2.5-32B model cache on disk (30GB
+   free → 48GB free) to make room for DeepSeek's ~19.3GB download.
+2. Replaced `vllm_models` in `host_vars/astro-orbiter/vars.yml`: primary
+   entry now `DeepSeek-R1-Distill-Qwen-32B-AWQ`, aux (`Qwen3-8B-AWQ`) and
+   embedding (`nomic-embed-text-v1.5`) both `enabled: false`.
+3. Staged the model via `--tags vllm-models` (idempotent `hf download`,
+   ~19GB, confirmed via `du -sh` and snapshot-dir stat).
+4. **Three rounds of live VRAM-fit debugging** before a stable config was
+   found (documented inline in host_vars comments) — worth recording here
+   since the failure mode is non-obvious and will recur for future
+   32B-class models at high context on this 24GB card:
+   - **Round 1 (fp16 KV, gpu_memory_utilization 0.90/0.95/0.98):** vLLM's
+     own pre-flight check reported 18.17GiB weights + 8.0GiB KV cache
+     needed at 32768 ctx fp16 = 26.17GB — mathematically impossible on a
+     24GB card at ANY utilization percentage. Crash-looped every attempt.
+   - **Round 2 (`--kv-cache-dtype fp8`):** halved nominal KV cache to
+     ~4.0-4.3GiB, should fit with ~1GB margin. Still OOM'd — small
+     (~50-150MB) `cudaMalloc` failures during FlashInfer kernel warmup,
+     consistently, even when vLLM's own pre-flight math said it should
+     fit. Root cause: real GPU usage during warmup kernel compilation
+     exceeds what upfront profiling/reservation accounts for by roughly
+     ~1GB (unaccounted FlashInfer/sampler warmup workspace buffers).
+     Tried both the percentage knob AND vLLM's own suggested
+     `--kv-cache-memory-bytes` exact value — same failure either way,
+     confirming the gap wasn't a rounding/estimation error in the
+     percentage math, it was a real missing ~1GB of margin.
+   - **Round 3 (`--kv-cache-dtype int4_per_token_head`, fixed): SUCCESS.**
+     Switching from 8-bit to 4-bit KV cache roughly halves the KV
+     footprint again (~2GiB instead of ~4-4.3GiB), buying back enough
+     real headroom to absorb the unaccounted warmup overhead. Clean
+     single-attempt start, `NRestarts=0`, steady-state VRAM 23.2GB/24.576GB.
+5. **Full Ansible verify phase (`--tags vllm-api-key,vllm-verify`)**
+   passed: systemd unit active, `/health` 200, `/v1/models` returns
+   `DeepSeek-R1-Distill-Qwen-32B-AWQ` with `max_model_len: 32768`, live
+   `/v1/completions` smoke test HTTP 200, clean restart + re-run of
+   `--tags vllm-systemd` confirmed idempotent (`changed=0`,
+   `NRestarts=0`, same `ActiveEnterTimestamp` — no unnecessary restart).
+6. **Manual end-to-end generation test**, not inference: a real
+   `/v1/chat/completions` call ("What is 12*8?") returned a genuine
+   DeepSeek-R1 reasoning trace in `<think>` tags followed by the correct
+   answer (96) with correct step-by-step arithmetic shown — confirms the
+   model is not just health-check-alive but actually reasoning correctly.
+
+**Role/template changes (reusable for future models on this host):**
+- Added `kv_cache_dtype` (renders `--kv-cache-dtype`) and
+  `kv_cache_memory_bytes` (renders `--kv-cache-memory-bytes`) as new
+  optional per-model fields in `vllm.service.j2` — both are `{% if
+  ... is defined %}` guarded, no effect on models that don't set them.
+
+**Final production state on astro-orbiter (verified live, 2026-09-01):**
+- `vllm.service` (DeepSeek-R1-Distill-Qwen-32B-AWQ, :8000, `max_model_len:
+  32768`, `kv_cache_dtype: int4_per_token_head`): active, enabled,
+  boot-persistent, single model on the card
+- `vllm-nomic-embed-text-v1.5.service` (:8020): inactive, disabled
+- `vllm-Qwen3-8B-AWQ.service` (:8010): inactive, disabled (unchanged from prior state)
+- `llama-swap.service`: inactive, disabled (unchanged from prior state)
+- VRAM: ~23.2GB/24.576GB steady-state, no crash-looping, `NRestarts=0`
+
+**Not done in this task (flagging, not implied by this swap):**
+- Hindsight's `HINDSIGHT_API_LLM_MODEL` / `HINDSIGHT_API_LLM_BASE_URL`
+  cluster config still references `Qwen2.5-32B-Instruct-AWQ` — that model
+  is now gone from the card. Hindsight's LLM calls to astro-orbiter will
+  fail model-not-found until that GitOps config is updated to point at
+  `DeepSeek-R1-Distill-Qwen-32B-AWQ`. Not touched here — task scope was
+  the astro-orbiter model swap itself, cluster consumer cutover is a
+  separate, explicit follow-up (same boundary respected in the prior
+  t_5508360a section: this role does not own cluster-side config).
+- DeepSeek-R1's reasoning output uses `<think>` tags and the model card
+  recommends temperature 0.5-0.7 (not greedy/0) — neither is enforced
+  server-side; any consumer wiring this model into a Hermes profile or
+  application should account for both when parsing responses.
 
 ## Validation Log (2026-08-31, t_ca1af9fb)
 

ansible/roles/deploy-vllm/templates/vllm.service.j2

@@ -46,6 +46,12 @@ ExecStart={{ vllm_venv_path }}/bin/python -m vllm.entrypoints.openai.api_server
 {% endif %}
 {% if item.quantization is defined and item.quantization != 'none' %}
     --quantization {{ item.quantization }} \
+{% endif %}
+{% if item.kv_cache_dtype is defined %}
+    --kv-cache-dtype {{ item.kv_cache_dtype }} \
+{% endif %}
+{% if item.kv_cache_memory_bytes is defined %}
+    --kv-cache-memory-bytes {{ item.kv_cache_memory_bytes }} \
 {% endif %}
     --gpu-memory-utilization {{ item.gpu_memory_utilization }} \
     --max-model-len {{ item.max_model_len }} \