feat(astro-orbiter): add deploy-vllm Ansible role (t_ca1af9fb)

Idempotent vLLM OpenAI-compatible serving role, staged-first (does not
start/enable the systemd unit or touch production traffic by default).
Validated end-to-end against astro-orbiter in a brief shadow window
(llama-swap stopped ~5 min, per homelab-llm-inference skill's documented
shadow-validation pattern):
  - /health 200, /v1/models returns Qwen2.5-32B-Instruct-AWQ,
    /v1/completions live smoke test passes, clean journalctl
  - 3 consecutive full-role runs confirmed changed=0 (idempotent)
  - production restored: llama-swap active, /v1/embeddings against
    nomic-embed-text-v1.5 confirmed still working (Hindsight retain path)

Deviates from the original spec's model choices (Qwen2.5-32B-Instruct /
Qwen3-8B-Instruct bf16) to use the official Qwen AWQ pre-quantized variants
instead -- vLLM does not do safe on-the-fly quantization on this host
(bitsandbytes OOM history) and unquantized bf16 32B does not fit 24GB VRAM.

Two real bugs found+fixed during first-start validation (systemd-only
repro, not visible via interactive SSH testing):
  1. ninja not on systemd's minimal PATH -> vLLM torch.compile
     FileNotFoundError. Fixed via explicit PATH env in the unit.
  2. FlashInfer sampler JIT fails to compile on RTX 3090 (SM86) --
     known upstream issue class (vLLM GH #23023, #44305). Fixed via
     VLLM_USE_FLASHINFER_SAMPLER=0 (falls back to native sampler).

Also fixed a real idempotency bug: force-upgrading setuptools to latest
fought with vLLM's own setuptools<81.0.0 pin, causing an install/downgrade
flip-flop (changed:true) on every run.

vllm_service_enabled defaults to false -- a host reboot must not
auto-start vLLM and VRAM-collide with the still-live llama-swap production
service. Cutover (enabling + starting + migrating consumers) is an
explicit, separate step outside this role, gated on adding embedding-mode
support (--task embed) for nomic-embed-text-v1.5, which this role does
not yet implement (Hindsight retain still depends on llama-swap's
nomic-embed until that follow-up lands).

Role: roles/deploy-vllm/ (defaults/handlers/meta/tasks/templates/README)
Playbook: playbooks/day1_deploy_vllm.yml

ansible/roles/deploy-vllm/templates/vllm-workspace.sh.j2

@@ -0,0 +1,54 @@
+#!/usr/bin/env bash
+# ------------------------------------------------------------------------------
+# FILE: vllm-workspace.sh — deployed by roles/deploy-vllm to
+#       /home/{{ vllm_venv_owner }}/vllm-workspace.sh
+#
+# Convenience wrapper for manual debugging / smoke-testing the vLLM venv
+# without having to remember the venv path or model roster each time.
+# Regenerated on every Ansible run — do not hand-edit, edit the template
+# instead (roles/deploy-vllm/templates/vllm-workspace.sh.j2).
+# ------------------------------------------------------------------------------
+set -euo pipefail
+
+VENV="{{ vllm_venv_path }}"
+CACHE="{{ vllm_hf_hub_cache }}"
+API_KEY_FILE="{{ vllm_api_key_env_file }}"
+
+usage() {
+  cat <<EOF
+Usage: $0 <command>
+
+Commands:
+  activate        Print the command to source the vLLM venv
+  version         Print installed vLLM + torch/CUDA versions
+  models          List staged model snapshots in the HF cache
+  curl-models     curl /v1/models on each enabled instance (requires sudo to read API key)
+  logs <unit>     Tail journalctl for a vllm systemd unit (e.g. vllm.service)
+EOF
+}
+
+case "${1:-}" in
+  activate)
+    echo "source $VENV/bin/activate"
+    ;;
+  version)
+    "$VENV/bin/python" -c 'import vllm, torch; print("vllm", vllm.__version__); print("torch", torch.__version__, "cuda", torch.version.cuda, "available", torch.cuda.is_available())'
+    ;;
+  models)
+    find "$CACHE" -maxdepth 1 -type d -name 'models--*' -printf '%f\n' 2>/dev/null || echo "(no models staged yet)"
+    ;;
+  curl-models)
+{% for item in vllm_enabled_models | default([]) %}
+    echo "--- {{ item.id }} (:{{ item.port }}) ---"
+    curl -s -H "Authorization: Bearer $(sudo grep -oP '(?<=VLLM_API_KEY=).*' "$API_KEY_FILE")" \
+      http://127.0.0.1:{{ item.port }}/v1/models | python3 -m json.tool || true
+{% endfor %}
+    ;;
+  logs)
+    sudo journalctl -u "${2:-vllm.service}" -f
+    ;;
+  *)
+    usage
+    exit 1
+    ;;
+esac

ansible/roles/deploy-vllm/templates/vllm.service.j2

@@ -0,0 +1,56 @@
+[Unit]
+Description=vLLM OpenAI-compatible inference server — {{ item.id }} ({{ item.hf_repo }})
+After=network-online.target nvidia-persistenced.service
+Wants=network-online.target nvidia-persistenced.service
+
+[Service]
+Type=simple
+User={{ vllm_venv_owner }}
+Group={{ vllm_venv_owner }}
+EnvironmentFile={{ vllm_api_key_env_file }}
+Environment="HOME=/home/{{ vllm_venv_owner }}"
+Environment="HF_HUB_CACHE={{ vllm_hf_hub_cache }}"
+Environment="HF_HOME={{ vllm_cache_dir }}"
+# vLLM's torch.compile path shells out to `ninja` by bare name (not via
+# venv-relative path) — without the venv's bin/ on PATH, systemd's minimal
+# default PATH causes FileNotFoundError: 'ninja' deep in compile, even
+# though `pip install vllm` installs the ninja package (and its console
+# script) INTO the venv. Caught during Phase 5 validation (t_ca1af9fb,
+# 2026-08-31): interactive SSH sessions have a different PATH than systemd
+# services, so this only reproduces under systemd, not manual testing.
+Environment="PATH={{ vllm_venv_path }}/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
+# FlashInfer's bundled sampling.cu JIT-compiles against a cub template API
+# (BlockAdjacentDifference::FlagHeads) that this flashinfer/CUDA toolkit
+# combination does not provide on RTX 3090 (SM86) — 100 compile errors,
+# confirmed upstream-known (vLLM GH #23023, #44305: FlashInfer sampler JIT
+# breaks on various SM targets across flashinfer/vLLM version combos).
+# Falls back to vLLM's native PyTorch sampler, which is fully supported and
+# only marginally slower for single-request/low-concurrency serving. Caught
+# during Phase 5 validation (t_ca1af9fb, 2026-08-31).
+Environment="VLLM_USE_FLASHINFER_SAMPLER=0"
+
+ExecStart={{ vllm_venv_path }}/bin/python -m vllm.entrypoints.openai.api_server \
+    --model {{ item.hf_repo }} \
+    --served-model-name {{ item.id }} \
+    --host {{ vllm_serve_host }} \
+    --port {{ item.port }} \
+{% if item.quantization is defined and item.quantization != 'none' %}
+    --quantization {{ item.quantization }} \
+{% endif %}
+    --gpu-memory-utilization {{ item.gpu_memory_utilization }} \
+    --max-model-len {{ item.max_model_len }} \
+    --dtype {{ vllm_dtype }} \
+    --api-key ${VLLM_API_KEY} \
+    --enable-prefix-caching
+
+Restart={{ vllm_restart_policy }}
+RestartSec=10
+# vLLM torch.compile can take 4+ minutes before /health responds even after
+# weights are loaded (homelab-llm-inference skill pitfall) — give it room.
+TimeoutStartSec=600
+StandardOutput=journal
+StandardError=journal
+SyslogIdentifier=vllm-{{ item.id }}
+
+[Install]
+WantedBy=multi-user.target