60220e18b6
Idempotent vLLM OpenAI-compatible serving role, staged-first (does not
start/enable the systemd unit or touch production traffic by default).
Validated end-to-end against astro-orbiter in a brief shadow window
(llama-swap stopped ~5 min, per homelab-llm-inference skill's documented
shadow-validation pattern):
- /health 200, /v1/models returns Qwen2.5-32B-Instruct-AWQ,
/v1/completions live smoke test passes, clean journalctl
- 3 consecutive full-role runs confirmed changed=0 (idempotent)
- production restored: llama-swap active, /v1/embeddings against
nomic-embed-text-v1.5 confirmed still working (Hindsight retain path)
Deviates from the original spec's model choices (Qwen2.5-32B-Instruct /
Qwen3-8B-Instruct bf16) to use the official Qwen AWQ pre-quantized variants
instead -- vLLM does not do safe on-the-fly quantization on this host
(bitsandbytes OOM history) and unquantized bf16 32B does not fit 24GB VRAM.
Two real bugs found+fixed during first-start validation (systemd-only
repro, not visible via interactive SSH testing):
1. ninja not on systemd's minimal PATH -> vLLM torch.compile
FileNotFoundError. Fixed via explicit PATH env in the unit.
2. FlashInfer sampler JIT fails to compile on RTX 3090 (SM86) --
known upstream issue class (vLLM GH #23023, #44305). Fixed via
VLLM_USE_FLASHINFER_SAMPLER=0 (falls back to native sampler).
Also fixed a real idempotency bug: force-upgrading setuptools to latest
fought with vLLM's own setuptools<81.0.0 pin, causing an install/downgrade
flip-flop (changed:true) on every run.
vllm_service_enabled defaults to false -- a host reboot must not
auto-start vLLM and VRAM-collide with the still-live llama-swap production
service. Cutover (enabling + starting + migrating consumers) is an
explicit, separate step outside this role, gated on adding embedding-mode
support (--task embed) for nomic-embed-text-v1.5, which this role does
not yet implement (Hindsight retain still depends on llama-swap's
nomic-embed until that follow-up lands).
Role: roles/deploy-vllm/ (defaults/handlers/meta/tasks/templates/README)
Playbook: playbooks/day1_deploy_vllm.yml
39 lines
1.6 KiB
YAML
39 lines
1.6 KiB
YAML
# ------------------------------------------------------------------------------
|
|
# FILE: roles/deploy-vllm/tasks/main.yml
|
|
# ROLE: deploy-vllm — orchestrator. Phased, idempotent, mirrors the pattern
|
|
# used by roles/llm-inference and roles/llm-inference-multimodel:
|
|
# Phase 1: dependencies (Python/venv/vLLM/CUDA/nvidia-smi)
|
|
# Phase 2: model downloads (~/.vllm-cache, checksum-verified)
|
|
# Phase 3: systemd service(s)
|
|
# Phase 4: API key management (1Password)
|
|
# Phase 5: verification (health + smoke test)
|
|
# Each phase is a separate task file so a partial re-run / targeted --tags
|
|
# run is possible without re-reading the whole role.
|
|
# ------------------------------------------------------------------------------
|
|
|
|
- name: Compute enabled model list (available to every phase/tag combination)
|
|
ansible.builtin.set_fact:
|
|
vllm_enabled_models: "{{ vllm_models | selectattr('enabled', 'equalto', true) | list }}"
|
|
tags: [vllm, vllm-dependencies, vllm-models, vllm-api-key, vllm-systemd, vllm-verify]
|
|
|
|
- name: Phase 1 — Python & dependencies
|
|
ansible.builtin.import_tasks: dependencies.yml
|
|
tags: [vllm, vllm-dependencies]
|
|
|
|
- name: Phase 2 — Model downloads
|
|
ansible.builtin.import_tasks: models.yml
|
|
tags: [vllm, vllm-models]
|
|
|
|
- name: Phase 3 — API key management
|
|
ansible.builtin.import_tasks: api-key.yml
|
|
tags: [vllm, vllm-api-key]
|
|
|
|
- name: Phase 4 — vLLM systemd service(s)
|
|
ansible.builtin.import_tasks: systemd.yml
|
|
tags: [vllm, vllm-systemd]
|
|
|
|
- name: Phase 5 — Verification
|
|
ansible.builtin.import_tasks: verify.yml
|
|
tags: [vllm, vllm-verify]
|
|
when: vllm_service_state == 'started'
|