60220e18b6
Idempotent vLLM OpenAI-compatible serving role, staged-first (does not
start/enable the systemd unit or touch production traffic by default).
Validated end-to-end against astro-orbiter in a brief shadow window
(llama-swap stopped ~5 min, per homelab-llm-inference skill's documented
shadow-validation pattern):
- /health 200, /v1/models returns Qwen2.5-32B-Instruct-AWQ,
/v1/completions live smoke test passes, clean journalctl
- 3 consecutive full-role runs confirmed changed=0 (idempotent)
- production restored: llama-swap active, /v1/embeddings against
nomic-embed-text-v1.5 confirmed still working (Hindsight retain path)
Deviates from the original spec's model choices (Qwen2.5-32B-Instruct /
Qwen3-8B-Instruct bf16) to use the official Qwen AWQ pre-quantized variants
instead -- vLLM does not do safe on-the-fly quantization on this host
(bitsandbytes OOM history) and unquantized bf16 32B does not fit 24GB VRAM.
Two real bugs found+fixed during first-start validation (systemd-only
repro, not visible via interactive SSH testing):
1. ninja not on systemd's minimal PATH -> vLLM torch.compile
FileNotFoundError. Fixed via explicit PATH env in the unit.
2. FlashInfer sampler JIT fails to compile on RTX 3090 (SM86) --
known upstream issue class (vLLM GH #23023, #44305). Fixed via
VLLM_USE_FLASHINFER_SAMPLER=0 (falls back to native sampler).
Also fixed a real idempotency bug: force-upgrading setuptools to latest
fought with vLLM's own setuptools<81.0.0 pin, causing an install/downgrade
flip-flop (changed:true) on every run.
vllm_service_enabled defaults to false -- a host reboot must not
auto-start vLLM and VRAM-collide with the still-live llama-swap production
service. Cutover (enabling + starting + migrating consumers) is an
explicit, separate step outside this role, gated on adding embedding-mode
support (--task embed) for nomic-embed-text-v1.5, which this role does
not yet implement (Hindsight retain still depends on llama-swap's
nomic-embed until that follow-up lands).
Role: roles/deploy-vllm/ (defaults/handlers/meta/tasks/templates/README)
Playbook: playbooks/day1_deploy_vllm.yml
113 lines
4.4 KiB
YAML
113 lines
4.4 KiB
YAML
# ------------------------------------------------------------------------------
|
|
# FILE: roles/deploy-vllm/tasks/api-key.yml
|
|
# PHASE 3: API key management.
|
|
#
|
|
# Source of truth: 1Password op://mk-labs/vllm/api-key (Nick Fury manages).
|
|
# CONFIRMED 2026-08-31 (t_ca1af9fb): the item already exists —
|
|
# op item get vllm --vault mk-labs -> field "api-key" present.
|
|
# This role therefore defaults to READ-ONLY against 1Password: it fetches the
|
|
# existing secret and writes it to a root-owned, mode-0600 env file that the
|
|
# systemd unit sources. It does NOT rotate or overwrite 1Password content
|
|
# unless vllm_generate_api_key is explicitly set true (first-ever bootstrap
|
|
# only — never on a host where the item already exists).
|
|
#
|
|
# `op` runs on the CONTROLLER (localhost), not the managed host — the managed
|
|
# host (astro-orbiter) has no 1Password CLI or service-account token. The
|
|
# resolved secret is pushed to the host via `ansible.builtin.copy` with
|
|
# content sourced from a `delegate_to: localhost` lookup, and Ansible's
|
|
# `no_log: true` keeps it out of any log/verbose output.
|
|
# ------------------------------------------------------------------------------
|
|
|
|
- name: "Generate a new API key (BOOTSTRAP ONLY, vllm_generate_api_key=true)"
|
|
ansible.builtin.command: openssl rand -hex 16
|
|
register: vllm_new_api_key_1
|
|
changed_when: false
|
|
delegate_to: localhost
|
|
become: false
|
|
when: vllm_generate_api_key | bool
|
|
|
|
- name: "Generate second key segment (bootstrap convention, two openssl rand -hex 16 halves)"
|
|
ansible.builtin.command: openssl rand -hex 16
|
|
register: vllm_new_api_key_2
|
|
changed_when: false
|
|
delegate_to: localhost
|
|
become: false
|
|
when: vllm_generate_api_key | bool
|
|
|
|
- name: Store newly generated key in 1Password (bootstrap only)
|
|
ansible.builtin.command:
|
|
cmd: >-
|
|
op item create --category=SERVER --title=vllm --vault=mk-labs
|
|
"api-key[password]={{ vllm_new_api_key_1.stdout }}{{ vllm_new_api_key_2.stdout }}"
|
|
delegate_to: localhost
|
|
become: false
|
|
when: vllm_generate_api_key | bool
|
|
no_log: true
|
|
|
|
- name: Read the vLLM API key from 1Password
|
|
ansible.builtin.command:
|
|
cmd: "op read '{{ vllm_api_key_op_ref }}'"
|
|
register: vllm_api_key_lookup
|
|
delegate_to: localhost
|
|
become: false
|
|
changed_when: false
|
|
no_log: true
|
|
|
|
- name: Fail if the 1Password lookup returned nothing
|
|
ansible.builtin.fail:
|
|
msg: >-
|
|
op read {{ vllm_api_key_op_ref }} returned an empty value. Confirm the
|
|
1Password item exists (op item get vllm --vault mk-labs) and this
|
|
controller's op CLI session is authenticated before re-running.
|
|
when: vllm_api_key_lookup.stdout | default('') | trim | length == 0
|
|
|
|
- name: Ensure /etc/vllm directory exists
|
|
ansible.builtin.file:
|
|
path: "{{ vllm_api_key_env_file | dirname }}"
|
|
state: directory
|
|
owner: root
|
|
group: root
|
|
mode: "0750"
|
|
become: true
|
|
|
|
- name: Write API key env file (root-owned, 0600, not world-readable)
|
|
ansible.builtin.copy:
|
|
dest: "{{ vllm_api_key_env_file }}"
|
|
content: "VLLM_API_KEY={{ vllm_api_key_lookup.stdout }}\n"
|
|
owner: root
|
|
group: root
|
|
mode: "0600"
|
|
become: true
|
|
no_log: true
|
|
notify: restart vllm services
|
|
|
|
- name: Record quarterly rotation reminder doc (idempotent, content-driven)
|
|
ansible.builtin.copy:
|
|
dest: "/etc/vllm/API_KEY_ROTATION.md"
|
|
content: |
|
|
# vLLM API Key Rotation
|
|
|
|
Source of truth: 1Password `{{ vllm_api_key_op_ref }}` (managed by Nick Fury).
|
|
|
|
## Rotation procedure (target: quarterly)
|
|
|
|
1. Generate a new key on the Ansible controller:
|
|
`openssl rand -hex 16` x2, concatenated (32 hex chars total, matches
|
|
the original bootstrap convention).
|
|
2. Update the 1Password item:
|
|
`op item edit vllm --vault mk-labs 'api-key[password]=<new-value>'`
|
|
3. Re-run this role (`ansible-playbook ... --tags vllm-api-key,vllm-systemd`)
|
|
to push the new key to /etc/vllm/api-key.env and restart the vllm
|
|
service(s) with the new key.
|
|
4. Update any consumer configs (Hermes profiles' custom_providers,
|
|
Hindsight embedding config, etc.) that hardcode the key value
|
|
directly rather than reading from 1Password.
|
|
5. Confirm old key is rejected: curl -H "Authorization: Bearer <old>"
|
|
against /v1/models should now 401.
|
|
|
|
Last rotated: see 1Password item audit log (op item get vllm --vault mk-labs).
|
|
owner: root
|
|
group: root
|
|
mode: "0644"
|
|
become: true
|