REVERT: vLLM cannot be continuously resident alongside llama-swap (t_e6facb19)

Critical finding: flipping vllm_service_enabled/state=true/started and
restarting llama-swap alongside it broke llama-swap's ability to load
ANY of its own generative models -- every /v1/chat/completions request
against Qwen3.8-27B-Q4_K_M or Qwen3-8B aux models failed with
'upstream command exited prematurely' (llama-server OOM at spawn,
~1.8GB free on this 24GB card once vLLM's ~22.8GB was claimed).
Confirmed by direct A/B: same request 500s with vLLM running, 200s
seconds after stopping it.

This breaks 21 Hermes agent profiles' aux-model tasks (skills_hub,
approval, mcp, title_generation, profile_describer, compression) plus
OpenViking's VLM -- a far larger blast radius than Hindsight's single
LLM endpoint. Reverted:
- vllm_service_enabled/state back to role defaults (false/stopped) --
  vLLM stays staged, startable for a brief validated shadow window,
  NOT safe to leave resident in production.
- Hindsight's HINDSIGHT_API_LLM_BASE_URL back to llama-swap
  (astro-orbiter:8001, Qwen3.8-27B-Q4_K_M) and the API key secret
  source back to the Nous fallback item (pre-task state) --
  the vLLM cutover, while functionally validated in isolation
  (health, /v1/chat/completions, and a live hindsight_retain+recall
  round-trip all succeeded), requires continuous vLLM availability
  which is now known to be unsafe on this card.

Comment posted on t_6dff1ecc: teardown remains correctly blocked --
full cutover is not achievable within this card's VRAM budget as
currently scoped. Needs a human decision on aux-model migration
strategy (see roles/deploy-vllm README's 'Critical architectural
finding' section) before any further progress.
This commit is contained in:
Hermes Agent service account
2026-08-31 18:36:04 -05:00
parent f3a5687adf
commit 1af645d272
4 changed files with 136 additions and 74 deletions

View File

@@ -75,15 +75,15 @@ spec:
remoteRef:
key: hindsight
property: HINDSIGHT_API_MCP_AUTH_TOKEN
# Cutover to vLLM (t_e6facb19, 2026-08-31): astro-orbiter's llama-swap
# (which needed no real auth) is being retired for Hindsight's
# completions+embeddings roles. vLLM enforces a real API key — read the
# SAME key roles/deploy-vllm's api-key.yml phase writes to
# /etc/vllm/api-key.env on astro-orbiter, sourced from 1Password
# op://mk-labs/vllm/api-key (item "vllm", field "api-key", vault mk-labs).
# Replaces the prior "nous" item's api-key (Nous free-tier fallback,
# no longer the active LLM backend once this cutover lands).
# 1Password item "nous" (vault mk-labs), field "api-key" (Ryan-provisioned,
# decision 4). REVERTED to this source (t_e6facb19, 2026-08-31) after the
# vLLM cutover attempt was reverted -- llama-swap doesn't validate the
# API key at all, so this value is effectively unused by the live LLM
# path, but restoring the original source keeps this file's intent
# honest (Nous fallback item, not vLLM's real auth key) until a real
# vLLM cutover is safe to attempt again. See values.yaml's api.env
# comment for the full incident writeup.
- secretKey: HINDSIGHT_API_LLM_API_KEY
remoteRef:
key: vllm
key: nous
property: api-key