REVERT: vLLM cannot be continuously resident alongside llama-swap (t_e6facb19)
Critical finding: flipping vllm_service_enabled/state=true/started and restarting llama-swap alongside it broke llama-swap's ability to load ANY of its own generative models -- every /v1/chat/completions request against Qwen3.8-27B-Q4_K_M or Qwen3-8B aux models failed with 'upstream command exited prematurely' (llama-server OOM at spawn, ~1.8GB free on this 24GB card once vLLM's ~22.8GB was claimed). Confirmed by direct A/B: same request 500s with vLLM running, 200s seconds after stopping it. This breaks 21 Hermes agent profiles' aux-model tasks (skills_hub, approval, mcp, title_generation, profile_describer, compression) plus OpenViking's VLM -- a far larger blast radius than Hindsight's single LLM endpoint. Reverted: - vllm_service_enabled/state back to role defaults (false/stopped) -- vLLM stays staged, startable for a brief validated shadow window, NOT safe to leave resident in production. - Hindsight's HINDSIGHT_API_LLM_BASE_URL back to llama-swap (astro-orbiter:8001, Qwen3.8-27B-Q4_K_M) and the API key secret source back to the Nous fallback item (pre-task state) -- the vLLM cutover, while functionally validated in isolation (health, /v1/chat/completions, and a live hindsight_retain+recall round-trip all succeeded), requires continuous vLLM availability which is now known to be unsafe on this card. Comment posted on t_6dff1ecc: teardown remains correctly blocked -- full cutover is not achievable within this card's VRAM budget as currently scoped. Needs a human decision on aux-model migration strategy (see roles/deploy-vllm README's 'Critical architectural finding' section) before any further progress.
This commit is contained in:
@@ -75,15 +75,15 @@ spec:
|
||||
remoteRef:
|
||||
key: hindsight
|
||||
property: HINDSIGHT_API_MCP_AUTH_TOKEN
|
||||
# Cutover to vLLM (t_e6facb19, 2026-08-31): astro-orbiter's llama-swap
|
||||
# (which needed no real auth) is being retired for Hindsight's
|
||||
# completions+embeddings roles. vLLM enforces a real API key — read the
|
||||
# SAME key roles/deploy-vllm's api-key.yml phase writes to
|
||||
# /etc/vllm/api-key.env on astro-orbiter, sourced from 1Password
|
||||
# op://mk-labs/vllm/api-key (item "vllm", field "api-key", vault mk-labs).
|
||||
# Replaces the prior "nous" item's api-key (Nous free-tier fallback,
|
||||
# no longer the active LLM backend once this cutover lands).
|
||||
# 1Password item "nous" (vault mk-labs), field "api-key" (Ryan-provisioned,
|
||||
# decision 4). REVERTED to this source (t_e6facb19, 2026-08-31) after the
|
||||
# vLLM cutover attempt was reverted -- llama-swap doesn't validate the
|
||||
# API key at all, so this value is effectively unused by the live LLM
|
||||
# path, but restoring the original source keeps this file's intent
|
||||
# honest (Nous fallback item, not vLLM's real auth key) until a real
|
||||
# vLLM cutover is safe to attempt again. See values.yaml's api.env
|
||||
# comment for the full incident writeup.
|
||||
- secretKey: HINDSIGHT_API_LLM_API_KEY
|
||||
remoteRef:
|
||||
key: vllm
|
||||
key: nous
|
||||
property: api-key
|
||||
|
||||
Reference in New Issue
Block a user