hindsight: cut over LLM endpoint to vLLM (t_5508360a)

Companion commit to 6bfcc76 (vllm permanent residency cutover). Now that
llama-swap is stopped+disabled and vLLM is the permanent serving layer:

- HINDSIGHT_API_LLM_BASE_URL: llama-swap :8001 -> vLLM :8000
- HINDSIGHT_API_LLM_MODEL: Qwen3.8-27B-Q4_K_M -> Qwen2.5-32B-Instruct-AWQ
- Added HINDSIGHT_API_RETAIN_MAX_COMPLETION_TOKENS=4096 (vLLM's
  max_model_len=8192 is far below the 64000 Hindsight retain default;
  uncapped this 400s on long retains -- found in t_e6facb19's validation)
- ExternalSecret HINDSIGHT_API_LLM_API_KEY source: 1Password 'nous' item
  (unused fallback) -> 1Password 'vllm' item's api-key (vLLM validates
  its bearer token, unlike llama-swap which accepted anything)

Embeddings intentionally UNCHANGED -- Hindsight stays on its bundled
local 384-dim embedder (HINDSIGHT_API_EMBEDDINGS_PROVIDER still unset).
Switching to vLLM's nomic-embed (768-dim) is a separate, destructive,
explicitly-approved-only migration (~1300 rows across ~20 banks) --
out of scope here, not part of the dashboard's 'stop llama-swap /
start vLLM' instruction.
This commit is contained in:
Hermes Agent service account
2026-08-31 19:06:11 -05:00
parent 6bfcc76845
commit 39c5fdca69
2 changed files with 32 additions and 31 deletions

View File

@@ -23,9 +23,11 @@
# 1Password item "hindsight" fields (all CONCEALED):
# - postgres-password (32 chars, letters+digits only, URL-safe)
# - HINDSIGHT_API_MCP_AUTH_TOKEN (48 chars urlsafe bearer token)
# - HINDSIGHT_API_LLM_API_KEY (literal "local-placeholder" — astro-orbiter
# does not validate; must rotate if OpenRouter
# hosted fallback is ever enabled)
# HINDSIGHT_API_LLM_API_KEY now sourced from 1Password item "vllm" (vault
# mk-labs), field "api-key" -- vLLM DOES validate its bearer token (unlike
# llama-swap, which never checked one), so this must be a real matching
# secret post-cutover (t_5508360a, 2026-08-31). See values.yaml api.env
# comment and roles/deploy-vllm/README.md for the full cutover writeup.
# ============================================================================
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
@@ -75,15 +77,16 @@ spec:
remoteRef:
key: hindsight
property: HINDSIGHT_API_MCP_AUTH_TOKEN
# 1Password item "nous" (vault mk-labs), field "api-key" (Ryan-provisioned,
# decision 4). REVERTED to this source (t_e6facb19, 2026-08-31) after the
# vLLM cutover attempt was reverted -- llama-swap doesn't validate the
# API key at all, so this value is effectively unused by the live LLM
# path, but restoring the original source keeps this file's intent
# honest (Nous fallback item, not vLLM's real auth key) until a real
# vLLM cutover is safe to attempt again. See values.yaml's api.env
# comment for the full incident writeup.
# CUT OVER to vLLM's real auth key (t_5508360a, 2026-08-31): llama-swap
# is now stopped+disabled on astro-orbiter (dashboard-approved breaking
# change) and Hindsight's LLM path points at vLLM :8000
# (Qwen2.5-32B-Instruct-AWQ), which DOES validate its API key (unlike
# llama-swap, which ignored it). Source is the same 1Password item
# deploy-vllm's Ansible role reads at deploy time
# (op://mk-labs/vllm/api-key) -- keeps exactly one source of truth for
# the vLLM auth key across the Ansible-deployed server and this
# Kubernetes-deployed consumer.
- secretKey: HINDSIGHT_API_LLM_API_KEY
remoteRef:
key: nous
key: vllm
property: api-key