hindsight: cut over LLM endpoint to vLLM (t_5508360a)
Companion commit to 6bfcc76 (vllm permanent residency cutover). Now that
llama-swap is stopped+disabled and vLLM is the permanent serving layer:
- HINDSIGHT_API_LLM_BASE_URL: llama-swap :8001 -> vLLM :8000
- HINDSIGHT_API_LLM_MODEL: Qwen3.8-27B-Q4_K_M -> Qwen2.5-32B-Instruct-AWQ
- Added HINDSIGHT_API_RETAIN_MAX_COMPLETION_TOKENS=4096 (vLLM's
max_model_len=8192 is far below the 64000 Hindsight retain default;
uncapped this 400s on long retains -- found in t_e6facb19's validation)
- ExternalSecret HINDSIGHT_API_LLM_API_KEY source: 1Password 'nous' item
(unused fallback) -> 1Password 'vllm' item's api-key (vLLM validates
its bearer token, unlike llama-swap which accepted anything)
Embeddings intentionally UNCHANGED -- Hindsight stays on its bundled
local 384-dim embedder (HINDSIGHT_API_EMBEDDINGS_PROVIDER still unset).
Switching to vLLM's nomic-embed (768-dim) is a separate, destructive,
explicitly-approved-only migration (~1300 rows across ~20 banks) --
out of scope here, not part of the dashboard's 'stop llama-swap /
start vLLM' instruction.
This commit is contained in:
@@ -23,9 +23,11 @@
|
||||
# 1Password item "hindsight" fields (all CONCEALED):
|
||||
# - postgres-password (32 chars, letters+digits only, URL-safe)
|
||||
# - HINDSIGHT_API_MCP_AUTH_TOKEN (48 chars urlsafe bearer token)
|
||||
# - HINDSIGHT_API_LLM_API_KEY (literal "local-placeholder" — astro-orbiter
|
||||
# does not validate; must rotate if OpenRouter
|
||||
# hosted fallback is ever enabled)
|
||||
# HINDSIGHT_API_LLM_API_KEY now sourced from 1Password item "vllm" (vault
|
||||
# mk-labs), field "api-key" -- vLLM DOES validate its bearer token (unlike
|
||||
# llama-swap, which never checked one), so this must be a real matching
|
||||
# secret post-cutover (t_5508360a, 2026-08-31). See values.yaml api.env
|
||||
# comment and roles/deploy-vllm/README.md for the full cutover writeup.
|
||||
# ============================================================================
|
||||
apiVersion: external-secrets.io/v1beta1
|
||||
kind: ExternalSecret
|
||||
@@ -75,15 +77,16 @@ spec:
|
||||
remoteRef:
|
||||
key: hindsight
|
||||
property: HINDSIGHT_API_MCP_AUTH_TOKEN
|
||||
# 1Password item "nous" (vault mk-labs), field "api-key" (Ryan-provisioned,
|
||||
# decision 4). REVERTED to this source (t_e6facb19, 2026-08-31) after the
|
||||
# vLLM cutover attempt was reverted -- llama-swap doesn't validate the
|
||||
# API key at all, so this value is effectively unused by the live LLM
|
||||
# path, but restoring the original source keeps this file's intent
|
||||
# honest (Nous fallback item, not vLLM's real auth key) until a real
|
||||
# vLLM cutover is safe to attempt again. See values.yaml's api.env
|
||||
# comment for the full incident writeup.
|
||||
# CUT OVER to vLLM's real auth key (t_5508360a, 2026-08-31): llama-swap
|
||||
# is now stopped+disabled on astro-orbiter (dashboard-approved breaking
|
||||
# change) and Hindsight's LLM path points at vLLM :8000
|
||||
# (Qwen2.5-32B-Instruct-AWQ), which DOES validate its API key (unlike
|
||||
# llama-swap, which ignored it). Source is the same 1Password item
|
||||
# deploy-vllm's Ansible role reads at deploy time
|
||||
# (op://mk-labs/vllm/api-key) -- keeps exactly one source of truth for
|
||||
# the vLLM auth key across the Ansible-deployed server and this
|
||||
# Kubernetes-deployed consumer.
|
||||
- secretKey: HINDSIGHT_API_LLM_API_KEY
|
||||
remoteRef:
|
||||
key: nous
|
||||
key: vllm
|
||||
property: api-key
|
||||
|
||||
Reference in New Issue
Block a user