hindsight: revert embeddings cutover after dimension-mismatch crash
HINDSIGHT_API_EMBEDDINGS_PROVIDER=openai pointed at vLLM's nomic-embed-text-v1.5 (768-dim) crash-looped hindsight-api: 'Cannot change embedding dimension from 384 to 768: memory_units table contains 1289 rows with embeddings.' Hindsight was never actually using astro-orbiter for embeddings (defaults to a bundled local 384-dim sentence-transformers model when the env var is unset) -- this was a genuine architecture assumption error, not a config typo. Re-embedding all existing memory data across ~20 agent banks is a destructive, irreversible operation requiring its own explicit, approved task -- not something to trigger as a side effect of an infra migration. Keeps the LLM cutover (astro-orbiter:8000, vLLM Qwen2.5-32B-Instruct-AWQ) which is safe and already validated.
This commit is contained in:
@@ -55,15 +55,12 @@ spec:
|
||||
# env-var-named keys injected via envFrom
|
||||
HINDSIGHT_API_MCP_AUTH_TOKEN: "{{ .HINDSIGHT_API_MCP_AUTH_TOKEN }}"
|
||||
HINDSIGHT_API_LLM_API_KEY: "{{ .HINDSIGHT_API_LLM_API_KEY }}"
|
||||
# Cutover to vLLM (t_e6facb19, 2026-08-31): vLLM enforces its API key
|
||||
# on every request (unlike llama-swap, which accepted any/no key) —
|
||||
# confirmed empirically, a bad/missing key gets a real 401
|
||||
# {"error":"Unauthorized"}. Both the completions endpoint (:8000) and
|
||||
# the embeddings endpoint (:8020) are separate vLLM processes but
|
||||
# share the SAME key (api-key.env is written once, read by both
|
||||
# systemd units per roles/deploy-vllm/tasks/api-key.yml). Reuse
|
||||
# HINDSIGHT_API_LLM_API_KEY's value for the embeddings key too.
|
||||
HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY: "{{ .HINDSIGHT_API_LLM_API_KEY }}"
|
||||
# NOTE: no HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY here — Hindsight's
|
||||
# embeddings stayed on its bundled local provider after the vLLM
|
||||
# embeddings cutover attempt crash-looped the API on an embedding
|
||||
# dimension mismatch (384 existing vs 768 nomic-embed). See
|
||||
# values.yaml's api.env comment for the full incident writeup
|
||||
# (t_e6facb19, 2026-08-31).
|
||||
data:
|
||||
# 1Password item "hindsight", field "postgres-password".
|
||||
# letters+digits only / URL-safe: required because the chart interpolates the
|
||||
|
||||
Reference in New Issue
Block a user