2cc9370f3d
- vllm.service.j2: branch on role==embedding for --runner pooling --convert embed, --no-enable-prefix-caching, per-model trust_remote_code toggle (needed for nomic-embed-text-v1.5's custom NomicBertModel code), and enforce_eager toggle (needed to avoid CUDA graph capture OOM when co-resident with another vLLM process on this 24GB card). - tasks/verify.yml: split completions vs embedding smoke tests -- embedding-mode instances don't serve /v1/completions. Assert a non-empty embedding vector, not just HTTP 200. - host_vars/astro-orbiter: enable nomic-embed-text-v1.5 (port 8020), lower primary model's gpu_memory_utilization 0.95->0.90 + add enforce_eager after finding 0.95 crash-looped 6-7x before stabilizing with co-resident nomic-embed (real fix, confirmed via NRestarts=0 after clean stop/start, not luck). - Hindsight (values.yaml + externalsecret.yaml): cut LLM + embeddings over to vLLM (:8000, :8020), wire the previously-unset HINDSIGHT_API_EMBEDDINGS_* env vars for the first time, and swap the API key secret source from the Nous fallback item to vllm/api-key (vLLM enforces real auth, llama-swap did not). - README: document the embedding-mode branch, VRAM findings, and a genuine architecture gap -- vLLM's one-model-per-process design cannot replace llama-swap's 5-model LRU roster on this 24GB card, so 21 Hermes profiles' aux-model consumers (Qwen3-8B-no_think, Phi-3.5-mini, Meta-Llama-3.1-8B, Qwen2.5-Coder-14B) and OpenViking's VLM stay on llama-swap. Full teardown (t_6dff1ecc) needs a human decision on the aux-model strategy before it can proceed.
93 lines
4.7 KiB
YAML
93 lines
4.7 KiB
YAML
# ============================================================================
|
|
# ExternalSecret - Hindsight Credentials
|
|
# Peter Parker, Phase C, t_6d47a360
|
|
#
|
|
# Syncs Hindsight credentials from the 1Password `mk-labs` vault (item
|
|
# "hindsight", UUID q6pwoohexogdyvitt7sm2xcusu).
|
|
# Pattern: openviking proven pattern (cluster/platform/openviking/externalsecret.yaml).
|
|
# Store: onepassword-connect ClusterSecretStore.
|
|
# Namespace: hindsight.
|
|
#
|
|
# Wave: -1 — must sync BEFORE the chart's Deployments (which default to wave 0)
|
|
# so the materialized Secret exists before the api / control-plane pods attempt
|
|
# envFrom / secretKeyRef injection.
|
|
#
|
|
# The materialized Secret (hindsight-credentials) must carry keys in TWO shapes
|
|
# (Phase B record: inbox/ryan/2026-08-24-hindsight-phase-b-secrets.md):
|
|
# 1. `postgres-password` (lowercase, hyphenated) — read by the chart's EXPLICIT
|
|
# secretKeyRef for POSTGRES_PASSWORD, NOT via envFrom. A naming error here
|
|
# silently drops the DB password (pod starts, then fails to connect).
|
|
# 2. `HINDSIGHT_API_LLM_API_KEY` / `HINDSIGHT_API_MCP_AUTH_TOKEN` — env-var-named,
|
|
# injected via envFrom (always runs when existingSecret is set).
|
|
#
|
|
# 1Password item "hindsight" fields (all CONCEALED):
|
|
# - postgres-password (32 chars, letters+digits only, URL-safe)
|
|
# - HINDSIGHT_API_MCP_AUTH_TOKEN (48 chars urlsafe bearer token)
|
|
# - HINDSIGHT_API_LLM_API_KEY (literal "local-placeholder" — astro-orbiter
|
|
# does not validate; must rotate if OpenRouter
|
|
# hosted fallback is ever enabled)
|
|
# ============================================================================
|
|
apiVersion: external-secrets.io/v1beta1
|
|
kind: ExternalSecret
|
|
metadata:
|
|
name: hindsight-credentials
|
|
namespace: hindsight
|
|
annotations:
|
|
# Wave -1: ensure the ExternalSecret syncs BEFORE the chart's Deployments
|
|
# (wave 0). Guarantees the Secret exists before the api / control-plane
|
|
# pods attempt envFrom / secretKeyRef injection.
|
|
argocd.argoproj.io/sync-wave: "-1"
|
|
description: "Phase C secrets for Hindsight deployment (1Password mk-labs item: hindsight)"
|
|
spec:
|
|
refreshInterval: "1h"
|
|
secretStoreRef:
|
|
kind: ClusterSecretStore
|
|
name: onepassword-connect
|
|
target:
|
|
name: hindsight-credentials
|
|
creationPolicy: Owner
|
|
template:
|
|
engineVersion: v2
|
|
data:
|
|
# LOWERCASE, HYPHENATED — read by the chart's explicit secretKeyRef for
|
|
# POSTGRES_PASSWORD. Must be exactly "postgres-password".
|
|
postgres-password: "{{ .postgresPassword }}"
|
|
# env-var-named keys injected via envFrom
|
|
HINDSIGHT_API_MCP_AUTH_TOKEN: "{{ .HINDSIGHT_API_MCP_AUTH_TOKEN }}"
|
|
HINDSIGHT_API_LLM_API_KEY: "{{ .HINDSIGHT_API_LLM_API_KEY }}"
|
|
# Cutover to vLLM (t_e6facb19, 2026-08-31): vLLM enforces its API key
|
|
# on every request (unlike llama-swap, which accepted any/no key) —
|
|
# confirmed empirically, a bad/missing key gets a real 401
|
|
# {"error":"Unauthorized"}. Both the completions endpoint (:8000) and
|
|
# the embeddings endpoint (:8020) are separate vLLM processes but
|
|
# share the SAME key (api-key.env is written once, read by both
|
|
# systemd units per roles/deploy-vllm/tasks/api-key.yml). Reuse
|
|
# HINDSIGHT_API_LLM_API_KEY's value for the embeddings key too.
|
|
HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY: "{{ .HINDSIGHT_API_LLM_API_KEY }}"
|
|
data:
|
|
# 1Password item "hindsight", field "postgres-password".
|
|
# letters+digits only / URL-safe: required because the chart interpolates the
|
|
# value RAW into HINDSIGHT_API_DATABASE_URL (no URL-encoding).
|
|
- secretKey: postgresPassword
|
|
remoteRef:
|
|
key: hindsight
|
|
property: postgres-password
|
|
# 1Password item "hindsight", field "HINDSIGHT_API_MCP_AUTH_TOKEN"
|
|
# (MCP bearer token, D6 — enables bearer-auth on the /mcp endpoint)
|
|
- secretKey: HINDSIGHT_API_MCP_AUTH_TOKEN
|
|
remoteRef:
|
|
key: hindsight
|
|
property: HINDSIGHT_API_MCP_AUTH_TOKEN
|
|
# Cutover to vLLM (t_e6facb19, 2026-08-31): astro-orbiter's llama-swap
|
|
# (which needed no real auth) is being retired for Hindsight's
|
|
# completions+embeddings roles. vLLM enforces a real API key — read the
|
|
# SAME key roles/deploy-vllm's api-key.yml phase writes to
|
|
# /etc/vllm/api-key.env on astro-orbiter, sourced from 1Password
|
|
# op://mk-labs/vllm/api-key (item "vllm", field "api-key", vault mk-labs).
|
|
# Replaces the prior "nous" item's api-key (Nous free-tier fallback,
|
|
# no longer the active LLM backend once this cutover lands).
|
|
- secretKey: HINDSIGHT_API_LLM_API_KEY
|
|
remoteRef:
|
|
key: vllm
|
|
property: api-key
|