Hermes Agent service account ad70b3439c feat(llm): add nomic-embed-text-v1.5-Q4_K_M to astro-orbiter router ...

OpenViking Phase 1b (t_34b96e83) — Ryan-approved implementation.

Changes:
- roles/llm-inference-multimodel/templates/llama-server-router-preset.ini.j2:
  Add [nomic-embed-text-v1.5] section with embedding=true, n-gpu-layers=99,
  ctx-size=8192, load-on-startup=true, sleep-idle-seconds=-1. No flash-attn
  or KV cache params (embedding models use bidirectional forward pass, not
  autoregressive KV cache). Var: llm_router_nomic_ctx_size.

- roles/llm-inference-multimodel/defaults/main.yml:
  Add llm_router_nomic_ctx_size: 8192.

- host_vars/astro-orbiter/vars.yml:
  Add nomic-embed-text-v1.5-Q4_K_M.gguf to llm_staged_models list
  (size_bytes: 84106624, source: nomic-ai/nomic-embed-text-v1.5-GGUF).
  Update VRAM note to reflect 5 registered models (nomic adds ~84MB,
  negligible given sleep-idle-seconds=-1 / load-on-startup=true pinning).

- playbooks/day2_add_nomic_embed.yml:
  New day2 playbook following the coder-alias pattern:
  Phase 1: idempotent GGUF download (exact size check)
  Phase 2: redeploy preset INI
  Phase 3: redeploy + restart systemd unit
  Phase 4: /v1/models gate (all 5 models present)
  Phase 5: /v1/embeddings smoke test (vector returned, not empty)

VRAM: ~84MB, always pinned. No impact on generative model LRU behavior.
peter-parker Helm values already point at :8002 for the embedding endpoint.

2026-08-13 23:18:27 -05:00

9.5 KiB

Raw Blame History

View Raw