fix(openviking): cap embedding max_input_tokens at 1536 to stay under llama.cpp nomic-bert 2048 ctx limit

astro-orbiter's llama.cpp router hard-caps nomic-embed-text-v1.5 effective
context at 2048 tokens regardless of ctx-size (known nomic-bert/RoPE limitation
in llama.cpp, not fixable server-side). OpenViking chunks observed at
2000-3400 tokens were tripping 400 exceed_context_size_error and endless
circuit-breaker re-enqueue for viking://temp/default/08140552_5f1c9e/homelab.tar/*.

Set embedding.max_input_tokens: 1536 (well under 2048) since OpenViking's
chunk-time token estimator uses a different tokenizer than llama.cpp's context
counter, so token counts won't match 1:1 - 1536 leaves ~25% headroom.

Approved by Ryan as lowest-risk mitigation (does not touch astro-orbiter/
llama.cpp serving config, which is War Machine's domain and already fixed
separately for the ubatch-size issue).
This commit is contained in:
Hermes Agent service account
2026-08-14 23:22:12 -05:00
parent 170a31d090
commit 48536f2615

View File

@@ -146,6 +146,17 @@ config:
dimension: 768
input: "text"
max_concurrent: 5
# max_input_tokens caps the raw text tokens OpenViking sends per chunk to the
# embedding model. astro-orbiter's llama.cpp router hard-caps nomic-embed-text-v1.5's
# effective context at 2048 tokens regardless of ctx-size config (known llama.cpp
# nomic-bert limitation, not fixable via server flags). OpenViking's chunker was
# observed producing 2000-3400 token chunks, well over that ceiling, causing
# `400 exceed_context_size_error, n_ctx: 2048` and endless circuit-breaker re-enqueues.
# Set well under 2048 (1536) to leave headroom: OpenViking's chunk-time token
# estimator is not the same tokenizer llama.cpp uses to count context, so token
# counts won't match 1:1 between the two. Approved by Ryan as lowest-risk fix
# (option 1 of 3) vs. touching the astro-orbiter serving stack further.
max_input_tokens: 1536
# ============================================================================
# VLM / Summarization configuration (L0/L1/L2 generation)