Files
homelab/cluster
Hermes Agent service account f3a5687adf hindsight: cap RETAIN_MAX_COMPLETION_TOKENS for vLLM's 8192 ctx
Default 64000 exceeded vLLM Qwen2.5-32B-Instruct-AWQ's --max-model-len
8192, causing every retain call to 500 with
'max_tokens=64000 cannot be greater than max_model_len=8192'.
llama-swap's Qwen3.8-27B ran at ctx=65536 so this never surfaced
before the vLLM cutover. Lowered to 4096.
2026-08-31 18:29:22 -05:00
..
2026-05-19 22:25:00 -05:00
2026-05-18 22:01:18 -05:00