Commit Graph

6 Commits

Author SHA1 Message Date
Hermes Agent service account
b741f9b20b llm-inference-multimodel: repoint qwen shadow slot to Qwen2.5-14B-Instruct-1M (base Qwen disqualified, n_ctx_train=32768) 2026-08-06 10:39:45 -05:00
Hermes Agent service account
a3c1342837 llm-inference-multimodel: reset qwen shadow unit to disabled by default -- model disqualified (n_ctx_train=32768, not 64K+), leaving enabled would crash-loop on next playbook run 2026-08-06 09:38:32 -05:00
Hermes Agent service account
75cb93f25c llm-inference-multimodel: enable Qwen2.5-14B shadow instance (port 8002) for shadow-test window
Mistral (llama-server-toolcall, 8001) stopped temporarily on astro-orbiter to
free ~6.2GB VRAM headroom for this test window per Ryan/JARVIS approval.
Not a permanent decommission of Mistral.
2026-08-06 09:10:06 -05:00
Hermes Agent service account
d10255297c llm-inference-multimodel: add Qwen2.5-14B shadow instance (port 8002, gated off — VRAM co-residency not yet confirmed)
- New llama-server-qwen systemd unit template, gated by llm_qwen_service_enabled (default false)
- Idempotent GGUF download task (bartowski Qwen2.5-14B-Instruct-Q5_K_M, stat-guarded)
- Launch flags per local-llm-64k-context-recommendation.md: ctx-size 65536, flash-attn, q8_0 KV cache, batch 2048/ubatch 512, jinja, parallel 1
- verify.yml only starts/verifies the qwen unit when llm_qwen_service_enabled=true
- README: documents live VRAM gate finding (nvidia-smi 2026-08-06: Phi-4+Mistral already ~16.6/24GB, ~7.5GB free -- insufficient for Qwen weights concurrently) and options
- Does NOT touch llama-server-aux (8000) or llama-server-toolcall (8001) service state
2026-08-06 09:08:33 -05:00
Hermes Agent service account
5dc76a8348 llm-inference-multimodel: fix tool-calling support (jinja template + gpu-layers=20 for VRAM fit) 2026-08-05 17:13:32 -05:00
Hermes Agent service account
c3755aa29e llm-inference-multimodel: role + day1 playbook (phase 0 discover approved) 2026-08-05 15:53:31 -05:00