rblundon/homelab
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

llm-inference-multimodel: repoint qwen shadow slot to Qwen2.5-14B-Instruct-1M (base Qwen disqualified, n_ctx_train=32768)

Browse Source
This commit is contained in:
Hermes Agent service account
2026-08-06 10:39:45 -05:00
parent a3c1342837
commit b741f9b20b
2 changed files with 23 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
ansible/roles/llm-inference-multimodel/templates/llama-server-qwen.service.j2
View File

@@ -1,5 +1,5 @@
[Unit]
Description=llama-server (shadow) — Qwen2.5-14B-Instruct Q5_K_M (OpenAI-compatible inference, 64K ctx)
Description=llama-server (shadow) — Qwen2.5-14B-Instruct-1M Q5_K_M (OpenAI-compatible inference, 64K ctx)
After=network.target nvidia-persistenced.service
Wants=nvidia-persistenced.service