rblundon/homelab
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

llm-inference-multimodel: correct stale VRAM estimate for qwen-1m shadow slot

Browse Source
This commit is contained in:
Hermes Agent service account
2026-08-06 10:40:13 -05:00
parent b741f9b20b
commit b4bdb63e4a
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
ansible/roles/llm-inference-multimodel/defaults/main.yml
View File

@@ -93,7 +93,7 @@ llm_qwen_batch_size: 2048
llm_qwen_ubatch_size: 512 llm_qwen_ubatch_size: 512
llm_qwen_service_name: llama-server-qwen llm_qwen_service_name: llama-server-qwen
llm_qwen_model_id: qwen2.5-14b-instruct-1m llm_qwen_model_id: qwen2.5-14b-instruct-1m
llm_qwen_expected_vram_gb: 16.5 llm_qwen_expected_vram_gb: 11 # updated 2026-08-06 for -1M checkpoint per report math (~10-12GB); prior 16.5 was stale base-Qwen estimate
# --- Existing Gemma baseline (rollback target — never modified by this role) - # --- Existing Gemma baseline (rollback target — never modified by this role) -
# Populated by Phase 0 discovery (tasks/discover.yml) if not already known. # Populated by Phase 0 discovery (tasks/discover.yml) if not already known.