llm-inference-multimodel: add Qwen2.5-14B shadow instance (port 8002, gated off — VRAM co-residency not yet confirmed)
- New llama-server-qwen systemd unit template, gated by llm_qwen_service_enabled (default false) - Idempotent GGUF download task (bartowski Qwen2.5-14B-Instruct-Q5_K_M, stat-guarded) - Launch flags per local-llm-64k-context-recommendation.md: ctx-size 65536, flash-attn, q8_0 KV cache, batch 2048/ubatch 512, jinja, parallel 1 - verify.yml only starts/verifies the qwen unit when llm_qwen_service_enabled=true - README: documents live VRAM gate finding (nvidia-smi 2026-08-06: Phi-4+Mistral already ~16.6/24GB, ~7.5GB free -- insufficient for Qwen weights concurrently) and options - Does NOT touch llama-server-aux (8000) or llama-server-toolcall (8001) service state
This commit is contained in:
@@ -41,6 +41,18 @@
|
||||
notify:
|
||||
- reload systemd
|
||||
|
||||
- name: Deploy llama-server-qwen systemd unit (shadow, port 8002)
|
||||
ansible.builtin.template:
|
||||
src: llama-server-qwen.service.j2
|
||||
dest: "/etc/systemd/system/{{ llm_qwen_service_name }}.service"
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0644"
|
||||
become: true
|
||||
register: llm_qwen_unit_deployed
|
||||
notify:
|
||||
- reload systemd
|
||||
|
||||
- name: Flush handlers so daemon-reload lands before any later phase acts on unit state
|
||||
ansible.builtin.meta: flush_handlers
|
||||
|
||||
|
||||
Reference in New Issue
Block a user