Phase 2 revised: consolidate astro-orbiter to single Qwen2.5-14B-1M model (port 8002)
- Retire llama-server-aux (Phi-4, 8000) and llama-server-toolcall (Mistral-Small-24B, 8001): stopped, disabled, unit files removed from host and Ansible role - Promote llama-server-qwen (Qwen2.5-14B-Instruct-1M, port 8002) to sole production model, serving both friday and war-machine Hermes profiles - Verified live: n_ctx=65536/n_ctx_train=1010000, and tool_calls response via /v1/chat/completions probe (no hallucination) - Deleted superseded GGUF weights (phi-4, mistral-small, orphaned base-Qwen, gemma-2-27b) from astro-orbiter, ~45GB reclaimed - Updated friday and war-machine Hermes profile configs (model + compression + skills_hub aux) to point at 10.1.71.130:8002 - Ryan explicitly accepted single-model tradeoffs for both profiles
This commit is contained in:
@@ -1,47 +1,28 @@
|
||||
---
|
||||
# ------------------------------------------------------------------------------
|
||||
# FILE: roles/llm-inference-multimodel/tasks/systemd.yml
|
||||
# DESCRIPTION: Phase 2 — template + deploy both unit files.
|
||||
# DELIBERATELY DOES NOT START OR ENABLE either service — that is
|
||||
# Phase 4 (verify.yml)'s job, after Phase 3 firewall scoping is
|
||||
# in place. This keeps "units land on disk" and "processes
|
||||
# actually bind ports and load 20+GB into VRAM" as separately
|
||||
# reviewable checkpoints per Ryan's iterative-build preference.
|
||||
# DESCRIPTION: Phase 2 (REVISED 2026-08-06) — consolidated to a single
|
||||
# production unit: llama-server-qwen (Qwen2.5-14B-Instruct-1M,
|
||||
# port 8002), serving BOTH the friday and war-machine Hermes
|
||||
# profiles. Ryan explicitly accepted the tradeoffs of running
|
||||
# one model for both profiles instead of the original
|
||||
# aux+toolcall two-model split.
|
||||
#
|
||||
# Two independent units (llama-server-aux.service,
|
||||
# llama-server-toolcall.service) — NOT one unit with two
|
||||
# ExecStarts — so either can be stopped/restarted without
|
||||
# affecting the other (plan §2, §6 rollback requirement).
|
||||
# llama-server-aux (Phi-4, port 8000) and llama-server-toolcall
|
||||
# (Mistral-Small-24B, port 8001) are RETIRED: services stopped
|
||||
# + disabled, unit files removed from astro-orbiter, and their
|
||||
# GGUF weights deleted from /opt/models (~45GB reclaimed).
|
||||
# Ansible no longer templates or manages either unit — see git
|
||||
# log for the prior task definitions if a future rollback needs
|
||||
# them restored.
|
||||
#
|
||||
# The pre-existing Gemma unit (whatever discover.yml found it to
|
||||
# be) is never templated, restarted, or disabled by this file.
|
||||
# The pre-existing Gemma unit/weights (llama-server.service,
|
||||
# gemma-2-27b-it-Q4_K_M.gguf) were ALSO removed as part of this
|
||||
# consolidation (superseded baseline, no longer a rollback
|
||||
# target once Qwen was accepted as sole production model).
|
||||
# ------------------------------------------------------------------------------
|
||||
|
||||
- name: Deploy llama-server-aux systemd unit
|
||||
ansible.builtin.template:
|
||||
src: llama-server-aux.service.j2
|
||||
dest: "/etc/systemd/system/{{ llm_aux_service_name }}.service"
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0644"
|
||||
become: true
|
||||
register: llm_aux_unit_deployed
|
||||
notify:
|
||||
- reload systemd
|
||||
|
||||
- name: Deploy llama-server-toolcall systemd unit
|
||||
ansible.builtin.template:
|
||||
src: llama-server-toolcall.service.j2
|
||||
dest: "/etc/systemd/system/{{ llm_toolcall_service_name }}.service"
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0644"
|
||||
become: true
|
||||
register: llm_toolcall_unit_deployed
|
||||
notify:
|
||||
- reload systemd
|
||||
|
||||
- name: Deploy llama-server-qwen systemd unit (shadow, port 8002)
|
||||
- name: Deploy llama-server-qwen systemd unit (production, port 8002)
|
||||
ansible.builtin.template:
|
||||
src: llama-server-qwen.service.j2
|
||||
dest: "/etc/systemd/system/{{ llm_qwen_service_name }}.service"
|
||||
|
||||
Reference in New Issue
Block a user