Phase 2 revised: consolidate astro-orbiter to single Qwen2.5-14B-1M model (port 8002)

- Retire llama-server-aux (Phi-4, 8000) and llama-server-toolcall (Mistral-Small-24B, 8001): stopped, disabled, unit files removed from host and Ansible role
- Promote llama-server-qwen (Qwen2.5-14B-Instruct-1M, port 8002) to sole production model, serving both friday and war-machine Hermes profiles
- Verified live: n_ctx=65536/n_ctx_train=1010000, and tool_calls response via /v1/chat/completions probe (no hallucination)
- Deleted superseded GGUF weights (phi-4, mistral-small, orphaned base-Qwen, gemma-2-27b) from astro-orbiter, ~45GB reclaimed
- Updated friday and war-machine Hermes profile configs (model + compression + skills_hub aux) to point at 10.1.71.130:8002
- Ryan explicitly accepted single-model tradeoffs for both profiles
This commit is contained in:
Hermes Agent service account
2026-08-06 11:42:34 -05:00
parent b4bdb63e4a
commit d1f97ad5ac
5 changed files with 94 additions and 320 deletions

View File

@@ -1,47 +1,28 @@
---
# ------------------------------------------------------------------------------
# FILE: roles/llm-inference-multimodel/tasks/systemd.yml
# DESCRIPTION: Phase 2 — template + deploy both unit files.
# DELIBERATELY DOES NOT START OR ENABLE either service — that is
# Phase 4 (verify.yml)'s job, after Phase 3 firewall scoping is
# in place. This keeps "units land on disk" and "processes
# actually bind ports and load 20+GB into VRAM" as separately
# reviewable checkpoints per Ryan's iterative-build preference.
# DESCRIPTION: Phase 2 (REVISED 2026-08-06) — consolidated to a single
# production unit: llama-server-qwen (Qwen2.5-14B-Instruct-1M,
# port 8002), serving BOTH the friday and war-machine Hermes
# profiles. Ryan explicitly accepted the tradeoffs of running
# one model for both profiles instead of the original
# aux+toolcall two-model split.
#
# Two independent units (llama-server-aux.service,
# llama-server-toolcall.service) — NOT one unit with two
# ExecStarts — so either can be stopped/restarted without
# affecting the other (plan §2, §6 rollback requirement).
# llama-server-aux (Phi-4, port 8000) and llama-server-toolcall
# (Mistral-Small-24B, port 8001) are RETIRED: services stopped
# + disabled, unit files removed from astro-orbiter, and their
# GGUF weights deleted from /opt/models (~45GB reclaimed).
# Ansible no longer templates or manages either unit — see git
# log for the prior task definitions if a future rollback needs
# them restored.
#
# The pre-existing Gemma unit (whatever discover.yml found it to
# be) is never templated, restarted, or disabled by this file.
# The pre-existing Gemma unit/weights (llama-server.service,
# gemma-2-27b-it-Q4_K_M.gguf) were ALSO removed as part of this
# consolidation (superseded baseline, no longer a rollback
# target once Qwen was accepted as sole production model).
# ------------------------------------------------------------------------------
- name: Deploy llama-server-aux systemd unit
ansible.builtin.template:
src: llama-server-aux.service.j2
dest: "/etc/systemd/system/{{ llm_aux_service_name }}.service"
owner: root
group: root
mode: "0644"
become: true
register: llm_aux_unit_deployed
notify:
- reload systemd
- name: Deploy llama-server-toolcall systemd unit
ansible.builtin.template:
src: llama-server-toolcall.service.j2
dest: "/etc/systemd/system/{{ llm_toolcall_service_name }}.service"
owner: root
group: root
mode: "0644"
become: true
register: llm_toolcall_unit_deployed
notify:
- reload systemd
- name: Deploy llama-server-qwen systemd unit (shadow, port 8002)
- name: Deploy llama-server-qwen systemd unit (production, port 8002)
ansible.builtin.template:
src: llama-server-qwen.service.j2
dest: "/etc/systemd/system/{{ llm_qwen_service_name }}.service"