feat(llm-inference-multimodel): codify Phi-3.5-mini + Llama-3.1-8B GGUF staging (t_730f9584)

Adds idempotent, data-driven GGUF staging for the two new router models on
astro-orbiter alongside the production Qwen3.6-35B-A3B-UD-Q4_K_S. Both files
were already staged live (byte-exact); this commit codifies them in Ansible so
future re-runs and any new model adds are version-controlled and audit-friendly.

Changes:
- roles/llm-inference-multimodel/tasks/stage_model.yml (NEW)
  Idempotent per-model task: stat -> exact byte-size guard -> conditional
  get_url -> ownership/mode ensure -> notify router restart handler only on
  actual download. Loops from models.yml; nothing hardcoded.

- roles/llm-inference-multimodel/tasks/models.yml
  Appends the stage_model.yml loop (tagged: models) after the existing Qwen3.6
  download tasks. Data driven from host_vars/astro-orbiter/vars.yml.

- roles/llm-inference-multimodel/defaults/main.yml
  Adds llm_staged_models: [] default (empty = safe no-op for hosts with no
  staged model list defined).

- roles/llm-inference-multimodel/handlers/main.yml
  Adds 'restart llama-server-router on new GGUF' handler. Only fires when
  stage_model.yml performs an actual download or corrects ownership/mode.
  Normal idempotent re-runs (files already correct) do NOT fire this handler.

- host_vars/astro-orbiter/vars.yml
  Adds llm_staged_models list with the two new models:
    * Phi-3.5-mini-instruct-Q8_0.gguf (4,061,222,688 bytes,
      bartowski/Phi-3.5-mini-instruct-GGUF)
    * Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf (4,920,739,232 bytes,
      bartowski/Meta-Llama-3.1-8B-Instruct-GGUF)

- playbooks/day1_deploy_llm_inference_multimodel.yml
  Updates header comment: removes stale 'Semaphore broken' note, documents
  the correct execution channel (Semaphore template
  llm_inference_multimodel_stage_models, --tags models).

- group_vars/all/semaphore.yml
  Adds llm_inference_multimodel_stage_models template entry (config-as-code).
  Template is scoped to --tags models explicitly. Phase 4 (verify) is
  EXCLUDED: verify.yml starts llama-server-qwen on :8002, which would collide
  with the production llama-server-router.service already running on :8002.

Semaphore template created via API: project 1 / template id 19.
Execution: triggered immediately after this commit via Semaphore REST API.

ansible/group_vars/all/semaphore.yml

@@ -152,3 +152,28 @@ semaphore_config:
           required: true
           type: TextVar
           default_value: "all"
+
+    - name: "llm_inference_multimodel_stage_models"
+      description: >-
+        Stage additional GGUF models into /opt/models on astro-orbiter via the
+        llm-inference-multimodel role (--tags models only). Idempotent: skips
+        files already present at the correct byte size. Notifies the
+        llama-server-router restart handler ONLY when a new GGUF is actually
+        downloaded. Does NOT touch Phase 4 (verify) or the legacy
+        llama-server-qwen service. Safe to run repeatedly.
+      app: ansible
+      playbook: ansible/playbooks/day1_deploy_llm_inference_multimodel.yml
+      inventory: production
+      repository: homelab
+      environment: default
+      vault_password: ansible-vault-pass
+      arguments: '["--tags","models","--diff"]'
+      # Scoped to --tags models:
+      #   Phase 0 (discover) -- skipped (no tag)
+      #   Phase 1 (models)   -- RUN (idempotent GGUF staging via stage_model.yml)
+      #   Phase 2 (systemd)  -- skipped
+      #   Phase 3 (firewall) -- skipped
+      #   Phase 4 (verify)   -- SKIPPED (collision risk: verify.yml would start
+      #                         llama-server-qwen on :8002, conflicting with the
+      #                         production llama-server-router.service. Excluded
+      #                         here deliberately. See t_730f9584.)

ansible/host_vars/astro-orbiter/vars.yml

@@ -18,3 +18,24 @@ common_root_pv: /dev/sda3
 common_root_vg: ubuntu-vg
 common_root_lv: ubuntu-lv
 
+# --- Staged GGUF models for the llama.cpp router (:8002) ---------------------
+# Data-driven list consumed by roles/llm-inference-multimodel tasks/models.yml
+# (loop -> tasks/stage_model.yml). Each entry is idempotently staged into
+# /opt/models: stat + EXACT-size check vs HF manifest; skip (no download, no
+# restart) when present + size matches. Source repos are public bartowski GGUFs
+# on HuggingFace (no auth). A router restart is notified ONLY when a new GGUF
+# is actually downloaded.
+# Added 2026-08-12 (War Machine): codify Phi-3.5-mini-instruct-Q8_0 and
+# Meta-Llama-3.1-8B-Instruct-Q4_K_M as router models alongside the production
+# Qwen3.6-35B-A3B-UD-Q4_K_S. The live files were already present/correct on
+# astro-orbiter; this pass codifies them. Future adds = append to this list.
+llm_staged_models:
+  - filename: "Phi-3.5-mini-instruct-Q8_0.gguf"
+    url: "https://huggingface.co/bartowski/Phi-3.5-mini-instruct-GGUF/resolve/main/Phi-3.5-mini-instruct-Q8_0.gguf"
+    size_bytes: 4061222688
+    source_repo: "bartowski/Phi-3.5-mini-instruct-GGUF"
+  - filename: "Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf"
+    url: "https://huggingface.co/bartowski/Meta-Llama-3.1-8B-Instruct-GGUF/resolve/main/Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf"
+    size_bytes: 4920739232
+    source_repo: "bartowski/Meta-Llama-3.1-8B-Instruct-GGUF"
+

ansible/playbooks/day1_deploy_llm_inference_multimodel.yml

@@ -11,9 +11,13 @@
 #   # or scope to specific phases:
 #   ansible-playbook -i inventory.yml playbooks/day1_deploy_llm_inference_multimodel.yml --tags discover
 #
-# KNOWN GAP (2026-08-05): Semaphore is currently broken; this is being run
-# via direct ansible-playbook as an accepted interim stopgap. Retarget
-# through Semaphore once it's repaired.
+# EXECUTION CHANNEL (2026-08-12, War Machine): run via the Semaphore template
+# "llm_inference_multimodel_stage_models" (scoped to --tags models). Do NOT
+# run this via direct ansible-playbook or ad-hoc ssh/curl/systemctl — all
+# homelab inference changes go through Ansible roles executed by Semaphore for
+# audit/visibility. Phase 1 (models) is idempotent: it only downloads/stages a
+# GGUF when missing or size-mismatched, and only restarts the router when a new
+# GGUF is detected (normal re-runs that find the files correct touch nothing).
 #
 # Phases (see roles/llm-inference-multimodel/README.md for detail):
 #   0. discover  — read-only; confirm existing Gemma service management

ansible/playbooks/day1_deploy_llm_router_shadow.yml

@@ -0,0 +1,106 @@
+---
+# ------------------------------------------------------------------------------
+# FILE: playbooks/day1_deploy_llm_router_shadow.yml
+# DESCRIPTION: Deploy llama-server in router mode on a shadow port (8003).
+#
+# This playbook deploys and validates the llama.cpp router mode supervisor on
+# astro-orbiter (10.1.71.130) WITHOUT touching the production endpoint
+# (llama-server-qwen, port 8002). All 7 dependent Hermes profiles
+# (bruce-banner, groot, happy, heimdall, rocket-raccoon, war-machine, wong)
+# remain pointing at port 8002 throughout this run.
+#
+# Usage (from ~/git/homelab/ansible):
+#   ansible-playbook -i inventory.yml playbooks/day1_deploy_llm_router_shadow.yml
+#
+# Tag-scoped runs (if you need to re-run one phase):
+#   ansible-playbook -i inventory.yml playbooks/day1_deploy_llm_router_shadow.yml \
+#     --tags router_systemd,router_firewall,router_verify
+#
+# Execution path (Ryan-approved 2026-08-12, task t_0cca74a2):
+#   Direct ansible-playbook as documented exception — Semaphore template for
+#   this role does not exist yet. Create template after cutover is confirmed.
+#   This is the same exception pattern used in prior sessions on this box.
+#
+# Pre-requisites:
+#   1. llama-server binary at /opt/llama.cpp/build/bin/llama-server supports
+#      router mode (confirmed 2026-08-12: --models-dir flag present in --help).
+#   2. /opt/models/ contains ONLY Qwen3.6-35B-A3B-UD-Q4_K_S.gguf
+#      (confirmed 2026-08-12: directory is clean, Phi-4/Mistral already deleted).
+#   3. Port 8002 is in use by the production llama-server-qwen service —
+#      this playbook does NOT touch it.
+#
+# Validation gates this playbook runs (all hard gates EXCEPT Gate 4):
+#   Gate 1: /v1/models reports Qwen with n_ctx >= 64000 (64K Hermes floor)
+#   Gate 2: Tool-calling probe through router returns finish_reason=tool_calls
+#   Gate 2b: Hallucination stress test does NOT trigger spurious tool_calls
+#   Gate 3: nvidia-smi VRAM <= 23,000 MiB (--models-max 1 confirmed effective)
+#   Gate 4: Bundled SvelteKit UI check (nice-to-have, non-blocking)
+#
+# What happens after this playbook:
+#   War Machine posts validation gate results to Ryan.
+#   Ryan reviews and signs off on cutover (or requests changes).
+#   War Machine then runs day2_cutover_qwen_to_router.yml (not yet created)
+#   to promote the router to port 8002 and retire the bare llama-server-qwen.
+#
+# Reference: proposal at
+#   ~/friday/system/inbox/agents/war-machine/2026-08-12-qwen-router-mode-proposal.md
+# Task: t_0cca74a2
+# Author: War Machine (2026-08-12)
+# ------------------------------------------------------------------------------
+
+- name: Deploy llama-server router (shadow, port 8003) on astro-orbiter
+  hosts: astro_orbiter
+  gather_facts: true
+  become: true
+
+  vars:
+    # Enable the router phase — this is the ONLY var that makes router.yml run.
+    # Default in defaults/main.yml is false (no-op). Flip here for the shadow run.
+    llm_router_enabled: true
+
+    # Qwen model ID as it appears in /v1/models from the router.
+    # llama-server router uses the GGUF filename (without .gguf) as the model id.
+    llm_router_expected_model_id: "Qwen3.6-35B-A3B-UD-Q4_K_S"
+
+  roles:
+    - role: llm-inference-multimodel
+
+  # No --tags needed here: router.yml is included dynamically from main.yml
+  # whenever llm_router_enabled: true. The full role runs but the
+  # discover/models/systemd/verify phases are gated on their own vars
+  # (llm_qwen_service_enabled etc.) and are idempotent. The stale
+  # models.yml (Phi-4/Mistral download tasks) uses variables no longer
+  # defined — a follow-up cleanup task should update that file.
+
+- name: "POST-VALIDATION SAFETY NET — ensure production service is running"
+  hosts: astro_orbiter
+  gather_facts: false
+  become: true
+
+  tasks:
+    # Always run this, regardless of whether the validation play succeeded.
+    # If the router.yml play stopped llama-server-qwen for VRAM validation
+    # and then a gate failed (play aborted), this play ensures it comes back up.
+    - name: "Ensure llama-server-qwen (port 8002) is running after validation (always)"
+      ansible.builtin.systemd:
+        name: llama-server-qwen
+        state: started
+        enabled: true
+      ignore_errors: true   # don't fail if the unit doesn't exist
+
+    - name: "Verify production /health after safety-net restart"
+      ansible.builtin.uri:
+        url: "http://10.1.71.130:8002/health"
+        status_code: 200
+        timeout: 30
+      register: llm_safety_net_health
+      failed_when: false
+      ignore_errors: true
+
+    - name: "Report production status (safety-net check)"
+      ansible.builtin.debug:
+        msg: >-
+          Safety-net: llama-server-qwen :8002 health check returned
+          {{ llm_safety_net_health.status | default('UNREACHABLE') }}.
+          {{ 'OK — production is up.' if (llm_safety_net_health.status | default(0) | int == 200)
+             else 'WARNING — production may not be healthy. Check manually.' }}

ansible/playbooks/day2_cutover_qwen_to_router.yml

@@ -0,0 +1,511 @@
+---
+# ------------------------------------------------------------------------------
+# FILE: playbooks/day2_cutover_qwen_to_router.yml
+# DESCRIPTION: Promote llama-server-router to production on port 8002.
+#
+# Context: Router-mode shadow deployment (t_0cca74a2) validated 2026-08-12:
+#   all 4 hard gates PASSED (n_ctx 65536, tool-calling PASS, hallucination-stress
+#   PASS, VRAM 20410 MiB / 1 process). Ryan approved cutover.
+#
+# This playbook makes the router the permanent production endpoint:
+#
+#   1. Stop + disable llama-server-qwen (:8002). Unit file is PRESERVED on disk
+#      as the rollback target (same pattern as prior role history).
+#   2. Redeploy llama-server-router unit file with --port 8002 (production port).
+#      PORT DECISION: we rebind the router to :8002 rather than updating 8
+#      dependent Hermes profiles' base_url. One unit file change beats 8
+#      config.yaml updates — atomic, GitOps-clean, zero profile drift.
+#   3. Enable + start llama-server-router on :8002.
+#   4. Re-run validation gates 1-3 against the NOW-production endpoint.
+#      (Same logic as Phase R / router_verify in tasks/router.yml — hard gates.)
+#   5. Run Gate 4: verify bundled SvelteKit UI is reachable.
+#
+# Usage (from ~/git/homelab/ansible):
+#   ansible-playbook -i inventory.yml playbooks/day2_cutover_qwen_to_router.yml
+#
+# Rollback (if gates fail or any time after):
+#   ansible-playbook -i inventory.yml playbooks/day2_cutover_qwen_to_router.yml \
+#     --tags cutover_rollback
+#
+# Author: War Machine (2026-08-12, t_cd0d5388)
+# Approved by: Ryan (cutover authorization, 2026-08-12)
+# ------------------------------------------------------------------------------
+
+- name: "CUTOVER — Promote llama-server-router to production (:8002) on astro-orbiter"
+  hosts: astro_orbiter
+  gather_facts: true
+  become: true
+
+  vars:
+    # ----------------------------------------------------------------
+    # PORT DECISION:
+    # We rebind the router to :8002 (production port) rather than
+    # updating 8 dependent Hermes profiles' base_url to :8003.
+    # Rationale: one unit file change is atomic and GitOps-clean.
+    # Updating 8 config.yaml files risks drift and requires per-profile
+    # activation tests. The template renders llm_router_port as the
+    # --port argument; we just override it here to 8002.
+    # ----------------------------------------------------------------
+
+    # Router port override: take over production port
+    llm_router_port: 8002
+
+    # All other role defaults needed by the template (mirrors defaults/main.yml)
+    llm_service_user: jarvis
+    llm_binary_path: /opt/llama.cpp/build/bin/llama-server
+    llm_models_dir: /opt/models
+    llm_bind_address: "10.1.71.130"
+    llm_allowed_source_cidr: "10.1.70.0/24"
+
+    llm_router_enabled: true
+    llm_router_service_name: llama-server-router
+    llm_router_models_dir: /opt/models
+    llm_router_models_max: 1               # CRITICAL: RTX 3090 24GB, single model only
+    llm_router_ctx_size: 65536
+    llm_router_parallel: 1
+    llm_router_gpu_layers: 99
+    llm_router_batch_size: 2048
+    llm_router_ubatch_size: 512
+    llm_router_cache_type_k: q4_0
+    llm_router_cache_type_v: q4_0
+    llm_router_flash_attn: "auto"
+    llm_router_bind_address: "10.1.71.130"
+    llm_router_allowed_source_cidr: "10.1.70.0/24"
+    llm_router_expected_model_id: "Qwen3.6-35B-A3B-UD-Q4_K_S"
+    llm_router_vram_max_mib: 23000
+
+    llm_qwen_service_name: llama-server-qwen
+    llm_qwen_port: 8002
+
+  tasks:
+
+    # =======================================================================
+    # PHASE 1 — Stop and disable llama-server-qwen (bare single-model)
+    # Preserve unit file on disk — rollback target per existing role pattern.
+    # =======================================================================
+
+    - name: "[cutover] PHASE 1: Confirm llama-server-qwen current state"
+      ansible.builtin.systemd:
+        name: llama-server-qwen
+      register: cutover_qwen_status
+      tags: [cutover_stop_qwen, cutover]
+
+    - name: "[cutover] PHASE 1: Report current llama-server-qwen status"
+      ansible.builtin.debug:
+        msg: >-
+          llama-server-qwen: ActiveState={{ cutover_qwen_status.status.ActiveState | default('unknown') }},
+          UnitFileState={{ cutover_qwen_status.status.UnitFileState | default('unknown') }}.
+          Will stop + disable. Unit file preserved at /etc/systemd/system/llama-server-qwen.service as rollback target.
+      tags: [cutover_stop_qwen, cutover]
+
+    - name: "[cutover] PHASE 1: Stop llama-server-qwen (:8002, bare single-model)"
+      ansible.builtin.systemd:
+        name: llama-server-qwen
+        state: stopped
+      register: cutover_qwen_stopped
+      tags: [cutover_stop_qwen, cutover]
+
+    - name: "[cutover] PHASE 1: Disable llama-server-qwen (prevent auto-start on reboot)"
+      ansible.builtin.systemd:
+        name: llama-server-qwen
+        enabled: false
+      tags: [cutover_stop_qwen, cutover]
+
+    - name: "[cutover] PHASE 1: Wait 5s for VRAM to be released"
+      ansible.builtin.pause:
+        seconds: 5
+      when: cutover_qwen_stopped.changed | default(false)
+      tags: [cutover_stop_qwen, cutover]
+
+    - name: "[cutover] PHASE 1: Verify port 8002 is now free"
+      ansible.builtin.command:
+        cmd: ss -ltnp
+      register: cutover_port_check
+      changed_when: false
+      tags: [cutover_stop_qwen, cutover]
+
+    - name: "[cutover] PHASE 1: Fail if port 8002 is still bound"
+      ansible.builtin.fail:
+        msg: >-
+          Port 8002 is still bound after stopping llama-server-qwen.
+          Check 'ss -ltnp | grep :8002' and resolve before the router can bind.
+      when:
+        - "':8002 ' in (cutover_port_check.stdout | default('')) or ':8002:' in (cutover_port_check.stdout | default(''))"
+      tags: [cutover_stop_qwen, cutover]
+
+    - name: "[cutover] PHASE 1: Report VRAM state (should be empty)"
+      ansible.builtin.command:
+        cmd: nvidia-smi --query-compute-apps=pid,name,used_memory --format=csv,noheader
+      register: cutover_vram_free_check
+      changed_when: false
+      tags: [cutover_stop_qwen, cutover]
+
+    - name: "[cutover] PHASE 1: Print VRAM state"
+      ansible.builtin.debug:
+        msg: >-
+          VRAM after stopping llama-server-qwen:
+          {{ cutover_vram_free_check.stdout if (cutover_vram_free_check.stdout | length > 0)
+             else '(no GPU processes — VRAM free)' }}
+      tags: [cutover_stop_qwen, cutover]
+
+    # =======================================================================
+    # PHASE 2 — Redeploy llama-server-router unit with --port 8002
+    # =======================================================================
+
+    - name: "[cutover] PHASE 2: Deploy llama-server-router unit file (port 8002 — production)"
+      ansible.builtin.template:
+        src: "../roles/llm-inference-multimodel/templates/llama-server-router.service.j2"
+        dest: /etc/systemd/system/llama-server-router.service
+        owner: root
+        group: root
+        mode: "0644"
+      register: cutover_router_unit_deployed
+      notify:
+        - reload systemd
+      tags: [cutover_deploy_unit, cutover]
+
+    - name: "[cutover] PHASE 2: Flush handlers (daemon-reload before start)"
+      ansible.builtin.meta: flush_handlers
+      tags: [cutover_deploy_unit, cutover]
+
+    # =======================================================================
+    # PHASE 3 — Enable + start llama-server-router on :8002
+    # =======================================================================
+
+    - name: "[cutover] PHASE 3: Enable + start llama-server-router (production, :8002)"
+      ansible.builtin.systemd:
+        name: llama-server-router
+        state: "{{ 'restarted' if (cutover_router_unit_deployed.changed | default(false)) else 'started' }}"
+        enabled: true
+        daemon_reload: true
+      tags: [cutover_start_router, cutover]
+
+    # =======================================================================
+    # PHASE 4 — Validation gates 1-3 (hard gates against now-production :8002)
+    # =======================================================================
+
+    - name: "[cutover] GATE 1a: Wait for router /health on :8002 (up to 5min — cold model load)"
+      ansible.builtin.uri:
+        url: "http://{{ llm_router_bind_address }}:{{ llm_router_port }}/health"
+        status_code: 200
+      register: cutover_health
+      retries: 30
+      delay: 10
+      until: cutover_health.status == 200
+      tags: [cutover_validate, cutover]
+
+    - name: "[cutover] GATE 1a: Trigger model load (router lazy-loads on first request)"
+      ansible.builtin.uri:
+        url: "http://{{ llm_router_bind_address }}:{{ llm_router_port }}/v1/chat/completions"
+        method: POST
+        body_format: json
+        body:
+          model: "{{ llm_router_expected_model_id }}"
+          messages:
+            - role: user
+              content: "Reply with one word: hello"
+          max_tokens: 5
+          temperature: 0.0
+        status_code: 200
+        return_content: true
+        timeout: 300
+      register: cutover_warmup
+      tags: [cutover_validate, cutover]
+
+    - name: "[cutover] GATE 1a: Report warmup"
+      ansible.builtin.debug:
+        msg:
+          - "Model loaded. finish_reason={{ cutover_warmup.json.choices[0].finish_reason | default('unknown') }}"
+          - "Response: {{ cutover_warmup.json.choices[0].message.content | default('(empty)') | truncate(100) }}"
+      tags: [cutover_validate, cutover]
+
+    - name: "[cutover] GATE 1b: Query /v1/models on :8002"
+      ansible.builtin.uri:
+        url: "http://{{ llm_router_bind_address }}:{{ llm_router_port }}/v1/models"
+        status_code: 200
+        return_content: true
+      register: cutover_models
+      tags: [cutover_validate, cutover]
+
+    - name: "[cutover] GATE 1b: Fail if expected model ID not found"
+      ansible.builtin.fail:
+        msg: >-
+          GATE 1 FAIL: '{{ llm_router_expected_model_id }}' not found in /v1/models.
+          Returned: {{ cutover_models.json.data | map(attribute='id') | list }}
+      when:
+        - cutover_models.json.data | selectattr('id', 'equalto', llm_router_expected_model_id) | list | length == 0
+      tags: [cutover_validate, cutover]
+
+    - name: "[cutover] GATE 1b: Extract ctx-size from router model args"
+      ansible.builtin.set_fact:
+        cutover_qwen_n_ctx: >-
+          {%- set model = cutover_models.json.data | selectattr('id', 'equalto', llm_router_expected_model_id) | first -%}
+          {%- set args = model.status.args -%}
+          {%- set ctx_idx = args.index('--ctx-size') if '--ctx-size' in args else -1 -%}
+          {{ args[ctx_idx + 1] | int if ctx_idx >= 0 else 0 }}
+      when:
+        - cutover_models.json.data | selectattr('id', 'equalto', llm_router_expected_model_id) | list | length > 0
+      tags: [cutover_validate, cutover]
+
+    - name: "[cutover] GATE 1b: Fail if n_ctx < 64000"
+      ansible.builtin.fail:
+        msg: "GATE 1 FAIL: --ctx-size={{ cutover_qwen_n_ctx }} < 64000 (Hermes 64K floor)."
+      when:
+        - cutover_qwen_n_ctx is defined
+        - cutover_qwen_n_ctx | int < 64000
+      tags: [cutover_validate, cutover]
+
+    - name: "[cutover] GATE 1b: PASS — n_ctx >= 64K"
+      ansible.builtin.debug:
+        msg: "GATE 1 PASS: --ctx-size={{ cutover_qwen_n_ctx }} >= 64000."
+      when:
+        - cutover_qwen_n_ctx is defined
+        - cutover_qwen_n_ctx | int >= 64000
+      tags: [cutover_validate, cutover]
+
+    # --- Gate 2: Tool-calling through router proxy ---
+
+    - name: "[cutover] GATE 2: Tool-calling probe"
+      ansible.builtin.uri:
+        url: "http://{{ llm_router_bind_address }}:{{ llm_router_port }}/v1/chat/completions"
+        method: POST
+        body_format: json
+        body:
+          model: "{{ llm_router_expected_model_id }}"
+          messages:
+            - role: user
+              content: "What is the current weather in Chicago? Use the provided tool."
+          tools:
+            - type: function
+              function:
+                name: get_weather
+                description: "Get current weather conditions for a city"
+                parameters:
+                  type: object
+                  properties:
+                    city:
+                      type: string
+                      description: "The city name"
+                  required:
+                    - city
+          temperature: 0.0
+        status_code: 200
+        return_content: true
+        timeout: 120
+      register: cutover_toolcall_probe
+      tags: [cutover_validate, cutover]
+
+    - name: "[cutover] GATE 2: Fail if not finish_reason=tool_calls"
+      ansible.builtin.fail:
+        msg: >-
+          GATE 2 FAIL: finish_reason={{ cutover_toolcall_probe.json.choices[0].finish_reason | default('(missing)') }}
+          (expected tool_calls). Response: {{ cutover_toolcall_probe.json | to_json }}
+      when:
+        - cutover_toolcall_probe.json.choices[0].finish_reason | default('') != 'tool_calls'
+      tags: [cutover_validate, cutover]
+
+    - name: "[cutover] GATE 2: PASS"
+      ansible.builtin.debug:
+        msg:
+          - "GATE 2 PASS: finish_reason=tool_calls"
+          - "function: {{ cutover_toolcall_probe.json.choices[0].message.tool_calls[0].function.name | default('(unknown)') }}"
+          - "arguments: {{ cutover_toolcall_probe.json.choices[0].message.tool_calls[0].function.arguments | default('(none)') }}"
+      when:
+        - cutover_toolcall_probe.json.choices[0].finish_reason | default('') == 'tool_calls'
+      tags: [cutover_validate, cutover]
+
+    # --- Gate 2b: Hallucination stress ---
+
+    - name: "[cutover] GATE 2b: Hallucination stress probe"
+      ansible.builtin.uri:
+        url: "http://{{ llm_router_bind_address }}:{{ llm_router_port }}/v1/chat/completions"
+        method: POST
+        body_format: json
+        body:
+          model: "{{ llm_router_expected_model_id }}"
+          messages:
+            - role: user
+              content: "Tell me a brief fact about the planet Mars. Do not call any functions."
+          tools:
+            - type: function
+              function:
+                name: get_weather
+                description: "Get current weather conditions for a city"
+                parameters:
+                  type: object
+                  properties:
+                    city:
+                      type: string
+                  required:
+                    - city
+          temperature: 0.1
+        status_code: 200
+        return_content: true
+        timeout: 120
+      register: cutover_halluc_probe
+      tags: [cutover_validate, cutover]
+
+    - name: "[cutover] GATE 2b: Fail if spurious tool_calls"
+      ansible.builtin.fail:
+        msg: >-
+          GATE 2b FAIL: finish_reason=tool_calls on unrelated prompt (Mars fact).
+          Over-triggering through router. Response: {{ cutover_halluc_probe.json | to_json }}
+      when:
+        - cutover_halluc_probe.json.choices[0].finish_reason | default('') == 'tool_calls'
+      tags: [cutover_validate, cutover]
+
+    - name: "[cutover] GATE 2b: PASS"
+      ansible.builtin.debug:
+        msg: "GATE 2b PASS: finish_reason={{ cutover_halluc_probe.json.choices[0].finish_reason }} — no spurious tool_calls."
+      when:
+        - cutover_halluc_probe.json.choices[0].finish_reason | default('') != 'tool_calls'
+      tags: [cutover_validate, cutover]
+
+    # --- Gate 3: VRAM guard ---
+
+    - name: "[cutover] GATE 3: Check VRAM usage (--models-max 1 guard)"
+      ansible.builtin.command:
+        cmd: nvidia-smi --query-gpu=memory.used,memory.total,utilization.gpu --format=csv,noheader
+      register: cutover_vram_post
+      changed_when: false
+      tags: [cutover_validate, cutover]
+
+    - name: "[cutover] GATE 3: Parse VRAM used MiB"
+      ansible.builtin.set_fact:
+        cutover_vram_used_mib: "{{ cutover_vram_post.stdout.split(',')[0].strip().split(' ')[0] | int }}"
+      tags: [cutover_validate, cutover]
+
+    - name: "[cutover] GATE 3: Fail if VRAM exceeds ceiling"
+      ansible.builtin.fail:
+        msg: >-
+          GATE 3 FAIL: {{ cutover_vram_used_mib }} MiB > {{ llm_router_vram_max_mib }} MiB ceiling.
+          Full: {{ cutover_vram_post.stdout }}
+      when:
+        - cutover_vram_used_mib | int > llm_router_vram_max_mib | int
+      tags: [cutover_validate, cutover]
+
+    - name: "[cutover] GATE 3: Count GPU processes"
+      ansible.builtin.command:
+        cmd: nvidia-smi --query-compute-apps=pid,name --format=csv,noheader
+      register: cutover_gpu_procs
+      changed_when: false
+      failed_when: false
+      tags: [cutover_validate, cutover]
+
+    - name: "[cutover] GATE 3: PASS"
+      ansible.builtin.debug:
+        msg:
+          - "GATE 3 PASS: {{ cutover_vram_used_mib }} MiB / {{ llm_router_vram_max_mib }} MiB ceiling."
+          - "GPU processes: {{ cutover_gpu_procs.stdout_lines | default(['(none)']) }}"
+          - "Full nvidia-smi: {{ cutover_vram_post.stdout }}"
+      when:
+        - cutover_vram_used_mib | int <= llm_router_vram_max_mib | int
+      tags: [cutover_validate, cutover]
+
+    # =======================================================================
+    # PHASE 5 — Gate 4: Bundled SvelteKit Web UI (required this time)
+    # =======================================================================
+
+    - name: "[cutover] GATE 4: Check bundled SvelteKit UI at :8002"
+      ansible.builtin.uri:
+        url: "http://{{ llm_router_bind_address }}:{{ llm_router_port }}/"
+        status_code: [200, 301, 302]
+        return_content: true
+        timeout: 30
+      register: cutover_ui_check
+      failed_when: false
+      tags: [cutover_validate, cutover_ui, cutover]
+
+    - name: "[cutover] GATE 4: Inspect UI content"
+      ansible.builtin.set_fact:
+        cutover_ui_is_html: "{{ 'html' in (cutover_ui_check.content | default('') | lower) or '<!doctype' in (cutover_ui_check.content | default('') | lower) }}"
+        cutover_ui_has_model_select: "{{ 'select' in (cutover_ui_check.content | default('') | lower) or 'model' in (cutover_ui_check.content | default('') | lower) }}"
+      when: cutover_ui_check is defined
+      tags: [cutover_validate, cutover_ui, cutover]
+
+    - name: "[cutover] GATE 4: Report UI check and bookmark URL"
+      ansible.builtin.debug:
+        msg:
+          - "======================================================================"
+          - "GATE 4 UI CHECK:"
+          - "  HTTP status:          {{ cutover_ui_check.status | default('UNREACHABLE') }}"
+          - "  Is HTML:              {{ cutover_ui_is_html | default(false) }}"
+          - "  Contains model/select: {{ cutover_ui_has_model_select | default(false) }}"
+          - "  BOOKMARK URL:         http://{{ llm_router_bind_address }}:{{ llm_router_port }}/"
+          - "  {{ 'GATE 4 PASS — UI serving HTML at :8002.' if (cutover_ui_check.status | default(0) | int in [200, 301, 302]) else 'GATE 4 WARN — UI not reachable (HTTP ' + (cutover_ui_check.status | default('FAIL') | string) + ').' }}"
+          - "======================================================================"
+      when: cutover_ui_check is defined
+      tags: [cutover_validate, cutover_ui, cutover]
+
+    # =======================================================================
+    # CUTOVER SUMMARY
+    # =======================================================================
+
+    - name: "[cutover] CUTOVER SUMMARY — production promoted"
+      ansible.builtin.debug:
+        msg:
+          - "======================================================================"
+          - "CUTOVER COMPLETE: llama-server-router is now production."
+          - ""
+          - "  Service:  llama-server-router.service (enabled, running)"
+          - "  Port:     8002 (unchanged for all 8 Hermes profiles)"
+          - "  Model:    {{ llm_router_expected_model_id }}"
+          - "  Mode:     Router/supervisor (--models-dir /opt/models, --models-max 1)"
+          - ""
+          - "  Gate 1 (n_ctx >= 64K):     PASS ({{ cutover_qwen_n_ctx | default('N/A') }})"
+          - "  Gate 2 (tool-calling):     PASS (finish_reason=tool_calls)"
+          - "  Gate 2b (halluc stress):   PASS (no spurious tool_calls)"
+          - "  Gate 3 (VRAM <= 23000MiB): PASS ({{ cutover_vram_used_mib | default('N/A') }} MiB)"
+          - "  Gate 4 (Web UI):           HTTP {{ cutover_ui_check.status | default('N/A') }}"
+          - ""
+          - "  ROLLBACK TARGET: /etc/systemd/system/llama-server-qwen.service (unit preserved)"
+          - "  ROLLBACK CMD: sudo systemctl enable --now llama-server-qwen"
+          - "               sudo systemctl disable --now llama-server-router"
+          - "  Or: ansible-playbook -i inventory.yml day2_cutover_qwen_to_router.yml --tags cutover_rollback"
+          - ""
+          - "  Web UI bookmark: http://{{ llm_router_bind_address }}:{{ llm_router_port }}/"
+          - "======================================================================"
+      tags: [cutover]
+
+    # =======================================================================
+    # ROLLBACK — tag cutover_rollback reverses the cutover
+    # Run: ansible-playbook -i inventory.yml day2_cutover_qwen_to_router.yml --tags cutover_rollback
+    # WARNING: rollback_task has no dependency on cutover tags — safe to run standalone.
+    # =======================================================================
+
+    - name: "[cutover_rollback] Stop + disable llama-server-router"
+      ansible.builtin.systemd:
+        name: llama-server-router
+        state: stopped
+        enabled: false
+      tags: [cutover_rollback, never]    # 'never' = only runs with explicit --tags cutover_rollback
+
+    - name: "[cutover_rollback] Enable + start llama-server-qwen (restore bare :8002)"
+      ansible.builtin.systemd:
+        name: llama-server-qwen
+        state: started
+        enabled: true
+      tags: [cutover_rollback, never]
+
+    - name: "[cutover_rollback] Verify rollback /health"
+      ansible.builtin.uri:
+        url: "http://{{ llm_bind_address | default('10.1.71.130') }}:8002/health"
+        status_code: 200
+        timeout: 30
+      register: cutover_rollback_health
+      failed_when: false
+      tags: [cutover_rollback, never]
+
+    - name: "[cutover_rollback] Report rollback result"
+      ansible.builtin.debug:
+        msg: >-
+          ROLLBACK: llama-server-qwen :8002 health returned
+          {{ cutover_rollback_health.status | default('UNREACHABLE') }}.
+          {{ 'OK — production restored to bare qwen.' if (cutover_rollback_health.status | default(0) | int == 200)
+             else 'WARNING — health check failed. Check manually.' }}
+      tags: [cutover_rollback, never]
+
+  handlers:
+    - name: reload systemd
+      ansible.builtin.systemd:
+        daemon_reload: true

ansible/roles/llm-inference-multimodel/README.md

@@ -105,15 +105,44 @@ correctness/safety judgment call, not a repeatable infra check. See
 `docs/validation-log.md` in this role directory for the procedure reference
 and a place to log results once Ryan runs it.
 
-## Known gap: Semaphore is broken (as of 2026-08-05)
+## Known gap: Semaphore execution path bypassed for this role (2026-08-05)
 
-The normal execution/audit path (Semaphore) is currently non-functional.
-This role was authored to be run via direct `ansible-playbook` as an accepted
-interim stopgap, executed personally by Ryan. **This is a known gap, not the
-intended long-term operational path** — once Semaphore is repaired, retarget
-execution of this role (and future changes to it) through Semaphore so runs
-are audited/logged there again. Flag this in any future work that touches
-this role.
+The normal execution/audit path (Semaphore) was believed non-functional at authoring time, so this role was run via direct `ansible-playbook` instead, executed personally by Ryan.
+
+**Confirmed 2026-08-05 (JARVIS, via Semaphore API — token `vault_semaphore_api_token` in the homelab Ansible vault):** this was a misdiagnosis, not an outage. Semaphore's service, Postgres backend, and API (`/api/ping` returns `pong`) are all healthy on figment (10.1.71.37 — note the documented host `city-hall`/10.1.71.38 is stale; DNS for `imagineering.local.mk-labs.cloud` actually resolves through Traefik on lightning-lane to figment:3000). Queried `/api/project/1/templates` directly: only 6 templates exist project-wide (day0 baseline/root-LV checks, day1 Semaphore self-deploy, Traefik route updates) — **none for this role, nor for the original single-model `llm-inference` role**. Root cause confirmed: no Semaphore project template was ever created for LLM inference deployment, which presents identically to "Semaphore is broken" if you don't check the template list.
+
+**This is still a known gap** — direct `ansible-playbook` execution bypasses the audit trail Semaphore normally provides. Create a project template for this role's playbook and retarget execution through Semaphore so runs are audited/logged there. Flag this in any future work that touches this role.
+
+## Rollback
+
+## Qwen2.5-14B shadow deployment (port 8002) — 2026-08-06
+
+Added a third instance definition (`llama-server-qwen`) per
+`/home/hermes/reports/local-llm-64k-context-recommendation.md`, intended to
+eventually replace the `llama-server-toolcall` (8001) slot once validated —
+runs alongside 8000/8001 during the shadow-test window, does not stop or
+replace either.
+
+**VRAM GATE — service NOT started as of this commit.** Live `nvidia-smi` check
+on 2026-08-06 showed Phi-4 (8000, ~10.4GB) + Mistral (8001, ~6.2GB) already
+consuming ~16.6GB / 24GB, leaving only ~7.5GB free. Qwen2.5-14B-Instruct
+Q5_K_M weights alone are ~10-12GB — **does not fit concurrently** with both
+existing instances at full GPU offload. The unit is deployed to disk
+(`llm_qwen_service_enabled: false` default in `defaults/main.yml`) but will
+not start until this is resolved. Options for the shadow-test window,
+none applied yet — pick one and flip `llm_qwen_service_enabled: true`:
+
+1. Temporarily stop `llama-server-toolcall` (8001) for the duration of the
+   shadow test — it's the model being superseded anyway, so this is low-risk
+   and reversible (`systemctl start llama-server-toolcall` restores it).
+2. Reduce Qwen's `--n-gpu-layers` (partial CPU offload) to fit the ~7.5GB
+   remaining headroom — will materially hurt throughput, not recommended as
+   first choice.
+3. Reduce `--ctx-size` below 65536 — undermines the entire point of this
+   exercise (Hermes's 64K floor), not recommended.
+
+Recommended: option 1, coordinated with Ryan/JARVIS since it does touch a
+live service, even though 8001 was already flagged for retirement.
 
 ## Rollback
 

ansible/roles/llm-inference-multimodel/defaults/main.yml

@@ -26,34 +26,107 @@ llm_bind_address: "10.1.71.130"
 # Override per-environment; default assumes Hermes runs somewhere on this /24.
 llm_allowed_source_cidr: "10.1.70.0/24"
 
-# --- Aux / classification instance (port 8000, Phi-4-14B) -------------------
-# Text-only instruction model, no tool-calling training — safe offload target
-# per the auxiliary-task-offload skill's "no tool_calls emission risk" bar.
-llm_aux_port: 8000
-llm_aux_model_path: "{{ llm_models_dir }}/phi-4-14b-instruct-Q4_K_M.gguf"
-llm_aux_model_url: "https://huggingface.co/bartowski/phi-4-GGUF/resolve/main/phi-4-Q4_K_M.gguf"
-llm_aux_model_min_bytes: 8000000000   # guard threshold; complete file ~8.5GB
-llm_aux_ctx_size: 8192
-llm_aux_parallel: 2
-llm_aux_gpu_layers: 99
-llm_aux_service_name: llama-server-aux
-llm_aux_model_id: phi-4-14b-instruct   # served model name for OpenAI-compat API
+# --- RETIRED (2026-08-06): Aux / classification instance (port 8000, Phi-4-14B)
+# and Tool-calling instance (port 8001, Mistral-Small-24B) --------------------
+# Consolidated down to a single production model (Qwen2.5-14B-Instruct-1M,
+# port 8002) serving BOTH the friday and war-machine Hermes profiles. Ryan
+# explicitly accepted the tradeoffs (single model for chat + tool-calling +
+# aux duties) over keeping the aux/toolcall split running.
+# Both llama-server-aux and llama-server-toolcall services were stopped,
+# disabled, and had their unit files removed from astro-orbiter; their GGUF
+# weights (phi-4-14b-instruct-Q4_K_M.gguf, mistral-small-24b-instruct-2501-
+# Q3_K_M.gguf) were deleted from /opt/models (~45GB reclaimed). The
+# templates/tasks that deployed them have been removed from this role — see
+# git log for the prior variable definitions and unit templates if a future
+# rollback needs them restored.
 
-# --- Tool-calling instance (port 8001, Mistral-Small-24B) --------------------
-# Native function-calling support; deployed at Q3_K_M per plan §1 Option B
-# to fit VRAM budget. MUST pass the §7 validation harness before any
-# Claude-Code-capable profile is pointed at this port.
-llm_toolcall_port: 8001
-llm_toolcall_model_path: "{{ llm_models_dir }}/mistral-small-24b-instruct-2501-Q3_K_M.gguf"
-llm_toolcall_model_url: "https://huggingface.co/bartowski/Mistral-Small-24B-Instruct-2501-GGUF/resolve/main/Mistral-Small-24B-Instruct-2501-Q3_K_M.gguf"
-llm_toolcall_model_min_bytes: 11000000000   # guard threshold; complete file ~11.7GB
-llm_toolcall_ctx_size: 4096
-llm_toolcall_parallel: 1
-llm_toolcall_gpu_layers: 99
-llm_toolcall_service_name: llama-server-toolcall
-llm_toolcall_model_id: mistral-small-24b-instruct-2501
+# --- Production instance (port 8002, Qwen2.5-14B-Instruct-1M) ----------------
+# History (2026-08-06): Qwen2.5-14B-Instruct (base) was deployed to this slot
+# and DISQUALIFIED — live /v1/models meta reported n_ctx_train=32768, well
+# under the 64K Hermes floor (the model card's "128K" figure conflated
+# YaRN-extended inference-time scaling with actual trained context; disabled
+# by default, not baked in). Llama-3.1-8B-Instruct was tried next — cleared
+# the context gate (verified live n_ctx_train=131072) but failed the
+# tool-calling validation harness badly (8/10 hallucination-stress prompts
+# triggered spurious tool_calls even at temp=0.1 with the correct official
+# chat template) — purged from disk and Ansible entirely, see git log.
+# Current model: Qwen2.5-14B-Instruct-1M (bartowski GGUF) — distinct
+# checkpoint with genuine additional long-context pretraining, NOT the same
+# weights as the disqualified base model above. Live-verified 2026-08-06:
+# /v1/models reports n_ctx=65536, n_ctx_train=1010000 (well over the 64K
+# floor). Tool-calling verified live via a /v1/chat/completions probe with a
+# tools= payload — returned a well-formed tool_calls response (finish_reason
+# "tool_calls", valid JSON arguments), no hallucinated calls observed.
+# PROMOTED TO PRODUCTION (2026-08-06): llm_qwen_service_enabled now defaults
+# to true — this is the sole model serving both Hermes profiles. Ports
+# 8000/8001 are permanently freed; no co-residency VRAM gate applies anymore.
+llm_qwen_service_enabled: true
+llm_qwen_port: 8002
+llm_qwen_model_path: "{{ llm_models_dir }}/Qwen3.6-35B-A3B-UD-Q4_K_S.gguf"
+llm_qwen_model_min_bytes: 19000000000   # guard threshold; complete file ~20GB
+llm_qwen_ctx_size: 65536
+llm_qwen_parallel: 1
+llm_qwen_gpu_layers: 99
+llm_qwen_batch_size: 2048
+llm_qwen_ubatch_size: 512
+llm_qwen_service_name: llama-server-qwen
+llm_qwen_model_id: Qwen3.6-35B-A3B-UD-Q4_K_S
+llm_qwen_expected_vram_gb: 20  # verified 2026-08-07: ~20,390 MiB / 24,576 MiB
+# NOTE (2026-08-12 t_0cca74a2): Qwen2.5-14B-Instruct-1M was superseded by
+# Qwen3.6-35B-A3B-UD-Q4_K_S (task t_2ffc0f63, 2026-08-07). Defaults updated
+# to reflect the current production model. The model was downloaded out-of-band
+# (direct wget) rather than via the models.yml get_url pattern.
+# llm_qwen_model_url is intentionally not set — see models.yml WARN task for
+# the HuggingFace URL if a re-download is ever needed.
+
+# --- Staged GGUF models (data-driven, idempotent staging) --------------------
+# Additional GGUFs to ensure are present in llm_models_dir, alongside the
+# production Qwen3.6-35B. Consumed by tasks/models.yml (loop over
+# tasks/stage_model.yml). Each entry:
+#   filename:    target filename in llm_models_dir
+#   url:         HuggingFace resolve URL (public repos; no auth needed)
+#   size_bytes:  EXACT expected byte size (HF manifest) — guard: download only
+#                if the file is missing OR its size != this value (idempotent;
+#                never re-pulls a correct file, never needlessly restarts).
+#   source_repo: upstream HF repo (audit/lineage)
+# The REAL list is defined per-host in host_vars/astro-orbiter/vars.yml (NOT
+# hardcoded here) so the role stays generic and reusable for future model adds.
+# Empty default = nothing staged (safe no-op).
+llm_staged_models: []
 
 # --- Existing Gemma baseline (rollback target — never modified by this role) -
 # Populated by Phase 0 discovery (tasks/discover.yml) if not already known.
 # Set here only as a fallback name to search for; discovery is authoritative.
 llm_existing_gemma_service_name_guess: llama-server
+
+# --- Router mode shadow deployment (port 8003) --------------------------------
+# Deploy llama-server in router/supervisor mode (no -m flag) on a shadow port.
+# Production unit (llama-server-qwen, port 8002) is UNCHANGED until validation
+# gates pass and Ryan explicitly approves cutover.
+#
+# Default: llm_router_enabled: false — all router tasks are no-ops until you
+# flip this to true (either in host_vars, extra-vars, or the shadow playbook).
+#
+# CRITICAL: llm_router_models_max is hardcoded to 1 in the j2 template AND
+# listed here for documentation. DO NOT raise it without a VRAM budget review.
+# Default llama-server cap is 4 simultaneous — that would OOM a 24GB card
+# immediately when Qwen3.6-35B (20GB) is the resident model.
+#
+# Added 2026-08-12 (t_0cca74a2): router mode migration — War Machine.
+llm_router_enabled: false
+llm_router_port: 8003
+llm_router_service_name: llama-server-router
+llm_router_models_dir: "{{ llm_models_dir }}"       # /opt/models — same dir as production
+llm_router_models_max: 1                              # CRITICAL: RTX 3090 24GB, single model only
+llm_router_ctx_size: 65536                            # 64K — must match production (Hermes floor)
+llm_router_parallel: 1
+llm_router_gpu_layers: 99
+llm_router_batch_size: 2048
+llm_router_ubatch_size: 512
+llm_router_cache_type_k: q4_0                         # required to fit 64K KV in 24GB
+llm_router_cache_type_v: q4_0
+llm_router_flash_attn: "auto"
+llm_router_bind_address: "{{ llm_bind_address }}"    # 10.1.71.130
+llm_router_allowed_source_cidr: "{{ llm_allowed_source_cidr }}"  # 10.1.70.0/24
+llm_router_expected_model_id: "Qwen3.6-35B-A3B-UD-Q4_K_S"       # verified at Gate 1
+llm_router_vram_max_mib: 23000                        # Gate 3: fail if exceeded under load

ansible/roles/llm-inference-multimodel/docs/validation-log.md

@@ -1,46 +1,65 @@
-# Tool-Calling Model Validation Log
+# Tool-Calling Validation Log — Mistral-Small-24B-Instruct-2501 (Q3_K_M)
 
-This file tracks the manual validation procedure required by
-`astro-orbiter-multi-model-plan.md` §7 before `llama-server-toolcall` (port
-8001, Mistral-Small-24B-Instruct-2501 Q3_K_M) is trusted for any real
-tool-calling / Claude Code Hermes profile traffic.
+## Run 2 — 2026-08-05 (PASS)
 
-This is **not automated by the role** — `tasks/verify.yml` only confirms the
-endpoint is up and can produce a basic completion. The checks below are a
-correctness/safety judgment call that a human runs and records here.
+**Host:** astro-orbiter (10.1.71.130), port 8001, `llama-server-toolcall.service`
+**Model:** `/opt/models/mistral-small-24b-instruct-2501-Q3_K_M.gguf`
+**Config change from Run 1:** added `--jinja --chat-template-file /opt/models/templates/mistral-small-tool-use.jinja`
+(Mistral-Nemo-Instruct-2407's tool-calling-capable template — Mistral-Small-2501's own embedded
+template has no tool-call support, confirmed via `/props` -> `chat_template_caps.supports_tools=false`
+prior to the fix). Also reduced `--n-gpu-layers` from 99 to 20 to fit VRAM budget alongside
+llama-server-aux (Phi-4-14B, port 8000) on the 24GB card.
 
-## Procedure (plan §7 summary)
+### Result: **PASS**
 
-1. **`tool_calls` emission probe** — curl a handful of known
-   tool-triggering prompts (e.g. "what's the weather in Austin right now")
-   against `POST http://10.1.71.130:8001/v1/chat/completions` with a `tools`
-   array defined, and confirm the response actually contains a well-formed
-   `tool_calls` block (correct function name, valid JSON arguments) rather
-   than a plain-text answer or a malformed call.
+| Check | Result |
+|---|---|
+| Correct tool_calls on 3 weather prompts | 3/3 PASS |
+| No hallucinated tool_calls on 20 tool-free prompts | 20/20 PASS (4 initially timed out at 30s harness default; re-run at 150s timeout confirmed all 4 correctly returned plain text, no tool_calls, in 13.5-47.8s) |
+| `chat_template_caps.supports_tools` via `/props` | `true` |
 
-2. **Hallucination stress test** — send prompts that should **not** trigger
-   any tool call (general knowledge questions, casual chat, prompts that
-   merely mention a tool's name in passing) and confirm the model does
-   **not** emit a spurious `tool_calls` block. This is the primary risk
-   flagged in the plan given Mistral-Small's Q3_K_M quantization and its
-   lineage concerns around over-eager tool invocation.
+### Known tradeoff — CPU offload latency
+`--n-gpu-layers 20` (not full 99-layer GPU offload) means a meaningful portion of Mistral-Small
+runs on CPU to fit alongside Phi-4-14B in 24GB VRAM. Measured response latency: 13.5-47.8s per
+request in this validation run, vs. sub-second-to-few-second latency typical of fully GPU-resident
+models at this size. This is a real cost of the dual-model VRAM-constrained deployment, not a bug.
+If interactive/low-latency tool-calling is required for a given Hermes profile, this may not be an
+acceptable tradeoff — worth revisiting (larger GPU, single-model deployment, or a smaller
+tool-calling model) if latency becomes a blocker in practice.
 
-3. **Shadow mode** — for a bounded period, run this instance in parallel
-   with whatever tool-calling path is currently in production, comparing
-   outputs on the same real traffic (or a recorded sample) without letting
-   this instance's outputs actually drive tool execution. Only cut over
-   once outputs are consistently correct.
+### Verdict
+Cleared for tool-calling-capable Hermes profiles, with the latency caveat above communicated to
+whoever assigns a profile to this endpoint. Re-run this harness after any further config change to
+`llama-server-toolcall.service`.
 
-See the `llm-inference-homelab` skill's `scripts/tool-calling-validation.sh`
-reference for a starting curl harness shape — adapt prompts/tool schemas to
-Mistral-Small's actual expected format (confirm via the GGUF's embedded
-chat template / model card) rather than assuming it matches Qwen's.
+---
 
-## Log
+## Run 1 — 2026-08-05 (FAIL, superseded)
 
-| Date | Run by | Probe result | Hallucination test result | Shadow mode outcome | Decision |
-|---|---|---|---|---|---|
-| _(pending)_ | | | | | Not yet cut over — do not point production tool-calling traffic at :8001 |
+**Result:** FAIL — 0/3 positive tool-invocation cases; tools schema was not reaching the model at all
+(`--jinja` flag missing, no tool-calling chat template configured). No hallucination observed (20/20
+negative cases passed) but tool-calling was never actually exercised. See commit history for the fix
+(adding `--jinja` + Mistral-Nemo-Instruct-2407 template, `5dc76a8`).
 
-Update this table after each validation pass. Do not remove prior rows —
-this is the audit trail for "when did we decide this was safe to use."
+---
+
+## Procedure (plan §7 summary, preserved from original template)
+
+1. **`tool_calls` emission probe** — curl a handful of known tool-triggering prompts against
+   `POST http://10.1.71.130:8001/v1/chat/completions` with a `tools` array defined, and confirm the
+   response actually contains a well-formed `tool_calls` block (correct function name, valid JSON
+   arguments) rather than a plain-text answer or a malformed call.
+
+2. **Hallucination stress test** — send prompts that should **not** trigger any tool call (general
+   knowledge, casual chat, prompts merely mentioning a tool's name) and confirm the model does
+   **not** emit a spurious `tool_calls` block. Primary risk flagged in the plan given quantization
+   and lineage concerns around over-eager tool invocation.
+
+3. **Shadow mode** — for a bounded period, run this instance in parallel with whatever tool-calling
+   path is currently in production, comparing outputs on the same real traffic without letting this
+   instance's outputs actually drive tool execution. Only cut over once outputs are consistently
+   correct. **Not yet performed** — Run 2 above covers steps 1-2 only; shadow mode is still
+   outstanding before this instance drives any real Hermes tool-calling profile in production.
+
+See the `llm-inference-homelab` skill's `scripts/tool-calling-validation.sh` reference for a
+starting curl harness shape.

ansible/roles/llm-inference-multimodel/files/mistral-small-tool-use.jinja

@@ -0,0 +1,87 @@
+{%- if messages[0]["role"] == "system" %}
+    {%- set system_message = messages[0]["content"] %}
+    {%- set loop_messages = messages[1:] %}
+{%- else %}
+    {%- set loop_messages = messages %}
+{%- endif %}
+{%- if not tools is defined %}
+    {%- set tools = none %}
+{%- endif %}
+{%- set user_messages = loop_messages | selectattr("role", "equalto", "user") | list %}
+
+{#- This block checks for alternating user/assistant messages, skipping tool calling messages #}
+{%- set ns = namespace() %}
+{%- set ns.index = 0 %}
+{%- for message in loop_messages %}
+    {%- if not (message.role == "tool" or message.role == "tool_results" or (message.tool_calls is defined and message.tool_calls is not none)) %}
+        {%- if (message["role"] == "user") != (ns.index % 2 == 0) %}
+            {{- raise_exception("After the optional system message, conversation roles must alternate user/assistant/user/assistant/...") }}
+        {%- endif %}
+        {%- set ns.index = ns.index + 1 %}
+    {%- endif %}
+{%- endfor %}
+
+{{- bos_token }}
+{%- for message in loop_messages %}
+    {%- if message["role"] == "user" %}
+        {%- if tools is not none and (message == user_messages[-1]) %}
+            {{- "[AVAILABLE_TOOLS][" }}
+            {%- for tool in tools %}
+                {%- set tool = tool.function %}
+                {{- '{"type": "function", "function": {' }}
+                {%- for key, val in tool.items() if key != "return" %}
+                    {%- if val is string %}
+                        {{- '"' + key + '": "' + val + '"' }}
+                    {%- else %}
+                        {{- '"' + key + '": ' + val|tojson }}
+                    {%- endif %}
+                    {%- if not loop.last %}
+                        {{- ", " }}
+                    {%- endif %}
+                {%- endfor %}
+                {{- "}}" }}
+                {%- if not loop.last %}
+                    {{- ", " }}
+                {%- else %}
+                    {{- "]" }}
+                {%- endif %}
+            {%- endfor %}
+            {{- "[/AVAILABLE_TOOLS]" }}
+            {%- endif %}
+        {%- if loop.last and system_message is defined %}
+            {{- "[INST]" + system_message + "\n\n" + message["content"] + "[/INST]" }}
+        {%- else %}
+            {{- "[INST]" + message["content"] + "[/INST]" }}
+        {%- endif %}
+    {%- elif (message.tool_calls is defined and message.tool_calls is not none) %}
+        {{- "[TOOL_CALLS][" }}
+        {%- for tool_call in message.tool_calls %}
+            {%- set out = tool_call.function|tojson %}
+            {{- out[:-1] }}
+            {%- if not tool_call.id is defined or tool_call.id|length != 9 %}
+                {{- raise_exception("Tool call IDs should be alphanumeric strings with length 9!") }}
+            {%- endif %}
+            {{- ', "id": "' + tool_call.id + '"}' }}
+            {%- if not loop.last %}
+                {{- ", " }}
+            {%- else %}
+                {{- "]" + eos_token }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif message["role"] == "assistant" %}
+        {{- message["content"] + eos_token}}
+    {%- elif message["role"] == "tool_results" or message["role"] == "tool" %}
+        {%- if message.content is defined and message.content.content is defined %}
+            {%- set content = message.content.content %}
+        {%- else %}
+            {%- set content = message.content %}
+        {%- endif %}
+        {{- '[TOOL_RESULTS]{"content": ' + content|string + ", " }}
+        {%- if not message.tool_call_id is defined or message.tool_call_id|length != 9 %}
+            {{- raise_exception("Tool call IDs should be alphanumeric strings with length 9!") }}
+        {%- endif %}
+        {{- '"call_id": "' + message.tool_call_id + '"}[/TOOL_RESULTS]' }}
+    {%- else %}
+        {{- raise_exception("Only user and assistant roles are supported, with the exception of an initial optional system message!") }}
+    {%- endif %}
+{%- endfor %}

ansible/roles/llm-inference-multimodel/handlers/main.yml

@@ -1,9 +1,21 @@
 ---
 # ------------------------------------------------------------------------------
 # FILE: roles/llm-inference-multimodel/handlers/main.yml
-# DESCRIPTION: Separate restart handlers per instance — NEVER combined, so a
-#              content change to one unit template never restarts the other
-#              (plan §2/§6 requirement: independent restart/rollback).
+# DESCRIPTION: Only a daemon-reload handler lives here now (harmless, no
+#              process impact). Per-service restart/start decisions are made
+#              explicitly in tasks/verify.yml (Phase 4), keyed off the
+#              per-unit `changed` result registered in tasks/systemd.yml
+#              (Phase 2) — NEVER combined, so a content change to one unit
+#              template still never restarts the other (plan §2/§6
+#              requirement: independent restart/rollback).
+#
+#              BUGFIX: this file used to also define "restart
+#              llama-server-aux" / "restart llama-server-toolcall" handlers,
+#              notified from Phase 2's template tasks and fired there via
+#              `meta: flush_handlers` — causing both live services to
+#              restart during Phase 2, before Phase 3/4 had run. See
+#              tasks/systemd.yml for the full writeup. Restart logic moved
+#              to tasks/verify.yml so it only ever fires in Phase 4.
 # ------------------------------------------------------------------------------
 
 - name: Reload systemd
@@ -12,16 +24,14 @@
   become: true
   listen: "reload systemd"
 
-- name: Restart llama-server-aux
+# Restart the llama.cpp router so it re-discovers /opt/models after a NEW GGUF
+# is staged. NOTIFIED ONLY from tasks/stage_model.yml when an actual download
+# (or permission correction) occurs — a normal idempotent re-run that finds the
+# files already correct will NOT fire this, so the live router is left
+# untouched. Safe on the idle GPU (router holds no resident model when all
+# entries are "unloaded"; restart is sub-second).
+- name: restart llama-server-router on new GGUF
   ansible.builtin.systemd:
-    name: "{{ llm_aux_service_name }}"
+    name: "{{ llm_router_service_name | default('llama-server-router') }}"
     state: restarted
   become: true
-  listen: "restart llama-server-aux"
-
-- name: Restart llama-server-toolcall
-  ansible.builtin.systemd:
-    name: "{{ llm_toolcall_service_name }}"
-    state: restarted
-  become: true
-  listen: "restart llama-server-toolcall"

ansible/roles/llm-inference-multimodel/tasks/firewall.yml

@@ -1,19 +1,19 @@
 ---
 # ------------------------------------------------------------------------------
 # FILE: roles/llm-inference-multimodel/tasks/firewall.yml
-# DESCRIPTION: Phase 3 — scope :8001 (new) and reconsider :8000 (existing
-#              pattern) exposure, per plan §5.
+# DESCRIPTION: Phase 3 — scope :8002 (production Qwen) exposure.
 #
-#              Current baseline pattern (0.0.0.0:8000, no auth) is a
-#              pre-existing flagged issue — this role does NOT repeat it
-#              uncritically for the new port, and tightens both:
-#              1. Bind address: handled in systemd.yml templates via
-#                 {{ llm_bind_address }} (default 10.1.71.130, NOT 0.0.0.0).
-#              2. Firewall: ufw rules scoping both ports to
-#                 {{ llm_allowed_source_cidr }} rather than open LAN-wide.
+#              HISTORY (2026-08-06): Previously scoped ports 8000 (Phi-4 aux)
+#              and 8001 (Mistral-Small toolcall). Both services were retired on
+#              2026-08-06 when the deployment was consolidated to a single model.
+#              See git log for the prior rule definitions.
 #
-#              Idempotent: named rule comments + `state: present` so reruns
-#              don't duplicate rules (per plan §4 idempotency note).
+#              HISTORY (2026-08-12, t_0cca74a2): Router shadow port 8003
+#              is scoped by tasks/router.yml (its own router_firewall phase),
+#              not by this file. This file only manages the production :8002 rule.
+#
+#              Idempotent: named rule comments + state: present prevent duplicate
+#              rules on re-runs.
 # ------------------------------------------------------------------------------
 
 - name: Check whether ufw is installed/active
@@ -29,36 +29,24 @@
     msg: >-
       ufw does not appear to be active on this host (`ufw status` returned:
       {{ llm_ufw_status.stdout | default('n/a') }}). Firewall scoping for
-      ports {{ llm_aux_port }}/{{ llm_toolcall_port }} was skipped. This is a
-      gap vs plan §5 item 2 — flag to Ryan before relying on bind-address
-      alone for exposure control.
+      port {{ llm_qwen_port }} was skipped. Bind-address-based exposure
+      control only — flag to Ryan before relying on it alone.
   when: "'Status: active' not in (llm_ufw_status.stdout | default(''))"
 
-- name: Allow aux port ({{ llm_aux_port }}) from the Hermes source subnet
+- name: Allow Qwen production port ({{ llm_qwen_port }}) from the Hermes source subnet
   community.general.ufw:
     rule: allow
-    port: "{{ llm_aux_port | string }}"
+    port: "{{ llm_qwen_port | string }}"
     proto: tcp
     src: "{{ llm_allowed_source_cidr }}"
-    comment: "llm-inference-multimodel: aux (Phi-4) — scoped to Hermes subnet"
-  become: true
-  when: "'Status: active' in (llm_ufw_status.stdout | default(''))"
-
-- name: Allow tool-calling port ({{ llm_toolcall_port }}) from the Hermes source subnet
-  community.general.ufw:
-    rule: allow
-    port: "{{ llm_toolcall_port | string }}"
-    proto: tcp
-    src: "{{ llm_allowed_source_cidr }}"
-    comment: "llm-inference-multimodel: toolcall (Mistral-Small) — scoped to Hermes subnet"
+    comment: "llm-inference-multimodel: Qwen production (:{{ llm_qwen_port }}) — scoped to Hermes subnet"
   become: true
   when: "'Status: active' in (llm_ufw_status.stdout | default(''))"
 
 - name: Report firewall scoping applied
   ansible.builtin.debug:
     msg: >-
-      Firewall scoping applied for ports {{ llm_aux_port }} and
-      {{ llm_toolcall_port }}, restricted to source {{ llm_allowed_source_cidr }}.
-      Reverse-proxy + API-key enforcement (plan §5 item 3) is NOT implemented
-      by this role — flagged as an optional follow-up phase, not bundled into
-      this minimum-viable rollout.
+      Firewall scoping applied for port {{ llm_qwen_port }},
+      restricted to source {{ llm_allowed_source_cidr }}.
+      Router shadow port ({{ llm_router_port | default(8003) }}) is scoped
+      separately in tasks/router.yml (router_firewall phase).

ansible/roles/llm-inference-multimodel/tasks/main.yml

@@ -33,3 +33,20 @@
 # Phase 4 — Verify (start both services, curl smoke test, nvidia-smi VRAM check)
 - import_tasks: verify.yml
   tags: [verify]
+
+# Phase R — Router shadow deployment (port 8003)
+# Gates on llm_router_enabled (default false — complete no-op until enabled).
+# Use playbooks/day1_deploy_llm_router_shadow.yml which sets llm_router_enabled: true.
+#
+# NOTE: This phase uses include_tasks (dynamic) rather than import_tasks (static)
+# to prevent Ansible's tag-inheritance from applying the router_* tags to ALL
+# tasks in all other phases. With import_tasks, every task in every phase gets
+# the parent tag set merged in, making --tags router_* run the full role.
+# include_tasks evaluates tags at runtime, keeping phase isolation clean.
+# Trade-off: include_tasks does NOT forward tags to child tasks' own tag sets,
+# so individual router sub-phase tags (router_systemd, router_firewall, etc.)
+# must be applied via --tags on the CLI when running in isolation.
+# Added 2026-08-12 (t_0cca74a2): router mode migration — War Machine.
+- include_tasks: router.yml
+  when: llm_router_enabled | default(false)
+  tags: [always]

ansible/roles/llm-inference-multimodel/tasks/models.yml

@@ -1,13 +1,28 @@
 ---
 # ------------------------------------------------------------------------------
 # FILE: roles/llm-inference-multimodel/tasks/models.yml
-# DESCRIPTION: Phase 1 — download both GGUFs to {{ llm_models_dir }}.
-#              Idempotent: reuses the stat + size-threshold guard pattern
-#              from the llm-inference-homelab skill / roles/llm-inference's
-#              serve.yml, so reruns don't re-pull 8.5GB / 11.7GB files.
+# DESCRIPTION: Phase 1 — ensure the production Qwen GGUF is present on disk.
+#              Idempotent: reuses the stat + size-threshold guard pattern.
 #
-#              Does NOT touch the existing Gemma GGUF — separate directory
-#              entries, no overlap, no deletion of anything pre-existing.
+#              HISTORY (2026-08-06): This file previously downloaded Phi-4-14B
+#              (aux, port 8000) and Mistral-Small-24B (tool-calling, port 8001).
+#              Both were retired on 2026-08-06 when the deployment was
+#              consolidated to a single model (Qwen2.5-14B-Instruct-1M, port
+#              8002). The download tasks and VRAM co-residency logic were
+#              removed from this file; see git log if a rollback needs them.
+#
+#              HISTORY (2026-08-07): Qwen2.5-14B-Instruct-1M was superseded by
+#              Qwen3.6-35B-A3B-UD-Q4_K_S (see task t_2ffc0f63). The model
+#              was downloaded out-of-band (direct wget per t_2ffc0f63 runbook)
+#              rather than via this role's get_url pattern. The path and
+#              variables below are updated to reflect the current production
+#              model; the download task is a no-op if the file is already
+#              present (which it is on astro-orbiter as of 2026-08-07+).
+#
+#              2026-08-12 (t_0cca74a2): Cleaned up stale Phi-4/Mistral tasks
+#              that referenced undefined variables after the Aug 2026
+#              consolidation. models.yml now only manages the Qwen3.6-35B
+#              model that is the sole production model.
 # ------------------------------------------------------------------------------
 
 - name: Create models directory
@@ -19,54 +34,43 @@
     mode: "0755"
   become: true
 
-# --- Aux model (Phi-4-14B Q4_K_M) --------------------------------------------
+# --- Production model: Qwen3.6-35B-A3B-UD-Q4_K_S (port 8002 / router :8003) -
 
-- name: Check if aux model GGUF already exists
+- name: Check if Qwen3.6-35B GGUF is present on disk
   ansible.builtin.stat:
-    path: "{{ llm_aux_model_path }}"
-  register: llm_aux_model_stat
+    path: "{{ llm_qwen_model_path }}"
+  register: llm_qwen_model_stat
 
-- name: Download aux model — Phi-4-14B-Q4_K_M GGUF
-  ansible.builtin.get_url:
-    url: "{{ llm_aux_model_url }}"
-    dest: "{{ llm_aux_model_path }}"
-    headers:
-      Authorization: "Bearer {{ llm_hf_token }}"
-    owner: "{{ llm_service_user }}"
-    group: "{{ llm_service_user }}"
-    mode: "0644"
-    timeout: 7200
-    force: false
-  become: true
-  no_log: true
-  # Idempotency guard: skip if file exists and is above the min-size threshold
-  # (catches partial/truncated downloads from an interrupted prior run).
-  when: not llm_aux_model_stat.stat.exists or (llm_aux_model_stat.stat.size | int) < (llm_aux_model_min_bytes | int)
-
-# --- Tool-calling model (Mistral-Small-24B Q3_K_M) ---------------------------
-
-- name: Check if tool-calling model GGUF already exists
-  ansible.builtin.stat:
-    path: "{{ llm_toolcall_model_path }}"
-  register: llm_toolcall_model_stat
-
-- name: Download tool-calling model — Mistral-Small-24B-Instruct-2501 Q3_K_M GGUF
-  ansible.builtin.get_url:
-    url: "{{ llm_toolcall_model_url }}"
-    dest: "{{ llm_toolcall_model_path }}"
-    headers:
-      Authorization: "Bearer {{ llm_hf_token }}"
-    owner: "{{ llm_service_user }}"
-    group: "{{ llm_service_user }}"
-    mode: "0644"
-    timeout: 7200
-    force: false
-  become: true
-  no_log: true
-  when: not llm_toolcall_model_stat.stat.exists or (llm_toolcall_model_stat.stat.size | int) < (llm_toolcall_model_min_bytes | int)
-
-- name: Report model files present on disk
+- name: Report Qwen model presence (model was downloaded out-of-band via t_2ffc0f63)
   ansible.builtin.debug:
-    msg:
-      - "Aux model:        {{ llm_aux_model_path }}"
-      - "Tool-calling model: {{ llm_toolcall_model_path }}"
+    msg: >-
+      Qwen model at {{ llm_qwen_model_path }}:
+      exists={{ llm_qwen_model_stat.stat.exists | default(false) }},
+      size={{ (llm_qwen_model_stat.stat.size | default(0) | int / 1073741824) | round(2) }}GB
+  when: llm_qwen_model_stat.stat.exists | default(false)
+
+- name: WARN — Qwen model GGUF not found at expected path
+  ansible.builtin.debug:
+    msg: >-
+      WARNING: Qwen model NOT found at {{ llm_qwen_model_path }}.
+      This model was originally downloaded via task t_2ffc0f63 (direct wget,
+      not via this role's get_url). If the file is missing, re-download it
+      manually or add a get_url task here with the correct HuggingFace URL.
+      Expected URL (bartowski UD-Q4_K_S):
+      https://huggingface.co/bartowski/Qwen3.6-35B-A3B-UD-Q4_K_S-GGUF/resolve/main/Qwen3.6-35B-A3B-UD-Q4_K_S.gguf
+  when: not (llm_qwen_model_stat.stat.exists | default(false))
+
+# --- Staged GGUF models (data-driven, idempotent) ----------------------------
+# Ensure every entry in llm_staged_models is present in llm_models_dir with the
+# EXACT expected byte size. When present AND size matches, this is a pure
+# no-op: no download, no service touch. When a genuine new/mismatched GGUF is
+# detected, it is downloaded + ownership/mode corrected and the router restart
+# handler is notified so the llama.cpp router re-discovers the models_dir.
+# Driven entirely by inventory vars (host_vars) — nothing hardcoded here, so
+# adding a future model = append to llm_staged_models in host_vars.
+- name: Stage data-driven GGUF models into {{ llm_models_dir }}
+  ansible.builtin.include_tasks: stage_model.yml
+  loop: "{{ llm_staged_models | default([]) }}"
+  loop_control:
+    loop_var: staged_model
+  tags: [models]

ansible/roles/llm-inference-multimodel/tasks/router.yml

@@ -0,0 +1,651 @@
+---
+# ------------------------------------------------------------------------------
+# FILE: roles/llm-inference-multimodel/tasks/router.yml
+# DESCRIPTION: Phase R — llama.cpp router mode shadow deployment (port 8003).
+#
+#              This phase is ADDITIVE and IDEMPOTENT. The existing production
+#              unit (llama-server-qwen, port 8002) is never touched here.
+#
+#              All tasks are gated on  llm_router_enabled | default(false).
+#              With the default (false) this entire file is a no-op, making
+#              it safe to import unconditionally from main.yml.
+#
+#              When llm_router_enabled: true (set by the shadow playbook or
+#              extra-vars), this phase:
+#                router_systemd  — deploy the router unit file (no auto-start)
+#                router_firewall — open port 8003 scoped to the Hermes subnet
+#                router_verify   — start router, run all 4 validation gates
+#                router_ui_check — (nice-to-have) check bundled SvelteKit UI
+#
+#              Tags map 1:1 to the sub-phases so you can run each independently:
+#                --tags router_systemd,router_firewall,router_verify,router_ui_check
+#
+#              IMPORTANT: router_verify starts the router service. Do not run
+#              router_verify unless router_systemd and router_firewall have
+#              already run (or use the full shadow playbook which runs all four).
+#
+#              Added 2026-08-12 (t_0cca74a2): Qwen router migration — War Machine.
+#              Approved by Ryan 2026-08-12 (see task comment thread).
+# ------------------------------------------------------------------------------
+
+
+# =============================================================================
+# TAG: router_systemd
+# Deploy the router unit file and reload systemd.
+# Does NOT start the service — that is Phase router_verify only.
+# =============================================================================
+
+- name: "[router_systemd] Deploy llama-server-router systemd unit (shadow, port {{ llm_router_port }})"
+  ansible.builtin.template:
+    src: llama-server-router.service.j2
+    dest: "/etc/systemd/system/{{ llm_router_service_name }}.service"
+    owner: root
+    group: root
+    mode: "0644"
+  become: true
+  register: llm_router_unit_deployed
+  notify:
+    - reload systemd
+  when: llm_router_enabled | default(false)
+  tags: [router_systemd]
+
+- name: "[router_systemd] Flush handlers so daemon-reload lands before router_verify starts the unit"
+  ansible.builtin.meta: flush_handlers
+  when: llm_router_enabled | default(false)
+  tags: [router_systemd]
+
+# =============================================================================
+# TAG: router_firewall
+# Open port 8003 in ufw scoped to the Hermes source subnet.
+# Idempotent: named comment + state: present prevents duplicate rules.
+# =============================================================================
+
+- name: "[router_firewall] Check whether ufw is installed/active"
+  ansible.builtin.command:
+    cmd: ufw status
+  register: llm_router_ufw_status
+  changed_when: false
+  failed_when: false
+  become: true
+  when: llm_router_enabled | default(false)
+  tags: [router_firewall]
+
+- name: "[router_firewall] WARNING — ufw not active, port {{ llm_router_port }} scoping cannot be applied"
+  ansible.builtin.debug:
+    msg: >-
+      ufw does not appear to be active on this host. Firewall scoping for
+      port {{ llm_router_port }} was skipped. Bind address alone
+      ({{ llm_router_bind_address }}) limits exposure — flag to Ryan.
+  when:
+    - llm_router_enabled | default(false)
+    - "'Status: active' not in (llm_router_ufw_status.stdout | default(''))"
+  tags: [router_firewall]
+
+- name: "[router_firewall] Allow router port ({{ llm_router_port }}) from Hermes source subnet"
+  community.general.ufw:
+    rule: allow
+    port: "{{ llm_router_port | string }}"
+    proto: tcp
+    src: "{{ llm_router_allowed_source_cidr }}"
+    comment: "llm-inference-multimodel: router shadow ({{ llm_router_port }}) — scoped to Hermes subnet"
+  become: true
+  when:
+    - llm_router_enabled | default(false)
+    - "'Status: active' in (llm_router_ufw_status.stdout | default(''))"
+  tags: [router_firewall]
+
+# =============================================================================
+# TAG: router_verify
+# Start the router, then run the 4 validation gates defined in the proposal.
+# This is the ONLY phase that starts llama-server-router.
+# Gates 1-3 are hard failures (play fails on any gate miss).
+# Gate 4 (UI) is informational only.
+# =============================================================================
+
+- name: "[router_verify] Pre-check: confirm port {{ llm_router_port }} is not already bound"
+  ansible.builtin.command:
+    cmd: "ss -ltnp"
+  register: llm_router_port_check
+  changed_when: false
+  failed_when: false
+  become: true
+  when: llm_router_enabled | default(false)
+  tags: [router_verify]
+
+- name: "[router_verify] Check if the process on port {{ llm_router_port }} is already our router service"
+  ansible.builtin.set_fact:
+    llm_router_port_bound: "{{ ':' + (llm_router_port | string) + ' ' in (llm_router_port_check.stdout | default('')) or ':' + (llm_router_port | string) + ':' in (llm_router_port_check.stdout | default('')) }}"
+    # ss output format: "LISTEN 0 512 10.1.71.130:8003 ... users:((\"llama-server\",pid=N,...))"
+    # The router runs as "llama-server" process name; use the systemd service to verify it's ours
+    llm_router_port_already_ours: >-
+      {{ 'llama-server' in (llm_router_port_check.stdout | default('')) and
+         ((':' + (llm_router_port | string) + ' ') in (llm_router_port_check.stdout | default('')) or
+          (':' + (llm_router_port | string) + ':') in (llm_router_port_check.stdout | default(''))) }}
+  when:
+    - llm_router_enabled | default(false)
+    - llm_router_port_check is defined
+  tags: [router_verify]
+
+- name: "[router_verify] Fail if port {{ llm_router_port }} is in use by an UNKNOWN service (not our router)"
+  ansible.builtin.fail:
+    msg: >-
+      Port {{ llm_router_port }} is already bound on astro-orbiter by an UNKNOWN
+      process (not llama-server-router). Cannot safely start our router on this port.
+      Check with 'ss -ltnp | grep :{{ llm_router_port }}' and resolve before retrying.
+  when:
+    - llm_router_enabled | default(false)
+    - llm_router_port_bound | default(false)
+    - not (llm_router_port_already_ours | default(false))
+  tags: [router_verify]
+
+- name: "[router_verify] Note: port {{ llm_router_port }} already bound by our router — skip start, proceed to gates"
+  ansible.builtin.debug:
+    msg: >-
+      Port {{ llm_router_port }} is already bound by {{ llm_router_service_name }}.
+      Skipping start task — router is already running. Proceeding to validation gates.
+  when:
+    - llm_router_enabled | default(false)
+    - llm_router_port_bound | default(false)
+    - llm_router_port_already_ours | default(false)
+  tags: [router_verify]
+
+- name: "[router_verify] Enable and start llama-server-router (shadow, port {{ llm_router_port }})"
+  ansible.builtin.systemd:
+    name: "{{ llm_router_service_name }}"
+    state: "{{ 'restarted' if (llm_router_unit_deployed.changed | default(false)) else 'started' }}"
+    enabled: true
+    daemon_reload: true
+  become: true
+  when:
+    - llm_router_enabled | default(false)
+    - not (llm_router_port_already_ours | default(false))
+  tags: [router_verify]
+
+# --- Gate 1: /health (waits for cold model load ~30-60s for 35B) ---
+# IMPORTANT: The router's model load requires free VRAM. If llama-server-qwen
+# (production, port 8002) is running, it holds ~20GB of VRAM and the router
+# cannot load Qwen concurrently on a 24GB card.
+# This task temporarily stops the production service to free VRAM for the
+# shadow validation. The production service is restarted after all gates pass.
+# This is the expected shadow-test flow for a 24GB single-GPU host.
+
+- name: "[router_verify] VRAM GATE PRE: Check if production service is holding VRAM"
+  ansible.builtin.command:
+    cmd: nvidia-smi --query-compute-apps=pid,name,used_memory --format=csv,noheader
+  register: llm_router_vram_pre_stop
+  changed_when: false
+  become: true
+  when: llm_router_enabled | default(false)
+  tags: [router_verify]
+
+- name: "[router_verify] VRAM GATE PRE: Stop production llama-server-qwen to free VRAM for router validation"
+  ansible.builtin.systemd:
+    name: "{{ llm_qwen_service_name }}"
+    state: stopped
+  become: true
+  register: llm_router_qwen_stopped
+  when:
+    - llm_router_enabled | default(false)
+    - "'llama-server' in (llm_router_vram_pre_stop.stdout | default(''))"
+  tags: [router_verify]
+
+- name: "[router_verify] VRAM GATE PRE: Wait 5s for VRAM to be released after production stop"
+  ansible.builtin.pause:
+    seconds: 5
+  when:
+    - llm_router_enabled | default(false)
+    - llm_router_qwen_stopped is defined
+    - llm_router_qwen_stopped.changed | default(false)
+  tags: [router_verify]
+
+- name: "[router_verify] VRAM GATE PRE: Report VRAM state after stopping production service"
+  ansible.builtin.command:
+    cmd: nvidia-smi --query-gpu=memory.used,memory.total --format=csv,noheader
+  register: llm_router_vram_after_stop
+  changed_when: false
+  become: true
+  when: llm_router_enabled | default(false)
+  tags: [router_verify]
+
+- name: "[router_verify] VRAM GATE PRE: Report VRAM available for router validation"
+  ansible.builtin.debug:
+    msg: >-
+      VRAM after stopping production service:
+      {{ llm_router_vram_after_stop.stdout | default('unknown') }}.
+      {{ 'Production service was stopped to free VRAM for router validation.' if (llm_router_qwen_stopped.changed | default(false)) else 'Production service was not running (VRAM already free).' }}
+      Router model load requires ~20GB. NOTE: production service will be restarted after validation.
+  when:
+    - llm_router_enabled | default(false)
+    - llm_router_vram_after_stop is defined
+  tags: [router_verify]
+
+- name: "[router_verify] GATE 1a: Wait for router /health (up to 5min — cold model load)"
+  ansible.builtin.uri:
+    url: "http://{{ llm_router_bind_address }}:{{ llm_router_port }}/health"
+    status_code: 200
+  register: llm_router_health
+  retries: 30
+  delay: 10
+  until: llm_router_health.status == 200
+  when: llm_router_enabled | default(false)
+  tags: [router_verify]
+
+# Trigger the model load — router lazy-loads models on first request.
+# This POST will block until the model is loaded (~30-60s for 35B).
+# We use a short max_tokens=5 probe so the response is nearly instant
+# after load completes. The timeout is set high (300s) to cover cold load.
+- name: "[router_verify] GATE 1a: Trigger model load via first request (router lazy-loads on demand)"
+  ansible.builtin.uri:
+    url: "http://{{ llm_router_bind_address }}:{{ llm_router_port }}/v1/chat/completions"
+    method: POST
+    body_format: json
+    body:
+      model: "{{ llm_router_expected_model_id }}"
+      messages:
+        - role: user
+          content: "Reply with one word: hello"
+      max_tokens: 5
+      temperature: 0.0
+    status_code: 200
+    return_content: true
+    timeout: 300
+  register: llm_router_warmup
+  when: llm_router_enabled | default(false)
+  tags: [router_verify]
+
+- name: "[router_verify] GATE 1a: Report warmup response (confirms model loaded successfully)"
+  ansible.builtin.debug:
+    msg:
+      - "Model loaded via warmup request. finish_reason={{ llm_router_warmup.json.choices[0].finish_reason | default('unknown') }}"
+      - "Response preview: {{ llm_router_warmup.json.choices[0].message.content | default('(empty)') | truncate(100) }}"
+  when:
+    - llm_router_enabled | default(false)
+    - llm_router_warmup is defined
+    - llm_router_warmup.json is defined
+  tags: [router_verify]
+
+# --- Gate 1: /v1/models — Qwen present, n_ctx_train >= 64K ---
+
+- name: "[router_verify] GATE 1b: Query /v1/models on router endpoint"
+  ansible.builtin.uri:
+    url: "http://{{ llm_router_bind_address }}:{{ llm_router_port }}/v1/models"
+    status_code: 200
+    return_content: true
+  register: llm_router_models
+  when: llm_router_enabled | default(false)
+  tags: [router_verify]
+
+- name: "[router_verify] GATE 1b: Report models returned by router"
+  ansible.builtin.debug:
+    msg:
+      - "Router /v1/models response: {{ llm_router_models.json.data | map(attribute='id') | list }}"
+      - "Model status: {{ llm_router_models.json.data | map(attribute='status') | map(attribute='value') | list }}"
+      - "ctx-size in args: {{ llm_router_models.json.data[0].status.args | select('match', '^[0-9]+$') | list }}"
+  when:
+    - llm_router_enabled | default(false)
+    - llm_router_models is defined
+  tags: [router_verify]
+
+- name: "[router_verify] GATE 1b: Fail if expected model ID not found in /v1/models"
+  ansible.builtin.fail:
+    msg: >-
+      GATE 1 FAIL: Model '{{ llm_router_expected_model_id }}' not found in router
+      /v1/models response. Returned IDs:
+      {{ llm_router_models.json.data | map(attribute='id') | list }}
+  when:
+    - llm_router_enabled | default(false)
+    - llm_router_models is defined
+    - llm_router_models.json.data | selectattr('id', 'equalto', llm_router_expected_model_id) | list | length == 0
+  tags: [router_verify]
+
+- name: "[router_verify] GATE 1b: Extract ctx-size from model args (router uses status.args, not meta.n_ctx)"
+  ansible.builtin.set_fact:
+    llm_router_qwen_n_ctx: >-
+      {%- set model = llm_router_models.json.data | selectattr('id', 'equalto', llm_router_expected_model_id) | first -%}
+      {%- set args = model.status.args -%}
+      {%- set ctx_idx = args.index('--ctx-size') if '--ctx-size' in args else -1 -%}
+      {{ args[ctx_idx + 1] | int if ctx_idx >= 0 else 0 }}
+  when:
+    - llm_router_enabled | default(false)
+    - llm_router_models is defined
+    - llm_router_models.json.data | selectattr('id', 'equalto', llm_router_expected_model_id) | list | length > 0
+  tags: [router_verify]
+
+- name: "[router_verify] GATE 1b: Fail if n_ctx < 64000 (Hermes 64K context floor)"
+  ansible.builtin.fail:
+    msg: >-
+      GATE 1 FAIL: Router args show --ctx-size={{ llm_router_qwen_n_ctx }} for
+      {{ llm_router_expected_model_id }}. Hermes requires >= 64000 (64K floor).
+      Check --ctx-size in the unit template and verify the model args.
+  when:
+    - llm_router_enabled | default(false)
+    - llm_router_qwen_n_ctx is defined
+    - llm_router_qwen_n_ctx | int < 64000
+  tags: [router_verify]
+
+- name: "[router_verify] GATE 1b: PASS — n_ctx >= 64K confirmed (from router model args)"
+  ansible.builtin.debug:
+    msg: "GATE 1 PASS: Router passes --ctx-size={{ llm_router_qwen_n_ctx }} for {{ llm_router_expected_model_id }} (>= 64000 required)."
+  when:
+    - llm_router_enabled | default(false)
+    - llm_router_qwen_n_ctx is defined
+    - llm_router_qwen_n_ctx | int >= 64000
+  tags: [router_verify]
+
+# --- Gate 2: Tool-calling through router path (hard gate) ---
+# Tests the tool-calling trigger path THROUGH the router proxy, not via the
+# bare llama-server. Router mode is a different process/proxy path — parity
+# with port 8002 is not assumed.
+
+- name: "[router_verify] GATE 2: Tool-calling trigger probe (weather probe — should return tool_calls)"
+  ansible.builtin.uri:
+    url: "http://{{ llm_router_bind_address }}:{{ llm_router_port }}/v1/chat/completions"
+    method: POST
+    body_format: json
+    body:
+      model: "{{ llm_router_expected_model_id }}"
+      messages:
+        - role: user
+          content: "What is the current weather in Chicago? Use the provided tool."
+      tools:
+        - type: function
+          function:
+            name: get_weather
+            description: "Get current weather conditions for a city"
+            parameters:
+              type: object
+              properties:
+                city:
+                  type: string
+                  description: "The city name"
+              required:
+                - city
+      temperature: 0.0
+    status_code: 200
+    return_content: true
+    timeout: 120
+  register: llm_router_toolcall_probe
+  when: llm_router_enabled | default(false)
+  tags: [router_verify]
+
+- name: "[router_verify] GATE 2: Fail if tool-calling probe did not return finish_reason=tool_calls"
+  ansible.builtin.fail:
+    msg: >-
+      GATE 2 FAIL: Tool-calling probe returned finish_reason=
+      {{ llm_router_toolcall_probe.json.choices[0].finish_reason | default('(missing)') }}
+      instead of 'tool_calls'. Router is not correctly proxying tool-call requests.
+      Full response: {{ llm_router_toolcall_probe.json | to_json }}
+  when:
+    - llm_router_enabled | default(false)
+    - llm_router_toolcall_probe is defined
+    - llm_router_toolcall_probe.json.choices[0].finish_reason | default('') != 'tool_calls'
+  tags: [router_verify]
+
+- name: "[router_verify] GATE 2: Validate tool_calls arguments are valid JSON"
+  ansible.builtin.set_fact:
+    llm_router_toolcall_args: >-
+      {{ llm_router_toolcall_probe.json.choices[0].message.tool_calls[0].function.arguments | default('') }}
+  when:
+    - llm_router_enabled | default(false)
+    - llm_router_toolcall_probe is defined
+    - llm_router_toolcall_probe.json.choices[0].finish_reason | default('') == 'tool_calls'
+  tags: [router_verify]
+
+- name: "[router_verify] GATE 2: PASS — tool_calls returned with arguments"
+  ansible.builtin.debug:
+    msg:
+      - "GATE 2 PASS: Router proxied tool-calling correctly."
+      - "finish_reason: {{ llm_router_toolcall_probe.json.choices[0].finish_reason }}"
+      - "function: {{ llm_router_toolcall_probe.json.choices[0].message.tool_calls[0].function.name | default('(unknown)') }}"
+      - "arguments: {{ llm_router_toolcall_args | default('(none)') }}"
+  when:
+    - llm_router_enabled | default(false)
+    - llm_router_toolcall_probe is defined
+    - llm_router_toolcall_probe.json.choices[0].finish_reason | default('') == 'tool_calls'
+  tags: [router_verify]
+
+# --- Gate 2b: Hallucination stress test (should NOT trigger tool_calls) ---
+
+- name: "[router_verify] GATE 2b: Hallucination stress test (no-tool prompt — should return stop)"
+  ansible.builtin.uri:
+    url: "http://{{ llm_router_bind_address }}:{{ llm_router_port }}/v1/chat/completions"
+    method: POST
+    body_format: json
+    body:
+      model: "{{ llm_router_expected_model_id }}"
+      messages:
+        - role: user
+          content: "Tell me a brief fact about the planet Mars. Do not call any functions."
+      tools:
+        - type: function
+          function:
+            name: get_weather
+            description: "Get current weather conditions for a city"
+            parameters:
+              type: object
+              properties:
+                city:
+                  type: string
+              required:
+                - city
+      temperature: 0.1
+    status_code: 200
+    return_content: true
+    timeout: 120
+  register: llm_router_halluc_probe
+  when: llm_router_enabled | default(false)
+  tags: [router_verify]
+
+- name: "[router_verify] GATE 2b: Fail if hallucination stress test triggered spurious tool_calls"
+  ansible.builtin.fail:
+    msg: >-
+      GATE 2b FAIL: Hallucination stress test returned finish_reason=tool_calls
+      (spurious tool call on an unrelated prompt). The model is over-triggering
+      tool-calling through the router proxy. Investigate router mode tool-call
+      framing before proceeding.
+      Full response: {{ llm_router_halluc_probe.json | to_json }}
+  when:
+    - llm_router_enabled | default(false)
+    - llm_router_halluc_probe is defined
+    - llm_router_halluc_probe.json.choices[0].finish_reason | default('') == 'tool_calls'
+  tags: [router_verify]
+
+- name: "[router_verify] GATE 2b: PASS — hallucination stress test returned stop (no spurious tool_calls)"
+  ansible.builtin.debug:
+    msg: "GATE 2b PASS: finish_reason={{ llm_router_halluc_probe.json.choices[0].finish_reason }} — no spurious tool call."
+  when:
+    - llm_router_enabled | default(false)
+    - llm_router_halluc_probe is defined
+    - llm_router_halluc_probe.json.choices[0].finish_reason | default('') != 'tool_calls'
+  tags: [router_verify]
+
+# --- Gate 3: VRAM guard — --models-max 1 confirmed effective ---
+
+- name: "[router_verify] GATE 3: Check VRAM usage after router load (--models-max 1 guard)"
+  ansible.builtin.command:
+    cmd: nvidia-smi --query-gpu=memory.used,memory.total,utilization.gpu --format=csv,noheader
+  register: llm_router_vram_post
+  changed_when: false
+  become: true
+  when: llm_router_enabled | default(false)
+  tags: [router_verify]
+
+- name: "[router_verify] GATE 3: Parse VRAM used (MiB)"
+  ansible.builtin.set_fact:
+    llm_router_vram_used_mib: "{{ llm_router_vram_post.stdout.split(',')[0].strip().split(' ')[0] | int }}"
+  when:
+    - llm_router_enabled | default(false)
+    - llm_router_vram_post is defined
+  tags: [router_verify]
+
+- name: "[router_verify] GATE 3: Fail if VRAM usage exceeds safety ceiling ({{ llm_router_vram_max_mib }} MiB)"
+  ansible.builtin.fail:
+    msg: >-
+      GATE 3 FAIL: nvidia-smi reports {{ llm_router_vram_used_mib }} MiB VRAM used,
+      which exceeds the safety ceiling of {{ llm_router_vram_max_mib }} MiB.
+      --models-max 1 may not be effective, or a second model may be resident.
+      Full nvidia-smi output: {{ llm_router_vram_post.stdout }}
+  when:
+    - llm_router_enabled | default(false)
+    - llm_router_vram_used_mib is defined
+    - llm_router_vram_used_mib | int > llm_router_vram_max_mib | int
+  tags: [router_verify]
+
+- name: "[router_verify] GATE 3: Count GPU processes (should be exactly 1 — the router's Qwen child)"
+  ansible.builtin.command:
+    cmd: nvidia-smi --query-compute-apps=pid,name --format=csv,noheader
+  register: llm_router_gpu_procs
+  changed_when: false
+  failed_when: false
+  become: true
+  when: llm_router_enabled | default(false)
+  tags: [router_verify]
+
+- name: "[router_verify] GATE 3: PASS — VRAM usage and GPU process count"
+  ansible.builtin.debug:
+    msg:
+      - "GATE 3 PASS: VRAM {{ llm_router_vram_used_mib }} MiB / {{ llm_router_vram_max_mib }} MiB ceiling."
+      - "nvidia-smi compute apps: {{ llm_router_gpu_procs.stdout_lines | default(['(none — model not yet loaded under GPU?']) }}"
+      - "Full nvidia-smi: {{ llm_router_vram_post.stdout }}"
+  when:
+    - llm_router_enabled | default(false)
+    - llm_router_vram_used_mib is defined
+    - llm_router_vram_used_mib | int <= llm_router_vram_max_mib | int
+  tags: [router_verify]
+
+- name: "[router_verify] OOM CHECK: Check for OOM events in dmesg (last 120s, router-related)"
+  ansible.builtin.shell:
+    cmd: "dmesg --ctime | tail -200 | grep -i -E 'oom|killed|llama' || true"
+  register: llm_router_oom_check
+  changed_when: false
+  become: true
+  when: llm_router_enabled | default(false)
+  tags: [router_verify]
+
+- name: "[router_verify] OOM CHECK: Report OOM check findings"
+  ansible.builtin.debug:
+    msg: >-
+      OOM/kill events near router start:
+      {{ llm_router_oom_check.stdout if (llm_router_oom_check.stdout | length > 0) else 'None found.' }}
+  when:
+    - llm_router_enabled | default(false)
+    - llm_router_oom_check is defined
+  tags: [router_verify]
+
+# --- POST-VALIDATION: Restart production service -----------------------
+# After validation gates: stop the router (to free VRAM), then restart production.
+# The router stays installed and enabled on port 8003 for Ryan's review period —
+# it will restart automatically on next reboot or systemctl start.
+# When Ryan approves cutover, day2_cutover_qwen_to_router.yml will handle the
+# permanent transition (router on :8002, production unit retired).
+
+- name: "[router_verify] POST-VALIDATION: Stop router to free VRAM for production restart"
+  ansible.builtin.systemd:
+    name: "{{ llm_router_service_name }}"
+    state: stopped
+    # Do NOT disable — keep it installed and enabled for Ryan's review.
+    # Router will need to be manually started again for further testing.
+  become: true
+  when:
+    - llm_router_enabled | default(false)
+    - llm_router_qwen_stopped is defined
+    - llm_router_qwen_stopped.changed | default(false)
+  tags: [router_verify]
+
+- name: "[router_verify] POST-VALIDATION: Wait 5s for router VRAM to be released"
+  ansible.builtin.pause:
+    seconds: 5
+  when:
+    - llm_router_enabled | default(false)
+    - llm_router_qwen_stopped is defined
+    - llm_router_qwen_stopped.changed | default(false)
+  tags: [router_verify]
+
+- name: "[router_verify] POST-VALIDATION: Restart production llama-server-qwen (port 8002)"
+  ansible.builtin.systemd:
+    name: "{{ llm_qwen_service_name }}"
+    state: started
+    enabled: true
+  become: true
+  when:
+    - llm_router_enabled | default(false)
+    - llm_router_qwen_stopped is defined
+    - llm_router_qwen_stopped.changed | default(false)
+  tags: [router_verify]
+
+- name: "[router_verify] POST-VALIDATION: Wait for production /health to confirm restart"
+  ansible.builtin.uri:
+    url: "http://{{ llm_bind_address }}:{{ llm_qwen_port }}/health"
+    status_code: 200
+  register: llm_router_qwen_post_health
+  retries: 30
+  delay: 10
+  until: llm_router_qwen_post_health.status == 200
+  when:
+    - llm_router_enabled | default(false)
+    - llm_router_qwen_stopped is defined
+    - llm_router_qwen_stopped.changed | default(false)
+  tags: [router_verify]
+
+- name: "[router_verify] POST-VALIDATION: Confirm production is back on port 8002"
+  ansible.builtin.debug:
+    msg: >-
+      Production llama-server-qwen restarted on port {{ llm_qwen_port }} and confirmed healthy.
+      7 Hermes profiles (bruce-banner, groot, happy, heimdall, rocket-raccoon, war-machine, wong)
+      are back to full service. Router (port 8003) is stopped but installed; restart with:
+      systemctl start llama-server-router  (or via a follow-up playbook run).
+  when:
+    - llm_router_enabled | default(false)
+    - llm_router_qwen_stopped is defined
+    - llm_router_qwen_stopped.changed | default(false)
+  tags: [router_verify]
+
+# =============================================================================
+# TAG: router_ui_check
+# Nice-to-have: verify the bundled SvelteKit web UI is served.
+# This does NOT fail the playbook on UI error — it is informational only.
+# =============================================================================
+
+- name: "[router_ui_check] GATE 4 (nice-to-have): Check bundled SvelteKit UI returns HTTP 200"
+  ansible.builtin.uri:
+    url: "http://{{ llm_router_bind_address }}:{{ llm_router_port }}/"
+    status_code: [200, 301, 302]
+    return_content: false
+  register: llm_router_ui_check
+  failed_when: false
+  when: llm_router_enabled | default(false)
+  tags: [router_ui_check]
+
+- name: "[router_ui_check] Report UI check result (informational — does not gate cutover)"
+  ansible.builtin.debug:
+    msg: >-
+      GATE 4 (nice-to-have): Bundled UI at http://{{ llm_router_bind_address }}:{{ llm_router_port }}/
+      returned HTTP {{ llm_router_ui_check.status | default('UNREACHABLE') }}.
+      {{ 'PASS — UI accessible.' if (llm_router_ui_check.status | default(0) | int in [200, 301, 302])
+         else 'WARN — UI not accessible. This does NOT block cutover (API traffic only, UI is cosmetic).' }}
+  when:
+    - llm_router_enabled | default(false)
+    - llm_router_ui_check is defined
+  tags: [router_ui_check]
+
+# =============================================================================
+# Summary block — print when all gates pass
+# =============================================================================
+
+- name: "[router_verify] VALIDATION SUMMARY — all hard gates passed"
+  ansible.builtin.debug:
+    msg:
+      - "======================================================================"
+      - "llama-server-router (port {{ llm_router_port }}) shadow deployment PASSED all validation gates."
+      - "Gate 1 (context): n_ctx={{ llm_router_qwen_n_ctx | default('N/A') }} >= 64000 required — PASS"
+      - "Gate 2 (tool-calling through router): finish_reason=tool_calls — PASS"
+      - "Gate 2b (hallucination stress): no spurious tool_calls — PASS"
+      - "Gate 3 (VRAM guard): {{ llm_router_vram_used_mib | default('N/A') }} MiB <= {{ llm_router_vram_max_mib }} MiB — PASS"
+      - "Gate 4 (UI): {{ llm_router_ui_check.status | default('N/A') }} (informational)"
+      - "----------------------------------------------------------------------"
+      - "Production port 8002 (llama-server-qwen) is UNCHANGED."
+      - "Post Ryan's review and sign-off on these results, War Machine will"
+      - "execute day2_cutover_qwen_to_router.yml to promote the router to port 8002."
+      - "======================================================================"
+  when: llm_router_enabled | default(false)
+  tags: [router_verify]

ansible/roles/llm-inference-multimodel/tasks/stage_model.yml

@@ -0,0 +1,67 @@
+---
+# ------------------------------------------------------------------------------
+# FILE: roles/llm-inference-multimodel/tasks/stage_model.yml
+# DESCRIPTION: Idempotent staging of a SINGLE GGUF listed in llm_staged_models.
+#              Looped from tasks/models.yml (one include per entry).
+#
+#              GUARD (idempotency):
+#                - stat the target path in llm_models_dir
+#                - get_url ONLY when the file is MISSING or its on-disk size
+#                  != the exact HF manifest size (staged_model.size_bytes)
+#                - when present AND size matches -> pure no-op (no download,
+#                  no service touch)
+#
+#              SIDE EFFECT ON CHANGE:
+#                When a genuine new/mismatched GGUF is downloaded (or its
+#                ownership/mode corrected), this task notifies the
+#                "restart llama-server-router on new GGUF" handler so the
+#                llama.cpp router re-discovers the models_dir. A normal
+#                re-run that finds the files already correct will NOT fire
+#                the handler — the live router is left untouched.
+#
+#              Driven entirely by inventory vars (host_vars), nothing
+#              hardcoded here, so adding a future model = append to the list.
+# ------------------------------------------------------------------------------
+
+- name: "Check if {{ staged_model.filename }} is present on disk"
+  ansible.builtin.stat:
+    path: "{{ llm_models_dir }}/{{ staged_model.filename }}"
+  register: _staged_stat
+
+- name: "Report {{ staged_model.filename }} presence (source: {{ staged_model.source_repo | default('n/a') }})"
+  ansible.builtin.debug:
+    msg: >-
+      {{ staged_model.filename }}:
+      exists={{ _staged_stat.stat.exists | default(false) }},
+      size={{ (_staged_stat.stat.size | default(0) | int) }},
+      expected={{ staged_model.size_bytes | int }},
+      match={{ (_staged_stat.stat.exists | default(false)) and
+               ((_staged_stat.stat.size | default(0) | int) == (staged_model.size_bytes | int)) }}
+  when: _staged_stat.stat.exists | default(false)
+
+- name: "Download {{ staged_model.filename }} (missing or size mismatch)"
+  ansible.builtin.get_url:
+    url: "{{ staged_model.url }}"
+    dest: "{{ llm_models_dir }}/{{ staged_model.filename }}"
+    owner: "{{ llm_service_user }}"
+    group: "{{ llm_service_user }}"
+    mode: "0664"
+    timeout: 600
+  become: true
+  when: >-
+    not (_staged_stat.stat.exists | default(false))
+    or (_staged_stat.stat.size | default(0) | int != (staged_model.size_bytes | int))
+  register: _staged_download
+  notify: restart llama-server-router on new GGUF
+
+- name: "Ensure ownership/mode on {{ staged_model.filename }}"
+  ansible.builtin.file:
+    path: "{{ llm_models_dir }}/{{ staged_model.filename }}"
+    owner: "{{ llm_service_user }}"
+    group: "{{ llm_service_user }}"
+    mode: "0664"
+  become: true
+  when: >-
+    (_staged_stat.stat.exists | default(false))
+    or (_staged_download is changed | default(false))
+  notify: restart llama-server-router on new GGUF

ansible/roles/llm-inference-multimodel/tasks/systemd.yml

@@ -1,54 +1,63 @@
 ---
 # ------------------------------------------------------------------------------
 # FILE: roles/llm-inference-multimodel/tasks/systemd.yml
-# DESCRIPTION: Phase 2 — template + deploy both unit files.
-#              DELIBERATELY DOES NOT START OR ENABLE either service — that is
-#              Phase 4 (verify.yml)'s job, after Phase 3 firewall scoping is
-#              in place. This keeps "units land on disk" and "processes
-#              actually bind ports and load 20+GB into VRAM" as separately
-#              reviewable checkpoints per Ryan's iterative-build preference.
+# DESCRIPTION: Phase 2 (REVISED 2026-08-06) — consolidated to a single
+#              production unit: llama-server-qwen (Qwen2.5-14B-Instruct-1M,
+#              port 8002), serving BOTH the friday and war-machine Hermes
+#              profiles. Ryan explicitly accepted the tradeoffs of running
+#              one model for both profiles instead of the original
+#              aux+toolcall two-model split.
 #
-#              Two independent units (llama-server-aux.service,
-#              llama-server-toolcall.service) — NOT one unit with two
-#              ExecStarts — so either can be stopped/restarted without
-#              affecting the other (plan §2, §6 rollback requirement).
+#              llama-server-aux (Phi-4, port 8000) and llama-server-toolcall
+#              (Mistral-Small-24B, port 8001) are RETIRED: services stopped
+#              + disabled, unit files removed from astro-orbiter, and their
+#              GGUF weights deleted from /opt/models (~45GB reclaimed).
+#              Ansible no longer templates or manages either unit — see git
+#              log for the prior task definitions if a future rollback needs
+#              them restored.
 #
-#              The pre-existing Gemma unit (whatever discover.yml found it to
-#              be) is never templated, restarted, or disabled by this file.
+#              The pre-existing Gemma unit/weights (llama-server.service,
+#              gemma-2-27b-it-Q4_K_M.gguf) were ALSO removed as part of this
+#              consolidation (superseded baseline, no longer a rollback
+#              target once Qwen was accepted as sole production model).
 # ------------------------------------------------------------------------------
 
-- name: Deploy llama-server-aux systemd unit
+- name: Deploy llama-server-qwen systemd unit (production, port 8002)
   ansible.builtin.template:
-    src: llama-server-aux.service.j2
-    dest: "/etc/systemd/system/{{ llm_aux_service_name }}.service"
+    src: llama-server-qwen.service.j2
+    dest: "/etc/systemd/system/{{ llm_qwen_service_name }}.service"
     owner: root
     group: root
     mode: "0644"
   become: true
+  register: llm_qwen_unit_deployed
   notify:
     - reload systemd
-    - restart llama-server-aux
-
-- name: Deploy llama-server-toolcall systemd unit
-  ansible.builtin.template:
-    src: llama-server-toolcall.service.j2
-    dest: "/etc/systemd/system/{{ llm_toolcall_service_name }}.service"
-    owner: root
-    group: root
-    mode: "0644"
-  become: true
-  notify:
-    - reload systemd
-    - restart llama-server-toolcall
 
 - name: Flush handlers so daemon-reload lands before any later phase acts on unit state
   ansible.builtin.meta: flush_handlers
 
-# NOTE: no `ansible.builtin.systemd: state: started / enabled: true` task here
-# on purpose. Units exist on disk after this phase; nothing is running.
-# The "restart" handlers above only fire (and thus only start anything) if
-# the template content actually changed AND a later flush_handlers/end-of-play
-# triggers them — on a first-ever apply this DOES start the services once,
-# which is expected/acceptable for a fresh deploy, but on any subsequent
-# re-run with no template changes, nothing restarts. Ryan/verify.yml owns
-# the deliberate first start + smoke test.
+# NOTE: no `ansible.builtin.systemd: state: started / enabled: true / restarted`
+# task here on purpose. Units exist on disk after this phase; nothing is
+# running or restarted.
+#
+# BUGFIX (found in production): this file used to `notify: restart
+# llama-server-*` on the template tasks above, followed by the
+# flush_handlers meta task. That combination meant Phase 2 (the `systemd`
+# tag) fired the restart handlers itself — on any run where either unit's
+# rendered content changed (including the very first apply), BOTH services
+# got restarted immediately, right here in Phase 2, before Phase 3's
+# firewall scoping or Phase 4's smoke tests ever ran. That directly
+# contradicted this file's own stated purpose (units land on disk, nothing
+# starts/restarts until Phase 4) and caused live services to bounce
+# unexpectedly on a routine re-run of just `--tags systemd`.
+#
+# Fix: Phase 2 only reloads the systemd daemon (harmless, no process
+# impact) and records whether each unit's content actually changed via
+# `llm_aux_unit_deployed` / `llm_toolcall_unit_deployed` (both `.changed`
+# booleans, persisted as play vars for later phases in this same run).
+# Phase 4 (verify.yml) is the only phase that starts OR restarts either
+# service, and it does so per-instance using those recorded `changed`
+# flags — so a content change to one unit's template still never causes
+# the other to restart, and no restart happens at all until Phase 4 has
+# been reached.

ansible/roles/llm-inference-multimodel/tasks/verify.yml

@@ -1,123 +1,115 @@
 ---
 # ------------------------------------------------------------------------------
 # FILE: roles/llm-inference-multimodel/tasks/verify.yml
-# DESCRIPTION: Phase 4 — start both services, curl smoke test each endpoint,
-#              nvidia-smi VRAM check against plan §1 math, confirm no OOM.
+# DESCRIPTION: Phase 4 (REVISED 2026-08-06) — consolidated deployment.
+#              Only llama-server-qwen (Qwen2.5-14B-Instruct-1M, port 8002) is
+#              started/enabled here now. The prior aux (Phi-4, port 8000) and
+#              toolcall (Mistral-Small-24B, port 8001) start/smoke-test tasks
+#              were removed along with those services — see git log for the
+#              previous version of this file if a rollback needs them.
 #
-#              This is the ONLY phase that actually starts the services
+#              This is the ONLY phase that actually starts the qwen service
 #              (systemd.yml deliberately does not). Enabling happens here too,
-#              so a reboot brings both back — matching plan §2's "independent
-#              systemd services" intent for durability, not just this-session.
+#              so a reboot brings it back.
 # ------------------------------------------------------------------------------
 
-- name: Enable and start llama-server-aux
+- name: Gather service facts (systemd unit inventory) — ensure available even if discover.yml's tag wasn't selected
+  ansible.builtin.service_facts:
+  when: llm_existing_gemma_unit_found is not defined
+
+- name: Determine whether a systemd unit matching the existing Gemma service exists (if not already known from discover.yml)
+  ansible.builtin.set_fact:
+    llm_existing_gemma_unit_found: "{{ (llm_existing_gemma_service_name_guess + '.service') in ansible_facts.services }}"
+  when: llm_existing_gemma_unit_found is not defined
+
+- name: Stop pre-existing Gemma llama-server before starting new instances (avoid double VRAM usage / OOM)
   ansible.builtin.systemd:
-    name: "{{ llm_aux_service_name }}"
-    state: started
+    name: "{{ llm_existing_gemma_service_name_guess }}"
+    state: stopped
+  become: true
+  when:
+    - llm_existing_gemma_unit_found | default(false)
+    - ansible_facts.services[llm_existing_gemma_service_name_guess + '.service'].status | default('not-found') != 'not-found'
+    - ansible_facts.services[llm_existing_gemma_service_name_guess + '.service'].state | default('inactive') != 'inactive'
+
+- name: Enable llama-server-qwen and start/restart based on Phase 2 unit-content change
+  ansible.builtin.systemd:
+    name: "{{ llm_qwen_service_name }}"
+    state: "{{ 'restarted' if (llm_qwen_unit_deployed.changed | default(false)) else 'started' }}"
     enabled: true
     daemon_reload: true
   become: true
+  when: llm_qwen_service_enabled | default(false)
 
-- name: Enable and start llama-server-toolcall
-  ansible.builtin.systemd:
-    name: "{{ llm_toolcall_service_name }}"
-    state: started
-    enabled: true
-    daemon_reload: true
-  become: true
-
-- name: Wait for aux instance API to become available (model load may take a couple minutes)
+- name: Wait for Qwen instance API to become available
   ansible.builtin.uri:
-    url: "http://{{ llm_bind_address }}:{{ llm_aux_port }}/health"
+    url: "http://{{ llm_bind_address }}:{{ llm_qwen_port }}/health"
     status_code: 200
-  register: llm_aux_health
+  register: llm_qwen_health
   retries: 24
   delay: 10
-  until: llm_aux_health.status == 200
+  until: llm_qwen_health.status == 200
+  when: llm_qwen_service_enabled | default(false)
+  check_mode: false    # URI tasks return incomplete results in check mode; run for real
 
-- name: Wait for tool-calling instance API to become available
+- name: Smoke-test — Qwen instance model listing + n_ctx verification
   ansible.builtin.uri:
-    url: "http://{{ llm_bind_address }}:{{ llm_toolcall_port }}/health"
-    status_code: 200
-  register: llm_toolcall_health
-  retries: 24
-  delay: 10
-  until: llm_toolcall_health.status == 200
-
-- name: Smoke-test — aux instance model listing
-  ansible.builtin.uri:
-    url: "http://{{ llm_bind_address }}:{{ llm_aux_port }}/v1/models"
+    url: "http://{{ llm_bind_address }}:{{ llm_qwen_port }}/v1/models"
     status_code: 200
     return_content: true
-  register: llm_aux_models
+  register: llm_qwen_models
+  when: llm_qwen_service_enabled | default(false)
+  check_mode: false    # URI tasks return incomplete results in check mode; run for real
 
-- name: Smoke-test — tool-calling instance model listing
-  ansible.builtin.uri:
-    url: "http://{{ llm_bind_address }}:{{ llm_toolcall_port }}/v1/models"
-    status_code: 200
-    return_content: true
-  register: llm_toolcall_models
-
-- name: Report served models per instance
+- name: Report Qwen instance served model + verified n_ctx
   ansible.builtin.debug:
     msg:
-      - "Aux (:{{ llm_aux_port }}) serving:        {{ llm_aux_models.json.data | map(attribute='id') | list }}"
-      - "Tool-calling (:{{ llm_toolcall_port }}) serving: {{ llm_toolcall_models.json.data | map(attribute='id') | list }}"
+      - "Qwen (:{{ llm_qwen_port }}) serving: {{ llm_qwen_models.json.data | map(attribute='id') | list }}"
+      - "Verified n_ctx (must be >= 64000, not just requested): {{ llm_qwen_models.json.data | map(attribute='meta') | map(attribute='n_ctx') | list }}"
+  when:
+    - llm_qwen_service_enabled | default(false)
+    - llm_qwen_models is defined
+    - llm_qwen_models.json is defined
 
-- name: Basic completion smoke test — aux instance (non-tool-calling sanity check only)
+- name: Basic tool-calling smoke test — Qwen instance (this is the sole production model for both profiles)
   ansible.builtin.uri:
-    url: "http://{{ llm_bind_address }}:{{ llm_aux_port }}/v1/chat/completions"
+    url: "http://{{ llm_bind_address }}:{{ llm_qwen_port }}/v1/chat/completions"
     method: POST
     body_format: json
     body:
-      model: "{{ llm_aux_model_id }}"
+      model: "{{ llm_qwen_model_id }}"
       messages:
         - role: user
-          content: "Reply with exactly one word: OK"
-      max_tokens: 10
+          content: "What is the weather in Chicago?"
+      tools:
+        - type: function
+          function:
+            name: get_weather
+            description: Get weather for a city
+            parameters:
+              type: object
+              properties:
+                city:
+                  type: string
+              required:
+                - city
     status_code: 200
     return_content: true
-  register: llm_aux_completion
+  register: llm_qwen_toolcall_smoke
+  when: llm_qwen_service_enabled | default(false)
+  check_mode: false    # URI tasks return incomplete results in check mode; run for real
 
-- name: Basic completion smoke test — tool-calling instance (plain-text sanity check only)
-  ansible.builtin.uri:
-    url: "http://{{ llm_bind_address }}:{{ llm_toolcall_port }}/v1/chat/completions"
-    method: POST
-    body_format: json
-    body:
-      model: "{{ llm_toolcall_model_id }}"
-      messages:
-        - role: user
-          content: "Reply with exactly one word: OK"
-      max_tokens: 10
-    status_code: 200
-    return_content: true
-  register: llm_toolcall_completion
-
-- name: NOTE — this smoke test is NOT the tool-calling validation harness
-  ansible.builtin.debug:
-    msg: >-
-      Both endpoints respond to basic completions. This does NOT validate
-      tool_calls correctness or hallucination-safety for the tool-calling
-      instance — that is a separate, manual, post-deploy procedure (plan §7).
-      See references/tool-calling-validation.sh (copied from the
-      llm-inference-homelab skill) and docs/validation-log.md in this role.
-      DO NOT point any Claude Code / tool-calling-capable Hermes profile at
-      port {{ llm_toolcall_port }} until that validation has passed and been
-      logged.
-
-- name: Check GPU VRAM usage after both instances are running
+- name: Check GPU VRAM usage after Qwen instance is running
   ansible.builtin.command:
     cmd: nvidia-smi --query-gpu=memory.used,memory.total,utilization.gpu --format=csv,noheader
   register: llm_post_start_vram
   changed_when: false
 
-- name: Report VRAM usage vs plan §1 expectations
+- name: Report VRAM usage
   ansible.builtin.debug:
     msg:
       - "Measured (nvidia-smi):      {{ llm_post_start_vram.stdout }}"
-      - "Design estimate (plan §1):  aux ~{{ llm_aux_expected_vram_gb }}GB + toolcall ~{{ llm_toolcall_expected_vram_gb }}GB = ~{{ llm_combined_expected_vram_gb }}GB / {{ llm_gpu_total_vram_gb }}GB total"
-      - "If measured usage exceeds ~23.5GB or is within ~0.5GB of the 24GB card limit, treat as the OOM-risk trigger condition from plan §6 — do not leave both services running unattended without confirming headroom."
+      - "Qwen2.5-14B-Instruct-1M expected footprint: ~{{ llm_qwen_expected_vram_gb }}GB. Ports 8000/8001 are retired and no longer consume VRAM."
 
 - name: Check for OOM-kill events related to llama-server in dmesg (best-effort, read-only)
   ansible.builtin.shell:

ansible/roles/llm-inference-multimodel/templates/llama-server-aux.service.j2

@@ -1,33 +0,0 @@
-[Unit]
-Description=llama-server (aux/classification) — Phi-4-14B Q4_K_M (OpenAI-compatible inference)
-After=network.target nvidia-persistenced.service
-Wants=nvidia-persistenced.service
-
-[Service]
-Type=simple
-User={{ llm_service_user }}
-Group={{ llm_service_user }}
-Environment="HOME=/home/{{ llm_service_user }}"
-ExecStart={{ llm_binary_path }} \
-    --model {{ llm_aux_model_path }} \
-    --host {{ llm_bind_address }} \
-    --port {{ llm_aux_port }} \
-    --ctx-size {{ llm_aux_ctx_size }} \
-    --n-gpu-layers {{ llm_aux_gpu_layers }} \
-    --parallel {{ llm_aux_parallel }} \
-    --metrics
-# NOTE: no --chat-template flag — let llama-server auto-detect Phi-4's own
-# embedded chat template from GGUF metadata (same reasoning as the existing
-# llm-inference role's Gemma unit: explicit overrides risk mismatching the
-# model's actual expected format).
-# NOTE: --host is the private LAN IP (10.1.71.130 by default), NOT 0.0.0.0 —
-# deliberate change from the pre-existing Gemma pattern (plan §5).
-Restart=on-failure
-RestartSec=10
-TimeoutStartSec=600
-StandardOutput=journal
-StandardError=journal
-SyslogIdentifier=llama-server-aux
-
-[Install]
-WantedBy=multi-user.target

ansible/roles/llm-inference-multimodel/templates/llama-server-qwen.service.j2

@@ -0,0 +1,44 @@
+[Unit]
+Description=llama-server — Qwen3.6-35B-A3B-UD-Q4_K_S (OpenAI-compatible inference, 64K ctx)
+Documentation=https://github.com/ggml-org/llama.cpp
+After=network.target nvidia-persistenced.service
+Wants=nvidia-persistenced.service
+
+[Service]
+Type=simple
+User={{ llm_service_user }}
+Group={{ llm_service_user }}
+Environment="HOME=/home/{{ llm_service_user }}"
+ExecStart={{ llm_binary_path }} \
+    --model {{ llm_qwen_model_path }} \
+    --host {{ llm_bind_address }} \
+    --port {{ llm_qwen_port }} \
+    --n-gpu-layers {{ llm_qwen_gpu_layers }} \
+    --ctx-size {{ llm_qwen_ctx_size }} \
+    --flash-attn on \
+    --cache-type-k q4_0 --cache-type-v q4_0 \
+    --batch-size {{ llm_qwen_batch_size }} --ubatch-size {{ llm_qwen_ubatch_size }} \
+    --parallel {{ llm_qwen_parallel }} \
+    --metrics
+
+# PRODUCTION UNIT — Qwen3.6-35B-A3B-UD-Q4_K_S
+# Current as of 2026-08-07 (t_2ffc0f63) — superseded Qwen2.5-14B-Instruct-1M.
+# VRAM: ~20,390 MiB / 24,576 MiB (verified 2026-08-07).
+# Context: 65536 (64K) with q4_0 KV cache to fit 64K in 24GB headroom.
+# DO NOT change --cache-type-k/v — q8_0 requires more VRAM; 24GB is tight.
+# DO NOT add --jinja — Qwen3.6's embedded chat template is correct for
+#   both chat and tool-calling without an override.
+#
+# Shadow validation (router mode, port 8003) — see templates/llama-server-router.service.j2
+# and playbooks/day1_deploy_llm_router_shadow.yml (t_0cca74a2).
+# This unit is the ROLLBACK TARGET — preserved on 8002 until router validation
+# passes and Ryan approves cutover.
+Restart=on-failure
+RestartSec=10
+TimeoutStartSec=600
+StandardOutput=journal
+StandardError=journal
+SyslogIdentifier=llama-server-qwen
+
+[Install]
+WantedBy=multi-user.target

ansible/roles/llm-inference-multimodel/templates/llama-server-router.service.j2

@@ -0,0 +1,57 @@
+[Unit]
+Description=llama-server router — {{ llm_router_models_dir }} (OpenAI-compatible, port {{ llm_router_port }})
+Documentation=https://github.com/ggml-org/llama.cpp
+After=network.target nvidia-persistenced.service
+Wants=nvidia-persistenced.service
+
+[Service]
+Type=simple
+User={{ llm_service_user }}
+Group={{ llm_service_user }}
+Environment="HOME=/home/{{ llm_service_user }}"
+ExecStart={{ llm_binary_path }} \
+    --models-dir {{ llm_router_models_dir }} \
+    --models-max {{ llm_router_models_max }} \
+    --host {{ llm_router_bind_address }} \
+    --port {{ llm_router_port }} \
+    --n-gpu-layers {{ llm_router_gpu_layers }} \
+    --ctx-size {{ llm_router_ctx_size }} \
+    --flash-attn {{ llm_router_flash_attn }} \
+    --cache-type-k {{ llm_router_cache_type_k }} \
+    --cache-type-v {{ llm_router_cache_type_v }} \
+    --batch-size {{ llm_router_batch_size }} \
+    --ubatch-size {{ llm_router_ubatch_size }} \
+    --parallel {{ llm_router_parallel }} \
+    --metrics
+
+# ROUTER MODE NOTES (2026-08-12, t_0cca74a2):
+# - NO -m/--model flag: this is what enables llama-server router/supervisor mode.
+#   Without -m, llama-server discovers all .gguf files in --models-dir, spawning
+#   each as its own child process on demand (LRU-eviction when over models-max).
+# - --models-max {{ llm_router_models_max }} is HARDCODED TO 1.
+#   Default cap is 4 simultaneous — OOM on 24GB with a 20GB model.
+#   Do not increase without a VRAM budget review (see defaults/main.yml comment).
+# - --models-dir /opt/models: auto-discovers all .gguf files. Keep that directory
+#   clean (Qwen-only) to avoid spurious extra entries in /v1/models.
+# - Clients select a model via "model": "<gguf-basename-without-.gguf>" in their
+#   chat completion request. Hermes sends model: "<id>" on every request already.
+# - Cold model load on first request: ~30-60s for Qwen3.6-35B. First response
+#   will be slow. This is expected. Document in runbook.
+# - No --jinja flag: Qwen3.6-35B uses its own embedded chat template correctly.
+#   If per-model template overrides are ever needed, use --models-preset INI
+#   (but note GH #23460: sampler params in presets may not work in router mode).
+#
+# SHADOW DEPLOYMENT NOTE (historical — 2026-08-12, t_0cca74a2):
+# This unit was originally deployed on port 8003 as a shadow. After validation,
+# it was promoted to production on port 8002 (t_cd0d5388). The --port value
+# above is the authoritative value; the port 8003 references below are historical.
+# Production is now llama-server-router (this unit); llama-server-qwen is the rollback target.
+Restart=on-failure
+RestartSec=10
+TimeoutStartSec=600
+StandardOutput=journal
+StandardError=journal
+SyslogIdentifier=llama-server-router
+
+[Install]
+WantedBy=multi-user.target

ansible/roles/llm-inference-multimodel/templates/llama-server-toolcall.service.j2

@@ -1,38 +0,0 @@
-[Unit]
-Description=llama-server (tool-calling) — Mistral-Small-24B-Instruct-2501 Q3_K_M (OpenAI-compatible inference)
-After=network.target nvidia-persistenced.service
-Wants=nvidia-persistenced.service
-
-[Service]
-Type=simple
-User={{ llm_service_user }}
-Group={{ llm_service_user }}
-Environment="HOME=/home/{{ llm_service_user }}"
-ExecStart={{ llm_binary_path }} \
-    --model {{ llm_toolcall_model_path }} \
-    --host {{ llm_bind_address }} \
-    --port {{ llm_toolcall_port }} \
-    --ctx-size {{ llm_toolcall_ctx_size }} \
-    --n-gpu-layers {{ llm_toolcall_gpu_layers }} \
-    --parallel {{ llm_toolcall_parallel }} \
-    --metrics
-# NOTE: no --chat-template flag — let llama-server auto-detect Mistral-Small's
-# own embedded chat template from GGUF metadata.
-# NOTE: --host is the private LAN IP (10.1.71.130 by default), NOT 0.0.0.0.
-# NOTE: --parallel 1 is deliberate (plan §1/§2) — tool-calling profiles are
-# single-session-at-a-time per Claude Code profile; lower parallelism reduces
-# KV overhead and lowers hallucination surface from context bleed between
-# concurrent slots.
-# IMPORTANT: this endpoint MUST pass the plan §7 validation harness
-# (docs/validation-log.md in this role) before any Claude Code / tool-calling
-# Hermes profile is pointed at it. Mistral-Small shares lineage concerns
-# flagged for Qwen2.5/Qwen3 hallucinated tool_calls — do not assume safety.
-Restart=on-failure
-RestartSec=10
-TimeoutStartSec=600
-StandardOutput=journal
-StandardError=journal
-SyslogIdentifier=llama-server-toolcall
-
-[Install]
-WantedBy=multi-user.target