39 lines
1.5 KiB
YAML
39 lines
1.5 KiB
YAML
---
|
|
# ------------------------------------------------------------------------------
|
|
# Playbook: day2_qwen38_ctx128k.yml
|
|
# Purpose: Bump Qwen3.8-27B-Q4_K_M ctx-size from 32768 to 131072 (128K)
|
|
# on astro-orbiter's production router (port 8002).
|
|
#
|
|
# What this playbook does:
|
|
# 1. Renders the updated llama-server-router-preset.ini.j2 (now with
|
|
# llm_router_qwen38_ctx_size: 131072) to /opt/llama-server-router-preset.ini.
|
|
# 2. Restarts llama-server-router.service.
|
|
# 3. Verifies the router loads Qwen3.8-27B at ctx=131072 in status.args.
|
|
#
|
|
# Context:
|
|
# - Empirical VRAM test (t_4455a44c): 131072 ctx = 20,282 MiB Qwen3.8
|
|
# + 558 MiB nomic-embed = ~20.8GB total; ~3.2GB headroom on 24GB RTX 3090.
|
|
# Co-resident with nomic-embed: comfortably fits.
|
|
# - Ryan approved this deployment.
|
|
# - Semaphore SSH gap for astro-orbiter still applies (t_730f9584 / t_33acbb2e);
|
|
# running direct CLI Ansible per standing exception.
|
|
#
|
|
# Run:
|
|
# cd /home/hermes/git/homelab/ansible
|
|
# env -u ANSIBLE_VAULT_PASSWORD_FILE ansible-playbook \
|
|
# -i inventory.yml \
|
|
# playbooks/day2_qwen38_ctx128k.yml
|
|
#
|
|
# Task reference: t_441470b9 — War Machine, 2026-08-16
|
|
# ------------------------------------------------------------------------------
|
|
- name: Bump Qwen3.8-27B ctx-size to 131072 on astro-orbiter
|
|
hosts: astro-orbiter
|
|
become: true
|
|
vars:
|
|
llm_router_preset_enabled: true
|
|
llm_router_qwen38_ctx_size: 131072
|
|
|
|
roles:
|
|
- role: llm-inference-multimodel
|
|
tags: [preset, systemd, verify]
|