rblundon/homelab
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

llm-inference-multimodel: role + day1 playbook (phase 0 discover approved)

Browse Source
This commit is contained in:
Hermes Agent service account
2026-08-05 15:53:31 -05:00
parent 782cbe33d1
commit c3755aa29e
15 changed files with 850 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
ansible/roles/llm-inference-multimodel/meta/main.yml Normal file
View File

@@ -0,0 +1,17 @@
---
# ------------------------------------------------------------------------------
# FILE: roles/llm-inference-multimodel/meta/main.yml
# ------------------------------------------------------------------------------
galaxy_info:
role_name: llm_inference_multimodel
author: rblundon
license: MIT
description: >
Deploys two independent llama-server instances on astro-orbiter's RTX 3090:
an aux/classification instance (Phi-4-14B Q4_K_M, port 8000) and a
tool-calling instance (Mistral-Small-24B-Instruct-2501 Q3_K_M, port 8001).
Built alongside roles/llm-inference (not a replacement); assumes that
role's CUDA build/driver work is already done. See
/home/hermes/astro-orbiter-multi-model-plan.md for the full design.
min_ansible_version: "2.15"
dependencies: []