18 lines
799 B
YAML
18 lines
799 B
YAML
---
|
|
# ------------------------------------------------------------------------------
|
|
# FILE: roles/llm-inference-multimodel/meta/main.yml
|
|
# ------------------------------------------------------------------------------
|
|
galaxy_info:
|
|
role_name: llm_inference_multimodel
|
|
author: rblundon
|
|
license: MIT
|
|
description: >
|
|
Deploys two independent llama-server instances on astro-orbiter's RTX 3090:
|
|
an aux/classification instance (Phi-4-14B Q4_K_M, port 8000) and a
|
|
tool-calling instance (Mistral-Small-24B-Instruct-2501 Q3_K_M, port 8001).
|
|
Built alongside roles/llm-inference (not a replacement); assumes that
|
|
role's CUDA build/driver work is already done. See
|
|
/home/hermes/astro-orbiter-multi-model-plan.md for the full design.
|
|
min_ansible_version: "2.15"
|
|
dependencies: []
|