fix: correct astro-orbiter llama-server scrape target for router cutover

Port 8000 (gemma-2-27b-it-GGUF) is dead after the day2 router cutover on
2026-08-12. Production inference now runs through llama-server-router on
port 8002.

The router exposes per-model Prometheus metrics via /metrics?model=<id>.
Since a single /metrics request without ?model returns HTTP 400, replaced
the single stale job with three per-model jobs — one per model registered
in the router per /v1/models:
  - Qwen3.6-35B-A3B-UD-Q4_K_S  (currently unloaded but registered)
  - Meta-Llama-3.1-8B-Instruct-Q4_K_M  (loaded)
  - Phi-3.5-mini-instruct-Q8_0  (loaded)

Static 'model' label carries the canonical llama.cpp model id (not alias).
Added 'endpoint: astro-orbiter-router' to identify the scrape origin.
Removed dead :8000 target entirely.
This commit is contained in:
Hermes Agent service account
2026-08-13 09:08:01 -05:00
parent 7aea88724f
commit a87da82ebd

View File

@@ -232,15 +232,50 @@ prometheus:
hostname: astro-orbiter
gpu: rtx3090
- job_name: llama-server-astro-orbiter
# Router exposes per-model Prometheus metrics via /metrics?model=<id>.
# The old :8000 target (gemma-2-27b-it-GGUF) was removed after the day2
# cutover (2026-08-12) to llama-server-router on :8002. Three separate
# jobs — one per model registered in the router — are used so each gets
# its own model label. The static `model` label is set to the canonical
# llama.cpp model id (the `id` field from /v1/models, not the alias).
- job_name: llama-server-astro-orbiter-qwen3
scrape_interval: 15s
metrics_path: /metrics
params:
model: ["Qwen3.6-35B-A3B-UD-Q4_K_S"]
static_configs:
- targets:
- 10.1.71.130:8000
- 10.1.71.130:8002
labels:
hostname: astro-orbiter
model: bartowski/gemma-2-27b-it-GGUF
endpoint: astro-orbiter-router
model: Qwen3.6-35B-A3B-UD-Q4_K_S
- job_name: llama-server-astro-orbiter-llama3
scrape_interval: 15s
metrics_path: /metrics
params:
model: ["Meta-Llama-3.1-8B-Instruct-Q4_K_M"]
static_configs:
- targets:
- 10.1.71.130:8002
labels:
hostname: astro-orbiter
endpoint: astro-orbiter-router
model: Meta-Llama-3.1-8B-Instruct-Q4_K_M
- job_name: llama-server-astro-orbiter-phi35
scrape_interval: 15s
metrics_path: /metrics
params:
model: ["Phi-3.5-mini-instruct-Q8_0"]
static_configs:
- targets:
- 10.1.71.130:8002
labels:
hostname: astro-orbiter
endpoint: astro-orbiter-router
model: Phi-3.5-mini-instruct-Q8_0
# ─── Grafana ──────────────────────────────────────────────────────────────────
grafana: