fix: correct astro-orbiter llama-server scrape target for router cutover
Port 8000 (gemma-2-27b-it-GGUF) is dead after the day2 router cutover on 2026-08-12. Production inference now runs through llama-server-router on port 8002. The router exposes per-model Prometheus metrics via /metrics?model=<id>. Since a single /metrics request without ?model returns HTTP 400, replaced the single stale job with three per-model jobs — one per model registered in the router per /v1/models: - Qwen3.6-35B-A3B-UD-Q4_K_S (currently unloaded but registered) - Meta-Llama-3.1-8B-Instruct-Q4_K_M (loaded) - Phi-3.5-mini-instruct-Q8_0 (loaded) Static 'model' label carries the canonical llama.cpp model id (not alias). Added 'endpoint: astro-orbiter-router' to identify the scrape origin. Removed dead :8000 target entirely.
This commit is contained in:
@@ -232,15 +232,50 @@ prometheus:
|
|||||||
hostname: astro-orbiter
|
hostname: astro-orbiter
|
||||||
gpu: rtx3090
|
gpu: rtx3090
|
||||||
|
|
||||||
- job_name: llama-server-astro-orbiter
|
# Router exposes per-model Prometheus metrics via /metrics?model=<id>.
|
||||||
|
# The old :8000 target (gemma-2-27b-it-GGUF) was removed after the day2
|
||||||
|
# cutover (2026-08-12) to llama-server-router on :8002. Three separate
|
||||||
|
# jobs — one per model registered in the router — are used so each gets
|
||||||
|
# its own model label. The static `model` label is set to the canonical
|
||||||
|
# llama.cpp model id (the `id` field from /v1/models, not the alias).
|
||||||
|
- job_name: llama-server-astro-orbiter-qwen3
|
||||||
scrape_interval: 15s
|
scrape_interval: 15s
|
||||||
metrics_path: /metrics
|
metrics_path: /metrics
|
||||||
|
params:
|
||||||
|
model: ["Qwen3.6-35B-A3B-UD-Q4_K_S"]
|
||||||
static_configs:
|
static_configs:
|
||||||
- targets:
|
- targets:
|
||||||
- 10.1.71.130:8000
|
- 10.1.71.130:8002
|
||||||
labels:
|
labels:
|
||||||
hostname: astro-orbiter
|
hostname: astro-orbiter
|
||||||
model: bartowski/gemma-2-27b-it-GGUF
|
endpoint: astro-orbiter-router
|
||||||
|
model: Qwen3.6-35B-A3B-UD-Q4_K_S
|
||||||
|
|
||||||
|
- job_name: llama-server-astro-orbiter-llama3
|
||||||
|
scrape_interval: 15s
|
||||||
|
metrics_path: /metrics
|
||||||
|
params:
|
||||||
|
model: ["Meta-Llama-3.1-8B-Instruct-Q4_K_M"]
|
||||||
|
static_configs:
|
||||||
|
- targets:
|
||||||
|
- 10.1.71.130:8002
|
||||||
|
labels:
|
||||||
|
hostname: astro-orbiter
|
||||||
|
endpoint: astro-orbiter-router
|
||||||
|
model: Meta-Llama-3.1-8B-Instruct-Q4_K_M
|
||||||
|
|
||||||
|
- job_name: llama-server-astro-orbiter-phi35
|
||||||
|
scrape_interval: 15s
|
||||||
|
metrics_path: /metrics
|
||||||
|
params:
|
||||||
|
model: ["Phi-3.5-mini-instruct-Q8_0"]
|
||||||
|
static_configs:
|
||||||
|
- targets:
|
||||||
|
- 10.1.71.130:8002
|
||||||
|
labels:
|
||||||
|
hostname: astro-orbiter
|
||||||
|
endpoint: astro-orbiter-router
|
||||||
|
model: Phi-3.5-mini-instruct-Q8_0
|
||||||
|
|
||||||
# ─── Grafana ──────────────────────────────────────────────────────────────────
|
# ─── Grafana ──────────────────────────────────────────────────────────────────
|
||||||
grafana:
|
grafana:
|
||||||
|
|||||||
Reference in New Issue
Block a user