fix: correct astro-orbiter llama-server scrape target for router cutover
Port 8000 (gemma-2-27b-it-GGUF) is dead after the day2 router cutover on 2026-08-12. Production inference now runs through llama-server-router on port 8002. The router exposes per-model Prometheus metrics via /metrics?model=<id>. Since a single /metrics request without ?model returns HTTP 400, replaced the single stale job with three per-model jobs — one per model registered in the router per /v1/models: - Qwen3.6-35B-A3B-UD-Q4_K_S (currently unloaded but registered) - Meta-Llama-3.1-8B-Instruct-Q4_K_M (loaded) - Phi-3.5-mini-instruct-Q8_0 (loaded) Static 'model' label carries the canonical llama.cpp model id (not alias). Added 'endpoint: astro-orbiter-router' to identify the scrape origin. Removed dead :8000 target entirely.
This commit is contained in:
@@ -232,15 +232,50 @@ prometheus:
|
||||
hostname: astro-orbiter
|
||||
gpu: rtx3090
|
||||
|
||||
- job_name: llama-server-astro-orbiter
|
||||
# Router exposes per-model Prometheus metrics via /metrics?model=<id>.
|
||||
# The old :8000 target (gemma-2-27b-it-GGUF) was removed after the day2
|
||||
# cutover (2026-08-12) to llama-server-router on :8002. Three separate
|
||||
# jobs — one per model registered in the router — are used so each gets
|
||||
# its own model label. The static `model` label is set to the canonical
|
||||
# llama.cpp model id (the `id` field from /v1/models, not the alias).
|
||||
- job_name: llama-server-astro-orbiter-qwen3
|
||||
scrape_interval: 15s
|
||||
metrics_path: /metrics
|
||||
params:
|
||||
model: ["Qwen3.6-35B-A3B-UD-Q4_K_S"]
|
||||
static_configs:
|
||||
- targets:
|
||||
- 10.1.71.130:8000
|
||||
- 10.1.71.130:8002
|
||||
labels:
|
||||
hostname: astro-orbiter
|
||||
model: bartowski/gemma-2-27b-it-GGUF
|
||||
endpoint: astro-orbiter-router
|
||||
model: Qwen3.6-35B-A3B-UD-Q4_K_S
|
||||
|
||||
- job_name: llama-server-astro-orbiter-llama3
|
||||
scrape_interval: 15s
|
||||
metrics_path: /metrics
|
||||
params:
|
||||
model: ["Meta-Llama-3.1-8B-Instruct-Q4_K_M"]
|
||||
static_configs:
|
||||
- targets:
|
||||
- 10.1.71.130:8002
|
||||
labels:
|
||||
hostname: astro-orbiter
|
||||
endpoint: astro-orbiter-router
|
||||
model: Meta-Llama-3.1-8B-Instruct-Q4_K_M
|
||||
|
||||
- job_name: llama-server-astro-orbiter-phi35
|
||||
scrape_interval: 15s
|
||||
metrics_path: /metrics
|
||||
params:
|
||||
model: ["Phi-3.5-mini-instruct-Q8_0"]
|
||||
static_configs:
|
||||
- targets:
|
||||
- 10.1.71.130:8002
|
||||
labels:
|
||||
hostname: astro-orbiter
|
||||
endpoint: astro-orbiter-router
|
||||
model: Phi-3.5-mini-instruct-Q8_0
|
||||
|
||||
# ─── Grafana ──────────────────────────────────────────────────────────────────
|
||||
grafana:
|
||||
|
||||
Reference in New Issue
Block a user