Hermes Agent service account 6455d22752 feat(llm-router): add Meta-Llama-3.1-8B-Instruct-4bit alias; document Phi json_schema limitation (t_9adf0889) ...

- Add Meta-Llama-3.1-8B-Instruct-4bit alias on Meta-Llama-3.1-8B-Instruct-Q4_K_M
  entry in the preset INI (per Ryan dashboard note).
- Document known issue: Phi-3.5-mini json_schema grammar sampler incompatibility
  in router mode (GH #23460 variant — chat template token format mismatch).
  Meta-Llama works with json_schema response_format; confirmed via live test.
- Phi-3.5-mini-instruct-8bit alias remains working for model routing;
  structured output (json_schema) fails due to the model's token format.

2026-08-12 23:33:20 -05:00

4.9 KiB

Raw Blame History

View Raw