Hermes Agent service account 6455d22752 feat(llm-router): add Meta-Llama-3.1-8B-Instruct-4bit alias; document Phi json_schema limitation (t_9adf0889) ...

- Add Meta-Llama-3.1-8B-Instruct-4bit alias on Meta-Llama-3.1-8B-Instruct-Q4_K_M
  entry in the preset INI (per Ryan dashboard note).
- Document known issue: Phi-3.5-mini json_schema grammar sampler incompatibility
  in router mode (GH #23460 variant — chat template token format mismatch).
  Meta-Llama works with json_schema response_format; confirmed via live test.
- Phi-3.5-mini-instruct-8bit alias remains working for model routing;
  structured output (json_schema) fails due to the model's token format.

2026-08-12 23:33:20 -05:00

..

llama-server-qwen.service.j2

feat(llm-inference): add llama.cpp router mode shadow deployment

2026-08-12 20:23:00 -05:00

llama-server-router-preset.ini.j2

feat(llm-router): add Meta-Llama-3.1-8B-Instruct-4bit alias; document Phi json_schema limitation (t_9adf0889)

2026-08-12 23:33:20 -05:00

llama-server-router.service.j2

feat(llm-router): switch to --models-preset mode; add Phi-3.5-mini-instruct-8bit alias (t_9adf0889)

2026-08-12 22:58:12 -05:00