rblundon/homelab
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(honcho): cover all LLM subsystems, enable flush, disable embeddings

Browse Source 
Smoke-testing the first deploy uncovered three default-config issues
that no amount of Quadlet tuning would have caught:

  1. DIALECTIC subsystem ignored DERIVER_MODEL_CONFIG__*. Honcho splits
     dialectic into five reasoning levels (minimal/low/medium/high/max)
     each with its own MODEL_CONFIG that defaults to OpenAI. Without
     overrides, every /chat call fails with: ValidationException:
     Missing API key for openai model config. Now setting all five
     DIALECTIC_LEVELS__<level>__MODEL_CONFIG__* env vars to anthropic.

  2. DERIVER batches representation tasks until a token threshold is
     reached. For low-volume homelab use (one chatty operator), tasks
     can sit unprocessed forever. Add DERIVER_FLUSH_ENABLED knob,
     default true.

  3. Embeddings default to OpenAI text-embedding-3-small. Anthropic
     has no embedding API, so without an OpenAI key the embed step
     fails the entire derivation. Default EMBED_MESSAGES=false until
     a separate embedding provider is wired up (OpenAI for embeds-only
     or a local BGE endpoint on astro-orbiter).

defaults/main.yml documents all three issues and the migration path
back to embeddings when ready.
This commit is contained in:
JARVIS
2026-05-31 00:10:59 -05:00
parent ac955d327f
commit f8cf139b10
3 changed files with 81 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files

39
ansible/roles/honcho/defaults/main.yml
View File

@@ -68,17 +68,50 @@ honcho_postgres_initdb_args: "--encoding=UTF8 --locale=C"
# ---------------------------------------------------------------------------
# Supported transports: openai, anthropic, gemini. We start with anthropic
# and can flip to a local OpenAI-compatible endpoint later by changing
# these two values plus the API-key env-var name.
# these knobs plus the API-key env-var name.
#
# Honcho has THREE distinct LLM subsystems, each with its own settings
# class and env-var prefix:
#
# DERIVER — generates observations from messages (background worker)
# SUMMARY — periodic session summaries
# DIALECTIC— answers theory-of-mind queries (the `peers/{peer}/chat` API)
#
# Honcho's dialectic subsystem uses PER-LEVEL config (minimal/low/medium/
# high/max) under DIALECTIC_LEVELS__<level>__MODEL_CONFIG__*. Defaults
# point at OpenAI, so any honcho.dev install without an OpenAI key will
# fail dialectic queries with "Missing API key for openai model config".
# We override all five levels to Anthropic below in the API template.
honcho_llm_transport: "anthropic"
honcho_deriver_model: "claude-sonnet-4-5"
honcho_summary_model: "claude-sonnet-4-5"
honcho_dialectic_model: "claude-sonnet-4-5"
# Deriver tuning — number of workers, polling cadence, etc. Conservative
# defaults appropriate for a small homelab deployment.
# Deriver tuning. Honcho batches representation tasks until the per-batch
# token threshold is reached. For homelab use with one chatty operator,
# that means short bursts of conversation can sit unprocessed forever.
# DERIVER_FLUSH_ENABLED=true bypasses the batching threshold so each
# message is processed promptly.
honcho_deriver_enabled: true
honcho_deriver_workers: 1
honcho_deriver_polling_seconds: "1.0"
honcho_deriver_flush_enabled: true
# ---------------------------------------------------------------------------
# Embeddings
# ---------------------------------------------------------------------------
# Anthropic does not provide an embedding API. Honcho defaults to
# OpenAI text-embedding-3-small, which fails without an OpenAI key.
# Three options going forward:
# 1. Set EMBED_MESSAGES=false — disables semantic search/vector recall
# but theory-of-mind derivation still works. Default for now.
# 2. Provide an OpenAI API key purely for embeddings (cheap; embeddings
# are ~$0.02 per million tokens). Set honcho_embed_messages=true and
# add vault_honcho_openai_api_key.
# 3. Stand up a local BGE/nomic embedding endpoint (e.g. on astro-orbiter
# once GPU is ready) and point Honcho at it via
# EMBEDDING_MODEL_CONFIG__OVERRIDES__BASE_URL.
honcho_embed_messages: false
# ---------------------------------------------------------------------------
# Auth