fix(honcho): cover all LLM subsystems, enable flush, disable embeddings

Smoke-testing the first deploy uncovered three default-config issues
that no amount of Quadlet tuning would have caught:

  1. DIALECTIC subsystem ignored DERIVER_MODEL_CONFIG__*. Honcho splits
     dialectic into five reasoning levels (minimal/low/medium/high/max)
     each with its own MODEL_CONFIG that defaults to OpenAI. Without
     overrides, every /chat call fails with: ValidationException:
     Missing API key for openai model config. Now setting all five
     DIALECTIC_LEVELS__<level>__MODEL_CONFIG__* env vars to anthropic.

  2. DERIVER batches representation tasks until a token threshold is
     reached. For low-volume homelab use (one chatty operator), tasks
     can sit unprocessed forever. Add DERIVER_FLUSH_ENABLED knob,
     default true.

  3. Embeddings default to OpenAI text-embedding-3-small. Anthropic
     has no embedding API, so without an OpenAI key the embed step
     fails the entire derivation. Default EMBED_MESSAGES=false until
     a separate embedding provider is wired up (OpenAI for embeds-only
     or a local BGE endpoint on astro-orbiter).

defaults/main.yml documents all three issues and the migration path
back to embeddings when ready.
This commit is contained in:
JARVIS
2026-05-31 00:10:59 -05:00
parent ac955d327f
commit f8cf139b10
3 changed files with 81 additions and 19 deletions

View File

@@ -1,4 +1,3 @@
# {{ ansible_managed }}
# mk-labs Honcho deriver worker (background queue consumer)
# Managed by Ansible - honcho role
#
@@ -17,9 +16,10 @@ Image={{ honcho_image }}
ContainerName={{ honcho_deriver_container_name }}
Network={{ honcho_network_name }}
# Run the deriver script instead of the FastAPI server. The deriver
# polls the queue and processes work in a loop. Number of concurrent
# workers controlled via DERIVER_WORKERS.
# Run the deriver via the package's __main__ (not the helpers module).
# src/deriver/deriver.py has no __main__ guard — loading it exits 0 in
# ~3s and systemd treats that as a crash loop. The actual entry point
# lives in src/deriver/__main__.py and is invoked via "python -m src.deriver".
WorkingDir=/app
Exec=/app/.venv/bin/python -m src.deriver
@@ -29,21 +29,29 @@ Environment=DB_CONNECTION_URI=postgresql+psycopg://{{ honcho_db_user }}:{{ honch
# -- Auth -------------------------------------------------------------------
# Deriver does not serve HTTP, but it reads the same config object — keep
# secrets aligned with the API container.
Environment=AUTH_USE_AUTH={{ honcho_auth_enabled | string | lower }}
Environment=AUTH_JWT_SECRET={{ honcho_jwt_secret }}
Environment=AUTH_USE_AUTH=*** honcho_auth_enabled | string | lower }}
Environment=AUTH_JWT_SECRET=*** honcho_jwt_secret }}
# -- LLM provider -----------------------------------------------------------
# -- LLM provider keys ------------------------------------------------------
{% if honcho_llm_transport == 'anthropic' %}
Environment=LLM_ANTHROPIC_API_KEY={{ honcho_anthropic_api_key }}
Environment=LLM_ANTHROPIC_API_KEY=*** honcho_anthropic_api_key }}
{% endif %}
# -- Deriver runtime --------------------------------------------------------
Environment=DERIVER_ENABLED={{ honcho_deriver_enabled | string | lower }}
Environment=DERIVER_WORKERS={{ honcho_deriver_workers }}
Environment=DERIVER_POLLING_SLEEP_INTERVAL_SECONDS={{ honcho_deriver_polling_seconds }}
Environment=DERIVER_FLUSH_ENABLED={{ honcho_deriver_flush_enabled | string | lower }}
Environment=DERIVER_MODEL_CONFIG__TRANSPORT={{ honcho_llm_transport }}
Environment=DERIVER_MODEL_CONFIG__MODEL={{ honcho_deriver_model }}
# -- Summary subsystem (the deriver triggers summaries periodically) --------
Environment=SUMMARY_MODEL_CONFIG__TRANSPORT={{ honcho_llm_transport }}
Environment=SUMMARY_MODEL_CONFIG__MODEL={{ honcho_summary_model }}
# -- Embeddings (off by default — see defaults/main.yml for rationale) ------
Environment=EMBED_MESSAGES={{ honcho_embed_messages | string | lower }}
# -- Vector store -----------------------------------------------------------
Environment=VECTOR_STORE_TYPE=pgvector