hindsight: route ALL LLM stages to local gemma3:4b

Retain/extraction moved off the Kimi wrapper (hindsight-llm:8012) onto
ollama/gemma3:4b via LiteLLM, joining consolidation + reflect which were
already local (kb#88). Kimi is no longer in the Hindsight path at all, so
the memory backend consumes zero quota and Adolf's 5h window is left
entirely for conversation.

Accepted tradeoff: kb#84/kb#88 kept retain on Kimi for fact quality, since
gemma3:4b's Russian fluency was never verified and this bank is largely
Russian. A dry-run extraction on Russian input after the change returned
clean, parseable output but captured only one of two facts in the sample,
so extraction is coherent but lossier. Revert this first if facts degrade.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-26 07:59:44 +00:00
parent 0f9d83f3db
commit 59af13f1fe

View File

@@ -304,17 +304,21 @@ services:
container_name: hindsight
restart: unless-stopped
environment:
# ALL stages on the local model (alvis, 2026-07-26): retain/extraction
# moved OFF Kimi (hindsight-llm:8012) onto ollama/gemma3:4b via LiteLLM,
# joining consolidation + reflect which were already local. Kimi is no
# longer in the Hindsight path at all, so the memory backend costs zero
# quota and Adolf's 5h window is left entirely for conversation.
#
# ⚠️ Accepted tradeoff: the kb#88/kb#84 rationale for keeping retain on
# Kimi was fact QUALITY — gemma3:4b's Russian fluency was never verified
# (it was originally picked only to dodge qwen3:8b's <think>-token bug),
# and this bank's content is largely Russian. Watch extraction quality on
# the next retains; if facts degrade, this is the first thing to revert.
- HINDSIGHT_API_LLM_PROVIDER=openai
- HINDSIGHT_API_LLM_BASE_URL=http://hindsight-llm:8012/v1
- HINDSIGHT_API_LLM_MODEL=openai/hindsight-llm
# hindsight-llm ignores the key entirely (Kimi CLI wrapper, no real
# OpenAI auth) — dummy value, non-empty so the client constructs.
- HINDSIGHT_API_LLM_API_KEY=sk-hindsight-llm-local
# Per-stage LLM routing (kb#88): the heavy BACKGROUND stages —
# consolidation (~930 calls/3h) + reflect/mental-models (~177/3h) —
# were burning the Kimi 5h window (hit 100%). Route them to a cheap
# LOCAL model via LiteLLM/ollama (GPU, free); RETAIN extraction inherits
# the main HINDSIGHT_API_LLM_* above (Kimi) to keep fact quality.
- HINDSIGHT_API_LLM_BASE_URL=http://litellm:4000/v1
- HINDSIGHT_API_LLM_MODEL=ollama/gemma3:4b
- HINDSIGHT_API_LLM_API_KEY=sk-fjQC1BxAiGFSMs
- HINDSIGHT_API_CONSOLIDATION_LLM_PROVIDER=openai
- HINDSIGHT_API_CONSOLIDATION_LLM_BASE_URL=http://litellm:4000/v1
- HINDSIGHT_API_CONSOLIDATION_LLM_MODEL=ollama/gemma3:4b