ai: migrate LLM backbone from Kimi CLI to Codex CLI

Retires the Moonshot/Kimi subscription in favour of the already-paid ChatGPT plan. Both CLI wrappers now run `codex exec`; the kimi-agent container is gone. adolf-llm + hindsight-llm: - runKimi -> runCodex (`codex exec --json --skip-git-repo-check`), resume via `codex exec resume <thread_id>`. - MCP moves from a per-session .mcp.json (a workaround for Kimi having no --mcp-config-file flag) to a $CODEX_HOME/config.toml generated once at startup from shared-mcp.json. Field translation is load-bearing: bearerTokenEnvVar -> bearer_token_env_var, enabledTools -> enabled_tools. - approval_policy="never" + sandbox_mode required, or unattended turns block on an approval prompt nobody can answer. kimi-agent removed. It was the ONLY large-tier deployment behind LiteLLM, so deleting it outright would have silently degraded every large-tier request to the local 4B model via the existing fallbacks. tier-large, the auto_router complex-reasoning route and their fallbacks now point at the codex-backed adolf-llm wrapper (model_name: codex-agent). Three environment blockers fixed along the way: - OpenAI geo-blocks this host (403 unsupported_country_region_territory). Both containers now egress via the host xray proxy, with NO_PROXY keeping MCP and *.alogins.net traffic off the tunnel. - node:22-slim ships no system CA store; the Rust codex binary validates TLS against it, so every HTTPS call failed with a generic transport error while Node's own fetch worked. ca-certificates added to both images. - `codex exec resume` rejects -C/--cd (plain `codex exec` accepts it), which broke follow-up turns while first turns succeeded. Known regression: Kimi's managed-usage API has no Codex equivalent, so the /usage route returns 501 and there is no quota probe for the codex model. The two quota plugins degrade quietly to no output. Also: stop tracking cognee.env (live LLM + JWT secrets) and gitignore it. The secrets remain in earlier history and should be rotated. Verified live: plain turn, SSE streaming, session resume, MCP tool call, bearer-token MCP call, and completions through both LiteLLM routes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014Y5QPagv4iun1ghpwM96Ff
2026-08-01 06:13:27 +00:00
parent a27bae828a
commit 9094d71e2f
66 changed files with 653 additions and 851 deletions
--- a/ai/model-registry.yaml
+++ b/ai/model-registry.yaml
@@ -0,0 +1,279 @@
+# Model registry — models are the scarce queued resource.
+#
+# Per DESIGN-a2a-agents.md v2.1 §2-3b (commit df2071d5), kanboard task #133
+# (A2A-1). Version-controlled here; the "model plane" (§3) and the fabric's
+# workers/routers read this data — they do not duplicate it. Lifecycle a(t)
+# *probe mechanics* (QuotaProbe, GPUResidencyProbe, ...) live in
+# kanboard/bin/kb_worker.py; this registry supplies the *parameters* those
+# probes consume (commands, fields, thresholds, VRAM footprints).
+#
+# Scope constraint (alvis, §3a): NO METERED API BY DEFAULT. The workflow is
+# Claude Code (a flat-subscription runtime -> agent registry #134, not here)
+# + the Kimi wrapper + a local GPU embedder + a small weak local model. The
+# governor arbitrates quota and GPU, not money. Any metered model below is
+# `metered: true, opt_in_required: true` and carries no default route to it
+# (see routing.metered_opt_in: [] at the bottom — empty means unreachable).
+#
+# Read with model_registry.py (same directory): resolve(), preload_check().
+#
+# ── Coverage vs litellm-config.yaml (kb#195, 2026-07-26 audit) ──────────
+# Every model_name litellm-config.yaml defines must appear either as a
+# `litellm_model_name` below or in this exclusion list. litellm_key_spec()
+# default-denies anything not reachable via routing.tiers, so an excluded
+# model stays ungoverned-but-inert until someone wires it up (add it here
+# and to routing.tiers first).
+#
+#   GOVERNED (present below):
+#     - ollama/gemma3:4b -> id: local-small (hot path: Hindsight LLM/
+#       consolidation/reflect all route here as of 2026-07-26)
+#     - judge -> id: paid-fallback (metered; see kb#164 for the fact that
+#       the no-metered-API constraint has no runtime enforcement yet)
+#     - codex-agent -> id: codex-agent (LiteLLM-routed large deployment; see
+#       below. Was kimi-agent + its own container until the 2026-08-01 purge)
+#     - bge-m3 -> id: bge-m3 (kb#164, 2026-07-26: wired into litellm-config
+#       .yaml pointing at ollama on 11436, the real embedder/routing
+#       classifier; litellm_model_name below updated from null to "bge-m3")
+#
+#   INTENTIONAL EXCLUSIONS (not governed by this registry, by design):
+#     - tip-generator (ollama/qwen2.5:1.5b), embedder (ollama/nomic-embed-
+#       text): aliases consumed by the separate oO ml/serving project, not
+#       the a2a fabric. Tracked in oO/CLAUDE.md, not duplicated here.
+#     - Raw ollama/* passthrough exposures — ollama/qwen3.5:4b,
+#       ollama/qwen3:8b, ollama/qwen2.5:1.5b, ollama/qwen2.5:0.5b,
+#       ollama/gemma3:1b, ollama/nomic-embed-text — manual/dev-console
+#       access to the ollama instances for ad-hoc testing. No agent or
+#       fabric workflow is registered against them (grepped agent-registry
+#       .yaml and openai/*.py: no hits). Not in routing.tiers, so
+#       litellm_key_spec() grants no agent access to them either way.
+#       If one of these becomes a real dependency (as ollama/gemma3:4b
+#       did), give it its own registry entry at that point.
+#     - The 12 OpenRouter `*:free` models (meta-llama/llama-3.3-70b-
+#       instruct:free, meta-llama/llama-3.2-3b-instruct:free, deepseek/
+#       deepseek-r1:free, qwen/qwen3-4b:free, qwen/qwen3-coder:free,
+#       google/gemma-3-27b-it:free, google/gemma-3-12b-it:free, mistralai/
+#       mistral-small-3.1-24b-instruct:free, nvidia/nemotron-3-super-
+#       120b-a12b:free, openai/gpt-oss-120b:free, minimax/minimax-m2.5:free,
+#       nousresearch/hermes-3-llama-3.1-405b:free) — human-facing manual-
+#       selection models (e.g.
+#       via Open WebUI), outside the agent fabric's model plane. Not
+#       referenced by any agent registry entry, not in routing.tiers, so
+#       resolve()/litellm_key_spec() never route an agent to them. Free
+#       tier, so this is not the kb#164 metered-enforcement gap — flag
+#       for a proper entry only if an agent workflow starts depending on
+#       one of these.
+
+schema_version: 1
+
+models:
+  # ── codex — main reasoning ─────────────────────────────────────────────
+  # Flat ChatGPT subscription via `codex login`, wrapped by two independent
+  # Codex-CLI containers (own creds volume each). Not behind LiteLLM today —
+  # callers hit the wrapper HTTP endpoints directly.
+  #
+  # Migrated from Kimi CLI 2026-07-31 for cost: this retires the separate
+  # Moonshot subscription in favour of the already-paid ChatGPT plan. The id
+  # changed `kimi` -> `codex`; routing.tiers below refers to it by id.
+  - id: codex
+    role: "main reasoning (adolf-llm / hindsight-llm Codex-CLI wrappers)"
+    litellm_model_name: null
+    endpoints:
+      - name: adolf-llm
+        purpose: "Adolf's conversational backbone"
+        url: "http://adolf-llm:8010"
+        # Route retained but returns HTTP 501 since the Codex migration —
+        # no machine-readable quota on this backend. See quota: below.
+        usage_url: "http://localhost:8010/usage"
+      - name: hindsight-llm
+        purpose: "Hindsight's structured-extraction LLM (HINDSIGHT_API_LLM_MODEL)"
+        url: "http://hindsight-llm:8012/v1"
+        model_name: "openai/hindsight-llm"
+    tier: large
+    context_tokens: 400000  # GPT-5-Codex context window; re-verify if the CLI's pinned model changes
+    tool_use_quality: high
+    lifecycle: quota-gated
+    quota:
+      # UNRESOLVED at the Codex migration (2026-07-31): the Kimi backend had a
+      # machine-readable managed-usage API that adolf-llm's /usage route
+      # normalized into these fields. Codex exposes no equivalent endpoint, so
+      # /usage now returns HTTP 501 and there is currently NO quota probe for
+      # this model. Governor quota-gating on `codex` is therefore blind — it
+      # will not see the ChatGPT plan's rate limits until a signal is found.
+      probe_command: null
+      windows: []
+      threshold_pct: 95
+    gpu_residency: null
+    cost_class: subscription   # flat-rate, not metered — quota is the constraint, not spend
+    metered: false
+    opt_in_required: false
+
+  # ── codex-agent — the LiteLLM-routed large deployment ──────────────────
+  # Replaces the retired `kimi-agent` container (2026-08-01 Kimi purge). That
+  # container was the ONLY large-tier deployment behind LiteLLM, backing
+  # `tier-large`, the auto_router's complex-reasoning route and their
+  # fallbacks — removing it without a replacement would have silently degraded
+  # every large-tier request to the local 4B model. Rather than stand up a
+  # third CLI container with its own login, this now points at the existing
+  # codex-backed adolf-llm wrapper (:8010, model id "adolf").
+  #
+  # Same underlying ChatGPT subscription as `codex` above — the two ids differ
+  # only in call path (this one via LiteLLM, `codex` direct to the wrappers),
+  # so their quota is shared and neither has a probe.
+  - id: codex-agent
+    role: "LiteLLM-routed large deployment; proxies to the codex-backed adolf-llm wrapper"
+    litellm_model_name: "codex-agent"   # openai/litellm-config.yaml model_list entry
+    endpoints:
+      - name: adolf-llm
+        url: "http://adolf-llm:8010/v1"
+    tier: large
+    context_tokens: 400000  # GPT-5-Codex context window; re-verify if the CLI's pinned model changes
+    tool_use_quality: high
+    lifecycle: quota-gated
+    quota:
+      probe_command: null   # no machine-readable quota on the Codex backend — see `codex`
+      windows: []
+      threshold_pct: null
+    gpu_residency: null
+    cost_class: subscription
+    metered: false
+    opt_in_required: false
+
+  # ── local-small — the cheap tier ───────────────────────────────────────
+  # ollama/gemma3:4b on the GPU ollama instance. Already the live model for
+  # Hindsight consolidation/reflect (HINDSIGHT_API_CONSOLIDATION_LLM_MODEL /
+  # HINDSIGHT_API_REFLECT_LLM_MODEL, kb#88) and exposed via LiteLLM.
+  - id: local-small
+    role: "cheap tier — ollama small/weak local model (background extraction, consolidation, reflect)"
+    litellm_model_name: "ollama/gemma3:4b"   # openai/litellm-config.yaml model_list entry
+    endpoints:
+      - name: ollama-direct
+        url: "http://host.docker.internal:11436"
+      - name: via-litellm
+        url: "http://litellm:4000/v1"
+    tier: small
+    context_tokens: 8192   # gemma3:4b default ctx; re-verify with `ollama show gemma3:4b` if raised
+    tool_use_quality: low
+    lifecycle: always-on
+    quota: null
+    gpu_residency:
+      vram_mb: 4000              # approx measured footprint, within the shared 8GB card (see gpu_residency_policy below)
+      never_evict: false         # evictable — a bigger model may push it out; that's a silent regression to catch, not prevent here
+      co_residency_group: interactive-local
+    cost_class: free
+    metered: false
+    opt_in_required: false
+
+  # ── bge-m3 — embedder + routing classifier ─────────────────────────────
+  # Never-evict: it's both Hindsight's recall embedder AND (design §3a) the
+  # embedding model LiteLLM Auto Router's semantic-router classifier will
+  # use for tier/complexity routing — losing it degrades both recall AND
+  # routing at once.
+  - id: bge-m3
+    role: "embedder — also the routing classifier (§3a, LiteLLM Auto Router / semantic-router)"
+    litellm_model_name: "bge-m3"   # kb#164, 2026-07-26: wired into litellm-config.yaml (ollama/bge-m3 @ 11436) -- was null (unwired gap)
+    endpoints:
+      - name: ollama-direct
+        url: "http://host.docker.internal:11436"
+        openai_compatible_path: "/v1/embeddings"
+    tier: small
+    context_tokens: 8192
+    tool_use_quality: "n/a"   # embedder, not a chat/tool-use model
+    lifecycle: always-on
+    quota: null
+    gpu_residency:
+      vram_mb: 1200
+      never_evict: true
+      co_residency_group: interactive-local
+    cost_class: free
+    metered: false
+    opt_in_required: false
+
+  # ── tei-reranker — interactive-critical, never-evict ───────────────────
+  # Not an LLM (cross-encoder rerank sidecar for Hindsight recall, kb#87)
+  # but carries the same GPU-residency stakes as bge-m3, so it's tracked
+  # here rather than invented as a separate registry class.
+  - id: tei-reranker
+    role: "cross-encoder reranker sidecar for Hindsight recall (interactive-critical)"
+    litellm_model_name: null   # TEI-compatible /rerank API; not routed through LiteLLM
+    endpoints:
+      - name: tei-reranker
+        url: "http://tei-reranker:80"   # host-published :8014
+    tier: small
+    context_tokens: null
+    tool_use_quality: "n/a"
+    lifecycle: always-on
+    quota: null
+    gpu_residency:
+      vram_mb: 1000
+      never_evict: true
+      co_residency_group: interactive-local
+    cost_class: free
+    metered: false
+    opt_in_required: false
+
+  # ── paid-fallback — optional, opt-in only ──────────────────────────────
+  # §3a: "Any paid deployment in the LiteLLM config must be explicitly
+  # enabled per agent via its virtual key; nothing routes to a metered
+  # model implicitly." routing.metered_opt_in below is the enforcement
+  # point: empty list = no caller has opted in = unreachable by resolve().
+  - id: paid-fallback
+    role: "optional metered fallback (e.g. Haiku) — disabled by default"
+    litellm_model_name: "judge"   # litellm-config.yaml's existing entry (anthropic/claude-haiku-4-5-20251001)
+    endpoints: []
+    tier: large
+    context_tokens: 200000
+    tool_use_quality: high
+    lifecycle: cost-gated
+    quota:
+      probe_command: null   # wire to a LiteLLM virtual-key budget probe (kb_worker.py BudgetProbe) once a caller opts in
+      windows: []
+      threshold_pct: null
+    gpu_residency: null
+    cost_class: metered
+    metered: true
+    opt_in_required: true
+
+# ── GPU residency policy (§3b) ──────────────────────────────────────────
+# "a local model's a(t) is not 1": a(t) = f(VRAM headroom). Never-evict
+# models are excluded from eviction math entirely — their VRAM is a fixed
+# reservation. Everything else in a co-residency group must fit in what's
+# left. preload_check semantics documented here; implemented generically
+# in model_registry.py so it reads this data instead of hardcoding numbers.
+gpu_residency_policy:
+  card: "GTX 1070, 8192 MB (single GPU today; §8 — more GPUs become a placement problem, same policy, more slots)"
+  total_vram_mb: 8192
+  # Measured 2026-07-21: bge-m3 + gemma3:4b + tei-reranker ~= 6.2/8 GB.
+  # Loading something bigger than local-small's footprint on top evicts
+  # tei-reranker (LRU-ish ollama/torch behavior) -> silent recall-latency
+  # regression. This is the regression the pre-load check exists to catch.
+  measured_baseline_mb: 6200
+  never_evict_ids: [bge-m3, tei-reranker]
+  co_residency_groups:
+    interactive-local: [bge-m3, tei-reranker, local-small]
+  preload_check:
+    description: >
+      Before a worker pulls a candidate model onto the GPU it must pass
+      this check (see model_registry.py:preload_check): reserve every
+      never_evict model's vram_mb unconditionally, subtract whatever else
+      is currently resident, and require the candidate's own vram_mb to
+      fit in what's left of total_vram_mb. A failing check means "park,
+      don't load" — never silently evict a never-evict model.
+
+# ── routing ───────────────────────────────────────────────────────────────
+# Tier pools a caller can ask for without naming a model (design §2: "target
+# = constraint-set"). metered_opt_in lists the virtual keys that have
+# explicitly opted into paid-fallback; empty = no metered model is reachable
+# by anyone, satisfying the "no metered API by default" acceptance bar.
+routing:
+  tiers:
+    small: [local-small]
+    # paid-fallback listed as a large-tier candidate AFTER codex so resolve()
+    # can fail over to it when codex's a(t)=0 (quota parked) — but only for a
+    # caller that both passes allow_metered=True AND appears in
+    # metered_opt_in below. With metered_opt_in empty (the shipped default)
+    # resolve() skips it unconditionally, so it stays unreachable.
+    #
+    # NB: with the Codex migration there is no quota probe (see the `codex`
+    # entry), so a(t) never reads as parked and this failover cannot trigger
+    # on quota today.
+    large: [codex, paid-fallback]
+  metered_opt_in: []   # e.g. ["agent:torgash"] once a human explicitly opts a specific virtual key in