docs: v2.1 — no metered API by default; embedding-based routing
Some checks failed
ClawSweeper Dispatch / dispatch (push) Has been cancelled
CodeQL / Security High (actions) (push) Has been cancelled
CodeQL / Security High (channel-runtime-boundary) (push) Has been cancelled
CodeQL / Security High (core-auth-secrets) (push) Has been cancelled
CodeQL / Security High (mcp-process-tool-boundary) (push) Has been cancelled
CodeQL / Security High (network-ssrf-boundary) (push) Has been cancelled
CodeQL / Security High (plugin-trust-boundary) (push) Has been cancelled
CodeQL / Security High (process-exec-boundary) (push) Has been cancelled
Docs / docs (push) Has been cancelled
OpenClaw Stable Main Closeout / Resolve stable release closeout inputs (push) Has been cancelled
OpenClaw Stable Main Closeout / Verify stable main closeout (push) Has been cancelled
Workflow Sanity / no-tabs (push) Has been cancelled
Workflow Sanity / actionlint (push) Has been cancelled
Workflow Sanity / generated-doc-baselines (push) Has been cancelled
Stale / stale (push) Has been cancelled
Stale / backfill-stale-closures (push) Has been cancelled
Stale / lock-closed-issues (push) Has been cancelled
OpenClaw Scheduled Live And E2E Checks / live_and_openwebui_checks (push) Has been cancelled
Control UI Locale Refresh / plan (push) Has been cancelled
Control UI Locale Refresh / Refresh ${{ matrix.locale }} (push) Has been cancelled
Control UI Locale Refresh / Commit control UI locale refresh (push) Has been cancelled
QA-Lab - All Lanes / Run Telegram live QA lane with Convex leases (push) Has been cancelled
QA-Lab - All Lanes / Run Discord live QA lane with Convex leases (push) Has been cancelled
QA-Lab - All Lanes / Run WhatsApp live QA lane with Convex leases (push) Has been cancelled
QA-Lab - All Lanes / Run Slack live QA lane with Convex leases (push) Has been cancelled
QA-Lab - All Lanes / Authorize workflow actor (push) Has been cancelled
QA-Lab - All Lanes / Validate selected ref (push) Has been cancelled
QA-Lab - All Lanes / Run QA Lab mock parity lane (push) Has been cancelled
QA-Lab - All Lanes / Run live runtime token-efficiency lane (push) Has been cancelled
QA-Lab - All Lanes / Run Matrix live QA lane (push) Has been cancelled
QA-Lab - All Lanes / Run Matrix live QA lane (e2ee-cli) (push) Has been cancelled
QA-Lab - All Lanes / Run Matrix live QA lane (e2ee-deep) (push) Has been cancelled
QA-Lab - All Lanes / Run Matrix live QA lane (e2ee-smoke) (push) Has been cancelled
QA-Lab - All Lanes / Run Matrix live QA lane (media) (push) Has been cancelled
QA-Lab - All Lanes / Run Matrix live QA lane (transport) (push) Has been cancelled
OpenClaw Performance / Kova live OpenAI GPT 5.5 agent turn (push) Has been cancelled
OpenClaw Performance / Kova mock provider performance (push) Has been cancelled
OpenClaw Performance / Kova mock provider deep profile (push) Has been cancelled
CodeQL Critical Quality / Select Critical Quality shards (push) Has been cancelled
CodeQL Critical Quality / Critical Quality (core-auth-secrets) (push) Has been cancelled
CodeQL Critical Quality / Critical Quality (config-boundary) (push) Has been cancelled
CodeQL Critical Quality / Critical Quality (gateway-runtime-boundary) (push) Has been cancelled
CodeQL Critical Quality / Critical Quality (channel-runtime-boundary) (push) Has been cancelled
CodeQL Critical Quality / Critical Quality (network-runtime-boundary) (push) Has been cancelled
CodeQL Critical Quality / Critical Quality (agent-runtime-boundary) (push) Has been cancelled
CodeQL Critical Quality / Critical Quality (mcp-process-runtime-boundary) (push) Has been cancelled
CodeQL Critical Quality / Critical Quality (memory-runtime-boundary) (push) Has been cancelled
CodeQL Critical Quality / Critical Quality (session-diagnostics-boundary) (push) Has been cancelled
CodeQL Critical Quality / Critical Quality (plugin-sdk-reply-runtime) (push) Has been cancelled
CodeQL Critical Quality / Critical Quality (provider-runtime-boundary) (push) Has been cancelled
CodeQL Critical Quality / Critical Quality (ui-control-plane) (push) Has been cancelled
CodeQL Critical Quality / Critical Quality (web-media-runtime-boundary) (push) Has been cancelled
CodeQL Critical Quality / Critical Quality (plugin-boundary) (push) Has been cancelled
CodeQL Critical Quality / Critical Quality (plugin-sdk-package-contract) (push) Has been cancelled
CodeQL Android Critical Security / Critical Security (android) (push) Has been cancelled

alvis: the workflow is scoped to Claude Code + Kimi wrapper + local GPU
embedder + small local model. We do not pay per-token for API usage.

- NEW §3a cost model: the binding constraints are QUOTA and VRAM, not
  money. The "cheap tier" is the LOCAL small model, not a cheap paid
  one; paid deployments are opt-in fallback per virtual key and never
  route implicitly. Claude Code is a flat-subscription runtime, not a
  metered API.
- Routing classification is EMBEDDING-based on the already-resident
  local bge-m3 (LiteLLM Auto Router / semantic-router takes a
  configurable embedding_model) — no classifier LLM, no API spend.
  Keep a heuristic fallback: Auto Router v2 is a week old with an open
  embedding bug report.
- Budget governor reframed: arbitrates quota and GPU, not spend.
- GPU residency section renumbered 3b.
- Decision log 23-25 (incl. Hindsight caller-supplied cognition mode as
  an optional extra path, default API/queue path unchanged).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014t8Qg9gi7H7HtT8MncoXAB
This commit is contained in:
2026-07-21 09:02:24 +00:00
parent 1db3dd1b34
commit df2071d5ee

View File

@@ -103,6 +103,10 @@ task's acceptance criteria. Lifecycle: … → done (unverified) → closed
(Kimi): a(t)=0 when the window is spent — the queue **parks**, nothing fails, (Kimi): a(t)=0 when the window is spent — the queue **parks**, nothing fails,
drains on reset. cost-gated: a(t)=0 past budget. on-demand (remote llama): drains on reset. cost-gated: a(t)=0 past budget. on-demand (remote llama):
a(t)=0 until woken. Four lifecycles, one function. a(t)=0 until woken. Four lifecycles, one function.
**Note on "cost":** in this lab the binding constraint is **quota and VRAM,
not money** — see §3a. The cost-gated shape exists for the optional paid
fallback only; it is not the normal case.
3. **Hindsight reflect is just a submit** — `{intent: reflect, refs: bank+query, 3. **Hindsight reflect is just a submit** — `{intent: reflect, refs: bank+query,
constraints: tier≥large}`, async. Same for consolidation (low priority). constraints: tier≥large}`, async. Same for consolidation (low priority).
4. **Backbone swap is a constraint edit.** Persona agents name constraints, not 4. **Backbone swap is a constraint edit.** Persona agents name constraints, not
@@ -141,8 +145,12 @@ fabric free of micro-churn by construction.
│ specific backbone → pinned model_name │ │ specific backbone → pinned model_name │
│ “tier” routing → tier pool │ │ “tier” routing → tier pool │
│ automatic router → auto_router/complexity_router │ │ automatic router → auto_router/complexity_router │
│ Classification is EMBEDDING-based on the local bge-m3 (semantic- │
│ router): no classifier LLM, no API spend (§3a). │
│ Deployments: kimi (wrapper) | local small model | local embedder; │
│ metered/paid = opt-in fallback only, never implicit. │
│ The fabric owns everything LiteLLM cannot: ASYNC queueing, parking │ │ The fabric owns everything LiteLLM cannot: ASYNC queueing, parking │
│ across quota windows, leases, cross-agent budget arbitration. │ across quota windows, leases, cross-agent quota/GPU arbitration. │
└─────────────────────────────────────────────────────────────────────┘ └─────────────────────────────────────────────────────────────────────┘
┌─ Context stores ────────────────────────────────────────────────────┐ ┌─ Context stores ────────────────────────────────────────────────────┐
│ Hindsight banks (per-agent memory) · gitea (code, docs, this file) │ │ Hindsight banks (per-agent memory) · gitea (code, docs, this file) │
@@ -150,7 +158,34 @@ fabric free of micro-churn by construction.
└─────────────────────────────────────────────────────────────────────┘ └─────────────────────────────────────────────────────────────────────┘
``` ```
### GPU residency — a local model's a(t) is not 1 ### 3a. Cost model — no metered API by default
**Hard scope constraint (alvis):** the workflow is **Claude Code + the Kimi
wrapper + a local GPU embedder + a small weak local model**. We do **not** pay
per-token for API usage. Consequences that shape the design:
| Resource | Nature | Constraint |
|---|---|---|
| **Claude Code** | flat subscription; a *runtime*, not a metered API | the `claude-coder` agent's capacity |
| **Kimi** (via adolf-llm / hindsight-llm wrappers) | flat subscription, windowed | ~60 msgs/5h, ~300/wk — quota, not money |
| **local small model** (ollama) | free | GPU/VRAM contention (§3b) |
| **local embedder** (bge-m3) | free, already resident | never-evict |
| paid API (Haiku/Flash/…) | metered | **optional fallback only — disabled by default, explicitly opt-in** |
So the "cheap tier" is the **local small model**, not a cheap paid model. The
budget governor (§5) therefore arbitrates **quota and GPU**, not spend. Any
paid deployment in the LiteLLM config must be explicitly enabled per agent via
its virtual key; nothing routes to a metered model implicitly.
**Routing classification is embedding-based, not model-based.** LiteLLM's Auto
Router (built on `semantic-router`) takes a configurable `embedding_model` +
`match_threshold`, so complexity/semantic classification runs on the **local
bge-m3** we already keep resident — no classifier LLM call, no API spend, ~zero
marginal cost. (Auto Router v2 is new as of 2026-07-14 and has an open
embedding-related bug report: verify hands-on and keep a heuristic
keyword/length fallback for the classifier.)
### 3b. GPU residency — a local model's a(t) is not 1
Local "free" models contend for VRAM with interactive components (measured on Local "free" models contend for VRAM with interactive components (measured on
the 8 GB GTX 1070: bge-m3 + gemma3:4b + tei-reranker ≈ 6.2 GB; loading anything the 8 GB GTX 1070: bge-m3 + gemma3:4b + tei-reranker ≈ 6.2 GB; loading anything
@@ -409,3 +444,14 @@ Added in v2.1 (hardening review, same day):
22. **Missed crons are not a keeper duty**: schedules that must not be missed 22. **Missed crons are not a keeper duty**: schedules that must not be missed
are monitored in **Zabbix**; faults are incidents handled **per problem are monitored in **Zabbix**; faults are incidents handled **per problem
class** by a future **ops-agent** (§6d), never by a blanket catch-up policy. class** by a future **ops-agent** (§6d), never by a blanket catch-up policy.
23. **No metered API by default** (§3a): scope = Claude Code + Kimi wrapper +
local GPU embedder + small local model. The cheap tier is the *local*
model, not a paid one; paid deployments are opt-in fallback per virtual
key. The governor arbitrates **quota and GPU, not money**.
24. **Routing classification is embedding-based** on the local bge-m3 via
LiteLLM Auto Router (`semantic-router`): no classifier LLM call, no API
spend. Keep a heuristic fallback — Auto Router v2 is new (2026-07-14) with
an open embedding bug report.
25. Hindsight gains an **optional caller-supplied cognition mode** (#131) so it
can run inside a Claude Code session with zero extra API calls; the
API/queue path stays the default. Same verb, different `target`.