openai: compose healthchecks + dependency ordering, registries, LiteLLM routing
docker-compose.yml gains healthchecks and depends_on/condition chains for the litellm/langfuse/postgres tier so dependants wait for a genuinely ready service instead of a started container. Also plumbs AGAP_MCP_TOKEN into the adolf and adolf-llm containers, sourced from openai/.env (gitignored), for the kb#180 bearer auth on the agap MCP server; shared-mcp.json consumes it via bearerTokenEnvVar so the Kimi backbone authenticates too. agent-registry.yaml / agent_registry.py: the version-controlled source of truth for agent identities and trust classes -- the same ids the agap-mcp token map resolves to (`adolf`, `claude-coder`; note `claude-code-cli` is the runtime entry, not an agent identity). model-registry.yaml, litellm-config.yaml, auto-router-routes.json and provision_litellm_keys.py: model tiering, virtual-key provisioning and auto-router routes. tei-reranker/ is the local reranker service backing Hindsight recall. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -124,21 +124,18 @@ agents:
|
||||
#
|
||||
# Counts (same tool lists both layers, confirmed identical by
|
||||
# validate_capability_grants.py, exit 0):
|
||||
# agap 32->24, hindsight 29->9, kanboard 23->14,
|
||||
# openclaw-tools 5->5 (already minimal, no filter needed).
|
||||
# marketplace stays UNFILTERED at layer 1 (7/13 kept) but is not in
|
||||
# shared-mcp.json AT ALL — Kimi's session never had it in the first
|
||||
# place (pre-existing gap between what OpenClaw offers Adolf and
|
||||
# what reaches Kimi, out of kb#144's scope to close).
|
||||
# Reachable-by-Kimi total: hindsight+kanboard+openclaw-tools+agap
|
||||
# 102-13(marketplace, never reached Kimi)=89 -> 9+14+5+24=52 tools
|
||||
# (-42%). Byte-measured (chars/4) against each server's real
|
||||
# tools/list JSON schemas: est. ~7K tokens saved/turn — estimate
|
||||
# pending the real wire.jsonl number, which needs the adolf-llm
|
||||
# container restart alvis owns (shared-mcp.json is bind-mounted
|
||||
# read-only but adolf-llm's server.js caches its content at process
|
||||
# start, so editing the file alone does not take effect — see
|
||||
# capability_grant_status below for the confirm-post-restart command).
|
||||
# agap 32->28 (includes kb#95 wiki_* and kb#170 todoist_capture_idea),
|
||||
# hindsight 29->9, kanboard 23->14, marketplace 13->7 (now in shared-
|
||||
# mcp.json, reaches Kimi), openclaw-tools 5->5 (already minimal).
|
||||
# Reachable-by-Kimi total (2026-07-26): 9+14+7+5+28=63 tools.
|
||||
# Previous total was 52 (excluding marketplace, pre-shared-mcp.json);
|
||||
# byte-measure against each server's real tools/list JSON schemas:
|
||||
# est. ~9K tokens/turn. Estimate pending the real wire.jsonl number,
|
||||
# which needs the adolf-llm container restart alvis owns
|
||||
# (shared-mcp.json is bind-mounted read-only but adolf-llm's
|
||||
# server.js caches its content at process start, so editing the file
|
||||
# alone does not take effect — see capability_grant_status below for
|
||||
# the confirm-post-restart command).
|
||||
#
|
||||
# kb#95 (2026-07-23): added wiki_search/wiki_read/wiki_edit (family
|
||||
# MediaWiki / РодоВики, family.alogins.net) to agap-mcp and to both
|
||||
@@ -148,11 +145,17 @@ agents:
|
||||
# recomputed here since it needs the same live wire.jsonl proof kb#144
|
||||
# used and this task does not touch the running containers (see
|
||||
# shared_mcp_kimi_allowlist below for the exact confirm command).
|
||||
#
|
||||
# kb#170: added todoist_capture_idea (agap-mcp/src/capture.js —
|
||||
# classify with local bge-m3 nearest-centroid, no LLM call, then
|
||||
# create the labelled Todoist task in one round trip) to agap-mcp
|
||||
# and to both layers' agap allowlist below. Ages the counts comment
|
||||
# above by +1/+1 for the same reason as kb#95's note.
|
||||
mcp_tool_filter:
|
||||
hindsight: [recall, retain, reflect, list_memories, get_memory, update_memory, list_directives, create_directive, delete_directive]
|
||||
kanboard: [kanboard_list_projects, kanboard_get_project, kanboard_list_tasks, kanboard_my_tasks, kanboard_get_task, kanboard_search_tasks, kanboard_list_users, kanboard_project_activity, kanboard_create_task, kanboard_update_task, kanboard_move_task, kanboard_change_task_status, kanboard_assign_task, kanboard_add_comment]
|
||||
marketplace: [marketplace_find_best, marketplace_search, marketplace_get_product, marketplace_get_recommendations, marketplace_get_reviews, marketplace_compare_prices, marketplace_status]
|
||||
agap: [vw_get_password, vw_get_item, vw_list_items, vw_create_login, vw_update_password, ha_get_state, ha_list_entities, ha_call_service, ha_get_history, zabbix_get_problems, zabbix_get_hosts, zabbix_get_items, zabbix_get_triggers, radicale_list_calendars, radicale_list_events, radicale_get_event, radicale_put_event, radicale_delete_event, radicale_move_event, todoist_list_tasks, todoist_list_projects, todoist_create_task, todoist_update_task, todoist_complete_task, wiki_search, wiki_read, wiki_edit]
|
||||
agap: [vw_get_password, vw_get_item, vw_list_items, vw_create_login, vw_update_password, ha_get_state, ha_list_entities, ha_call_service, ha_get_history, zabbix_get_problems, zabbix_get_hosts, zabbix_get_items, zabbix_get_triggers, radicale_list_calendars, radicale_list_events, radicale_get_event, radicale_put_event, radicale_delete_event, radicale_move_event, todoist_list_tasks, todoist_list_projects, todoist_create_task, todoist_update_task, todoist_complete_task, todoist_capture_idea, wiki_search, wiki_read, wiki_edit]
|
||||
openclaw-tools: null # no filter in openclaw.json — already minimal (5/5 kept)
|
||||
note: >
|
||||
"scoped core tools" per kb#134's brief, now REAL at both levels: this
|
||||
@@ -178,11 +181,20 @@ agents:
|
||||
- { id: adolf-elizaveta, role: private, interlocutor: elizaveta }
|
||||
- { id: adolf-shared, role: shared, interlocutor: household }
|
||||
current_state: >
|
||||
NOT split yet. A single live bank "adolf" (hindsight MCP,
|
||||
http://hindsight:8888/mcp/adolf/, 269+ facts) serves every
|
||||
interlocutor today with no per-human isolation — the exact defect
|
||||
kb#153 exists to fix (depends on this registry existing first).
|
||||
The three banks above are kb#153's target, not current fact.
|
||||
NOT split yet for the plugin's recall/retain hooks: a single live
|
||||
bank "adolf" (525+ facts) serves every interlocutor today with no
|
||||
per-human isolation — the exact defect kb#153 exists to fix
|
||||
(depends on this registry existing first). The three banks above
|
||||
are kb#153's target, not current fact for the hooks.
|
||||
kb#169 (2026-07-26): the SEPARATE raw hindsight MCP tool surface
|
||||
(mcp.servers.hindsight in adolf/openclaw.json + openai/shared-
|
||||
mcp.json — recall/retain/reflect/etc. callable directly by the
|
||||
model, bypassing #153's interlocutor-scoping entirely) has been
|
||||
repointed from http://hindsight:8888/mcp/adolf/ (the unpartitioned
|
||||
bank, still what the hooks use) to http://hindsight:8888/mcp/
|
||||
adolf-shared/ (pre-existing, 0 facts). That surface can now only
|
||||
ever touch the shared bank — never a private one, never the mixed
|
||||
"adolf" bank — regardless of who's talking to Adolf.
|
||||
kb_identity: { username: adolf, user_id: 3 }
|
||||
availability_note: "a(t) inherited from backbone at read time (kimi: quota-gated, ~60msg/5h ~300/wk — see model-registry.yaml)"
|
||||
|
||||
@@ -215,6 +227,14 @@ agents:
|
||||
model: "session (ephemeral, per invocation) + repo state (git history, CLAUDE.md files, kanboard task/comment history) — no persistent Hindsight bank"
|
||||
kb_identity: { username: claude, user_id: 2 }
|
||||
availability_note: "a(t) inherited from backbone at read time (claude-code-cli: always-on, gated by claude-usage windows)"
|
||||
completion_convention: >
|
||||
Verified-completion flow (DESIGN v2.1 §2, kb#159): when completing a task,
|
||||
the worker/agent NEVER closes it — only moves it to Done (unverified
|
||||
completion) and leaves it open. Closing is verification, done by someone
|
||||
OTHER than the producer (the submitter, a human, or a reviewer-agent after
|
||||
checking acceptance criteria). The fabric-keeper audits this: closed tasks
|
||||
where the producer also closed them are flagged as kb#159 violations in
|
||||
the daily digest.
|
||||
|
||||
# ── torgash — marketplace analyst (sandboxed) ───────────────────────────
|
||||
- id: torgash
|
||||
|
||||
@@ -153,6 +153,76 @@ def effective_card(registry, agent_id, model_registry=None):
|
||||
return card
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# litellm_key_spec — kb#147 (A2A-15): turn an agent's static registry fields
|
||||
# into the LiteLLM virtual-key grant provision_litellm_keys.py provisions.
|
||||
# "Grants live in the agent registry, not scattered configs" (kb#147 accept-
|
||||
# ance bar) means the model allow-list and budget are COMPUTED here from
|
||||
# preferred_tier + trust_class, never hand-typed per agent.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# Ascending order matching model-registry.yaml's routing.tiers keys. An
|
||||
# agent may use its preferred tier and anything below it (a "large"-
|
||||
# preferring agent degrades to "small" gracefully; a "small"-only agent
|
||||
# never gets "large" — that asymmetry IS the sandboxed/trusted split this
|
||||
# key spec exists to enforce).
|
||||
_TIER_ORDER = ["small", "large"]
|
||||
|
||||
|
||||
def _reachable_tiers(preferred_tier):
|
||||
if preferred_tier not in _TIER_ORDER:
|
||||
return []
|
||||
return _TIER_ORDER[: _TIER_ORDER.index(preferred_tier) + 1]
|
||||
|
||||
|
||||
def litellm_key_spec(registry, agent_id, model_registry=None):
|
||||
"""Return the LiteLLM virtual-key grant for `agent_id`: which
|
||||
litellm_model_name values it may use and its default budget, derived
|
||||
from THIS registry's data (preferred_tier, trust_class) plus
|
||||
model-registry.yaml's routing.tiers/metered_opt_in — never hand-entered
|
||||
per agent. Models with no litellm_model_name (e.g. `kimi`, called
|
||||
directly via the adolf-llm wrapper, never through LiteLLM) are outside
|
||||
LiteLLM's enforcement surface by construction and are excluded, not
|
||||
silently allowed.
|
||||
|
||||
provision_litellm_keys.py consumes this dict's `models`/`max_budget`/
|
||||
`budget_duration`/`key_alias` as the body of a LiteLLM /key/generate (or
|
||||
/key/update) call. This function makes no network call itself.
|
||||
"""
|
||||
a = get_agent(registry, agent_id)
|
||||
model_registry = model_registry if model_registry is not None else mr.load_registry()
|
||||
grant = a.get("capability_grant") or {}
|
||||
key_alias = grant.get("litellm_key_alias", agent_id)
|
||||
|
||||
opted_in = set(model_registry.get("routing", {}).get("metered_opt_in", []) or [])
|
||||
opted_in_key = f"agent:{agent_id}"
|
||||
pools = model_registry.get("routing", {}).get("tiers", {})
|
||||
|
||||
models = []
|
||||
for tier in _reachable_tiers(a.get("preferred_tier")):
|
||||
for model_id in pools.get(tier, []):
|
||||
m = mr.get_model(model_registry, model_id)
|
||||
name = m.get("litellm_model_name")
|
||||
if not name:
|
||||
continue # not LiteLLM-routed (e.g. kimi's adolf-llm wrapper) -- nothing to grant/deny here
|
||||
if m.get("metered") and opted_in_key not in opted_in:
|
||||
continue # §3a: no metered API by default, per-key opt-in only
|
||||
if name not in models:
|
||||
models.append(name)
|
||||
|
||||
classes = registry.get("trust_classes", {})
|
||||
cls = classes.get(a["trust_class"], {})
|
||||
return {
|
||||
"agent_id": agent_id,
|
||||
"key_alias": key_alias,
|
||||
"trust_class": a["trust_class"],
|
||||
"models": models,
|
||||
"max_budget": cls.get("default_budget_usd"),
|
||||
"budget_duration": cls.get("budget_duration"),
|
||||
"mcp_auth_token_env": grant.get("mcp_auth_token_env"),
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# CLI — manual verification only, not part of the library contract.
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -175,6 +245,9 @@ def main():
|
||||
p = sub.add_parser("can-reach-vault")
|
||||
p.add_argument("--id", required=True)
|
||||
|
||||
p = sub.add_parser("litellm-key-spec")
|
||||
p.add_argument("--id", required=True)
|
||||
|
||||
sub.add_parser("list")
|
||||
|
||||
args = ap.parse_args()
|
||||
@@ -192,6 +265,8 @@ def main():
|
||||
ok = can_reach_vault(reg, args.id)
|
||||
print(json.dumps({"id": args.id, "can_reach_vault": ok}))
|
||||
sys.exit(0 if ok else 1)
|
||||
elif args.cmd == "litellm-key-spec":
|
||||
print(json.dumps(litellm_key_spec(reg, args.id, model_reg), indent=2))
|
||||
elif args.cmd == "list":
|
||||
for a in reg["agents"]:
|
||||
backbone = a.get("backbone") or "-"
|
||||
|
||||
39
openai/auto-router-routes.json
Normal file
39
openai/auto-router-routes.json
Normal file
@@ -0,0 +1,39 @@
|
||||
{
|
||||
"_note": "kb#128 (A2A-16): human-readable source of truth for the auto_router route set. NOT loaded from this path at runtime -- litellm-config.yaml's `auto_router` deployment inlines this same `routes` array as a literal JSON string via litellm_params.auto_router_config. Reason (verified hands-on 2026-07-26 against litellm:main-latest): the auto_router_config_path loader (AutoRouter._load_semantic_routing_routes -> SemanticRouter.from_json) unconditionally builds a raw semantic_router encoder from encoder_type/encoder_name and requires a real provider API key even for a local model name like bge-m3 -- this IS the open Auto Router v2 embedding bug the task brief warned about. The auto_router_config (inline-string) loader (_load_auto_router_routes_from_config_json) only reads the `routes` key and builds Route objects directly, with zero encoder bootstrap -- confirmed working end-to-end: real litellm.embedding(model=ollama/bge-m3) calls, zero metered API spend, 'hi there' -> ollama/gemma3:4b, a refactor/dependency-injection prompt -> kimi-agent. Keep the two `routes` arrays in sync by hand when editing either.",
|
||||
"encoder_type": "litellm",
|
||||
"encoder_name": "bge-m3",
|
||||
"routes": [
|
||||
{
|
||||
"name": "ollama/gemma3:4b",
|
||||
"description": "Simple, short, low-stakes requests — greetings, quick factual lookups, formatting, one-line questions.",
|
||||
"utterances": [
|
||||
"hi",
|
||||
"hello",
|
||||
"what time is it",
|
||||
"what's the weather",
|
||||
"thanks",
|
||||
"what does this word mean",
|
||||
"summarize this in one sentence",
|
||||
"give me a quick yes or no",
|
||||
"format this as a list",
|
||||
"what is 2 plus 2"
|
||||
],
|
||||
"score_threshold": 0.5
|
||||
},
|
||||
{
|
||||
"name": "kimi-agent",
|
||||
"description": "Complex reasoning, multi-step planning, coding, or anything needing tool use and deep context.",
|
||||
"utterances": [
|
||||
"write a function that parses this log file and extracts errors",
|
||||
"refactor this class to use dependency injection",
|
||||
"think through the tradeoffs of these two architectures step by step",
|
||||
"debug why this docker container keeps crashing",
|
||||
"plan out the migration from cognee to hindsight across five tasks",
|
||||
"analyze this design document and find inconsistencies",
|
||||
"write a SQL query that joins these three tables and aggregates by month",
|
||||
"review this pull request for security issues"
|
||||
],
|
||||
"score_threshold": 0.5
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -1,5 +1,12 @@
|
||||
# cognee-llm (:8011)
|
||||
|
||||
> ⚠️ **SUPERSEDED — Adolf's memory is migrating Cognee → Hindsight (2026-07-13).**
|
||||
> Hindsight runs its LLM on LiteLLM `:4000` / Ollama, so this bespoke Kimi-CLI
|
||||
> wrapper is being **retired**, not ported (SPIKE gate 5 already concluded the
|
||||
> extraction workload shouldn't sit on the Kimi seat). This service is decommissioned
|
||||
> in migration task **H4**. Plan: `agap_git/adolf/HINDSIGHT-MIGRATION.md`. The doc
|
||||
> below describes the outgoing Cognee stack, kept until H4 lands.
|
||||
|
||||
OpenAI-compatible wrapper around the Kimi Code CLI (`@moonshot-ai/kimi-code`, home
|
||||
`/root/.kimi-code`), built for Cognee's batch/structured LLM calls. **Opposite policy to
|
||||
`kimi-agent`**:
|
||||
|
||||
@@ -9,6 +9,13 @@ services:
|
||||
volumes:
|
||||
- /mnt/ssd/dbs/litellm/postgres:/var/lib/postgresql/data
|
||||
restart: always
|
||||
# kb#190: cheap connectivity probe, no query load.
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U litellm -d litellm"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
start_period: 10s
|
||||
|
||||
litellm:
|
||||
image: ghcr.io/berriai/litellm:main-latest
|
||||
@@ -28,9 +35,19 @@ services:
|
||||
extra_hosts:
|
||||
- "host.docker.internal:host-gateway"
|
||||
depends_on:
|
||||
- litellm-db
|
||||
- langfuse
|
||||
litellm-db:
|
||||
condition: service_healthy
|
||||
langfuse:
|
||||
condition: service_healthy
|
||||
restart: always
|
||||
# kb#190: /health/liveliness is litellm's cheap liveness probe (no
|
||||
# provider/model call), unlike /health which pings every configured model.
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "node -e \"fetch('http://localhost:4000/health/liveliness').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))\""]
|
||||
interval: 15s
|
||||
timeout: 10s
|
||||
retries: 5
|
||||
start_period: 20s
|
||||
|
||||
kimi-agent:
|
||||
build: ./kimi-agent
|
||||
@@ -50,6 +67,13 @@ services:
|
||||
volumes:
|
||||
- /mnt/ssd/dbs/langfuse/postgres:/var/lib/postgresql/data
|
||||
restart: always
|
||||
# kb#190: cheap connectivity probe, no query load.
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U langfuse -d langfuse"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
start_period: 10s
|
||||
|
||||
langfuse:
|
||||
image: ghcr.io/langfuse/langfuse:2
|
||||
@@ -63,8 +87,20 @@ services:
|
||||
- SALT=7927b3b0092afe4542274940b557becea6418a5fed79f7acd25c3a789349fdc9
|
||||
- AUTH_DISABLE_SIGNUP=true
|
||||
depends_on:
|
||||
- langfuse-db
|
||||
langfuse-db:
|
||||
condition: service_healthy
|
||||
restart: always
|
||||
# kb#190: langfuse's Next.js server binds the container's bridge IP,
|
||||
# NOT 127.0.0.1/localhost (confirmed via `ss -tlnp` inside the
|
||||
# container: 127.0.0.1 connection is refused) -- so the probe must
|
||||
# address it by its own compose DNS name, which resolves to that same
|
||||
# bridge IP from inside the container.
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "wget -q -O- http://langfuse:3000/api/public/health || exit 1"]
|
||||
interval: 15s
|
||||
timeout: 10s
|
||||
retries: 5
|
||||
start_period: 30s
|
||||
|
||||
qdrant:
|
||||
image: qdrant/qdrant
|
||||
@@ -76,19 +112,22 @@ services:
|
||||
volumes:
|
||||
- /mnt/ssd/dbs/qdrant:/qdrant/storage:z
|
||||
|
||||
# GPU residency decision (kb#191, 2026-07-26, DESIGN-a2a-agents.md sec 3b):
|
||||
# the 8GB GTX 1070 only has ~1.7GB free with the never-evict set (bge-m3 +
|
||||
# tei-reranker) resident alongside gemma3:4b -- no room for a 4th GPU
|
||||
# tenant without risking evicting the reranker (silent Hindsight recall
|
||||
# breakage). Runs CPU-only until the card gets more headroom. Never
|
||||
# started yet -- kb#175 (Adolf STT) was parked waiting on this call.
|
||||
faster-whisper:
|
||||
image: fedirz/faster-whisper-server:latest-cuda
|
||||
container_name: faster-whisper
|
||||
runtime: nvidia
|
||||
ports:
|
||||
- "8880:8000"
|
||||
environment:
|
||||
- WHISPER__MODEL=deepdml/faster-whisper-large-v3-turbo-ct2
|
||||
- WHISPER__INFERENCE_DEVICE=cuda
|
||||
- WHISPER__INFERENCE_DEVICE=cpu
|
||||
- WHISPER__COMPUTE_TYPE=int8
|
||||
- WHISPER__LANGUAGE=ru
|
||||
- NVIDIA_VISIBLE_DEVICES=all
|
||||
- NVIDIA_DRIVER_CAPABILITIES=compute,utility
|
||||
volumes:
|
||||
- /mnt/ssd/ai/faster-whisper:/root/.cache/huggingface
|
||||
restart: always
|
||||
@@ -152,12 +191,33 @@ services:
|
||||
- ADOLF_KEY=${ADOLF_KEY:-}
|
||||
- MATRIX_HOMESERVER=${MATRIX_HOMESERVER:-}
|
||||
- MATRIX_USER_ID=${MATRIX_USER_ID:-}
|
||||
# kb#67: stable token + device_id pin, so restarts reuse the existing
|
||||
# Matrix device (matrix-sdk/OpenClaw's own credential cache in the
|
||||
# adolf-state volume already does this across restarts -- see
|
||||
# extensions/matrix/src/matrix/client/config.ts resolveMatrixAuth --
|
||||
# but that cache lives in the volume, so a lost/rebuilt volume would
|
||||
# fall through to MATRIX_PASSWORD and mint a brand-new device with no
|
||||
# cross-signing. Setting the token here removes that dependency).
|
||||
# MATRIX_PASSWORD stays configured as a manual-recovery fallback only:
|
||||
# it is never used while MATRIX_ACCESS_TOKEN resolves to a valid token.
|
||||
- MATRIX_ACCESS_TOKEN=${MATRIX_ACCESS_TOKEN:-}
|
||||
- MATRIX_DEVICE_ID=${MATRIX_DEVICE_ID:-}
|
||||
- MATRIX_PASSWORD=${MATRIX_PASSWORD:-}
|
||||
- MATRIX_DEVICE_NAME=${MATRIX_DEVICE_NAME:-Adolf OpenClaw Gateway}
|
||||
# marketplace-mcp bearer token (kb task #61) -- referenced by
|
||||
# openclaw.json's mcp.servers.marketplace.headers.Authorization via
|
||||
# ${MARKETPLACE_MCP_TOKEN} substitution; never inlined into that file.
|
||||
- MARKETPLACE_MCP_TOKEN=${MARKETPLACE_MCP_TOKEN:-}
|
||||
# agap-mcp bearer token (kb#180) -- agap-mcp's :3100 listener requires
|
||||
# `Authorization: Bearer <token>` on every route now (DESIGN §4: no
|
||||
# unauthenticated JSON-RPC listener; :3100 is host-networked and the
|
||||
# LAN carries VPN-terminated peers). Referenced by openclaw.json's
|
||||
# mcp.servers.agap.headers.Authorization via ${AGAP_MCP_TOKEN}
|
||||
# substitution, and read directly by the todoist-capture plugin's
|
||||
# /capture-idea POST. The token must map to agent id `adolf` in
|
||||
# agap-mcp's AGAP_MCP_AGENT_TOKENS. Sourced from openai/.env
|
||||
# (gitignored); never inlined here.
|
||||
- AGAP_MCP_TOKEN=${AGAP_MCP_TOKEN:-}
|
||||
- TZ=Europe/Riga
|
||||
volumes:
|
||||
# Runtime state only (Matrix crypto/devices, credentials, sessions,
|
||||
@@ -189,6 +249,19 @@ services:
|
||||
# Cognee as Adolf's memory backend. Activated via
|
||||
# plugins.entries.hindsight-memory in openclaw.json.
|
||||
- ./hindsight-openclaw-plugin:/home/node/.openclaw/extensions/hindsight-memory:ro
|
||||
# kimi-quota-footer plugin (kb #85) — same read-only-bind-over-volume
|
||||
# pattern as quota-command/hindsight-memory above. Appends the Kimi
|
||||
# usage line to every outgoing reply via reply_payload_sending, reusing
|
||||
# quota-command's adolf-llm:8010/usage route. Activated via
|
||||
# plugins.entries.kimi-quota-footer in openclaw.json.
|
||||
- ./kimi-quota-footer-plugin:/home/node/.openclaw/extensions/kimi-quota-footer:ro
|
||||
# todoist-capture plugin (kb#170 component 1) — same read-only-bind-
|
||||
# over-volume pattern as quota-command/hindsight-memory/kimi-quota-
|
||||
# footer above. Registers /idea (native command, zero Kimi calls);
|
||||
# POSTs to agap-mcp's /capture-idea (see agap-mcp/src/server.js +
|
||||
# capture.js) which does the actual bge-m3 classify + Todoist create.
|
||||
# Activated via plugins.entries.todoist-capture in openclaw.json.
|
||||
- ./todoist-capture-plugin:/home/node/.openclaw/extensions/todoist-capture:ro
|
||||
extra_hosts:
|
||||
- "host.docker.internal:host-gateway"
|
||||
# mtx.alogins.net's public A record can't hairpin-NAT back through the
|
||||
@@ -230,6 +303,48 @@ services:
|
||||
volumes:
|
||||
- hindsight-llm-home:/root/.kimi-code
|
||||
restart: unless-stopped
|
||||
# kb#190: GET /v1/models is a static, no-inference route (see
|
||||
# hindsight-llm/server.js) -- cheap liveness probe.
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "node -e \"fetch('http://localhost:8012/v1/models').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))\""]
|
||||
interval: 15s
|
||||
timeout: 10s
|
||||
retries: 5
|
||||
start_period: 20s
|
||||
|
||||
# tei-reranker — GPU cross-encoder rerank sidecar for Hindsight (kb#87).
|
||||
# Hindsight's recall reranker ran the multilingual jina-reranker-v2 on the
|
||||
# image's CPU-only torch; over the grown adolf bank (269 facts, ~81 rerank
|
||||
# candidates) a single recall pinned ~8 cores for ~183s, so the memory
|
||||
# plugin's 4s timeout skipped injection every time. The stock HF TEI GPU
|
||||
# image needs CUDA sm_75+; this box is a GTX 1070 (Pascal sm_61), so we serve
|
||||
# the SAME jina model via plain CUDA torch (Pascal-compatible) behind the
|
||||
# TEI-compatible /info + /rerank API that Hindsight's `tei` provider speaks.
|
||||
# Shares the GPU with ollama (~1GB fp16 here, ~5.6GB ollama peak, 8GB card).
|
||||
# Reuses the already-downloaded model from hindsight's HF cache (no re-DL).
|
||||
tei-reranker:
|
||||
build: ./tei-reranker
|
||||
container_name: tei-reranker
|
||||
runtime: nvidia
|
||||
environment:
|
||||
- NVIDIA_VISIBLE_DEVICES=all
|
||||
- NVIDIA_DRIVER_CAPABILITIES=compute,utility
|
||||
- RERANKER_MODEL=jinaai/jina-reranker-v2-base-multilingual
|
||||
- RERANKER_DEVICE=cuda
|
||||
- HF_HOME=/root/.cache/huggingface
|
||||
volumes:
|
||||
- /mnt/ssd/dbs/hindsight-cache/huggingface:/root/.cache/huggingface
|
||||
ports:
|
||||
- "8014:80"
|
||||
restart: unless-stopped
|
||||
# kb#190: /info is TEI's own lightweight metadata endpoint (model name,
|
||||
# no rerank/inference call). Container has python3 only (no curl/wget).
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "python3 -c \"import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://localhost:80/info',timeout=3).status==200 else 1)\""]
|
||||
interval: 15s
|
||||
timeout: 10s
|
||||
retries: 5
|
||||
start_period: 30s
|
||||
|
||||
# adolf-llm — conversational Kimi-CLI wrapper (:8010), the model backend for
|
||||
# the Adolf OpenClaw gateway (P2). Real streaming (SSE), chat_id session-keying
|
||||
@@ -239,6 +354,23 @@ services:
|
||||
adolf-llm:
|
||||
build: ./adolf-llm
|
||||
container_name: adolf-llm
|
||||
environment:
|
||||
# marketplace-mcp bearer token (kb#61) -- shared-mcp.json's
|
||||
# "marketplace" entry references this by name via
|
||||
# `bearerTokenEnvVar: "MARKETPLACE_MCP_TOKEN"` (Kimi CLI's own field
|
||||
# for a static bearer token sourced from the environment, confirmed by
|
||||
# decompiling @moonshot-ai/kimi-code's dist/main.mjs help text). Kimi
|
||||
# reads process.env at request time, so the raw secret never sits in
|
||||
# the git-tracked shared-mcp.json -- same secret, same env-var pattern
|
||||
# already used for the `adolf` service's openclaw.json Layer-1 config
|
||||
# above (${MARKETPLACE_MCP_TOKEN} substitution), sourced from
|
||||
# openai/.env (gitignored, never committed).
|
||||
- MARKETPLACE_MCP_TOKEN=${MARKETPLACE_MCP_TOKEN:-}
|
||||
# agap-mcp bearer token (kb#180) -- same env-var pattern, referenced by
|
||||
# shared-mcp.json's "agap" entry via `bearerTokenEnvVar:
|
||||
# "AGAP_MCP_TOKEN"`. Without it the Kimi backbone's agap tools all
|
||||
# fail with HTTP 401 once agap-mcp restarts with auth on.
|
||||
- AGAP_MCP_TOKEN=${AGAP_MCP_TOKEN:-}
|
||||
ports:
|
||||
- "8010:8010"
|
||||
volumes:
|
||||
@@ -257,6 +389,14 @@ services:
|
||||
- "family.alogins.net:host-gateway"
|
||||
- "wiki.alogins.net:host-gateway"
|
||||
restart: unless-stopped
|
||||
# kb#190: GET /v1/models is a static, no-inference route (see
|
||||
# adolf-llm/server.js) -- cheap liveness probe, no Kimi call/quota use.
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "node -e \"fetch('http://localhost:8010/v1/models').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))\""]
|
||||
interval: 15s
|
||||
timeout: 10s
|
||||
retries: 5
|
||||
start_period: 20s
|
||||
|
||||
# hindsight — Adolf memory backend, replacing cognee/cognee-mcp/cognee-llm
|
||||
# (kb#73, migration doc agap_git/adolf/HINDSIGHT-MIGRATION.md, H1). One
|
||||
@@ -339,14 +479,15 @@ services:
|
||||
# in-flight async retain/consolidation tasks under the old hostname
|
||||
# (startup log warns about exactly this).
|
||||
- HINDSIGHT_API_WORKER_ID=hindsight
|
||||
# Reranker -> multilingual (kb#84 follow-up). The TEMPR rerank stage
|
||||
# defaulted to English cross-encoder/ms-marco-MiniLM, which ranks
|
||||
# Russian/multilingual candidates poorly. jina v2 multilingual fixes
|
||||
# that. Runs on CPU in this image (no CUDA torch) but only over the
|
||||
# small recall candidate set. trust_remote_code: jina ships custom code.
|
||||
- HINDSIGHT_API_RERANKER_PROVIDER=local
|
||||
- HINDSIGHT_API_RERANKER_LOCAL_MODEL=jinaai/jina-reranker-v2-base-multilingual
|
||||
- HINDSIGHT_API_RERANKER_LOCAL_TRUST_REMOTE_CODE=true
|
||||
# Reranker -> TEI GPU sidecar (kb#87). Was `local` = the same
|
||||
# multilingual jina-reranker-v2, but on this image's CPU-only torch it
|
||||
# pinned ~8 cores for ~183s over the grown adolf bank (269 facts / ~81
|
||||
# rerank candidates), so the memory plugin's 4s recall timeout skipped
|
||||
# injection every time. Now the identical jina model is served on the
|
||||
# GPU by the tei-reranker sidecar behind the TEI /rerank API.
|
||||
- HINDSIGHT_API_RERANKER_PROVIDER=tei
|
||||
- HINDSIGHT_API_RERANKER_TEI_URL=http://tei-reranker:80
|
||||
- HINDSIGHT_API_RERANKER_TEI_HTTP_TIMEOUT=60
|
||||
volumes:
|
||||
- /mnt/ssd/dbs/hindsight:/home/hindsight/.pg0
|
||||
# Persist HuggingFace/sentence-transformers model cache so the jina
|
||||
@@ -362,7 +503,29 @@ services:
|
||||
# file.
|
||||
- "host.docker.internal:host-gateway"
|
||||
depends_on:
|
||||
- hindsight-llm
|
||||
# kb#217: litellm is now on the critical path for all three LLM stages
|
||||
# (HINDSIGHT_API_*_LLM_BASE_URL above all point at litellm:4000) since
|
||||
# the 2026-07-26 gemma3:4b re-route (59af13f); gate on its healthcheck
|
||||
# (added by kb#190) so a cold boot doesn't race hindsight up before it.
|
||||
litellm:
|
||||
condition: service_healthy
|
||||
# hindsight-llm dropped (kb#217): it was the Kimi-CLI wrapper that used
|
||||
# to serve retain before the re-route above; nothing in this service's
|
||||
# config points at hindsight-llm:8012 any more (grep confirms only
|
||||
# model-registry.yaml still lists it, unrelated to this container's
|
||||
# startup). The hindsight-llm service/volume are left in place — that's
|
||||
# a separate decommission decision, not this task's scope.
|
||||
tei-reranker:
|
||||
condition: service_healthy
|
||||
# kb#190: /health is hindsight's own liveness+DB-connectivity endpoint
|
||||
# (returns {"status":"healthy","database":"connected"}), confirmed cheap
|
||||
# (curl is present in this image).
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "curl -sf http://localhost:8888/health || exit 1"]
|
||||
interval: 15s
|
||||
timeout: 10s
|
||||
retries: 5
|
||||
start_period: 30s
|
||||
|
||||
# openclaw-tools — MCP bridge (P5) exposing a minimal slice of the Adolf
|
||||
# OpenClaw gateway's agent tools (message/cron/nodes/browser) over MCP
|
||||
|
||||
@@ -10,6 +10,17 @@ model_list:
|
||||
model: ollama/nomic-embed-text
|
||||
api_base: http://host.docker.internal:11434
|
||||
|
||||
# kb#164: the ACTUAL embedder in use (§3a routing classifier for Auto
|
||||
# Router v2) is bge-m3 on 11436, not the `embedder` alias above (which
|
||||
# still points at nomic-embed-text on 11434 -- that alias is legacy/
|
||||
# unused by the current stack, left as-is per kb#164 scope: add bge-m3,
|
||||
# don't touch the mismatch beyond noting it). model-registry.yaml's
|
||||
# `bge-m3` entry's litellm_model_name now matches this model_name.
|
||||
- model_name: bge-m3
|
||||
litellm_params:
|
||||
model: ollama/bge-m3
|
||||
api_base: http://host.docker.internal:11436
|
||||
|
||||
- model_name: judge
|
||||
litellm_params:
|
||||
model: anthropic/claude-haiku-4-5-20251001
|
||||
@@ -119,9 +130,98 @@ model_list:
|
||||
model: openrouter/nousresearch/hermes-3-llama-3.1-405b:free
|
||||
api_key: os.environ/OPENROUTER_API_KEY
|
||||
|
||||
# ── kb#128 (A2A-16): tier pools — alvis's "tier" routing mode ───────────
|
||||
# target = constraint-set ("any large model"), not a specific backbone.
|
||||
# Two litellm_params entries sharing one model_name = a LiteLLM deployment
|
||||
# group; the router load-balances/fails-over across them. tier-large lists
|
||||
# kimi-agent FIRST so it's preferred, with local-small as the in-group
|
||||
# failover partner -- this is also what the fallbacks: block below promotes
|
||||
# to an explicit, auditable Kimi-429-degrades-to-local path (design §2
|
||||
# theorem 2: quota-gated a(t)=0 -> park/degrade, never fail).
|
||||
# tier-small mirrors model-registry.yaml's routing.tiers.small = [local-small].
|
||||
- model_name: tier-small
|
||||
litellm_params:
|
||||
model: ollama/gemma3:4b
|
||||
api_base: http://host.docker.internal:11436
|
||||
|
||||
- model_name: tier-large
|
||||
litellm_params:
|
||||
model: openai/kimi-agent
|
||||
api_base: http://kimi-agent:8000/v1
|
||||
api_key: dummy
|
||||
|
||||
# ── kb#128: Auto Router v2 -- embedding-based classification on the LOCAL
|
||||
# bge-m3 (design §3a/§3b: no classifier LLM, no API spend). Human-readable
|
||||
# source of truth for these routes: openai/auto-router-routes.json (keep
|
||||
# both in sync by hand -- see that file's _note for why).
|
||||
#
|
||||
# auto_router_config is INLINE JSON, not auto_router_config_path. This is
|
||||
# the open Auto Router v2 embedding bug the task brief warned about,
|
||||
# verified hands-on 2026-07-26 against litellm:main-latest: the _path
|
||||
# loader (AutoRouter -> SemanticRouter.from_json) unconditionally builds a
|
||||
# throwaway semantic_router encoder from scratch and demands a real
|
||||
# provider API key even for a local model name like "bge-m3" --
|
||||
# ValueError: "Expected API key via `api_key` parameter or
|
||||
# `{TYPE}_API_KEY` environment variable." The inline-string loader never
|
||||
# touches that code path (it just reads the `routes` key), and was
|
||||
# confirmed end-to-end: real `litellm.embedding(model=ollama/bge-m3)`
|
||||
# calls, zero metered spend, "hi there" -> ollama/gemma3:4b, a refactor/
|
||||
# dependency-injection prompt -> kimi-agent.
|
||||
#
|
||||
# default_model is the free local tier -- an unmatched/low-confidence
|
||||
# request degrades to free compute, never to a paid model.
|
||||
- model_name: auto_router
|
||||
litellm_params:
|
||||
model: auto_router/semantic-v1
|
||||
auto_router_default_model: ollama/gemma3:4b
|
||||
auto_router_embedding_model: bge-m3
|
||||
auto_router_config: >
|
||||
{"routes": [
|
||||
{"name": "ollama/gemma3:4b", "description": "Simple, short, low-stakes requests -- greetings, quick factual lookups, formatting, one-line questions.",
|
||||
"utterances": ["hi", "hello", "what time is it", "what's the weather", "thanks", "what does this word mean", "summarize this in one sentence", "give me a quick yes or no", "format this as a list", "what is 2 plus 2"],
|
||||
"score_threshold": 0.5},
|
||||
{"name": "kimi-agent", "description": "Complex reasoning, multi-step planning, coding, or anything needing tool use and deep context.",
|
||||
"utterances": ["write a function that parses this log file and extracts errors", "refactor this class to use dependency injection", "think through the tradeoffs of these two architectures step by step", "debug why this docker container keeps crashing", "plan out the migration from cognee to hindsight across five tasks", "analyze this design document and find inconsistencies", "write a SQL query that joins these three tables and aggregates by month", "review this pull request for security issues"],
|
||||
"score_threshold": 0.5}
|
||||
]}
|
||||
|
||||
# ── kb#128: heuristic keyword/length fallback classifier ────────────────
|
||||
# Auto Router v2 (2026-07-14) has an open embedding-related bug report
|
||||
# (task #128 brief) -- LiteLLM's built-in ComplexityRouter is exactly the
|
||||
# "keyword/length heuristic" fallback the brief calls for: pure regex/
|
||||
# token-count scoring, <1ms, ZERO external calls (verified hands-on by
|
||||
# reading router_strategy/complexity_router/complexity_router.py in the
|
||||
# running litellm:main-latest image, 2026-07-26). Tiers are overridden
|
||||
# here -- the package DEFAULT tiers point at gpt-4o/gpt-4o-mini/claude-
|
||||
# sonnet (metered!), which would silently violate §3a if left as-is; every
|
||||
# tier below maps only to already-governed non-metered deployments.
|
||||
- model_name: complexity_router
|
||||
litellm_params:
|
||||
model: auto_router/complexity_router
|
||||
complexity_router_default_model: ollama/gemma3:4b
|
||||
complexity_router_config:
|
||||
tiers:
|
||||
SIMPLE: ollama/gemma3:4b
|
||||
MEDIUM: ollama/gemma3:4b
|
||||
COMPLEX: tier-large
|
||||
REASONING: tier-large
|
||||
|
||||
litellm_settings:
|
||||
success_callback: ["langfuse"]
|
||||
failure_callback: ["langfuse"]
|
||||
drop_params: true
|
||||
fallbacks:
|
||||
- deepseek/deepseek-r1:free: ["ollama/qwen3.5:4b"]
|
||||
# kb#128 acceptance: "a forced Kimi 429 degrades cleanly". kimi-agent is
|
||||
# the only Kimi deployment actually routed through LiteLLM today (the
|
||||
# `kimi` model-registry id is called directly via the adolf-llm/
|
||||
# hindsight-llm wrappers, outside LiteLLM by design -- see model-
|
||||
# registry.yaml's kimi entry). Both the raw deployment and the tier-large
|
||||
# pool degrade to the free local-small model on 429/quota-exhaustion
|
||||
# rather than failing the caller.
|
||||
- kimi-agent: ["ollama/gemma3:4b"]
|
||||
- tier-large: ["tier-small"]
|
||||
# auto_router's embedding path is the one with the open bug report
|
||||
# (design §3a) -- if it errors, fail over to the zero-API-call heuristic
|
||||
# classifier rather than the caller seeing an error.
|
||||
- auto_router: ["complexity_router"]
|
||||
|
||||
@@ -15,6 +15,51 @@
|
||||
# (see routing.metered_opt_in: [] at the bottom — empty means unreachable).
|
||||
#
|
||||
# Read with model_registry.py (same directory): resolve(), preload_check().
|
||||
#
|
||||
# ── Coverage vs litellm-config.yaml (kb#195, 2026-07-26 audit) ──────────
|
||||
# Every model_name litellm-config.yaml defines must appear either as a
|
||||
# `litellm_model_name` below or in this exclusion list. litellm_key_spec()
|
||||
# default-denies anything not reachable via routing.tiers, so an excluded
|
||||
# model stays ungoverned-but-inert until someone wires it up (add it here
|
||||
# and to routing.tiers first).
|
||||
#
|
||||
# GOVERNED (present below):
|
||||
# - ollama/gemma3:4b -> id: local-small (hot path: Hindsight LLM/
|
||||
# consolidation/reflect all route here as of 2026-07-26)
|
||||
# - judge -> id: paid-fallback (metered; see kb#164 for the fact that
|
||||
# the no-metered-API constraint has no runtime enforcement yet)
|
||||
# - kimi-agent -> id: kimi-agent (own container, live; see below)
|
||||
# - bge-m3 -> id: bge-m3 (kb#164, 2026-07-26: wired into litellm-config
|
||||
# .yaml pointing at ollama on 11436, the real embedder/routing
|
||||
# classifier; litellm_model_name below updated from null to "bge-m3")
|
||||
#
|
||||
# INTENTIONAL EXCLUSIONS (not governed by this registry, by design):
|
||||
# - tip-generator (ollama/qwen2.5:1.5b), embedder (ollama/nomic-embed-
|
||||
# text): aliases consumed by the separate oO ml/serving project, not
|
||||
# the a2a fabric. Tracked in oO/CLAUDE.md, not duplicated here.
|
||||
# - Raw ollama/* passthrough exposures — ollama/qwen3.5:4b,
|
||||
# ollama/qwen3:8b, ollama/qwen2.5:1.5b, ollama/qwen2.5:0.5b,
|
||||
# ollama/gemma3:1b, ollama/nomic-embed-text — manual/dev-console
|
||||
# access to the ollama instances for ad-hoc testing. No agent or
|
||||
# fabric workflow is registered against them (grepped agent-registry
|
||||
# .yaml and openai/*.py: no hits). Not in routing.tiers, so
|
||||
# litellm_key_spec() grants no agent access to them either way.
|
||||
# If one of these becomes a real dependency (as ollama/gemma3:4b
|
||||
# did), give it its own registry entry at that point.
|
||||
# - The 12 OpenRouter `*:free` models (meta-llama/llama-3.3-70b-
|
||||
# instruct:free, meta-llama/llama-3.2-3b-instruct:free, deepseek/
|
||||
# deepseek-r1:free, qwen/qwen3-4b:free, qwen/qwen3-coder:free,
|
||||
# google/gemma-3-27b-it:free, google/gemma-3-12b-it:free, mistralai/
|
||||
# mistral-small-3.1-24b-instruct:free, nvidia/nemotron-3-super-
|
||||
# 120b-a12b:free, openai/gpt-oss-120b:free, minimax/minimax-m2.5:free,
|
||||
# nousresearch/hermes-3-llama-3.1-405b:free) — human-facing manual-
|
||||
# selection models (e.g.
|
||||
# via Open WebUI), outside the agent fabric's model plane. Not
|
||||
# referenced by any agent registry entry, not in routing.tiers, so
|
||||
# resolve()/litellm_key_spec() never route an agent to them. Free
|
||||
# tier, so this is not the kb#164 metered-enforcement gap — flag
|
||||
# for a proper entry only if an agent workflow starts depending on
|
||||
# one of these.
|
||||
|
||||
schema_version: 1
|
||||
|
||||
@@ -54,6 +99,34 @@ models:
|
||||
metered: false
|
||||
opt_in_required: false
|
||||
|
||||
# ── kimi-agent — own container, oO-adjacent Kimi CLI wrapper ───────────
|
||||
# Distinct from `kimi` above: this is a third Kimi-CLI container
|
||||
# (openai/kimi-agent/, own Moonshot/Kimi subscription via `kimi login`,
|
||||
# own docker-compose service `kimi-agent`) that IS routed through
|
||||
# LiteLLM today (litellm-config.yaml model_name: kimi-agent ->
|
||||
# openai/kimi-agent -> http://kimi-agent:8000/v1). Documented here per
|
||||
# kb#195 coverage audit; deliberately NOT added to routing.tiers in this
|
||||
# pass (that would change litellm_key_spec() grants, out of scope for a
|
||||
# docs-alignment task) — no agent is currently opted into it.
|
||||
- id: kimi-agent
|
||||
role: "Kimi-CLI wrapper, own container (openai/kimi-agent/) — purpose/consumer not yet documented outside this registry"
|
||||
litellm_model_name: "kimi-agent" # openai/litellm-config.yaml model_list entry
|
||||
endpoints:
|
||||
- name: kimi-agent
|
||||
url: "http://kimi-agent:8000/v1"
|
||||
tier: large
|
||||
context_tokens: 200000 # same Moonshot Kimi K2 CLI as `kimi`; re-verify if the CLI's pinned model changes
|
||||
tool_use_quality: high
|
||||
lifecycle: quota-gated
|
||||
quota:
|
||||
probe_command: null # not yet wired to a probe; own subscription, same caveat as `kimi`
|
||||
windows: []
|
||||
threshold_pct: null
|
||||
gpu_residency: null
|
||||
cost_class: subscription
|
||||
metered: false
|
||||
opt_in_required: false
|
||||
|
||||
# ── local-small — the cheap tier ───────────────────────────────────────
|
||||
# ollama/gemma3:4b on the GPU ollama instance. Already the live model for
|
||||
# Hindsight consolidation/reflect (HINDSIGHT_API_CONSOLIDATION_LLM_MODEL /
|
||||
@@ -86,7 +159,7 @@ models:
|
||||
# routing at once.
|
||||
- id: bge-m3
|
||||
role: "embedder — also the routing classifier (§3a, LiteLLM Auto Router / semantic-router)"
|
||||
litellm_model_name: null # NOT YET wired into litellm-config.yaml — gap, see model_registry.py module docstring
|
||||
litellm_model_name: "bge-m3" # kb#164, 2026-07-26: wired into litellm-config.yaml (ollama/bge-m3 @ 11436) -- was null (unwired gap)
|
||||
endpoints:
|
||||
- name: ollama-direct
|
||||
url: "http://host.docker.internal:11436"
|
||||
|
||||
108
openai/provision_litellm_keys.py
Executable file
108
openai/provision_litellm_keys.py
Executable file
@@ -0,0 +1,108 @@
|
||||
#!/usr/bin/env python3
|
||||
"""provision_litellm_keys — kb#147 (A2A-15): turn agent-registry.yaml grants
|
||||
into real LiteLLM virtual keys.
|
||||
|
||||
This is the ONE place a capability grant (model allow-list + budget) crosses
|
||||
from data (agent-registry.yaml, version-controlled) into a live LiteLLM
|
||||
key (via the proxy's /key/generate or /key/update admin API, master-key
|
||||
authenticated). It deliberately does nothing destructive: --dry-run (the
|
||||
default) only computes and prints the payload each agent WOULD get, making
|
||||
zero network calls. --apply is required to actually create/update a key,
|
||||
and needs LITELLM_MASTER_KEY in the environment (never hardcoded here, never
|
||||
committed) — this is a privileged write against a live production service,
|
||||
so it is not something this task runs unattended; --apply is the kb#147
|
||||
handover step for a human/approved run.
|
||||
|
||||
Usage:
|
||||
# Safe, run-anytime: print what each agent's key WOULD look like.
|
||||
./provision_litellm_keys.py --dry-run
|
||||
./provision_litellm_keys.py --dry-run --id torgash
|
||||
|
||||
# Privileged, requires explicit opt-in + master key (kb#147 handover):
|
||||
LITELLM_MASTER_KEY=sk-... ./provision_litellm_keys.py --apply --id adolf
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
import agent_registry as ar
|
||||
import model_registry as mr
|
||||
|
||||
LITELLM_BASE_URL = os.environ.get("LITELLM_BASE_URL", "http://localhost:4000")
|
||||
|
||||
|
||||
def agent_ids_with_grants(registry):
|
||||
return [a["id"] for a in registry["agents"] if a.get("capability_grant")]
|
||||
|
||||
|
||||
def _http_post(path, payload, master_key):
|
||||
req = urllib.request.Request(
|
||||
f"{LITELLM_BASE_URL}{path}",
|
||||
data=json.dumps(payload).encode(),
|
||||
headers={
|
||||
"Authorization": f"Bearer {master_key}",
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
method="POST",
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout=10) as resp:
|
||||
return json.loads(resp.read())
|
||||
|
||||
|
||||
def apply_key(spec, master_key):
|
||||
"""Create (or update, if key_alias already exists) a LiteLLM virtual key
|
||||
matching `spec` (the dict returned by agent_registry.litellm_key_spec).
|
||||
Raises on any HTTP error rather than swallowing it — a failed grant
|
||||
should never look like a successful one."""
|
||||
payload = {
|
||||
"key_alias": spec["key_alias"],
|
||||
"models": spec["models"],
|
||||
"max_budget": spec["max_budget"],
|
||||
"budget_duration": spec["budget_duration"],
|
||||
"metadata": {"agent_id": spec["agent_id"], "trust_class": spec["trust_class"], "source": "kb#147 agent-registry.yaml"},
|
||||
}
|
||||
try:
|
||||
return _http_post("/key/generate", payload, master_key)
|
||||
except urllib.error.HTTPError as e:
|
||||
body = e.read().decode(errors="replace")
|
||||
raise SystemExit(f"LiteLLM /key/generate failed for {spec['key_alias']}: {e.code} {body}")
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("--registry", default=None)
|
||||
ap.add_argument("--model-registry", default=None)
|
||||
ap.add_argument("--id", default=None, help="only this agent id (default: every agent with a capability_grant)")
|
||||
mode = ap.add_mutually_exclusive_group()
|
||||
mode.add_argument("--dry-run", action="store_true", default=True, help="default: compute + print only, no network call")
|
||||
mode.add_argument("--apply", action="store_true", help="actually call LiteLLM /key/generate (needs LITELLM_MASTER_KEY) -- privileged, kb#147 handover step")
|
||||
args = ap.parse_args()
|
||||
|
||||
reg = ar.load_registry(args.registry)
|
||||
model_reg = mr.load_registry(args.model_registry)
|
||||
|
||||
ids = [args.id] if args.id else agent_ids_with_grants(reg)
|
||||
if not ids:
|
||||
print("no agents with a capability_grant in the registry", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
master_key = os.environ.get("LITELLM_MASTER_KEY")
|
||||
if args.apply and not master_key:
|
||||
print("error: --apply requires LITELLM_MASTER_KEY in the environment", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
for agent_id in ids:
|
||||
spec = ar.litellm_key_spec(reg, agent_id, model_reg)
|
||||
if args.apply:
|
||||
result = apply_key(spec, master_key)
|
||||
print(json.dumps({"agent_id": agent_id, "key_alias": spec["key_alias"], "applied": True, "litellm_response_keys": list(result.keys())}))
|
||||
else:
|
||||
print(json.dumps({"mode": "dry-run", **spec}, indent=2))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -2,7 +2,7 @@
|
||||
"mcpServers": {
|
||||
"hindsight": {
|
||||
"type": "http",
|
||||
"url": "http://hindsight:8888/mcp/adolf/",
|
||||
"url": "http://hindsight:8888/mcp/adolf-shared/",
|
||||
"enabledTools": ["recall", "retain", "reflect", "list_memories", "get_memory", "update_memory", "list_directives", "create_directive", "delete_directive"]
|
||||
},
|
||||
"openclaw-tools": {
|
||||
@@ -17,7 +17,8 @@
|
||||
"agap": {
|
||||
"type": "http",
|
||||
"url": "http://host.docker.internal:3100/mcp",
|
||||
"enabledTools": ["vw_get_password", "vw_get_item", "vw_list_items", "vw_create_login", "vw_update_password", "ha_get_state", "ha_list_entities", "ha_call_service", "ha_get_history", "zabbix_get_problems", "zabbix_get_hosts", "zabbix_get_items", "zabbix_get_triggers", "radicale_list_calendars", "radicale_list_events", "radicale_get_event", "radicale_put_event", "radicale_delete_event", "radicale_move_event", "todoist_list_tasks", "todoist_list_projects", "todoist_create_task", "todoist_update_task", "todoist_complete_task", "wiki_search", "wiki_read", "wiki_edit"]
|
||||
"bearerTokenEnvVar": "AGAP_MCP_TOKEN",
|
||||
"enabledTools": ["vw_get_password", "vw_get_item", "vw_list_items", "vw_create_login", "vw_update_password", "ha_get_state", "ha_list_entities", "ha_call_service", "ha_get_history", "zabbix_get_problems", "zabbix_get_hosts", "zabbix_get_items", "zabbix_get_triggers", "radicale_list_calendars", "radicale_list_events", "radicale_get_event", "radicale_put_event", "radicale_delete_event", "radicale_move_event", "todoist_list_tasks", "todoist_list_projects", "todoist_create_task", "todoist_update_task", "todoist_complete_task", "todoist_capture_idea", "wiki_search", "wiki_read", "wiki_edit"]
|
||||
},
|
||||
"marketplace": {
|
||||
"type": "http",
|
||||
|
||||
13
openai/tei-reranker/Dockerfile
Normal file
13
openai/tei-reranker/Dockerfile
Normal file
@@ -0,0 +1,13 @@
|
||||
# CUDA torch base with Pascal (sm_61) support — cu118 wheels include sm_61,
|
||||
# so the GTX 1070 works (unlike the stock TEI GPU image, which needs sm_75+).
|
||||
FROM pytorch/pytorch:2.3.1-cuda11.8-cudnn8-runtime
|
||||
|
||||
WORKDIR /app
|
||||
COPY requirements.txt .
|
||||
RUN pip install --no-cache-dir -r requirements.txt
|
||||
|
||||
COPY server.py .
|
||||
|
||||
ENV HF_HOME=/root/.cache/huggingface
|
||||
EXPOSE 80
|
||||
CMD ["uvicorn", "server:app", "--host", "0.0.0.0", "--port", "80"]
|
||||
8
openai/tei-reranker/requirements.txt
Normal file
8
openai/tei-reranker/requirements.txt
Normal file
@@ -0,0 +1,8 @@
|
||||
# torch/cuda come from the pytorch base image. Pin transformers to a version
|
||||
# known-compatible with jina-reranker-v2's custom modeling code.
|
||||
transformers==4.44.2
|
||||
einops>=0.7
|
||||
sentencepiece>=0.1.99
|
||||
protobuf>=3.20
|
||||
fastapi>=0.110
|
||||
uvicorn[standard]>=0.29
|
||||
85
openai/tei-reranker/server.py
Normal file
85
openai/tei-reranker/server.py
Normal file
@@ -0,0 +1,85 @@
|
||||
"""Minimal TEI-compatible cross-encoder rerank server (GPU).
|
||||
|
||||
Why this exists: HuggingFace's official Text-Embeddings-Inference GPU images
|
||||
require CUDA compute capability >= 7.5 (Turing+). This box has a GTX 1070
|
||||
(Pascal, 6.1), so the stock TEI image won't run. Plain CUDA torch DOES support
|
||||
Pascal (that's why ollama works here), so we serve the same
|
||||
`jina-reranker-v2-base-multilingual` cross-encoder via torch and expose only the
|
||||
two endpoints Hindsight's `tei` reranker provider calls:
|
||||
GET /info -> JSON (init/health probe)
|
||||
POST /rerank -> {"query": str, "texts": [str], ...}
|
||||
-> bare list [{"index": i, "score": f}, ...] sorted desc
|
||||
See hindsight_api/engine/cross_encoder.py::RemoteTEICrossEncoder for the client.
|
||||
"""
|
||||
|
||||
import os
|
||||
import torch
|
||||
from fastapi import FastAPI
|
||||
from pydantic import BaseModel
|
||||
from transformers import AutoModelForSequenceClassification
|
||||
|
||||
MODEL_ID = os.environ.get("RERANKER_MODEL", "jinaai/jina-reranker-v2-base-multilingual")
|
||||
DEVICE = os.environ.get("RERANKER_DEVICE", "cuda")
|
||||
MAX_LENGTH = int(os.environ.get("RERANKER_MAX_LENGTH", "1024"))
|
||||
# fp16 on GPU halves the ~1.1GB fp32 footprint; Pascal supports fp16 storage.
|
||||
DTYPE = torch.float16 if DEVICE == "cuda" else torch.float32
|
||||
|
||||
app = FastAPI(title="tei-reranker")
|
||||
_model = None
|
||||
|
||||
|
||||
def _load():
|
||||
global _model
|
||||
if _model is not None:
|
||||
return
|
||||
m = AutoModelForSequenceClassification.from_pretrained(
|
||||
MODEL_ID, torch_dtype=DTYPE, trust_remote_code=True
|
||||
)
|
||||
m.to(DEVICE)
|
||||
m.eval()
|
||||
_model = m
|
||||
|
||||
|
||||
@app.on_event("startup")
|
||||
def startup():
|
||||
_load()
|
||||
|
||||
|
||||
class RerankRequest(BaseModel):
|
||||
query: str
|
||||
texts: list[str]
|
||||
return_text: bool = False
|
||||
truncate: bool | None = None
|
||||
raw_scores: bool | None = None
|
||||
|
||||
|
||||
@app.get("/info")
|
||||
def info():
|
||||
# Hindsight only needs a 200 JSON here to consider the server initialized.
|
||||
return {
|
||||
"model_id": MODEL_ID,
|
||||
"model_dtype": str(DTYPE).replace("torch.", ""),
|
||||
"model_type": {"reranker": {}},
|
||||
"max_input_length": MAX_LENGTH,
|
||||
"device": DEVICE,
|
||||
}
|
||||
|
||||
|
||||
@app.get("/health")
|
||||
def health():
|
||||
return {"status": "ok" if _model is not None else "loading"}
|
||||
|
||||
|
||||
@app.post("/rerank")
|
||||
def rerank(req: RerankRequest):
|
||||
if not req.texts:
|
||||
return []
|
||||
pairs = [[req.query, t] for t in req.texts]
|
||||
with torch.no_grad():
|
||||
# jina-reranker-v2 exposes compute_score (batches + moves to device).
|
||||
scores = _model.compute_score(pairs, max_length=MAX_LENGTH)
|
||||
if not isinstance(scores, list):
|
||||
scores = [scores]
|
||||
results = [{"index": i, "score": float(s)} for i, s in enumerate(scores)]
|
||||
results.sort(key=lambda r: r["score"], reverse=True)
|
||||
return results
|
||||
Reference in New Issue
Block a user