Compare commits
2 Commits
4a9ae75912
...
9094d71e2f
| Author | SHA1 | Date | |
|---|---|---|---|
| 9094d71e2f | |||
| a27bae828a |
3
.gitignore
vendored
3
.gitignore
vendored
@@ -16,3 +16,6 @@ __pycache__/
|
|||||||
# (e.g. docker-compose.yml.bak-20260704-141509, CLAUDE.md.bak-kb).
|
# (e.g. docker-compose.yml.bak-20260704-141509, CLAUDE.md.bak-kb).
|
||||||
*.bak
|
*.bak
|
||||||
*.bak-*
|
*.bak-*
|
||||||
|
|
||||||
|
# contains live LLM + JWT secrets — never commit
|
||||||
|
ai/cognee/cognee.env
|
||||||
|
|||||||
195
RESTORE-RUNBOOK.md
Normal file
195
RESTORE-RUNBOOK.md
Normal file
@@ -0,0 +1,195 @@
|
|||||||
|
# Restore Runbook — Kanboard, Vaultwarden, Seafile
|
||||||
|
|
||||||
|
Companion to each service's `backup.sh`. Covers kb#192.
|
||||||
|
|
||||||
|
Each service now has a `restore.sh` next to its `backup.sh`:
|
||||||
|
- `kanboard/restore.sh`
|
||||||
|
- `vaultwarden/restore.sh`
|
||||||
|
- `seafile/restore.sh`
|
||||||
|
|
||||||
|
All three follow the same convention: they default to the **live** container
|
||||||
|
names/paths, but every target is overridable via env vars, so the exact same
|
||||||
|
script restores into a real disaster or into a disposable/throwaway
|
||||||
|
container for a dry-run test. **Never invoke a restore.sh without env
|
||||||
|
overrides unless you are doing a real, intentional disaster recovery** —
|
||||||
|
the defaults point at production.
|
||||||
|
|
||||||
|
## Before you restore anything
|
||||||
|
|
||||||
|
1. `/mnt/backups/<service>/` is read-only in practice — copy the snapshot
|
||||||
|
you want out to a scratch dir first, don't operate on it in place.
|
||||||
|
2. Confirm you have the right snapshot: `ls /mnt/backups/<service>/` and
|
||||||
|
pick the newest dir, but **check its contents aren't empty** (see the
|
||||||
|
Seafile gotcha below — an empty dump looks like a valid directory).
|
||||||
|
3. Restoring into the live container is destructive and briefly stops the
|
||||||
|
service. Only do this for a real incident, and say so out loud before
|
||||||
|
running it.
|
||||||
|
|
||||||
|
## Kanboard
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cp -r /mnt/backups/kanboard/<snapshot> /tmp/kb-restore
|
||||||
|
# Real disaster recovery (overwrites the live "kanboard" container):
|
||||||
|
cd /home/alvis/agap_git/kanboard
|
||||||
|
./restore.sh /tmp/kb-restore
|
||||||
|
```
|
||||||
|
|
||||||
|
What it does: stops the `kanboard` container, replaces `db.sqlite` via
|
||||||
|
`docker cp`, restores `plugins.tar.gz` if present, restarts, then verifies
|
||||||
|
by querying `SELECT COUNT(*) FROM tasks` through the container's PHP PDO
|
||||||
|
sqlite driver (kanboard's image has no sqlite3 CLI).
|
||||||
|
|
||||||
|
Throwaway test (no live impact — no ports published, isolated volumes):
|
||||||
|
```bash
|
||||||
|
docker volume create kb_restore_test_data
|
||||||
|
docker volume create kb_restore_test_plugins
|
||||||
|
docker run -d --name kanboard-restore-test \
|
||||||
|
-v kb_restore_test_data:/var/www/app/data \
|
||||||
|
-v kb_restore_test_plugins:/var/www/app/plugins \
|
||||||
|
-e PLUGIN_INSTALLER=true kanboard/kanboard:latest
|
||||||
|
|
||||||
|
CONTAINER=kanboard-restore-test ./restore.sh /tmp/kb-restore
|
||||||
|
|
||||||
|
# Teardown
|
||||||
|
docker rm -f kanboard-restore-test
|
||||||
|
docker volume rm kb_restore_test_data kb_restore_test_plugins
|
||||||
|
```
|
||||||
|
|
||||||
|
**Verified 2026-07-30**: restored the 2026-07-28 03:00 snapshot into a
|
||||||
|
throwaway container this way. `tasks` table came back with 180 rows;
|
||||||
|
container started healthy. Throwaway container and volumes torn down after.
|
||||||
|
|
||||||
|
## Vaultwarden
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cp -r /mnt/backups/vaultwarden/<snapshot> /tmp/vw-restore
|
||||||
|
# Real disaster recovery (overwrites /mnt/ssd/dbs/vw-data and the live
|
||||||
|
# "vaultwarden" container — needs root; the data dir is root-owned):
|
||||||
|
cd /home/alvis/agap_git/vaultwarden
|
||||||
|
sudo ./restore.sh /tmp/vw-restore
|
||||||
|
```
|
||||||
|
|
||||||
|
What it does: stops the `vaultwarden` container, copies the `db_*.sqlite3`
|
||||||
|
snapshot to `db.sqlite3`, restores `config.json`, `rsa_key*`,
|
||||||
|
`attachments/`, `sends/`, restarts, then checks the DB file is in place
|
||||||
|
(the vaultwarden image has no `sqlite3` CLI either — verification falls
|
||||||
|
back to a file-presence/size check and reading the startup log for a
|
||||||
|
clean launch with no re-keying).
|
||||||
|
|
||||||
|
Throwaway test (isolated data dir, isolated container, no ports published):
|
||||||
|
```bash
|
||||||
|
mkdir -p /tmp/vw-data-test
|
||||||
|
docker run -d --name vaultwarden-restore-test --user 1000:1000 \
|
||||||
|
-v /tmp/vw-data-test:/data vaultwarden/server:latest
|
||||||
|
|
||||||
|
CONTAINER=vaultwarden-restore-test DATA_DIR=/tmp/vw-data-test \
|
||||||
|
./restore.sh /tmp/vw-restore
|
||||||
|
|
||||||
|
# Teardown
|
||||||
|
docker rm -f vaultwarden-restore-test
|
||||||
|
rm -rf /tmp/vw-data-test
|
||||||
|
```
|
||||||
|
Note: `--user 1000:1000` is only needed for the throwaway test so the bind
|
||||||
|
mount is writable by a non-root operator; the live container runs as root
|
||||||
|
and the live data dir is root-owned, so a real restore needs `sudo`.
|
||||||
|
|
||||||
|
**Verified 2026-07-30**: restored the 2026-07-28 02:00 snapshot into a
|
||||||
|
throwaway container this way. `db.sqlite3` landed at the correct size,
|
||||||
|
`config.json` was picked up ("Using saved config from `data/config.json`"
|
||||||
|
in the startup log), and the RSA key was reused rather than regenerated
|
||||||
|
(no "Private key created" line on the post-restore boot) — i.e. structural
|
||||||
|
restore confirmed. Row-level vault content was not inspected (per
|
||||||
|
Vaultwarden-handling rules — never surface real vault contents). Throwaway
|
||||||
|
container and scratch dir removed after.
|
||||||
|
|
||||||
|
## Seafile
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cp -r /mnt/backups/seafile/<snapshot> /tmp/sf-restore
|
||||||
|
# Real disaster recovery (drops+reloads ccnet_db/seafile_db/seahub_db in
|
||||||
|
# the live "seafile-mysql" container, and rsyncs the data dir back into
|
||||||
|
# /mnt/misc/seafile, stopping/starting the "seafile" container around it —
|
||||||
|
# needs root; data dir is root-owned):
|
||||||
|
cd /home/alvis/agap_git/seafile
|
||||||
|
sudo ./restore.sh /tmp/sf-restore
|
||||||
|
```
|
||||||
|
|
||||||
|
What it does: refuses to run if any of the three `*.sql` dumps in the
|
||||||
|
snapshot is missing/empty (see gotcha below), then for each of
|
||||||
|
`ccnet_db`/`seafile_db`/`seahub_db`: `DROP DATABASE IF EXISTS` +
|
||||||
|
`CREATE DATABASE` + reload from the dump, and reports table counts as a
|
||||||
|
sanity check. If the snapshot has a `data/` dir and `SEAFILE_CONTAINER` is
|
||||||
|
set (default), it also stops the seafile app container, rsyncs `data/`
|
||||||
|
into `DATA_DIR`, and restarts it.
|
||||||
|
|
||||||
|
**Env-var gotcha fixed during this task**: `SEAFILE_CONTAINER=""` used to
|
||||||
|
fall back to the live container name, because `${VAR:-default}` treats an
|
||||||
|
empty string as unset. It's now `${VAR-default}` so an explicit empty
|
||||||
|
string really means "skip the data-dir step." If you want a DB-only
|
||||||
|
restore, pass `SEAFILE_CONTAINER=""` explicitly.
|
||||||
|
|
||||||
|
Throwaway test (DB-only, fully isolated MariaDB container + scratch data dir):
|
||||||
|
```bash
|
||||||
|
docker volume create sf_restore_test_db
|
||||||
|
docker run -d --name seafile-mysql-restore-test \
|
||||||
|
-e MYSQL_ROOT_PASSWORD=<test-only-pw> \
|
||||||
|
-e MYSQL_USER=seafile -e MYSQL_PASSWORD=<matches SEAFILE_MYSQL_DB_PASSWORD> \
|
||||||
|
-e MYSQL_DATABASE=placeholder \
|
||||||
|
-v sf_restore_test_db:/var/lib/mysql mariadb:10.11
|
||||||
|
# grant seafile broad perms on this throwaway instance only:
|
||||||
|
docker exec seafile-mysql-restore-test mysql -u root -p<test-only-pw> \
|
||||||
|
-e "GRANT ALL PRIVILEGES ON *.* TO 'seafile'@'%'; FLUSH PRIVILEGES;"
|
||||||
|
|
||||||
|
mkdir -p /tmp/sf-data-test
|
||||||
|
MYSQL_CONTAINER=seafile-mysql-restore-test \
|
||||||
|
SEAFILE_CONTAINER=seafile-app-does-not-exist-test \
|
||||||
|
DATA_DIR=/tmp/sf-data-test \
|
||||||
|
./restore.sh /tmp/sf-restore
|
||||||
|
|
||||||
|
# Teardown
|
||||||
|
docker rm -f seafile-mysql-restore-test
|
||||||
|
docker volume rm sf_restore_test_db
|
||||||
|
rm -rf /tmp/sf-data-test
|
||||||
|
```
|
||||||
|
|
||||||
|
**Verified 2026-07-30 — partially**: DB restore was verified end-to-end
|
||||||
|
into a throwaway MariaDB container using the last known-good snapshot
|
||||||
|
(2026-07-04 02:00 — see the critical finding below): `ccnet_db` (13
|
||||||
|
tables), `seafile_db` (46 tables), `seahub_db` (127 tables) all restored
|
||||||
|
and importable. The `data/` directory step ran cleanly against an isolated
|
||||||
|
scratch dir (`/tmp/sf-data-test`, not `/mnt/misc/seafile`) — the on-disk
|
||||||
|
tree (`seafile-data/`, `seadoc-data/`, `onlyoffice-data/`) landed as
|
||||||
|
expected. **A full running Seafile app-stack test (seafile+redis+caddy
|
||||||
|
actually serving content from the restored data) was not attempted** — that
|
||||||
|
would need the full compose stack, matching `JWT_PRIVATE_KEY`/hostname
|
||||||
|
config from `.env`, and meaningfully more setup than the box's read-only
|
||||||
|
`/mnt/backups` + non-root constraints support cleanly in one pass. If a
|
||||||
|
full app-level restore drill is wanted, treat it as separate follow-up
|
||||||
|
work with root access.
|
||||||
|
|
||||||
|
### ⚠️ Critical finding: Seafile backups have been broken since 2026-07-07
|
||||||
|
|
||||||
|
Every `/mnt/backups/seafile/<snapshot>` from **2026-07-07 through the
|
||||||
|
latest, 2026-07-28**, contains only an **empty** `ccnet_db.sql` (0 bytes)
|
||||||
|
and nothing else — no `seafile_db.sql`, `seahub_db.sql`, or `data/`. The
|
||||||
|
last known-good snapshot is **2026-07-04 02:00**. `restore.sh` now refuses
|
||||||
|
to run against a snapshot with a missing/empty dump file rather than
|
||||||
|
silently "restoring" an empty database, but **the backup cron job itself
|
||||||
|
is still broken** and needs its own fix (likely the `mysqldump` credentials
|
||||||
|
in `seafile/backup.sh` no longer matching the live `seafile` MySQL user, or
|
||||||
|
similar — not diagnosed further here since reproducing it requires running
|
||||||
|
`mysqldump` against the live `seafile-mysql` container, which is out of
|
||||||
|
scope for a restore-focused task and was blocked by the sandbox's
|
||||||
|
action classifier during this session). **Recommend filing a new,
|
||||||
|
separate task to fix `seafile/backup.sh`** — this is a live gap: three
|
||||||
|
weeks of Seafile backups are currently useless.
|
||||||
|
|
||||||
|
## Files touched
|
||||||
|
|
||||||
|
- `/home/alvis/agap_git/kanboard/restore.sh` (new)
|
||||||
|
- `/home/alvis/agap_git/vaultwarden/restore.sh` (new)
|
||||||
|
- `/home/alvis/agap_git/seafile/restore.sh` (new)
|
||||||
|
- `/home/alvis/agap_git/RESTORE-RUNBOOK.md` (this file, new)
|
||||||
|
|
||||||
|
Per the standing rule for this repo, nothing above was committed — it's
|
||||||
|
left in the working tree for a human to review and commit.
|
||||||
@@ -9,7 +9,10 @@ import { join } from 'path';
|
|||||||
let _askpassPath = null;
|
let _askpassPath = null;
|
||||||
function askpassScript() {
|
function askpassScript() {
|
||||||
if (_askpassPath) return _askpassPath;
|
if (_askpassPath) return _askpassPath;
|
||||||
const dir = join(tmpdir(), 'agap-mcp-wiki');
|
// Deliberately separate from the wiki checkout dir (agap-mcp-wiki) below —
|
||||||
|
// sharing a dir made it non-empty before `git clone` ran, so clone always
|
||||||
|
// failed with "destination path already exists" on any fresh checkout.
|
||||||
|
const dir = join(tmpdir(), 'agap-mcp-wiki-askpass');
|
||||||
mkdirSync(dir, { recursive: true });
|
mkdirSync(dir, { recursive: true });
|
||||||
const scriptPath = join(dir, 'git-askpass.sh');
|
const scriptPath = join(dir, 'git-askpass.sh');
|
||||||
writeFileSync(scriptPath, '#!/bin/sh\nprintf %s "$GITEA_ASKPASS_TOKEN"\n', { mode: 0o700 });
|
writeFileSync(scriptPath, '#!/bin/sh\nprintf %s "$GITEA_ASKPASS_TOKEN"\n', { mode: 0o700 });
|
||||||
|
|||||||
@@ -493,6 +493,9 @@ if (isMainModule) {
|
|||||||
console.error(`agap-mcp refusing to start: ${e.message}`);
|
console.error(`agap-mcp refusing to start: ${e.message}`);
|
||||||
process.exit(1);
|
process.exit(1);
|
||||||
}
|
}
|
||||||
|
// kb#181: compute tool count from actual server.tool registrations at startup,
|
||||||
|
// before listening, so /health has the accurate count from the beginning
|
||||||
|
createServer(); // temporary server instance just to count tools; throws away the server
|
||||||
init()
|
init()
|
||||||
.then(() => {
|
.then(() => {
|
||||||
app.listen(PORT, () => console.log(`agap-mcp listening on :${PORT}`));
|
app.listen(PORT, () => console.log(`agap-mcp listening on :${PORT}`));
|
||||||
|
|||||||
24
ai/adolf-llm/Dockerfile
Normal file
24
ai/adolf-llm/Dockerfile
Normal file
@@ -0,0 +1,24 @@
|
|||||||
|
FROM node:22-slim
|
||||||
|
|
||||||
|
# ca-certificates is REQUIRED, not optional: node:22-slim ships no system CA
|
||||||
|
# store. Node bundles its own so JS fetch works, but the Codex CLI is a Rust
|
||||||
|
# binary and validates TLS against the system store — without this every HTTPS
|
||||||
|
# call (incl. `codex login`) dies after the TCP/proxy connect with a generic
|
||||||
|
# "error sending request". Cost us a long debug; do not drop it.
|
||||||
|
RUN apt-get update \
|
||||||
|
&& apt-get install -y --no-install-recommends ca-certificates \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
RUN npm install -g @openai/codex
|
||||||
|
|
||||||
|
WORKDIR /workspace
|
||||||
|
|
||||||
|
COPY server.js /app/server.js
|
||||||
|
|
||||||
|
# Codex reads auth + config from $CODEX_HOME (default ~/.codex). Pinned
|
||||||
|
# explicitly so the compose volume mount and the config writer agree.
|
||||||
|
ENV CODEX_HOME=/root/.codex
|
||||||
|
|
||||||
|
EXPOSE 8010
|
||||||
|
|
||||||
|
ENTRYPOINT ["node", "/app/server.js"]
|
||||||
@@ -4,11 +4,14 @@ const path = require('path');
|
|||||||
const crypto = require('crypto');
|
const crypto = require('crypto');
|
||||||
const { spawn } = require('child_process');
|
const { spawn } = require('child_process');
|
||||||
|
|
||||||
const PORT = 8010;
|
// Both overridable purely so the wrapper can be exercised outside the
|
||||||
|
// container (the defaults are the in-container values).
|
||||||
|
const PORT = Number(process.env.PORT) || 8010;
|
||||||
|
const SHARED_MCP_PATH = process.env.SHARED_MCP_PATH || '/shared-mcp.json';
|
||||||
const MODEL_ID = 'adolf';
|
const MODEL_ID = 'adolf';
|
||||||
const TIMEOUT_MS = 15 * 60 * 1000;
|
const TIMEOUT_MS = 15 * 60 * 1000;
|
||||||
|
|
||||||
const WORKSPACE = '/workspace';
|
const WORKSPACE = process.env.WORKSPACE || '/workspace';
|
||||||
const CONV_ROOT = path.join(WORKSPACE, 'conversations');
|
const CONV_ROOT = path.join(WORKSPACE, 'conversations');
|
||||||
const STATE_DIR = path.join(WORKSPACE, '.adolf-llm');
|
const STATE_DIR = path.join(WORKSPACE, '.adolf-llm');
|
||||||
const MAP_FILE = path.join(STATE_DIR, 'sessions.json');
|
const MAP_FILE = path.join(STATE_DIR, 'sessions.json');
|
||||||
@@ -18,51 +21,99 @@ fs.mkdirSync(CONV_ROOT, { recursive: true });
|
|||||||
fs.mkdirSync(STATE_DIR, { recursive: true });
|
fs.mkdirSync(STATE_DIR, { recursive: true });
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Shared MCP layer (Gate 1). Kimi Code CLI has NO `--mcp-config-file` flag and
|
// Shared MCP layer. Unlike Kimi Code CLI (which had no --mcp-config-file flag
|
||||||
// no `kimi mcp` subcommand; it auto-discovers a project-root `.mcp.json` by
|
// and forced a per-session `.mcp.json` dropped into each working directory),
|
||||||
// walking up from its cwd to the nearest `.git` (falling back to cwd itself
|
// Codex reads MCP servers from `$CODEX_HOME/config.toml` under `[mcp_servers.*]`
|
||||||
// when none is found). So we drop a `.mcp.json` into each session's working
|
// tables. So this is now written ONCE at startup instead of per session.
|
||||||
// directory before spawning kimi.
|
|
||||||
//
|
//
|
||||||
// Single source of truth: `/shared-mcp.json` (mounted read-only from the repo
|
// Single source of truth is unchanged: `/shared-mcp.json` (mounted read-only
|
||||||
// root's `shared-mcp.json`, the same file P6 wires into OpenClaw's own
|
// from the repo root's `shared-mcp.json`, the same file OpenClaw's own
|
||||||
// `mcp.servers` registry). Adding a server is then a one-file change — no
|
// `mcp.servers` registry uses). Adding a server stays a one-file change.
|
||||||
// server list is hardcoded here anymore.
|
|
||||||
//
|
//
|
||||||
// Gate-1 transport finding (P5, verified by decompiling the installed
|
// Transport mapping. shared-mcp.json entries are either:
|
||||||
// @moonshot-ai/kimi-code package, packages/agent-core/src/config/schema.ts's
|
// { command, args, env } -> stdio server
|
||||||
// McpServerConfigSchema): Kimi's own field name for remote MCP servers is
|
// { url, type: "http" } -> remote streamable-http server
|
||||||
// `transport` (literal "stdio" | "http" | "sse"), not `type`. When `transport`
|
// Codex expresses stdio servers as `command`/`args`/`env`, and remote servers
|
||||||
// is omitted, Kimi's config preprocessor infers it from shape: `command` ->
|
// as `url` with an optional `bearer_token_env_var`. It has no `type` key; the
|
||||||
// "stdio", `url` -> "http" (never "sse" — sse requires an explicit
|
// shape (command vs url) selects the transport, same inference Kimi did. The
|
||||||
// `transport: "sse"`). It does NOT recognize a `type` key at all; unknown keys
|
// `type: "http"` key that shared-mcp.json carries for OpenClaw's benefit is
|
||||||
// are silently stripped by the (non-strict) zod schema.
|
// simply not emitted here.
|
||||||
// OpenClaw's own canonical `mcp.servers` schema (docs/gateway/
|
const CODEX_HOME = process.env.CODEX_HOME || '/root/.codex';
|
||||||
// configuration-reference.md) uses different literals for the same
|
|
||||||
// transport: `transport: "streamable-http"` or `"sse"`, with `type: "http"`
|
|
||||||
// documented as a *CLI-native alias* that `openclaw mcp set` / `openclaw
|
|
||||||
// doctor --fix` normalize into canonical `transport: "streamable-http"`.
|
|
||||||
// So the two consumers disagree on the literal value for HTTP streaming
|
|
||||||
// ("http" vs "streamable-http") under the same field name `transport` --
|
|
||||||
// writing `transport` explicitly in shared-mcp.json would satisfy at most one
|
|
||||||
// side. `type: "http"` is the one shape both sides tolerate today: Kimi
|
|
||||||
// ignores the unrecognized `type` key and correctly infers transport "http"
|
|
||||||
// from the `url` field alone; OpenClaw recognizes `type` as its documented
|
|
||||||
// alias and normalizes it on its own terms (P6 concern, not touched here).
|
|
||||||
// Hence shared-mcp.json intentionally keeps `"type": "http"` for both cognee
|
|
||||||
// and openclaw-tools rather than switching to `transport`.
|
|
||||||
let SHARED_MCP_SERVERS = {};
|
let SHARED_MCP_SERVERS = {};
|
||||||
try {
|
try {
|
||||||
const raw = fs.readFileSync('/shared-mcp.json', 'utf8');
|
const raw = fs.readFileSync(SHARED_MCP_PATH, 'utf8');
|
||||||
SHARED_MCP_SERVERS = JSON.parse(raw).mcpServers || {};
|
SHARED_MCP_SERVERS = JSON.parse(raw).mcpServers || {};
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
console.error(`shared-mcp.json not loaded (${err.message}); sessions will get no shared MCP servers`);
|
console.error(`shared-mcp.json not loaded (${err.message}); sessions will get no shared MCP servers`);
|
||||||
}
|
}
|
||||||
|
|
||||||
function writeMcpConfig(dir) {
|
// Minimal TOML emitter — we only ever emit strings, string arrays and flat
|
||||||
const cfg = { mcpServers: SHARED_MCP_SERVERS };
|
// string maps, so a full TOML library would be dead weight.
|
||||||
fs.writeFileSync(path.join(dir, '.mcp.json'), JSON.stringify(cfg, null, 2));
|
function tomlString(s) {
|
||||||
|
return JSON.stringify(String(s)); // TOML basic strings share JSON escaping
|
||||||
}
|
}
|
||||||
|
function tomlValue(v) {
|
||||||
|
if (Array.isArray(v)) return `[${v.map(tomlString).join(', ')}]`;
|
||||||
|
return tomlString(v);
|
||||||
|
}
|
||||||
|
|
||||||
|
function renderMcpToml(servers) {
|
||||||
|
const lines = [
|
||||||
|
'# GENERATED by adolf-llm from /shared-mcp.json — do not edit by hand.',
|
||||||
|
'# Regenerated on every container start; manual edits are lost.',
|
||||||
|
'',
|
||||||
|
];
|
||||||
|
for (const [name, cfg] of Object.entries(servers)) {
|
||||||
|
lines.push(`[mcp_servers.${name}]`);
|
||||||
|
if (cfg.command) {
|
||||||
|
lines.push(`command = ${tomlValue(cfg.command)}`);
|
||||||
|
if (cfg.args && cfg.args.length) lines.push(`args = ${tomlValue(cfg.args)}`);
|
||||||
|
} else if (cfg.url) {
|
||||||
|
lines.push(`url = ${tomlValue(cfg.url)}`);
|
||||||
|
} else {
|
||||||
|
console.error(`shared-mcp.json: server "${name}" has neither command nor url; skipped`);
|
||||||
|
lines.pop();
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
// Field-name translation, Kimi -> Codex. shared-mcp.json is written in
|
||||||
|
// Kimi/OpenClaw's camelCase dialect; Codex's RawMcpServerConfig uses
|
||||||
|
// snake_case. Both keys are load-bearing:
|
||||||
|
// bearerTokenEnvVar -> bearer_token_env_var (agap + marketplace auth;
|
||||||
|
// without it every tool call on those servers returns HTTP 401)
|
||||||
|
// enabledTools -> enabled_tools (the capability allow-list
|
||||||
|
// that ai/agent-registry.yaml's mcp_tool_filter is validated against;
|
||||||
|
// dropping it would silently widen Adolf's tool access)
|
||||||
|
if (cfg.bearerTokenEnvVar) {
|
||||||
|
lines.push(`bearer_token_env_var = ${tomlValue(cfg.bearerTokenEnvVar)}`);
|
||||||
|
}
|
||||||
|
if (cfg.enabledTools && cfg.enabledTools.length) {
|
||||||
|
lines.push(`enabled_tools = ${tomlValue(cfg.enabledTools)}`);
|
||||||
|
}
|
||||||
|
if (cfg.env && Object.keys(cfg.env).length) {
|
||||||
|
lines.push(`[mcp_servers.${name}.env]`);
|
||||||
|
for (const [k, v] of Object.entries(cfg.env)) lines.push(`${k} = ${tomlValue(v)}`);
|
||||||
|
}
|
||||||
|
lines.push('');
|
||||||
|
}
|
||||||
|
return lines.join('\n');
|
||||||
|
}
|
||||||
|
|
||||||
|
// Write the Codex config once at startup: MCP servers + the headless-operation
|
||||||
|
// settings. `approval_policy = "never"` and `sandbox_mode` are load-bearing —
|
||||||
|
// Codex defaults to asking for approval before running a tool, and nobody is
|
||||||
|
// there to answer, so without these a turn hangs until the 15-minute timeout
|
||||||
|
// instead of failing loudly.
|
||||||
|
function writeCodexConfig() {
|
||||||
|
fs.mkdirSync(CODEX_HOME, { recursive: true });
|
||||||
|
const header = [
|
||||||
|
'approval_policy = "never"',
|
||||||
|
'sandbox_mode = "danger-full-access"',
|
||||||
|
'',
|
||||||
|
].join('\n');
|
||||||
|
fs.writeFileSync(path.join(CODEX_HOME, 'config.toml'), header + renderMcpToml(SHARED_MCP_SERVERS));
|
||||||
|
}
|
||||||
|
writeCodexConfig();
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Memory lives at the OpenClaw layer, not here (P8). The Adolf gateway loads
|
// Memory lives at the OpenClaw layer, not here (P8). The Adolf gateway loads
|
||||||
@@ -77,7 +128,7 @@ function writeMcpConfig(dir) {
|
|||||||
// were deleted when the plugin took over (P8).
|
// were deleted when the plugin took over (P8).
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Persistent conversation -> Kimi session map.
|
// Persistent conversation -> Codex session (thread) map.
|
||||||
// Primary key: `chat:<chat_id>` parsed from OpenClaw's "Conversation info" block
|
// Primary key: `chat:<chat_id>` parsed from OpenClaw's "Conversation info" block
|
||||||
// (Gate 2). Fallback key: `hist:<sha256(prior history)>` when no chat_id is
|
// (Gate 2). Fallback key: `hist:<sha256(prior history)>` when no chat_id is
|
||||||
// present (e.g. webchat surface / future OpenClaw layout change).
|
// present (e.g. webchat surface / future OpenClaw layout change).
|
||||||
@@ -271,31 +322,70 @@ async function buildPrompt(userMsg, dir) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Kimi invocation with REAL streaming. Parses `--output-format stream-json`
|
// Codex invocation with REAL streaming. Parses `codex exec --json` incrementally:
|
||||||
// incrementally: each complete stdout line is one JSON object.
|
// each complete stdout line is one JSON event.
|
||||||
// {"role":"assistant","content":"..."} -> emit as a delta
|
//
|
||||||
// {"type":"session.resume_hint","session_id":"..."} -> capture session id
|
// Codex 0.146 ships TWO event schemas and which one `--json` emits can change
|
||||||
|
// between releases, so we handle both rather than pinning to one:
|
||||||
|
//
|
||||||
|
// legacy "msg" schema:
|
||||||
|
// {"id":..,"msg":{"type":"agent_message_delta","delta":"..."}} -> delta
|
||||||
|
// {"id":..,"msg":{"type":"agent_message","message":"..."}} -> full text
|
||||||
|
// {"id":..,"msg":{"type":"session_configured","session_id":".."}}-> session id
|
||||||
|
// newer thread/turn/item schema:
|
||||||
|
// {"type":"thread.started","thread_id":"..."} -> session id
|
||||||
|
// {"type":"item.completed","item":{"type":"agent_message",...}} -> full text
|
||||||
|
//
|
||||||
|
// Non-assistant items (reasoning, command execution, MCP tool calls) are
|
||||||
|
// deliberately ignored — Adolf's users see the answer, not the agent's work.
|
||||||
|
//
|
||||||
|
// Sequencing note: when a run emits streaming deltas AND a terminal full-text
|
||||||
|
// message, the full text is the same content already streamed. We therefore
|
||||||
|
// prefer deltas when any arrived, and fall back to the terminal message only
|
||||||
|
// when none did — otherwise the reply would be duplicated.
|
||||||
|
//
|
||||||
// onDelta(chunk) is called per assistant content fragment as it arrives.
|
// onDelta(chunk) is called per assistant content fragment as it arrives.
|
||||||
// Resolves { text, sessionId } once the process closes.
|
// Resolves { text, sessionId } once the process closes.
|
||||||
function runKimi({ prompt, cwd, resumeId, onDelta, signal }) {
|
function runCodex({ prompt, cwd, resumeId, onDelta, signal }) {
|
||||||
return new Promise((resolve, reject) => {
|
return new Promise((resolve, reject) => {
|
||||||
if (signal?.aborted) { reject(new Error('aborted before start')); return; }
|
if (signal?.aborted) { reject(new Error('aborted before start')); return; }
|
||||||
const args = [];
|
// `exec resume <id>` must come before the prompt; `--skip-git-repo-check`
|
||||||
if (resumeId) args.push('-r', resumeId);
|
// is required because session dirs under /workspace are not git repos.
|
||||||
args.push('-p', prompt, '--output-format', 'stream-json');
|
//
|
||||||
|
// `-C/--cd` is accepted by `codex exec` but NOT by `codex exec resume` —
|
||||||
|
// passing it there fails with "unexpected argument '-C' found" and breaks
|
||||||
|
// every follow-up turn while first turns still work. The spawn cwd below
|
||||||
|
// already puts the process in the right directory, so -C is only an
|
||||||
|
// explicit belt-and-braces on the fresh-session path.
|
||||||
|
const args = ['exec'];
|
||||||
|
if (resumeId) {
|
||||||
|
args.push('resume', resumeId, '--json', '--skip-git-repo-check', prompt);
|
||||||
|
} else {
|
||||||
|
args.push('--json', '--skip-git-repo-check', '-C', cwd, prompt);
|
||||||
|
}
|
||||||
|
|
||||||
const child = spawn('kimi', args, { cwd, timeout: TIMEOUT_MS });
|
// stdin MUST be 'ignore'. With the default 'pipe', codex prints "Reading
|
||||||
|
// additional input from stdin..." and blocks waiting for EOF on a pipe this
|
||||||
|
// wrapper never writes to or closes — every turn would hang until the
|
||||||
|
// 15-minute timeout. (Kimi's CLI did not read stdin, so this is new.)
|
||||||
|
const child = spawn('codex', args, {
|
||||||
|
cwd,
|
||||||
|
timeout: TIMEOUT_MS,
|
||||||
|
stdio: ['ignore', 'pipe', 'pipe'],
|
||||||
|
});
|
||||||
|
|
||||||
let buf = '';
|
let buf = '';
|
||||||
let stderr = '';
|
let stderr = '';
|
||||||
const parts = [];
|
const parts = []; // streamed deltas, in order
|
||||||
|
let finalText = null; // terminal full-text message, if the run emits one
|
||||||
|
let errText = null; // structured error reported on the event stream
|
||||||
let sessionId = null;
|
let sessionId = null;
|
||||||
let settled = false;
|
let settled = false;
|
||||||
let aborted = false;
|
let aborted = false;
|
||||||
|
|
||||||
// If the caller aborts (the gateway/client disconnected — e.g. its idle
|
// If the caller aborts (the gateway/client disconnected — e.g. its idle
|
||||||
// watchdog gave up), kill the child so it doesn't keep grinding an
|
// watchdog gave up), kill the child so it doesn't keep grinding an
|
||||||
// orphaned agent turn to completion, wasting Kimi quota and streaming into
|
// orphaned agent turn to completion, wasting Codex quota and streaming into
|
||||||
// a dead socket. SIGTERM first, hard SIGKILL if it lingers.
|
// a dead socket. SIGTERM first, hard SIGKILL if it lingers.
|
||||||
const onAbort = () => {
|
const onAbort = () => {
|
||||||
aborted = true;
|
aborted = true;
|
||||||
@@ -309,11 +399,35 @@ function runKimi({ prompt, cwd, resumeId, onDelta, signal }) {
|
|||||||
if (!t) return;
|
if (!t) return;
|
||||||
let obj;
|
let obj;
|
||||||
try { obj = JSON.parse(t); } catch { return; }
|
try { obj = JSON.parse(t); } catch { return; }
|
||||||
if (obj.role === 'assistant' && typeof obj.content === 'string' && obj.content) {
|
|
||||||
parts.push(obj.content);
|
// --- legacy "msg" schema -------------------------------------------
|
||||||
if (onDelta) onDelta(obj.content);
|
const msg = obj.msg;
|
||||||
|
if (msg && typeof msg.type === 'string') {
|
||||||
|
if (msg.type === 'agent_message_delta' && typeof msg.delta === 'string' && msg.delta) {
|
||||||
|
parts.push(msg.delta);
|
||||||
|
if (onDelta) onDelta(msg.delta);
|
||||||
|
} else if (msg.type === 'agent_message' && typeof msg.message === 'string' && msg.message) {
|
||||||
|
finalText = msg.message;
|
||||||
|
} else if (msg.type === 'session_configured' && msg.session_id) {
|
||||||
|
sessionId = msg.session_id;
|
||||||
|
} else if (msg.type === 'error' && msg.message) {
|
||||||
|
errText = msg.message;
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- newer thread/turn/item schema ----------------------------------
|
||||||
|
if (obj.type === 'thread.started' && obj.thread_id) {
|
||||||
|
sessionId = obj.thread_id;
|
||||||
|
} else if (obj.type === 'item.completed' && obj.item) {
|
||||||
|
const item = obj.item;
|
||||||
|
if (item.type === 'agent_message') {
|
||||||
|
const text = typeof item.text === 'string' ? item.text : item.message;
|
||||||
|
if (typeof text === 'string' && text) finalText = text;
|
||||||
|
}
|
||||||
|
} else if (obj.type === 'turn.failed') {
|
||||||
|
errText = (obj.error && (obj.error.message || obj.error)) || 'turn.failed';
|
||||||
}
|
}
|
||||||
if (obj.type === 'session.resume_hint' && obj.session_id) sessionId = obj.session_id;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
child.stdout.on('data', d => {
|
child.stdout.on('data', d => {
|
||||||
@@ -338,11 +452,19 @@ function runKimi({ prompt, cwd, resumeId, onDelta, signal }) {
|
|||||||
settled = true;
|
settled = true;
|
||||||
if (signal) signal.removeEventListener('abort', onAbort);
|
if (signal) signal.removeEventListener('abort', onAbort);
|
||||||
if (buf) handleLine(buf); // flush any trailing partial line
|
if (buf) handleLine(buf); // flush any trailing partial line
|
||||||
const text = parts.join('').trim();
|
// Deltas win when present — the terminal agent_message repeats content
|
||||||
|
// already streamed to the client. Only fall back to it if nothing streamed.
|
||||||
|
const streamed = parts.join('').trim();
|
||||||
|
const text = streamed || (finalText || '').trim();
|
||||||
|
// A non-streaming run still has to reach the client: emit the terminal
|
||||||
|
// message as one delta so callers relying on onDelta aren't left empty.
|
||||||
|
if (!streamed && text && onDelta) onDelta(text);
|
||||||
if (aborted) {
|
if (aborted) {
|
||||||
reject(new Error('aborted: client disconnected'));
|
reject(new Error('aborted: client disconnected'));
|
||||||
} else if (!text && code !== 0) {
|
} else if (!text && code !== 0) {
|
||||||
reject(new Error(`kimi exited ${code}: ${stderr.slice(0, 2000)}`));
|
reject(new Error(`codex exited ${code}: ${(errText || stderr).slice(0, 2000)}`));
|
||||||
|
} else if (!text && errText) {
|
||||||
|
reject(new Error(`codex error: ${errText.slice(0, 2000)}`));
|
||||||
} else {
|
} else {
|
||||||
resolve({ text, sessionId });
|
resolve({ text, sessionId });
|
||||||
}
|
}
|
||||||
@@ -352,7 +474,7 @@ function runKimi({ prompt, cwd, resumeId, onDelta, signal }) {
|
|||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// One turn: resolve session (chat_id primary, history-hash fallback), persist
|
// One turn: resolve session (chat_id primary, history-hash fallback), persist
|
||||||
// media + .mcp.json, run kimi (streaming through onDelta), record the mapping,
|
// media, run codex (streaming through onDelta), record the mapping,
|
||||||
// and fire the async cognee ingest. Returns { text }.
|
// and fire the async cognee ingest. Returns { text }.
|
||||||
async function handleTurn(messages, onDelta, signal) {
|
async function handleTurn(messages, onDelta, signal) {
|
||||||
const turns = convTurns(messages);
|
const turns = convTurns(messages);
|
||||||
@@ -388,7 +510,7 @@ async function handleTurn(messages, onDelta, signal) {
|
|||||||
reseed = prior.length > 0;
|
reseed = prior.length > 0;
|
||||||
}
|
}
|
||||||
} else {
|
} else {
|
||||||
// Fallback: no chat_id -> forward history-hash mapping (kimi-agent style).
|
// Fallback: no chat_id -> forward history-hash mapping.
|
||||||
if (prior.length === 0) {
|
if (prior.length === 0) {
|
||||||
convId = crypto.randomUUID();
|
convId = crypto.randomUUID();
|
||||||
dir = path.join(CONV_ROOT, convId);
|
dir = path.join(CONV_ROOT, convId);
|
||||||
@@ -407,7 +529,6 @@ async function handleTurn(messages, onDelta, signal) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
fs.mkdirSync(dir, { recursive: true });
|
fs.mkdirSync(dir, { recursive: true });
|
||||||
writeMcpConfig(dir); // Gate 1: shared MCP via project-root .mcp.json
|
|
||||||
|
|
||||||
let prompt;
|
let prompt;
|
||||||
if (reseed) {
|
if (reseed) {
|
||||||
@@ -423,7 +544,7 @@ async function handleTurn(messages, onDelta, signal) {
|
|||||||
prompt = await buildPrompt(userMsg, dir);
|
prompt = await buildPrompt(userMsg, dir);
|
||||||
}
|
}
|
||||||
|
|
||||||
const { text, sessionId } = await runKimi({ prompt, cwd: dir, resumeId, onDelta, signal });
|
const { text, sessionId } = await runCodex({ prompt, cwd: dir, resumeId, onDelta, signal });
|
||||||
|
|
||||||
// Record the forward mapping.
|
// Record the forward mapping.
|
||||||
const entry = { convId, sessionId: sessionId || resumeId, dir, ts: Date.now() };
|
const entry = { convId, sessionId: sessionId || resumeId, dir, ts: Date.now() };
|
||||||
@@ -439,180 +560,21 @@ async function handleTurn(messages, onDelta, signal) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Kimi quota readout (kb #62). GET /usage — the claude-usage analog for
|
// Quota readout. The Kimi-specific implementation (kb #62/#87) was removed with
|
||||||
// Adolf. LLM-free: hits Kimi's own managed-usage endpoint directly, never
|
// the Codex migration: it authenticated against Kimi's managed-usage API using
|
||||||
// spawns `kimi`. Mirrors the parsing logic of the installed
|
// the Kimi CLI's OAuth creds file, and neither the endpoint nor the credential
|
||||||
// @moonshot-ai/kimi-code CLI itself (decompiled from dist/main.mjs's
|
// exists on this backend. Codex exposes no equivalent machine-readable quota
|
||||||
// parseManagedUsagePayload/toUsageRow/limitLabel/resetHintFrom — same
|
// endpoint, so /usage now reports "unsupported" rather than inventing numbers.
|
||||||
// endpoint, same response shape) so bucket labels/derivations stay in sync
|
|
||||||
// with what `kimi` would show via its own /usage-equivalent.
|
|
||||||
//
|
//
|
||||||
// Token source: the CLI's own OAuth creds file, kept fresh by the running
|
// The two consumers (kimi-quota-footer-plugin, quota-command-openclaw-plugin)
|
||||||
// `kimi` process (adolf-llm-home volume). We ONLY read the file's live
|
// both treat a non-OK /usage as "no data" and degrade quietly -- the footer is
|
||||||
// access_token and never refresh here. Kimi's OAuth rotates the refresh_token
|
// simply omitted. They still need a decision: retire them, or repoint them at
|
||||||
// on every refresh (single-use), so an independent refresh from this route
|
// whatever quota signal the Codex/ChatGPT plan actually exposes.
|
||||||
// invalidates the refresh_token the CLI's file still holds -> the CLI's next
|
const USAGE_UNSUPPORTED = {
|
||||||
// refresh fails `invalid_grant` and wipes the whole login (kb#87: this was the
|
error: 'usage_unsupported',
|
||||||
// recurring Adolf logout, incl. the 2026-07-17 06:15 wipe / task #86). Making
|
backend: 'codex',
|
||||||
// the CLI the sole refresher removes that race.
|
detail: 'Codex backend exposes no machine-readable quota endpoint.',
|
||||||
//
|
};
|
||||||
// Cost of that trade, measured 2026-07-22: the access token's `expires_in` is
|
|
||||||
// 900s, so it is only valid for 15 minutes after the CLI last refreshed it —
|
|
||||||
// i.e. only within 15 minutes of an actual Adolf turn. Adolf is idle most of
|
|
||||||
// the day, so a bare read failed far more often than it succeeded, which made
|
|
||||||
// quota gating effectively blind. Rather than refresh here (see above: that
|
|
||||||
// wipes the login), /usage now falls back to the LAST GOOD reading, clearly
|
|
||||||
// labelled `stale` with `as_of` + `age_s` so callers can decide whether it is
|
|
||||||
// fresh enough. The cache is written on every success and persisted to the
|
|
||||||
// workspace volume so it survives a container restart. Auth is untouched:
|
|
||||||
// this route still only ever READS the creds file.
|
|
||||||
const KIMI_CREDS_PATH = '/root/.kimi-code/credentials/kimi-code.json';
|
|
||||||
const KIMI_USAGES_URL = 'https://api.kimi.com/coding/v1/usages';
|
|
||||||
const KIMI_USAGE_CACHE_PATH = '/workspace/.adolf-llm/usage-cache.json';
|
|
||||||
|
|
||||||
// Last successful /usage payload, kept in memory and mirrored to disk.
|
|
||||||
let kimiUsageCache = null;
|
|
||||||
|
|
||||||
function readKimiUsageCache() {
|
|
||||||
if (kimiUsageCache) return kimiUsageCache;
|
|
||||||
try {
|
|
||||||
const parsed = JSON.parse(fs.readFileSync(KIMI_USAGE_CACHE_PATH, 'utf8'));
|
|
||||||
if (parsed && parsed.payload && parsed.cached_at) kimiUsageCache = parsed;
|
|
||||||
} catch { /* no cache yet, or unreadable — treated as "no cache" */ }
|
|
||||||
return kimiUsageCache;
|
|
||||||
}
|
|
||||||
|
|
||||||
function writeKimiUsageCache(payload) {
|
|
||||||
kimiUsageCache = { payload, cached_at: new Date().toISOString() };
|
|
||||||
try {
|
|
||||||
fs.mkdirSync(path.dirname(KIMI_USAGE_CACHE_PATH), { recursive: true });
|
|
||||||
fs.writeFileSync(KIMI_USAGE_CACHE_PATH, JSON.stringify(kimiUsageCache));
|
|
||||||
} catch { /* cache is best-effort; an unwritable volume must not break /usage */ }
|
|
||||||
}
|
|
||||||
|
|
||||||
async function loadKimiCreds() {
|
|
||||||
const raw = await fs.promises.readFile(KIMI_CREDS_PATH, 'utf8');
|
|
||||||
return JSON.parse(raw);
|
|
||||||
}
|
|
||||||
|
|
||||||
// Read the live access_token from the CLI's creds file. We deliberately do NOT
|
|
||||||
// refresh here (see the note above): the Kimi CLI is the sole refresher, so
|
|
||||||
// this route can never rotate the single-use refresh_token out from under it.
|
|
||||||
// A stale file token surfaces as an error -> /usage 502 -> "quota unavailable".
|
|
||||||
async function getKimiAccessToken() {
|
|
||||||
const creds = await loadKimiCreds();
|
|
||||||
const now = Math.floor(Date.now() / 1000);
|
|
||||||
if (creds.access_token && creds.expires_at && now < creds.expires_at - 30) {
|
|
||||||
return creds.access_token;
|
|
||||||
}
|
|
||||||
throw new Error('kimi access token stale (CLI refreshes on next use); quota temporarily unavailable');
|
|
||||||
}
|
|
||||||
|
|
||||||
async function fetchKimiUsagesRaw() {
|
|
||||||
const token = await getKimiAccessToken();
|
|
||||||
const res = await fetch(KIMI_USAGES_URL, { headers: { Authorization: `Bearer ${token}`, Accept: 'application/json' } });
|
|
||||||
if (!res.ok) {
|
|
||||||
const text = await res.text().catch(() => '');
|
|
||||||
throw new Error(`kimi /usages HTTP ${res.status}: ${text.slice(0, 500)}`);
|
|
||||||
}
|
|
||||||
return res.json();
|
|
||||||
}
|
|
||||||
|
|
||||||
function isRecord(v) { return v !== null && typeof v === 'object' && !Array.isArray(v); }
|
|
||||||
|
|
||||||
function toInt(v) {
|
|
||||||
if (typeof v === 'number') return Number.isFinite(v) ? Math.trunc(v) : null;
|
|
||||||
if (typeof v === 'string') { const n = Number(v); return Number.isFinite(n) ? Math.trunc(n) : null; }
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Port of the CLI's limitLabel(): prefer an explicit name/title/scope field,
|
|
||||||
// else derive "<N>h limit" / "<N>m limit" / "<N>d limit" from the window's
|
|
||||||
// duration+timeUnit.
|
|
||||||
function kimiLimitLabel(item, detail, window, idx) {
|
|
||||||
for (const key of ['name', 'title', 'scope']) {
|
|
||||||
const v = item[key] ?? detail[key];
|
|
||||||
if (typeof v === 'string' && v) return v;
|
|
||||||
}
|
|
||||||
const duration = toInt(window.duration ?? item.duration ?? detail.duration);
|
|
||||||
const rawUnit = window.timeUnit ?? item.timeUnit ?? detail.timeUnit;
|
|
||||||
const timeUnit = typeof rawUnit === 'string' ? rawUnit : '';
|
|
||||||
if (duration !== null) {
|
|
||||||
if (timeUnit.includes('MINUTE')) {
|
|
||||||
if (duration >= 60 && duration % 60 === 0) return `${duration / 60}h limit`;
|
|
||||||
return `${duration}m limit`;
|
|
||||||
}
|
|
||||||
if (timeUnit.includes('HOUR')) return `${duration}h limit`;
|
|
||||||
if (timeUnit.includes('DAY')) return `${duration}d limit`;
|
|
||||||
return `${duration}s limit`;
|
|
||||||
}
|
|
||||||
return `Limit #${idx + 1}`;
|
|
||||||
}
|
|
||||||
|
|
||||||
function kimiResetIso(raw) {
|
|
||||||
for (const key of ['reset_at', 'resetAt', 'reset_time', 'resetTime']) {
|
|
||||||
const v = raw[key];
|
|
||||||
if (typeof v === 'string' && v) return v;
|
|
||||||
}
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Port of the CLI's toUsageRow(): used = raw.used, or limit-remaining when
|
|
||||||
// used is absent.
|
|
||||||
function kimiUsageRow(raw, defaultLabel) {
|
|
||||||
if (!isRecord(raw)) return null;
|
|
||||||
const limit = toInt(raw.limit);
|
|
||||||
let used = toInt(raw.used);
|
|
||||||
const remaining = toInt(raw.remaining);
|
|
||||||
if (used === null && remaining !== null && limit !== null) used = limit - remaining;
|
|
||||||
if (used === null && limit === null) return null;
|
|
||||||
const name = typeof raw.name === 'string' ? raw.name : (typeof raw.title === 'string' ? raw.title : defaultLabel);
|
|
||||||
return {
|
|
||||||
label: name,
|
|
||||||
used: used ?? 0,
|
|
||||||
limit: limit ?? 0,
|
|
||||||
remaining: remaining !== null ? remaining : (limit !== null && used !== null ? limit - used : null),
|
|
||||||
resets: kimiResetIso(raw),
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
function kimiRowOut(row) {
|
|
||||||
if (!row) return null;
|
|
||||||
const pct = row.limit > 0 ? Math.round((row.used / row.limit) * 100) : null;
|
|
||||||
return { pct, used: row.used, limit: row.limit, remaining: row.remaining, resets: row.resets };
|
|
||||||
}
|
|
||||||
|
|
||||||
// Normalize Kimi's /usages payload ({ usage, limits: [...] }) into the
|
|
||||||
// claude-usage-analog shape: weekly / window_5h / window_7d, each
|
|
||||||
// pct/used/limit/remaining/resets, plus a raw `limits` passthrough so no
|
|
||||||
// bucket is lost if label text ever drifts from what we match on below.
|
|
||||||
function normalizeKimiUsage(payload) {
|
|
||||||
const rec = isRecord(payload) ? payload : {};
|
|
||||||
const summaryRow = kimiUsageRow(rec.usage, 'Weekly limit');
|
|
||||||
const limitRows = [];
|
|
||||||
const rawLimits = Array.isArray(rec.limits) ? rec.limits : [];
|
|
||||||
rawLimits.forEach((item, idx) => {
|
|
||||||
if (!isRecord(item)) return;
|
|
||||||
const detail = isRecord(item.detail) ? item.detail : item;
|
|
||||||
const window = isRecord(item.window) ? item.window : {};
|
|
||||||
const label = kimiLimitLabel(item, detail, window, idx);
|
|
||||||
const row = kimiUsageRow(detail, label);
|
|
||||||
if (row) limitRows.push(row);
|
|
||||||
});
|
|
||||||
|
|
||||||
const findByLabel = re => limitRows.find(r => re.test(r.label));
|
|
||||||
const weekly = summaryRow || findByLabel(/week/i) || null;
|
|
||||||
const window5h = findByLabel(/^5\s*h(our)?\b|5h limit/i) || null;
|
|
||||||
const window7d = findByLabel(/^7\s*d(ay)?\b|7d limit/i) || null;
|
|
||||||
|
|
||||||
return {
|
|
||||||
timestamp: new Date().toISOString(),
|
|
||||||
weekly: kimiRowOut(weekly),
|
|
||||||
window_5h: kimiRowOut(window5h),
|
|
||||||
window_7d: kimiRowOut(window7d),
|
|
||||||
limits: limitRows.map(r => ({ label: r.label, ...kimiRowOut(r) })),
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// OpenAI-compatible HTTP surface.
|
// OpenAI-compatible HTTP surface.
|
||||||
@@ -642,40 +604,17 @@ const server = http.createServer((req, res) => {
|
|||||||
res.writeHead(200, { 'Content-Type': 'application/json' });
|
res.writeHead(200, { 'Content-Type': 'application/json' });
|
||||||
res.end(JSON.stringify({
|
res.end(JSON.stringify({
|
||||||
object: 'list',
|
object: 'list',
|
||||||
data: [{ id: MODEL_ID, object: 'model', owned_by: 'moonshot' }],
|
data: [{ id: MODEL_ID, object: 'model', owned_by: 'openai' }],
|
||||||
}));
|
}));
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
if (req.method === 'GET' && req.url === '/usage') {
|
if (req.method === 'GET' && req.url === '/usage') {
|
||||||
(async () => {
|
// 501 rather than 502: this is not a transient upstream failure, it is a
|
||||||
try {
|
// capability the codex backend does not have. Consumers already treat any
|
||||||
const raw = await fetchKimiUsagesRaw();
|
// non-OK response as "no data" and omit the quota footer.
|
||||||
const out = normalizeKimiUsage(raw);
|
res.writeHead(501, { 'Content-Type': 'application/json' });
|
||||||
writeKimiUsageCache(out);
|
res.end(JSON.stringify(USAGE_UNSUPPORTED));
|
||||||
res.writeHead(200, { 'Content-Type': 'application/json' });
|
|
||||||
res.end(JSON.stringify({ ...out, stale: false }));
|
|
||||||
} catch (err) {
|
|
||||||
// Token stale (the common case when Adolf has been idle >15min) or Kimi
|
|
||||||
// unreachable. Serve the last good reading rather than nothing, labelled
|
|
||||||
// so a caller can reject it if it is too old to gate on.
|
|
||||||
const cached = readKimiUsageCache();
|
|
||||||
if (cached) {
|
|
||||||
const ageS = Math.max(0, Math.round((Date.now() - Date.parse(cached.cached_at)) / 1000));
|
|
||||||
res.writeHead(200, { 'Content-Type': 'application/json' });
|
|
||||||
res.end(JSON.stringify({
|
|
||||||
...cached.payload,
|
|
||||||
stale: true,
|
|
||||||
as_of: cached.cached_at,
|
|
||||||
age_s: ageS,
|
|
||||||
stale_reason: String(err.message || err),
|
|
||||||
}));
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
res.writeHead(502, { 'Content-Type': 'application/json' });
|
|
||||||
res.end(JSON.stringify({ error: String(err.message || err) }));
|
|
||||||
}
|
|
||||||
})();
|
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -695,7 +634,7 @@ const server = http.createServer((req, res) => {
|
|||||||
const messages = parsed.messages || [];
|
const messages = parsed.messages || [];
|
||||||
|
|
||||||
if (parsed.stream) {
|
if (parsed.stream) {
|
||||||
// Real streaming: open SSE, emit role chunk, then forward kimi deltas.
|
// Real streaming: open SSE, emit role chunk, then forward codex deltas.
|
||||||
res.writeHead(200, {
|
res.writeHead(200, {
|
||||||
'Content-Type': 'text/event-stream',
|
'Content-Type': 'text/event-stream',
|
||||||
'Cache-Control': 'no-cache',
|
'Cache-Control': 'no-cache',
|
||||||
@@ -706,7 +645,7 @@ const server = http.createServer((req, res) => {
|
|||||||
|
|
||||||
// Heartbeat keepalive: OpenClaw's LLM idle watchdog aborts a turn on
|
// Heartbeat keepalive: OpenClaw's LLM idle watchdog aborts a turn on
|
||||||
// any >120s gap between SSE stream events (default timeoutSeconds),
|
// any >120s gap between SSE stream events (default timeoutSeconds),
|
||||||
// not on total run length. During long thinking/tool/MCP phases Kimi
|
// not on total run length. During long thinking/tool/MCP phases Codex
|
||||||
// emits stream-json events we don't forward, so the SSE stream can go
|
// emits stream-json events we don't forward, so the SSE stream can go
|
||||||
// silent well past that window. Every write resets `lastWrite`; a 5s
|
// silent well past that window. Every write resets `lastWrite`; a 5s
|
||||||
// ticker emits an empty-content delta once 25s of silence elapses —
|
// ticker emits an empty-content delta once 25s of silence elapses —
|
||||||
@@ -723,7 +662,7 @@ const server = http.createServer((req, res) => {
|
|||||||
if (Date.now() - lastWrite >= 25_000) write({ content: '' }, null);
|
if (Date.now() - lastWrite >= 25_000) write({ content: '' }, null);
|
||||||
}, 5_000);
|
}, 5_000);
|
||||||
|
|
||||||
// Propagate a client/gateway disconnect down to the kimi child so an
|
// Propagate a client/gateway disconnect down to the codex child so an
|
||||||
// abandoned turn (e.g. OpenClaw's idle watchdog gave up) is killed
|
// abandoned turn (e.g. OpenClaw's idle watchdog gave up) is killed
|
||||||
// instead of finishing invisibly and burning quota. `done` guards
|
// instead of finishing invisibly and burning quota. `done` guards
|
||||||
// against the normal res.end() 'close' also aborting.
|
// against the normal res.end() 'close' also aborting.
|
||||||
26
ai/adolf-llm/service-block.yml
Normal file
26
ai/adolf-llm/service-block.yml
Normal file
@@ -0,0 +1,26 @@
|
|||||||
|
# adolf-llm — conversational Codex-CLI wrapper (P2, :8010). OpenAI-compatible,
|
||||||
|
# model id "adolf". Real streaming, chat_id session mapping (1:1 `codex exec
|
||||||
|
# resume`), media persistence, shared MCP via a generated $CODEX_HOME/config.toml.
|
||||||
|
# Needs `codex login` credentials seeded into its own home volume.
|
||||||
|
#
|
||||||
|
# Migrated off Kimi CLI (2026-07-31) for cost: the Moonshot subscription is
|
||||||
|
# replaced by the existing ChatGPT plan. NOTE the home volume changed from
|
||||||
|
# adolf-llm-home (/root/.kimi-code) to adolf-llm-codex-home (/root/.codex) —
|
||||||
|
# the old volume holds Kimi OAuth creds and can be dropped once this is verified.
|
||||||
|
#
|
||||||
|
# Orchestrator: merge this `adolf-llm` service + the two named volumes into
|
||||||
|
# ai/docker-compose.yml (do NOT edit that file here).
|
||||||
|
services:
|
||||||
|
adolf-llm:
|
||||||
|
build: ./adolf-llm
|
||||||
|
container_name: adolf-llm
|
||||||
|
ports:
|
||||||
|
- "8010:8010"
|
||||||
|
volumes:
|
||||||
|
- adolf-llm-workspace:/workspace
|
||||||
|
- adolf-llm-codex-home:/root/.codex
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
adolf-llm-workspace:
|
||||||
|
adolf-llm-codex-home:
|
||||||
@@ -87,7 +87,7 @@ agents:
|
|||||||
prompt_source:
|
prompt_source:
|
||||||
current: "container adolf:/home/node/.openclaw/workspace/SOUL.md — lives in the adolf-state Docker VOLUME, live-editable, zero git history. This is the kb#156 migration-debt state; recorded here per kb#134's brief, not migrated."
|
current: "container adolf:/home/node/.openclaw/workspace/SOUL.md — lives in the adolf-state Docker VOLUME, live-editable, zero git history. This is the kb#156 migration-debt state; recorded here per kb#134's brief, not migrated."
|
||||||
companion_files: ["AGENTS.md", "IDENTITY.md", "TOOLS.md", "USER.md", "HEARTBEAT.md"] # same workspace, same volume, same debt
|
companion_files: ["AGENTS.md", "IDENTITY.md", "TOOLS.md", "USER.md", "HEARTBEAT.md"] # same workspace, same volume, same debt
|
||||||
target: "git-controlled path once kb#156 lands (e.g. openai/personas/adolf/SOUL.md), deployed read-only into the volume"
|
target: "git-controlled path once kb#156 lands (e.g. ai/personas/adolf/SOUL.md), deployed read-only into the volume"
|
||||||
trust_class: trusted
|
trust_class: trusted
|
||||||
preferred_tier: large
|
preferred_tier: large
|
||||||
backbone: kimi # resolves via model-registry.yaml (adolf-llm Kimi-CLI wrapper endpoint). Swap this ONE field to rebackbone Adolf.
|
backbone: kimi # resolves via model-registry.yaml (adolf-llm Kimi-CLI wrapper endpoint). Swap this ONE field to rebackbone Adolf.
|
||||||
@@ -159,7 +159,7 @@ agents:
|
|||||||
openclaw-tools: null # no filter in openclaw.json — already minimal (5/5 kept)
|
openclaw-tools: null # no filter in openclaw.json — already minimal (5/5 kept)
|
||||||
note: >
|
note: >
|
||||||
"scoped core tools" per kb#134's brief, now REAL at both levels: this
|
"scoped core tools" per kb#134's brief, now REAL at both levels: this
|
||||||
IS openai/openclaw.json's live mcp.servers block (server selection)
|
IS ai/openclaw.json's live mcp.servers block (server selection)
|
||||||
plus its per-server toolFilter.include (tool selection, kb#144). Not
|
plus its per-server toolFilter.include (tool selection, kb#144). Not
|
||||||
a narrower aspirational allowlist — validate_capability_grants.py
|
a narrower aspirational allowlist — validate_capability_grants.py
|
||||||
cross-checks both against the live config. LiteLLM virtual-key
|
cross-checks both against the live config. LiteLLM virtual-key
|
||||||
@@ -171,7 +171,7 @@ agents:
|
|||||||
note: >
|
note: >
|
||||||
Reachable model tiers derived at read time from preferred_tier via
|
Reachable model tiers derived at read time from preferred_tier via
|
||||||
agent_registry.py:litellm_key_spec() — not duplicated here. Actual
|
agent_registry.py:litellm_key_spec() — not duplicated here. Actual
|
||||||
virtual-key provisioning happens via openai/provision_litellm_keys.py
|
virtual-key provisioning happens via ai/provision_litellm_keys.py
|
||||||
against the live LiteLLM proxy; NOT run automatically by this
|
against the live LiteLLM proxy; NOT run automatically by this
|
||||||
registry (privileged action, requires the LiteLLM master key —
|
registry (privileged action, requires the LiteLLM master key —
|
||||||
kb#147 handover step, see task comment).
|
kb#147 handover step, see task comment).
|
||||||
@@ -187,7 +187,7 @@ agents:
|
|||||||
(depends on this registry existing first). The three banks above
|
(depends on this registry existing first). The three banks above
|
||||||
are kb#153's target, not current fact for the hooks.
|
are kb#153's target, not current fact for the hooks.
|
||||||
kb#169 (2026-07-26): the SEPARATE raw hindsight MCP tool surface
|
kb#169 (2026-07-26): the SEPARATE raw hindsight MCP tool surface
|
||||||
(mcp.servers.hindsight in adolf/openclaw.json + openai/shared-
|
(mcp.servers.hindsight in adolf/openclaw.json + ai/shared-
|
||||||
mcp.json — recall/retain/reflect/etc. callable directly by the
|
mcp.json — recall/retain/reflect/etc. callable directly by the
|
||||||
model, bypassing #153's interlocutor-scoping entirely) has been
|
model, bypassing #153's interlocutor-scoping entirely) has been
|
||||||
repointed from http://hindsight:8888/mcp/adolf/ (the unpartitioned
|
repointed from http://hindsight:8888/mcp/adolf/ (the unpartitioned
|
||||||
@@ -436,7 +436,7 @@ capability_grant_status:
|
|||||||
each agent's model allow-list (derived from preferred_tier x
|
each agent's model allow-list (derived from preferred_tier x
|
||||||
model-registry.yaml routing.tiers, non-metered only unless opted in)
|
model-registry.yaml routing.tiers, non-metered only unless opted in)
|
||||||
and a default budget from trust_classes[...].default_budget_usd.
|
and a default budget from trust_classes[...].default_budget_usd.
|
||||||
openai/provision_litellm_keys.py turns that spec into LiteLLM
|
ai/provision_litellm_keys.py turns that spec into LiteLLM
|
||||||
/key/generate calls. Verified with --dry-run (prints the exact payload
|
/key/generate calls. Verified with --dry-run (prints the exact payload
|
||||||
per agent, no network call) — actually creating keys needs
|
per agent, no network call) — actually creating keys needs
|
||||||
LITELLM_MASTER_KEY against the live proxy, a privileged write this task
|
LITELLM_MASTER_KEY against the live proxy, a privileged write this task
|
||||||
@@ -448,7 +448,7 @@ capability_grant_status:
|
|||||||
adolf's real, live MCP surface (git-controlled): its server set matches
|
adolf's real, live MCP surface (git-controlled): its server set matches
|
||||||
tool_allowlist.mcp_servers, and each server's toolFilter.include (added
|
tool_allowlist.mcp_servers, and each server's toolFilter.include (added
|
||||||
kb#144 first pass) matches tool_allowlist.mcp_tool_filter — exactly.
|
kb#144 first pass) matches tool_allowlist.mcp_tool_filter — exactly.
|
||||||
openai/validate_capability_grants.py checks this automatically, read-only,
|
ai/validate_capability_grants.py checks this automatically, read-only,
|
||||||
no live changes. Real and correct for OpenClaw's OWN MCP client surface —
|
no live changes. Real and correct for OpenClaw's OWN MCP client surface —
|
||||||
but per the kb#144 first-pass release comment's wire.jsonl proof, this
|
but per the kb#144 first-pass release comment's wire.jsonl proof, this
|
||||||
layer alone does NOT reach the model on Adolf's kimi backbone (see
|
layer alone does NOT reach the model on Adolf's kimi backbone (see
|
||||||
@@ -462,7 +462,7 @@ capability_grant_status:
|
|||||||
wire.jsonl inspection) proved the first pass's openclaw.json-only fix
|
wire.jsonl inspection) proved the first pass's openclaw.json-only fix
|
||||||
left Kimi's actual tool bundle unchanged (23/32/29/5, not 14/24/9/5).
|
left Kimi's actual tool bundle unchanged (23/32/29/5, not 14/24/9/5).
|
||||||
Root cause: Kimi CLI auto-discovers a project-root `.mcp.json` that
|
Root cause: Kimi CLI auto-discovers a project-root `.mcp.json` that
|
||||||
adolf-llm/server.js's writeMcpConfig() seeds from openai/shared-mcp.json
|
adolf-llm/server.js's writeMcpConfig() seeds from ai/shared-mcp.json
|
||||||
— a completely separate config from openclaw.json, read by a separate
|
— a completely separate config from openclaw.json, read by a separate
|
||||||
MCP client (Kimi CLI inside the adolf-llm container, not OpenClaw inside
|
MCP client (Kimi CLI inside the adolf-llm container, not OpenClaw inside
|
||||||
the adolf container). shared-mcp.json now carries the same per-server
|
the adolf container). shared-mcp.json now carries the same per-server
|
||||||
@@ -470,14 +470,14 @@ capability_grant_status:
|
|||||||
McpServerCommonFields.enabledTools, applied via computeEnabledNames;
|
McpServerCommonFields.enabledTools, applied via computeEnabledNames;
|
||||||
confirmed by decompiling the installed @moonshot-ai/kimi-code package's
|
confirmed by decompiling the installed @moonshot-ai/kimi-code package's
|
||||||
dist/main.mjs, both copies of the function/schema, no live container
|
dist/main.mjs, both copies of the function/schema, no live container
|
||||||
touched). openai/validate_capability_grants.py now cross-checks THIS
|
touched). ai/validate_capability_grants.py now cross-checks THIS
|
||||||
file too (load_shared_mcp_enabled_tools), same exit-0-or-fail contract
|
file too (load_shared_mcp_enabled_tools), same exit-0-or-fail contract
|
||||||
as the openclaw.json check. marketplace is absent from shared-mcp.json
|
as the openclaw.json check. marketplace is absent from shared-mcp.json
|
||||||
entirely (pre-existing: Kimi's session never had it) — not asserted by
|
entirely (pre-existing: Kimi's session never had it) — not asserted by
|
||||||
the validator for that server, by design, not a gap this task opened.
|
the validator for that server, by design, not a gap this task opened.
|
||||||
NOT YET ACTIVATED: shared-mcp.json IS bind-mounted read-only into
|
NOT YET ACTIVATED: shared-mcp.json IS bind-mounted read-only into
|
||||||
adolf-llm (`./shared-mcp.json:/shared-mcp.json:ro` in
|
adolf-llm (`./shared-mcp.json:/shared-mcp.json:ro` in
|
||||||
openai/docker-compose.yml) so the file on disk is already what the
|
ai/docker-compose.yml) so the file on disk is already what the
|
||||||
container would read — but adolf-llm/server.js loads it ONCE into a
|
container would read — but adolf-llm/server.js loads it ONCE into a
|
||||||
module-level variable at process start (not per-request), so editing the
|
module-level variable at process start (not per-request), so editing the
|
||||||
file alone does not take effect; `docker compose restart adolf-llm` is
|
file alone does not take effect; `docker compose restart adolf-llm` is
|
||||||
@@ -210,6 +210,33 @@ def litellm_key_spec(registry, agent_id, model_registry=None):
|
|||||||
if name not in models:
|
if name not in models:
|
||||||
models.append(name)
|
models.append(name)
|
||||||
|
|
||||||
|
# kb#128 gap (flagged 2026-07-26, closed 2026-07-30): the raw litellm_
|
||||||
|
# model_names above (e.g. "ollama/gemma3:4b") are the BACKING deployments
|
||||||
|
# for openai/litellm-config.yaml's alias model_names -- tier-small/
|
||||||
|
# tier-large (alvis's "tier" routing mode) and auto_router/
|
||||||
|
# complexity_router (alvis's "automatic" routing mode). Without granting
|
||||||
|
# the aliases too, a provisioned key could reach a model directly but not
|
||||||
|
# by tier or through the router, so "all three routing modes exercisable"
|
||||||
|
# (kb#128 acceptance) wasn't actually true per-agent. Gate exactly like
|
||||||
|
# the raw grants above -- reachable tiers, not a separate allow-list --
|
||||||
|
# so an agent's routing-mode access never exceeds its direct-model access:
|
||||||
|
# - "small" reachable -> tier-small (mirrors the always-granted small
|
||||||
|
# pool; every agent with a backbone gets at least this).
|
||||||
|
# - "large" reachable -> tier-large, PLUS auto_router/complexity_router.
|
||||||
|
# Both routers' pools include tier-large in their upper bands (COMPLEX/
|
||||||
|
# REASONING, or the semantic "complex reasoning" route), so granting
|
||||||
|
# them to a small-only (sandboxed) agent would let automatic routing
|
||||||
|
# escalate it past its trust class -- exactly the asymmetry
|
||||||
|
# _reachable_tiers()/kb#147 exists to prevent. A small-only agent gets
|
||||||
|
# neither router: it can still call tier-small directly.
|
||||||
|
reachable = _reachable_tiers(a.get("preferred_tier"))
|
||||||
|
if "small" in reachable and "tier-small" not in models:
|
||||||
|
models.append("tier-small")
|
||||||
|
if "large" in reachable:
|
||||||
|
for alias in ("tier-large", "auto_router", "complexity_router"):
|
||||||
|
if alias not in models:
|
||||||
|
models.append(alias)
|
||||||
|
|
||||||
classes = registry.get("trust_classes", {})
|
classes = registry.get("trust_classes", {})
|
||||||
cls = classes.get(a["trust_class"], {})
|
cls = classes.get(a["trust_class"], {})
|
||||||
return {
|
return {
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
{
|
{
|
||||||
"_note": "kb#128 (A2A-16): human-readable source of truth for the auto_router route set. NOT loaded from this path at runtime -- litellm-config.yaml's `auto_router` deployment inlines this same `routes` array as a literal JSON string via litellm_params.auto_router_config. Reason (verified hands-on 2026-07-26 against litellm:main-latest): the auto_router_config_path loader (AutoRouter._load_semantic_routing_routes -> SemanticRouter.from_json) unconditionally builds a raw semantic_router encoder from encoder_type/encoder_name and requires a real provider API key even for a local model name like bge-m3 -- this IS the open Auto Router v2 embedding bug the task brief warned about. The auto_router_config (inline-string) loader (_load_auto_router_routes_from_config_json) only reads the `routes` key and builds Route objects directly, with zero encoder bootstrap -- confirmed working end-to-end: real litellm.embedding(model=ollama/bge-m3) calls, zero metered API spend, 'hi there' -> ollama/gemma3:4b, a refactor/dependency-injection prompt -> kimi-agent. Keep the two `routes` arrays in sync by hand when editing either.",
|
"_note": "kb#128 (A2A-16): human-readable source of truth for the auto_router route set. NOT loaded from this path at runtime -- litellm-config.yaml's `auto_router` deployment inlines this same `routes` array as a literal JSON string via litellm_params.auto_router_config. Reason (verified hands-on 2026-07-26 against litellm:main-latest): the auto_router_config_path loader (AutoRouter._load_semantic_routing_routes -> SemanticRouter.from_json) unconditionally builds a raw semantic_router encoder from encoder_type/encoder_name and requires a real provider API key even for a local model name like bge-m3 -- this IS the open Auto Router v2 embedding bug the task brief warned about. The auto_router_config (inline-string) loader (_load_auto_router_routes_from_config_json) only reads the `routes` key and builds Route objects directly, with zero encoder bootstrap -- confirmed working end-to-end: real litellm.embedding(model=ollama/bge-m3) calls, zero metered API spend, 'hi there' -> ollama/gemma3:4b, a refactor/dependency-injection prompt -> codex-agent (was kimi-agent until the 2026-08-01 Kimi purge). Keep the two `routes` arrays in sync by hand when editing either.",
|
||||||
"encoder_type": "litellm",
|
"encoder_type": "litellm",
|
||||||
"encoder_name": "bge-m3",
|
"encoder_name": "bge-m3",
|
||||||
"routes": [
|
"routes": [
|
||||||
@@ -21,7 +21,7 @@
|
|||||||
"score_threshold": 0.5
|
"score_threshold": 0.5
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"name": "kimi-agent",
|
"name": "codex-agent",
|
||||||
"description": "Complex reasoning, multi-step planning, coding, or anything needing tool use and deep context.",
|
"description": "Complex reasoning, multi-step planning, coding, or anything needing tool use and deep context.",
|
||||||
"utterances": [
|
"utterances": [
|
||||||
"write a function that parses this log file and extracts errors",
|
"write a function that parses this log file and extracts errors",
|
||||||
@@ -2,8 +2,8 @@
|
|||||||
# Backup script for hindsight (Adolf's long-term memory bank) and the
|
# Backup script for hindsight (Adolf's long-term memory bank) and the
|
||||||
# openai_adolf-state Docker volume (Matrix E2EE identity/sessions + config).
|
# openai_adolf-state Docker volume (Matrix E2EE identity/sessions + config).
|
||||||
# Mirrors the seafile/vaultwarden/openai-llm-dbs backup.sh pattern (same repo):
|
# Mirrors the seafile/vaultwarden/openai-llm-dbs backup.sh pattern (same repo):
|
||||||
# dump/tar via `docker exec`, gzip, retention of last 5, Zabbix freshness
|
# dump/tar via `docker exec`, gzip, retention of last 5. Backup-freshness
|
||||||
# trapper item per target.
|
# monitored via .age items.
|
||||||
#
|
#
|
||||||
# hindsight is an embedded Postgres (pg0) instance living at
|
# hindsight is an embedded Postgres (pg0) instance living at
|
||||||
# /mnt/ssd/dbs/hindsight on the host, bind-mounted into the `hindsight`
|
# /mnt/ssd/dbs/hindsight on the host, bind-mounted into the `hindsight`
|
||||||
@@ -17,7 +17,7 @@
|
|||||||
# (docker exec has access via the mount; no host-side permission needed).
|
# (docker exec has access via the mount; no host-side permission needed).
|
||||||
#
|
#
|
||||||
# Run every 3 days via root crontab (same schedule as sibling backups), e.g.:
|
# Run every 3 days via root crontab (same schedule as sibling backups), e.g.:
|
||||||
# 0 4 */3 * * /home/alvis/agap_git/openai/backup-hindsight-adolf.sh >> /mnt/backups/hindsight-adolf/backup.log 2>&1
|
# 0 4 */3 * * /home/alvis/agap_git/ai/backup-hindsight-adolf.sh >> /mnt/backups/hindsight-adolf/backup.log 2>&1
|
||||||
#
|
#
|
||||||
# Restore:
|
# Restore:
|
||||||
# # hindsight (drop+recreate the DB first if restoring into a fresh instance,
|
# # hindsight (drop+recreate the DB first if restoring into a fresh instance,
|
||||||
@@ -36,30 +36,14 @@
|
|||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
BACKUP_DIR="/mnt/backups/hindsight-adolf"
|
BACKUP_DIR="/mnt/backups/hindsight-adolf"
|
||||||
ZABBIX_TOKEN_FILE="/root/.zabbix_token"
|
|
||||||
ZABBIX_URL="http://192.168.1.4:81/api_jsonrpc.php"
|
|
||||||
|
|
||||||
DATE=$(date '+%Y%m%d-%H%M')
|
DATE=$(date '+%Y%m%d-%H%M')
|
||||||
DEST="$BACKUP_DIR/$DATE"
|
DEST="$BACKUP_DIR/$DATE"
|
||||||
|
|
||||||
mkdir -p "$DEST"
|
mkdir -p "$DEST"
|
||||||
|
# Backup-freshness monitoring is now done via .age items (calculated fields showing
|
||||||
notify_zabbix() {
|
# age of the backup). The .ts (timestamp) trappers were unreliable (history.push not
|
||||||
local itemid="$1" label="$2"
|
# landing); removed in kb#189 in favor of .age overdue triggers.
|
||||||
if [[ -f "$ZABBIX_TOKEN_FILE" ]]; then
|
|
||||||
local token now_epoch
|
|
||||||
token=$(cat "$ZABBIX_TOKEN_FILE")
|
|
||||||
now_epoch=$(date '+%s')
|
|
||||||
env -u HTTPS_PROXY -u HTTP_PROXY -u ALL_PROXY -u https_proxy -u http_proxy -u all_proxy \
|
|
||||||
curl -s -X POST "$ZABBIX_URL" \
|
|
||||||
-H "Content-Type: application/json" \
|
|
||||||
-H "Authorization: Bearer $token" \
|
|
||||||
-d "{\"jsonrpc\":\"2.0\",\"method\":\"history.push\",\"id\":1,\"params\":{\"itemid\":\"$itemid\",\"value\":$now_epoch}}" > /dev/null \
|
|
||||||
&& echo "Zabbix notified ($label=$now_epoch)."
|
|
||||||
else
|
|
||||||
echo "WARNING: $ZABBIX_TOKEN_FILE not found -- skipped Zabbix freshness push for $label." >&2
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
# --- hindsight (Postgres logical dump, live/read-only) ---
|
# --- hindsight (Postgres logical dump, live/read-only) ---
|
||||||
echo "Dumping hindsight..."
|
echo "Dumping hindsight..."
|
||||||
@@ -67,13 +51,11 @@ docker exec -e PGPASSWORD=hindsight hindsight \
|
|||||||
/home/hindsight/.pg0/installation/18.1.0/bin/pg_dump -U hindsight -h 127.0.0.1 -p 5432 hindsight \
|
/home/hindsight/.pg0/installation/18.1.0/bin/pg_dump -U hindsight -h 127.0.0.1 -p 5432 hindsight \
|
||||||
| gzip > "$DEST/hindsight.sql.gz"
|
| gzip > "$DEST/hindsight.sql.gz"
|
||||||
echo "Dumped: hindsight -> $DEST/hindsight.sql.gz"
|
echo "Dumped: hindsight -> $DEST/hindsight.sql.gz"
|
||||||
notify_zabbix "70639" "hindsight.backup.ts"
|
|
||||||
|
|
||||||
# --- adolf-state (tar the volume from inside the adolf container) ---
|
# --- adolf-state (tar the volume from inside the adolf container) ---
|
||||||
echo "Archiving adolf-state..."
|
echo "Archiving adolf-state..."
|
||||||
docker exec adolf tar czf - -C /home/node/.openclaw . > "$DEST/adolf-state.tar.gz"
|
docker exec adolf tar czf - -C /home/node/.openclaw . > "$DEST/adolf-state.tar.gz"
|
||||||
echo "Archived: adolf-state -> $DEST/adolf-state.tar.gz"
|
echo "Archived: adolf-state -> $DEST/adolf-state.tar.gz"
|
||||||
notify_zabbix "70640" "adolf-state.backup.ts"
|
|
||||||
|
|
||||||
echo "$(date): Backup complete: $DEST"
|
echo "$(date): Backup complete: $DEST"
|
||||||
ls -la "$DEST/"
|
ls -la "$DEST/"
|
||||||
@@ -2,12 +2,12 @@
|
|||||||
# Backup script for litellm-db and langfuse-db (openai stack postgres containers).
|
# Backup script for litellm-db and langfuse-db (openai stack postgres containers).
|
||||||
# litellm-db holds provisioned virtual keys + spend; langfuse-db holds all traces.
|
# litellm-db holds provisioned virtual keys + spend; langfuse-db holds all traces.
|
||||||
# Mirrors the seafile/vaultwarden backup.sh pattern (same repo): dump via
|
# Mirrors the seafile/vaultwarden backup.sh pattern (same repo): dump via
|
||||||
# `docker exec <container> pg_dump`, gzip, retention of last 5, Zabbix freshness
|
# `docker exec <container> pg_dump`, gzip, retention of last 5. Uses pg_dump (safe
|
||||||
# trapper item per DB. Uses pg_dump (safe against a live/running DB, no downtime
|
# against a live/running DB, no downtime needed — unlike gitea's stop-the-world dump).
|
||||||
# needed — unlike gitea's stop-the-world dump).
|
# Backup-freshness monitored via .age items.
|
||||||
#
|
#
|
||||||
# Run every 3 days via root crontab (same schedule as vaultwarden/seafile), e.g.:
|
# Run every 3 days via root crontab (same schedule as vaultwarden/seafile), e.g.:
|
||||||
# 0 3 */3 * * /home/alvis/agap_git/openai/backup-llm-dbs.sh >> /mnt/backups/openai-llm-dbs/backup.log 2>&1
|
# 0 3 */3 * * /home/alvis/agap_git/ai/backup-llm-dbs.sh >> /mnt/backups/openai-llm-dbs/backup.log 2>&1
|
||||||
#
|
#
|
||||||
# Restore (litellm-db example, langfuse-db is identical with its own container/user/db):
|
# Restore (litellm-db example, langfuse-db is identical with its own container/user/db):
|
||||||
# gunzip -c /mnt/backups/openai-llm-dbs/<DATE>/litellm-db.sql.gz | \
|
# gunzip -c /mnt/backups/openai-llm-dbs/<DATE>/litellm-db.sql.gz | \
|
||||||
@@ -21,42 +21,24 @@
|
|||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
BACKUP_DIR="/mnt/backups/openai-llm-dbs"
|
BACKUP_DIR="/mnt/backups/openai-llm-dbs"
|
||||||
ZABBIX_TOKEN_FILE="/root/.zabbix_token"
|
|
||||||
ZABBIX_URL="http://192.168.1.4:81/api_jsonrpc.php"
|
|
||||||
|
|
||||||
DATE=$(date '+%Y%m%d-%H%M')
|
DATE=$(date '+%Y%m%d-%H%M')
|
||||||
DEST="$BACKUP_DIR/$DATE"
|
DEST="$BACKUP_DIR/$DATE"
|
||||||
|
|
||||||
mkdir -p "$DEST"
|
mkdir -p "$DEST"
|
||||||
|
# Backup-freshness monitoring is now done via .age items (calculated fields showing
|
||||||
notify_zabbix() {
|
# age of the backup). The .ts (timestamp) trappers were unreliable (history.push not
|
||||||
local itemid="$1" label="$2"
|
# landing); removed in kb#189 in favor of .age overdue triggers.
|
||||||
if [[ -f "$ZABBIX_TOKEN_FILE" ]]; then
|
|
||||||
local token now_epoch
|
|
||||||
token=$(cat "$ZABBIX_TOKEN_FILE")
|
|
||||||
now_epoch=$(date '+%s')
|
|
||||||
env -u HTTPS_PROXY -u HTTP_PROXY -u ALL_PROXY -u https_proxy -u http_proxy -u all_proxy \
|
|
||||||
curl -s -X POST "$ZABBIX_URL" \
|
|
||||||
-H "Content-Type: application/json" \
|
|
||||||
-H "Authorization: Bearer $token" \
|
|
||||||
-d "{\"jsonrpc\":\"2.0\",\"method\":\"history.push\",\"id\":1,\"params\":{\"itemid\":\"$itemid\",\"value\":$now_epoch}}" > /dev/null \
|
|
||||||
&& echo "Zabbix notified ($label=$now_epoch)."
|
|
||||||
else
|
|
||||||
echo "WARNING: $ZABBIX_TOKEN_FILE not found -- skipped Zabbix freshness push for $label." >&2
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
# --- litellm-db ---
|
# --- litellm-db ---
|
||||||
echo "Dumping litellm-db..."
|
echo "Dumping litellm-db..."
|
||||||
docker exec litellm-db pg_dump -U litellm litellm | gzip > "$DEST/litellm-db.sql.gz"
|
docker exec litellm-db pg_dump -U litellm litellm | gzip > "$DEST/litellm-db.sql.gz"
|
||||||
echo "Dumped: litellm-db -> $DEST/litellm-db.sql.gz"
|
echo "Dumped: litellm-db -> $DEST/litellm-db.sql.gz"
|
||||||
notify_zabbix "70637" "litellm.db.backup.ts"
|
|
||||||
|
|
||||||
# --- langfuse-db ---
|
# --- langfuse-db ---
|
||||||
echo "Dumping langfuse-db..."
|
echo "Dumping langfuse-db..."
|
||||||
docker exec langfuse-db pg_dump -U langfuse langfuse | gzip > "$DEST/langfuse-db.sql.gz"
|
docker exec langfuse-db pg_dump -U langfuse langfuse | gzip > "$DEST/langfuse-db.sql.gz"
|
||||||
echo "Dumped: langfuse-db -> $DEST/langfuse-db.sql.gz"
|
echo "Dumped: langfuse-db -> $DEST/langfuse-db.sql.gz"
|
||||||
notify_zabbix "70638" "langfuse.db.backup.ts"
|
|
||||||
|
|
||||||
echo "$(date): Backup complete: $DEST"
|
echo "$(date): Backup complete: $DEST"
|
||||||
ls -la "$DEST/"
|
ls -la "$DEST/"
|
||||||
@@ -50,7 +50,7 @@ can't go through the agentic CLI).
|
|||||||
## Smoke test
|
## Smoke test
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd /home/alvis/agap_git/openai
|
cd /home/alvis/agap_git/ai
|
||||||
docker build -t cognee-llm:local ./cognee-llm
|
docker build -t cognee-llm:local ./cognee-llm
|
||||||
docker run --rm -d --name cognee-llm-smoke -p 18011:8011 cognee-llm:local
|
docker run --rm -d --name cognee-llm-smoke -p 18011:8011 cognee-llm:local
|
||||||
curl -s http://localhost:18011/v1/models
|
curl -s http://localhost:18011/v1/models
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
# Intended service block for /home/alvis/agap_git/openai/docker-compose.yml.
|
# Intended service block for /home/alvis/agap_git/ai/docker-compose.yml.
|
||||||
# Not wired in yet (see P3 task note) — orchestrator merges this in and adds
|
# Not wired in yet (see P3 task note) — orchestrator merges this in and adds
|
||||||
# `cognee-llm-home` to the top-level `volumes:` section.
|
# `cognee-llm-home` to the top-level `volumes:` section.
|
||||||
|
|
||||||
@@ -1,3 +1,10 @@
|
|||||||
|
# kb#220: dir renamed openai/ -> ai/ (nothing in it is OpenAI). Pin the
|
||||||
|
# compose project name explicitly so container/network/volume names
|
||||||
|
# (e.g. openai_adolf-state) stay stable across the rename -- otherwise
|
||||||
|
# Compose derives the project name from the directory basename and the
|
||||||
|
# rename would orphan the existing volume/network.
|
||||||
|
name: openai
|
||||||
|
|
||||||
services:
|
services:
|
||||||
litellm-db:
|
litellm-db:
|
||||||
image: postgres:16-alpine
|
image: postgres:16-alpine
|
||||||
@@ -49,13 +56,12 @@ services:
|
|||||||
retries: 5
|
retries: 5
|
||||||
start_period: 20s
|
start_period: 20s
|
||||||
|
|
||||||
kimi-agent:
|
# kimi-agent — REMOVED 2026-08-01 (Kimi purge). Was the only large-tier
|
||||||
build: ./kimi-agent
|
# deployment behind LiteLLM; `tier-large`, the auto_router complex route and
|
||||||
container_name: kimi-agent
|
# their fallbacks now point at the codex-backed adolf-llm wrapper instead
|
||||||
volumes:
|
# (litellm-config.yaml model_name: codex-agent). The kimi-agent-home volume
|
||||||
- /home/alvis/kimi-workspace:/workspace
|
# and /home/alvis/kimi-workspace are left on disk deliberately — drop them
|
||||||
- kimi-agent-home:/root/.kimi-code
|
# once the Codex path has proven itself.
|
||||||
restart: unless-stopped
|
|
||||||
|
|
||||||
langfuse-db:
|
langfuse-db:
|
||||||
image: postgres:16-alpine
|
image: postgres:16-alpine
|
||||||
@@ -75,20 +81,75 @@ services:
|
|||||||
retries: 5
|
retries: 5
|
||||||
start_period: 10s
|
start_period: 10s
|
||||||
|
|
||||||
|
# kb#148 (A2A-16): Langfuse v3 split the monolith into langfuse-web +
|
||||||
|
# langfuse-worker, and added ClickHouse (event/analytics store), Redis
|
||||||
|
# (queue) and S3-compatible blob storage (MinIO here) as hard
|
||||||
|
# dependencies -- Postgres alone is no longer sufficient, unlike v2.
|
||||||
|
# NOT YET ACTIVATED: v2's existing trace history (3175+ traces per
|
||||||
|
# DESIGN-a2a-agents.md §7, confirmed live 2026-07-26) lives only in the
|
||||||
|
# langfuse-db Postgres volume in v2's schema. Langfuse's official v2->v3
|
||||||
|
# upgrade path requires running the migration entrypoint once against
|
||||||
|
# this data (langfuse/langfuse:3's container runs pending Postgres
|
||||||
|
# migrations automatically on boot, but the ClickHouse backfill of
|
||||||
|
# historical trace data is a separate, explicit step -- see Langfuse's
|
||||||
|
# "Upgrade from v2 to v3" guide) BEFORE cutting traffic over, or the old
|
||||||
|
# traces are stranded. That migration is a live-data operation with real
|
||||||
|
# downtime and rollback risk, so it is out of scope for an unattended
|
||||||
|
# edit -- see kb#148's report for the exact handoff commands. New
|
||||||
|
# volumes (clickhouse/minio/redis below) also need their host dirs
|
||||||
|
# created + chowned first (root-gated, same pattern as kb#87's
|
||||||
|
# hindsight-cache dir).
|
||||||
|
langfuse-worker:
|
||||||
|
image: docker.io/langfuse/langfuse-worker:3
|
||||||
|
container_name: langfuse-worker
|
||||||
|
depends_on: &langfuse-depends-on
|
||||||
|
langfuse-db:
|
||||||
|
condition: service_healthy
|
||||||
|
langfuse-minio:
|
||||||
|
condition: service_healthy
|
||||||
|
langfuse-redis:
|
||||||
|
condition: service_healthy
|
||||||
|
langfuse-clickhouse:
|
||||||
|
condition: service_healthy
|
||||||
|
environment: &langfuse-worker-env
|
||||||
|
NEXTAUTH_URL: https://lf.alogins.net
|
||||||
|
DATABASE_URL: postgresql://langfuse:langfuse@langfuse-db:5432/langfuse
|
||||||
|
SALT: 7927b3b0092afe4542274940b557becea6418a5fed79f7acd25c3a789349fdc9
|
||||||
|
ENCRYPTION_KEY: 12056e4e3cf5b9d936fedca267d4bd877a4b79fb9ff0ff32859a623d5e96c814
|
||||||
|
CLICKHOUSE_MIGRATION_URL: clickhouse://langfuse-clickhouse:9000
|
||||||
|
CLICKHOUSE_URL: http://langfuse-clickhouse:8123
|
||||||
|
CLICKHOUSE_USER: clickhouse
|
||||||
|
CLICKHOUSE_PASSWORD: f1d3bd6dc01c9741b99c633b2e167d1d
|
||||||
|
LANGFUSE_S3_EVENT_UPLOAD_BUCKET: langfuse
|
||||||
|
LANGFUSE_S3_EVENT_UPLOAD_REGION: auto
|
||||||
|
LANGFUSE_S3_EVENT_UPLOAD_ACCESS_KEY_ID: minio
|
||||||
|
LANGFUSE_S3_EVENT_UPLOAD_SECRET_ACCESS_KEY: 078dd39aada907ab40c6a4d581033cfe
|
||||||
|
LANGFUSE_S3_EVENT_UPLOAD_ENDPOINT: http://langfuse-minio:9000
|
||||||
|
LANGFUSE_S3_EVENT_UPLOAD_FORCE_PATH_STYLE: "true"
|
||||||
|
LANGFUSE_S3_EVENT_UPLOAD_PREFIX: events/
|
||||||
|
LANGFUSE_S3_MEDIA_UPLOAD_BUCKET: langfuse
|
||||||
|
LANGFUSE_S3_MEDIA_UPLOAD_REGION: auto
|
||||||
|
LANGFUSE_S3_MEDIA_UPLOAD_ACCESS_KEY_ID: minio
|
||||||
|
LANGFUSE_S3_MEDIA_UPLOAD_SECRET_ACCESS_KEY: 078dd39aada907ab40c6a4d581033cfe
|
||||||
|
LANGFUSE_S3_MEDIA_UPLOAD_ENDPOINT: http://langfuse-minio:9000
|
||||||
|
LANGFUSE_S3_MEDIA_UPLOAD_FORCE_PATH_STYLE: "true"
|
||||||
|
LANGFUSE_S3_MEDIA_UPLOAD_PREFIX: media/
|
||||||
|
REDIS_HOST: langfuse-redis
|
||||||
|
REDIS_PORT: "6379"
|
||||||
|
REDIS_AUTH: 36471006ce5b95ed4f7fb769788fe91c
|
||||||
|
restart: always
|
||||||
|
|
||||||
langfuse:
|
langfuse:
|
||||||
image: ghcr.io/langfuse/langfuse:2
|
image: docker.io/langfuse/langfuse:3
|
||||||
container_name: langfuse
|
container_name: langfuse
|
||||||
|
depends_on: *langfuse-depends-on
|
||||||
ports:
|
ports:
|
||||||
- "3200:3000"
|
- "3200:3000"
|
||||||
environment:
|
environment:
|
||||||
- DATABASE_URL=postgresql://langfuse:langfuse@langfuse-db:5432/langfuse
|
<<: *langfuse-worker-env
|
||||||
- NEXTAUTH_URL=https://lf.alogins.net
|
NEXTAUTH_SECRET: 532a746b24ac40afa39f9d317031cab94d4d6881107ea3b1209b28020f1a9761
|
||||||
- NEXTAUTH_SECRET=532a746b24ac40afa39f9d317031cab94d4d6881107ea3b1209b28020f1a9761
|
AUTH_DISABLE_SIGNUP: "true"
|
||||||
- SALT=7927b3b0092afe4542274940b557becea6418a5fed79f7acd25c3a789349fdc9
|
LANGFUSE_S3_MEDIA_UPLOAD_ENDPOINT: https://lf.alogins.net
|
||||||
- AUTH_DISABLE_SIGNUP=true
|
|
||||||
depends_on:
|
|
||||||
langfuse-db:
|
|
||||||
condition: service_healthy
|
|
||||||
restart: always
|
restart: always
|
||||||
# kb#190: langfuse's Next.js server binds the container's bridge IP,
|
# kb#190: langfuse's Next.js server binds the container's bridge IP,
|
||||||
# NOT 127.0.0.1/localhost (confirmed via `ss -tlnp` inside the
|
# NOT 127.0.0.1/localhost (confirmed via `ss -tlnp` inside the
|
||||||
@@ -102,6 +163,59 @@ services:
|
|||||||
retries: 5
|
retries: 5
|
||||||
start_period: 30s
|
start_period: 30s
|
||||||
|
|
||||||
|
langfuse-clickhouse:
|
||||||
|
image: docker.io/clickhouse/clickhouse-server:25.12
|
||||||
|
container_name: langfuse-clickhouse
|
||||||
|
user: "101:101"
|
||||||
|
environment:
|
||||||
|
- CLICKHOUSE_DB=default
|
||||||
|
- CLICKHOUSE_USER=clickhouse
|
||||||
|
- CLICKHOUSE_PASSWORD=f1d3bd6dc01c9741b99c633b2e167d1d
|
||||||
|
volumes:
|
||||||
|
- /mnt/ssd/dbs/langfuse/clickhouse-data:/var/lib/clickhouse
|
||||||
|
- /mnt/ssd/dbs/langfuse/clickhouse-logs:/var/log/clickhouse-server
|
||||||
|
restart: always
|
||||||
|
healthcheck:
|
||||||
|
test: wget --no-verbose --tries=1 --spider http://localhost:8123/ping || exit 1
|
||||||
|
interval: 5s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 10
|
||||||
|
start_period: 1s
|
||||||
|
|
||||||
|
langfuse-minio:
|
||||||
|
image: cgr.dev/chainguard/minio
|
||||||
|
container_name: langfuse-minio
|
||||||
|
entrypoint: sh
|
||||||
|
# create the 'langfuse' bucket before starting the service
|
||||||
|
command: -c 'mkdir -p /data/langfuse && minio server --address ":9000" --console-address ":9001" /data'
|
||||||
|
environment:
|
||||||
|
- MINIO_ROOT_USER=minio
|
||||||
|
- MINIO_ROOT_PASSWORD=078dd39aada907ab40c6a4d581033cfe
|
||||||
|
volumes:
|
||||||
|
- /mnt/ssd/dbs/langfuse/minio:/data
|
||||||
|
restart: always
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "mc", "ready", "local"]
|
||||||
|
interval: 1s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 5
|
||||||
|
start_period: 1s
|
||||||
|
|
||||||
|
langfuse-redis:
|
||||||
|
image: docker.io/redis:7
|
||||||
|
container_name: langfuse-redis
|
||||||
|
command: >
|
||||||
|
--requirepass 36471006ce5b95ed4f7fb769788fe91c
|
||||||
|
--maxmemory-policy noeviction
|
||||||
|
volumes:
|
||||||
|
- /mnt/ssd/dbs/langfuse/redis:/data
|
||||||
|
restart: always
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "redis-cli", "ping"]
|
||||||
|
interval: 3s
|
||||||
|
timeout: 10s
|
||||||
|
retries: 10
|
||||||
|
|
||||||
qdrant:
|
qdrant:
|
||||||
image: qdrant/qdrant
|
image: qdrant/qdrant
|
||||||
container_name: qdrant
|
container_name: qdrant
|
||||||
@@ -164,12 +278,12 @@ services:
|
|||||||
# gateway config (Matrix channel + allow-list, model provider ->
|
# gateway config (Matrix channel + allow-list, model provider ->
|
||||||
# adolf-llm:8010, MCP registry, gateway.tools.allow for cron/nodes) is
|
# adolf-llm:8010, MCP registry, gateway.tools.allow for cron/nodes) is
|
||||||
# version-controlled at agap_git/adolf/openclaw.json (repo root, alongside
|
# version-controlled at agap_git/adolf/openclaw.json (repo root, alongside
|
||||||
# this openai/ project, not nested inside it) and bind-mounted read-only
|
# this ai/ project, not nested inside it) and bind-mounted read-only
|
||||||
# over the adolf-state volume (see volumes below), so git is the single
|
# over the adolf-state volume (see volumes below), so git is the single
|
||||||
# source of truth — not a hand-edited volume file. The volume still
|
# source of truth — not a hand-edited volume file. The volume still
|
||||||
# holds runtime state only (Matrix crypto/devices, credentials, sessions,
|
# holds runtime state only (Matrix crypto/devices, credentials, sessions,
|
||||||
# workspace/SOUL.md, logs). Matrix creds and ADOLF_KEY come from
|
# workspace/SOUL.md, logs). Matrix creds and ADOLF_KEY come from
|
||||||
# openai/.env (gitignored, never committed). Source tree: /home/alvis/adolf.
|
# ai/.env (gitignored, never committed). Source tree: /home/alvis/adolf.
|
||||||
# To change config: edit ../adolf/openclaw.json + restart adolf.
|
# To change config: edit ../adolf/openclaw.json + restart adolf.
|
||||||
adolf:
|
adolf:
|
||||||
build:
|
build:
|
||||||
@@ -215,53 +329,73 @@ services:
|
|||||||
# mcp.servers.agap.headers.Authorization via ${AGAP_MCP_TOKEN}
|
# mcp.servers.agap.headers.Authorization via ${AGAP_MCP_TOKEN}
|
||||||
# substitution, and read directly by the todoist-capture plugin's
|
# substitution, and read directly by the todoist-capture plugin's
|
||||||
# /capture-idea POST. The token must map to agent id `adolf` in
|
# /capture-idea POST. The token must map to agent id `adolf` in
|
||||||
# agap-mcp's AGAP_MCP_AGENT_TOKENS. Sourced from openai/.env
|
# agap-mcp's AGAP_MCP_AGENT_TOKENS. Sourced from ai/.env
|
||||||
# (gitignored); never inlined here.
|
# (gitignored); never inlined here.
|
||||||
- AGAP_MCP_TOKEN=${AGAP_MCP_TOKEN:-}
|
- AGAP_MCP_TOKEN=${AGAP_MCP_TOKEN:-}
|
||||||
- TZ=Europe/Riga
|
- TZ=Europe/Riga
|
||||||
volumes:
|
volumes:
|
||||||
# Runtime state only (Matrix crypto/devices, credentials, sessions,
|
# kb#219: permanent host mount, replacing the named Docker volume
|
||||||
# workspace, logs). The gateway config file itself is overlaid below.
|
# (openai_adolf-state) so Adolf's state is inspectable/backup-able on
|
||||||
|
# the host like every other Agap service (hindsight, litellm, qdrant,
|
||||||
|
# langfuse, ... all live under /mnt/ssd/dbs/<service>). Runtime state
|
||||||
|
# only (Matrix crypto/devices, credentials, sessions, workspace,
|
||||||
|
# logs). The gateway config file and personas are overlaid below.
|
||||||
|
# Migration: agap_git/ai/migrate-adolf-state.sh (copies the old
|
||||||
|
# openai_adolf-state volume here; run + verified before this bind
|
||||||
|
# mount is activated — see kb#219).
|
||||||
|
# NOT YET ACTIVE (2026-07-31): migrate-adolf-state.sh has not been run,
|
||||||
|
# so /mnt/ssd/dbs/adolf/ is empty and binding it starts Adolf with a
|
||||||
|
# clobbered config. Reverted to the named volume until kb#219's root
|
||||||
|
# setup + migration is done; re-swap these six lines then.
|
||||||
- adolf-state:/home/node/.openclaw
|
- adolf-state:/home/node/.openclaw
|
||||||
|
# kb#219: config binds folded from two scattered locations
|
||||||
|
# (agap_git/adolf/openclaw.json + four agap_git/ai/*-plugin dirs)
|
||||||
|
# into one coherent home, /mnt/ssd/dbs/adolf/config/. Git remains the
|
||||||
|
# single source of truth for content — these are symlinks back to the
|
||||||
|
# tracked agap_git paths (created by the root setup block in kb#219's
|
||||||
|
# report), not copies, so "edit the tracked file + restart" still
|
||||||
|
# applies unchanged. Only the mount *path* changed from five spread
|
||||||
|
# locations to one directory tree.
|
||||||
|
#
|
||||||
# Version-controlled OpenClaw gateway config, mounted read-only on top
|
# Version-controlled OpenClaw gateway config, mounted read-only on top
|
||||||
# of the state volume so it is the single source of truth. The gateway
|
# of the state mount so it is the single source of truth. The gateway
|
||||||
# reads this JSONC file and snapshots its own .last-good/.rejected
|
# reads this JSONC file and snapshots its own .last-good/.rejected
|
||||||
# copies into the volume dir (writable) — it never rewrites this file,
|
# copies into the state dir (writable) — it never rewrites this file,
|
||||||
# so read-only is safe. Edit the tracked file + restart to change config;
|
# so read-only is safe.
|
||||||
# runtime/UI edits are intentionally disabled by the ro mount.
|
|
||||||
- ../adolf/openclaw.json:/home/node/.openclaw/openclaw.json:ro
|
- ../adolf/openclaw.json:/home/node/.openclaw/openclaw.json:ro
|
||||||
# quota-command plugin (kb #62) — same read-only-bind-over-volume
|
# quota-command plugin (kb #62) — same read-only-bind pattern as
|
||||||
# pattern as openclaw.json above, applied to a single external plugin
|
# openclaw.json above, applied to a single external plugin dir.
|
||||||
# dir instead of the whole state tree. Previously the only precedent
|
# Activated via plugins.entries.quota-command in openclaw.json.
|
||||||
# (cognee-memory) was docker cp'd straight into the adolf-state volume
|
|
||||||
# at runtime with no git backing; this plugin is small enough (no
|
|
||||||
# node_modules — only Node built-ins/global fetch) to just bind-mount
|
|
||||||
# its tracked source directly at its extensions/<id> path, so git stays
|
|
||||||
# the single source of truth the same way it already is for
|
|
||||||
# openclaw.json. Activated via plugins.entries.quota-command in that file.
|
|
||||||
- ./quota-command-openclaw-plugin:/home/node/.openclaw/extensions/quota-command:ro
|
- ./quota-command-openclaw-plugin:/home/node/.openclaw/extensions/quota-command:ro
|
||||||
# hindsight-memory plugin (kb #75, H3) — same read-only-bind-over-volume
|
# hindsight-memory plugin (kb #75, H3) — same pattern. Forced hooks
|
||||||
# pattern as quota-command above. Structural successor to cognee-memory
|
# (before_prompt_build recall / agent_end retain) against the
|
||||||
# (still docker cp'd into the adolf-state volume, no git backing; that
|
# hindsight service (see that service's block below), replacing
|
||||||
# plugin's activation/container is decommissioned in H4, not here).
|
|
||||||
# Forced hooks (before_prompt_build recall / agent_end retain) against
|
|
||||||
# the hindsight service (see that service's block below), replacing
|
|
||||||
# Cognee as Adolf's memory backend. Activated via
|
# Cognee as Adolf's memory backend. Activated via
|
||||||
# plugins.entries.hindsight-memory in openclaw.json.
|
# plugins.entries.hindsight-memory in openclaw.json.
|
||||||
- ./hindsight-openclaw-plugin:/home/node/.openclaw/extensions/hindsight-memory:ro
|
- ./hindsight-openclaw-plugin:/home/node/.openclaw/extensions/hindsight-memory:ro
|
||||||
# kimi-quota-footer plugin (kb #85) — same read-only-bind-over-volume
|
# kimi-quota-footer plugin (kb #85) — same pattern. Appends the Kimi
|
||||||
# pattern as quota-command/hindsight-memory above. Appends the Kimi
|
|
||||||
# usage line to every outgoing reply via reply_payload_sending, reusing
|
# usage line to every outgoing reply via reply_payload_sending, reusing
|
||||||
# quota-command's adolf-llm:8010/usage route. Activated via
|
# quota-command's adolf-llm:8010/usage route. Activated via
|
||||||
# plugins.entries.kimi-quota-footer in openclaw.json.
|
# plugins.entries.kimi-quota-footer in openclaw.json.
|
||||||
- ./kimi-quota-footer-plugin:/home/node/.openclaw/extensions/kimi-quota-footer:ro
|
- ./kimi-quota-footer-plugin:/home/node/.openclaw/extensions/kimi-quota-footer:ro
|
||||||
# todoist-capture plugin (kb#170 component 1) — same read-only-bind-
|
# todoist-capture plugin (kb#170 component 1) — same pattern.
|
||||||
# over-volume pattern as quota-command/hindsight-memory/kimi-quota-
|
# Registers /idea (native command, zero Kimi calls); POSTs to
|
||||||
# footer above. Registers /idea (native command, zero Kimi calls);
|
# agap-mcp's /capture-idea (see agap-mcp/src/server.js + capture.js)
|
||||||
# POSTs to agap-mcp's /capture-idea (see agap-mcp/src/server.js +
|
# which does the actual bge-m3 classify + Todoist create. Activated
|
||||||
# capture.js) which does the actual bge-m3 classify + Todoist create.
|
# via plugins.entries.todoist-capture in openclaw.json.
|
||||||
# Activated via plugins.entries.todoist-capture in openclaw.json.
|
|
||||||
- ./todoist-capture-plugin:/home/node/.openclaw/extensions/todoist-capture:ro
|
- ./todoist-capture-plugin:/home/node/.openclaw/extensions/todoist-capture:ro
|
||||||
|
# kb#219 / kb#156: personas deploy read-only onto the mount from the
|
||||||
|
# alvis/agent-personas gitea repo (commit 936f655 at time of writing)
|
||||||
|
# via that repo's deploy/deploy-persona.sh, landing at
|
||||||
|
# /mnt/ssd/dbs/adolf/personas/adolf/*.md. Overlaid individually onto
|
||||||
|
# the corresponding workspace/*.md files so they stay read-only from
|
||||||
|
# Adolf's side and are written only by a git deploy — "who changed
|
||||||
|
# Adolf's soul" is answerable by `git log` in that repo. USER.md is
|
||||||
|
# deliberately NOT deployed (alvis, 2026-07-30): Hindsight's per-human
|
||||||
|
# bank (#153) is the single source of user facts now, USER.md was the
|
||||||
|
# stale unused template.
|
||||||
|
# (persona overlays deliberately not mounted until kb#219 lands — the
|
||||||
|
# personas currently live inside the adolf-state volume's workspace/)
|
||||||
extra_hosts:
|
extra_hosts:
|
||||||
- "host.docker.internal:host-gateway"
|
- "host.docker.internal:host-gateway"
|
||||||
# mtx.alogins.net's public A record can't hairpin-NAT back through the
|
# mtx.alogins.net's public A record can't hairpin-NAT back through the
|
||||||
@@ -292,16 +426,27 @@ services:
|
|||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
|
|
||||||
# hindsight-llm — standalone clone of cognee-llm (kb#76, H4 option B): the
|
# hindsight-llm — standalone clone of cognee-llm (kb#76, H4 option B): the
|
||||||
# dedicated Kimi-CLI wrapper that is now Hindsight's LLM, so the whole cognee
|
# dedicated Codex-CLI wrapper that is now Hindsight's LLM, so the whole cognee
|
||||||
# stack (incl. cognee-llm) can be decommissioned. Own port (:8012) + own
|
# stack (incl. cognee-llm) can be decommissioned. Own port (:8012) + own
|
||||||
# kimi-code volume; needs a one-time `kimi login` seeded into hindsight-llm-home.
|
# codex volume; needs a one-time `codex login` seeded into
|
||||||
|
# hindsight-llm-codex-home. Migrated off Kimi CLI 2026-07-31 for cost.
|
||||||
hindsight-llm:
|
hindsight-llm:
|
||||||
build: ./hindsight-llm
|
build: ./hindsight-llm
|
||||||
container_name: hindsight-llm
|
container_name: hindsight-llm
|
||||||
|
environment:
|
||||||
|
# Same OpenAI geo-block workaround as adolf-llm above — see the comment
|
||||||
|
# there. This wrapper makes no MCP calls, but NO_PROXY still keeps
|
||||||
|
# container-to-container traffic off the tunnel.
|
||||||
|
- HTTPS_PROXY=http://host.docker.internal:56928
|
||||||
|
- HTTP_PROXY=http://host.docker.internal:56928
|
||||||
|
- NO_PROXY=localhost,127.0.0.1,::1,host.docker.internal,.alogins.net,172.16.0.0/12,10.0.0.0/8,192.168.0.0/16
|
||||||
|
extra_hosts:
|
||||||
|
# Needed to reach the host's xray proxy (:56928) for OpenAI egress.
|
||||||
|
- "host.docker.internal:host-gateway"
|
||||||
ports:
|
ports:
|
||||||
- "8012:8012"
|
- "8012:8012"
|
||||||
volumes:
|
volumes:
|
||||||
- hindsight-llm-home:/root/.kimi-code
|
- hindsight-llm-codex-home:/root/.codex
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
# kb#190: GET /v1/models is a static, no-inference route (see
|
# kb#190: GET /v1/models is a static, no-inference route (see
|
||||||
# hindsight-llm/server.js) -- cheap liveness probe.
|
# hindsight-llm/server.js) -- cheap liveness probe.
|
||||||
@@ -346,43 +491,60 @@ services:
|
|||||||
retries: 5
|
retries: 5
|
||||||
start_period: 30s
|
start_period: 30s
|
||||||
|
|
||||||
# adolf-llm — conversational Kimi-CLI wrapper (:8010), the model backend for
|
# adolf-llm — conversational Codex-CLI wrapper (:8010), the model backend for
|
||||||
# the Adolf OpenClaw gateway (P2). Real streaming (SSE), chat_id session-keying
|
# the Adolf OpenClaw gateway (P2). Real streaming (SSE), chat_id session-keying
|
||||||
# + 1:1 kimi resume, media, per-session .mcp.json sourced from the shared
|
# + 1:1 `codex exec resume`, media, shared MCP via a generated
|
||||||
# shared-mcp.json contract (cognee-mcp P4, openclaw-tools P5). Needs
|
# $CODEX_HOME/config.toml sourced from the shared-mcp.json contract
|
||||||
# `kimi login` in adolf-llm-home.
|
# (cognee-mcp P4, openclaw-tools P5). Needs `codex login` in
|
||||||
|
# adolf-llm-codex-home.
|
||||||
|
#
|
||||||
|
# Migrated off Kimi CLI 2026-07-31 for cost (Moonshot subscription retired in
|
||||||
|
# favour of the existing ChatGPT plan).
|
||||||
adolf-llm:
|
adolf-llm:
|
||||||
build: ./adolf-llm
|
build: ./adolf-llm
|
||||||
container_name: adolf-llm
|
container_name: adolf-llm
|
||||||
environment:
|
environment:
|
||||||
# marketplace-mcp bearer token (kb#61) -- shared-mcp.json's
|
# marketplace-mcp bearer token (kb#61) -- shared-mcp.json's
|
||||||
# "marketplace" entry references this by name via
|
# "marketplace" entry references this by name via
|
||||||
# `bearerTokenEnvVar: "MARKETPLACE_MCP_TOKEN"` (Kimi CLI's own field
|
# `bearerTokenEnvVar: "MARKETPLACE_MCP_TOKEN"`, which adolf-llm's config
|
||||||
# for a static bearer token sourced from the environment, confirmed by
|
# writer translates to Codex's own `bearer_token_env_var` key. Codex
|
||||||
# decompiling @moonshot-ai/kimi-code's dist/main.mjs help text). Kimi
|
|
||||||
# reads process.env at request time, so the raw secret never sits in
|
# reads process.env at request time, so the raw secret never sits in
|
||||||
# the git-tracked shared-mcp.json -- same secret, same env-var pattern
|
# the git-tracked shared-mcp.json -- same secret, same env-var pattern
|
||||||
# already used for the `adolf` service's openclaw.json Layer-1 config
|
# already used for the `adolf` service's openclaw.json Layer-1 config
|
||||||
# above (${MARKETPLACE_MCP_TOKEN} substitution), sourced from
|
# above (${MARKETPLACE_MCP_TOKEN} substitution), sourced from
|
||||||
# openai/.env (gitignored, never committed).
|
# ai/.env (gitignored, never committed).
|
||||||
- MARKETPLACE_MCP_TOKEN=${MARKETPLACE_MCP_TOKEN:-}
|
- MARKETPLACE_MCP_TOKEN=${MARKETPLACE_MCP_TOKEN:-}
|
||||||
# agap-mcp bearer token (kb#180) -- same env-var pattern, referenced by
|
# agap-mcp bearer token (kb#180) -- same env-var pattern, referenced by
|
||||||
# shared-mcp.json's "agap" entry via `bearerTokenEnvVar:
|
# shared-mcp.json's "agap" entry via `bearerTokenEnvVar:
|
||||||
# "AGAP_MCP_TOKEN"`. Without it the Kimi backbone's agap tools all
|
# "AGAP_MCP_TOKEN"`. Without it the Codex backbone's agap tools all
|
||||||
# fail with HTTP 401 once agap-mcp restarts with auth on.
|
# fail with HTTP 401 once agap-mcp restarts with auth on.
|
||||||
- AGAP_MCP_TOKEN=${AGAP_MCP_TOKEN:-}
|
- AGAP_MCP_TOKEN=${AGAP_MCP_TOKEN:-}
|
||||||
|
# OpenAI egress proxy (Codex migration, 2026-07-31). OpenAI geo-blocks
|
||||||
|
# this host outright: a direct call returns HTTP 403
|
||||||
|
# `unsupported_country_region_territory`, so `codex login` and every model
|
||||||
|
# call fail without this. Routed through the same xray proxy on the host
|
||||||
|
# that Claude Code itself uses (:56928, listening on all interfaces);
|
||||||
|
# reached from the container via the host-gateway alias below.
|
||||||
|
# Codex is Rust/reqwest, which honours these vars natively.
|
||||||
|
- HTTPS_PROXY=http://host.docker.internal:56928
|
||||||
|
- HTTP_PROXY=http://host.docker.internal:56928
|
||||||
|
# NO_PROXY is load-bearing, not cosmetic: without it ALL egress —
|
||||||
|
# including MCP calls to hindsight/openclaw-tools/agap-mcp and the local
|
||||||
|
# *.alogins.net services — would be tunnelled through xray, which is both
|
||||||
|
# slow and likely to fail. Only OpenAI should take the tunnel.
|
||||||
|
- NO_PROXY=localhost,127.0.0.1,::1,host.docker.internal,hindsight,openclaw-tools,.alogins.net,172.16.0.0/12,10.0.0.0/8,192.168.0.0/16
|
||||||
ports:
|
ports:
|
||||||
- "8010:8010"
|
- "8010:8010"
|
||||||
volumes:
|
volumes:
|
||||||
- adolf-llm-workspace:/workspace
|
- adolf-llm-workspace:/workspace
|
||||||
- adolf-llm-home:/root/.kimi-code
|
- adolf-llm-codex-home:/root/.codex
|
||||||
- ./shared-mcp.json:/shared-mcp.json:ro
|
- ./shared-mcp.json:/shared-mcp.json:ro
|
||||||
extra_hosts:
|
extra_hosts:
|
||||||
# Needed to reach kanboard-mcp-adolf (:3104, network_mode: host, outside
|
# Needed to reach kanboard-mcp-adolf (:3104, network_mode: host, outside
|
||||||
# this compose project's network) via shared-mcp.json's "kanboard"
|
# this compose project's network) via shared-mcp.json's "kanboard"
|
||||||
# entry — same host-gateway trick used by adolf/cognee/pipecat above.
|
# entry — same host-gateway trick used by adolf/cognee/pipecat above.
|
||||||
- "host.docker.internal:host-gateway"
|
- "host.docker.internal:host-gateway"
|
||||||
# Local *.alogins.net web services: the Kimi CLI's own web-fetch tool
|
# Local *.alogins.net web services: the Codex CLI's own web-fetch tool
|
||||||
# runs IN THIS container, so it needs the same hairpin-NAT dodge as the
|
# runs IN THIS container, so it needs the same hairpin-NAT dodge as the
|
||||||
# adolf gateway (the public A record can't loop back through the router).
|
# adolf gateway (the public A record can't loop back through the router).
|
||||||
# Route to the host gateway where Caddy terminates TLS on :443.
|
# Route to the host gateway where Caddy terminates TLS on :443.
|
||||||
@@ -390,7 +552,7 @@ services:
|
|||||||
- "wiki.alogins.net:host-gateway"
|
- "wiki.alogins.net:host-gateway"
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
# kb#190: GET /v1/models is a static, no-inference route (see
|
# kb#190: GET /v1/models is a static, no-inference route (see
|
||||||
# adolf-llm/server.js) -- cheap liveness probe, no Kimi call/quota use.
|
# adolf-llm/server.js) -- cheap liveness probe, no Codex call/quota use.
|
||||||
healthcheck:
|
healthcheck:
|
||||||
test: ["CMD-SHELL", "node -e \"fetch('http://localhost:8010/v1/models').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))\""]
|
test: ["CMD-SHELL", "node -e \"fetch('http://localhost:8010/v1/models').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))\""]
|
||||||
interval: 15s
|
interval: 15s
|
||||||
@@ -547,8 +709,22 @@ services:
|
|||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
|
|
||||||
volumes:
|
volumes:
|
||||||
|
# No longer mounted by any service (kimi-agent removed 2026-08-01). Left
|
||||||
|
# declared so the volume survives as a rollback source; `docker volume rm`
|
||||||
|
# is a human decision after a soak period.
|
||||||
kimi-agent-home:
|
kimi-agent-home:
|
||||||
|
# kb#219: no longer mounted by the adolf service (replaced by the
|
||||||
|
# /mnt/ssd/dbs/adolf/state bind mount above). Left declared, not removed,
|
||||||
|
# so the volume itself survives as a rollback source until a human
|
||||||
|
# explicitly `docker volume rm adolf-state` after a soak period — see the
|
||||||
|
# rollback procedure in the kb#219 report. Removing this declaration is
|
||||||
|
# a later cleanup step, not part of this migration.
|
||||||
adolf-state:
|
adolf-state:
|
||||||
hindsight-llm-home:
|
# Replaces hindsight-llm-home (/root/.kimi-code) at the Codex migration; the
|
||||||
|
# old volume still holds the Kimi login until this is verified.
|
||||||
|
hindsight-llm-codex-home:
|
||||||
adolf-llm-workspace:
|
adolf-llm-workspace:
|
||||||
adolf-llm-home:
|
# Replaces adolf-llm-home (/root/.kimi-code) at the Codex migration. The old
|
||||||
|
# volume still exists and holds the Kimi OAuth login; drop it once the Codex
|
||||||
|
# backend is verified working.
|
||||||
|
adolf-llm-codex-home:
|
||||||
19
ai/hindsight-llm/Dockerfile
Normal file
19
ai/hindsight-llm/Dockerfile
Normal file
@@ -0,0 +1,19 @@
|
|||||||
|
FROM node:22-slim
|
||||||
|
|
||||||
|
# Required: node:22-slim has no system CA store and the Codex CLI is a Rust
|
||||||
|
# binary that validates TLS against it. See adolf-llm/Dockerfile for detail.
|
||||||
|
RUN apt-get update \
|
||||||
|
&& apt-get install -y --no-install-recommends ca-certificates \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
RUN npm install -g @openai/codex
|
||||||
|
|
||||||
|
WORKDIR /workspace
|
||||||
|
|
||||||
|
COPY server.js /app/server.js
|
||||||
|
|
||||||
|
ENV CODEX_HOME=/root/.codex
|
||||||
|
|
||||||
|
EXPOSE 8012
|
||||||
|
|
||||||
|
ENTRYPOINT ["node", "/app/server.js"]
|
||||||
@@ -66,17 +66,26 @@ function withSlot(fn) {
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
// --- kimi invocation: stateless one-shot, no resume --------------------------
|
// --- codex invocation: stateless one-shot, no resume -------------------------
|
||||||
// Fresh temp dir per call, NO -r/-S session flag, discard the dir after.
|
// Fresh temp dir per call, no `exec resume`, discard the dir after.
|
||||||
// Returns the assembled text from --output-format stream-json:
|
// Parses `codex exec --json`, which ships two event schemas depending on
|
||||||
// {"role":"assistant","content":"..."}
|
// release (see adolf-llm/server.js for the same dual handling):
|
||||||
// (reuses the same parse core as kimi-agent/server.js's runKimi, minus the
|
// legacy: {"msg":{"type":"agent_message_delta","delta":"..."}}
|
||||||
// resume/session-id bookkeeping that wrapper needs and this one deliberately
|
// {"msg":{"type":"agent_message","message":"..."}}
|
||||||
// does not).
|
// newer: {"type":"item.completed","item":{"type":"agent_message","text":...}}
|
||||||
function runKimi({ prompt, cwd }) {
|
// Deltas are preferred when present; the terminal full message is a fallback,
|
||||||
|
// never an addition, or the extraction output would be duplicated.
|
||||||
|
function runCodex({ prompt, cwd }) {
|
||||||
return new Promise((resolve, reject) => {
|
return new Promise((resolve, reject) => {
|
||||||
const args = ['-p', prompt, '--output-format', 'stream-json'];
|
// --skip-git-repo-check: the per-call temp dir is not a git repo.
|
||||||
const child = spawn('kimi', args, { cwd, timeout: TIMEOUT_MS });
|
const args = ['exec', '--json', '--skip-git-repo-check', '-C', cwd, prompt];
|
||||||
|
// stdin 'ignore': otherwise codex waits for EOF on an unused pipe and the
|
||||||
|
// call hangs until TIMEOUT_MS. See adolf-llm/server.js for detail.
|
||||||
|
const child = spawn('codex', args, {
|
||||||
|
cwd,
|
||||||
|
timeout: TIMEOUT_MS,
|
||||||
|
stdio: ['ignore', 'pipe', 'pipe'],
|
||||||
|
});
|
||||||
|
|
||||||
let stdout = '';
|
let stdout = '';
|
||||||
let stderr = '';
|
let stderr = '';
|
||||||
@@ -86,16 +95,26 @@ function runKimi({ prompt, cwd }) {
|
|||||||
child.on('error', reject);
|
child.on('error', reject);
|
||||||
child.on('close', code => {
|
child.on('close', code => {
|
||||||
const parts = [];
|
const parts = [];
|
||||||
|
let finalText = null;
|
||||||
for (const line of stdout.split('\n')) {
|
for (const line of stdout.split('\n')) {
|
||||||
const t = line.trim();
|
const t = line.trim();
|
||||||
if (!t) continue;
|
if (!t) continue;
|
||||||
let obj;
|
let obj;
|
||||||
try { obj = JSON.parse(t); } catch { continue; }
|
try { obj = JSON.parse(t); } catch { continue; }
|
||||||
if (obj.role === 'assistant' && obj.content) parts.push(obj.content);
|
const msg = obj.msg;
|
||||||
|
if (msg && typeof msg.type === 'string') {
|
||||||
|
if (msg.type === 'agent_message_delta' && msg.delta) parts.push(msg.delta);
|
||||||
|
else if (msg.type === 'agent_message' && msg.message) finalText = msg.message;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (obj.type === 'item.completed' && obj.item && obj.item.type === 'agent_message') {
|
||||||
|
const text = typeof obj.item.text === 'string' ? obj.item.text : obj.item.message;
|
||||||
|
if (text) finalText = text;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
const text = parts.join('').trim();
|
const text = (parts.join('').trim() || (finalText || '').trim());
|
||||||
if (!text && code !== 0) {
|
if (!text && code !== 0) {
|
||||||
reject(new Error(`kimi exited ${code}: ${stderr.slice(0, 2000)}`));
|
reject(new Error(`codex exited ${code}: ${stderr.slice(0, 2000)}`));
|
||||||
} else {
|
} else {
|
||||||
resolve(text);
|
resolve(text);
|
||||||
}
|
}
|
||||||
@@ -109,7 +128,7 @@ async function handleTurn(messages) {
|
|||||||
const dir = path.join(WORKSPACE, reqId);
|
const dir = path.join(WORKSPACE, reqId);
|
||||||
fs.mkdirSync(dir, { recursive: true });
|
fs.mkdirSync(dir, { recursive: true });
|
||||||
try {
|
try {
|
||||||
return await withSlot(() => runKimi({ prompt, cwd: dir }));
|
return await withSlot(() => runCodex({ prompt, cwd: dir }));
|
||||||
} finally {
|
} finally {
|
||||||
// Stateless one-shot: nothing about this call is meant to survive it, so
|
// Stateless one-shot: nothing about this call is meant to survive it, so
|
||||||
// the temp dir is discarded unconditionally, success or failure.
|
// the temp dir is discarded unconditionally, success or failure.
|
||||||
@@ -138,7 +157,7 @@ const server = http.createServer((req, res) => {
|
|||||||
res.writeHead(200, { 'Content-Type': 'application/json' });
|
res.writeHead(200, { 'Content-Type': 'application/json' });
|
||||||
res.end(JSON.stringify({
|
res.end(JSON.stringify({
|
||||||
object: 'list',
|
object: 'list',
|
||||||
data: [{ id: MODEL_ID, object: 'model', owned_by: 'moonshot' }],
|
data: [{ id: MODEL_ID, object: 'model', owned_by: 'openai' }],
|
||||||
}));
|
}));
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
@@ -21,16 +21,22 @@ model_list:
|
|||||||
model: ollama/bge-m3
|
model: ollama/bge-m3
|
||||||
api_base: http://host.docker.internal:11436
|
api_base: http://host.docker.internal:11436
|
||||||
|
|
||||||
- model_name: judge
|
# kb#164: the `judge` alias (anthropic/claude-haiku-4-5, metered) was removed
|
||||||
litellm_params:
|
# 2026-07-30 by alvis's decision. No ANTHROPIC_API_KEY was ever set in this
|
||||||
model: anthropic/claude-haiku-4-5-20251001
|
# container or .env, so it could not spend; it was kept only as a latent
|
||||||
api_key: os.environ/ANTHROPIC_API_KEY
|
# paid-fallback footgun. Per design §3a (no metered API by default), do not
|
||||||
|
# re-add a metered deployment without an explicit opt-in decision.
|
||||||
|
|
||||||
# Kimi Code CLI agent (own container, own Moonshot/Kimi subscription via `kimi login`)
|
# Codex CLI agent. Replaces the retired `kimi-agent` container (2026-08-01,
|
||||||
- model_name: kimi-agent
|
# Kimi purge): that was the ONLY large-tier deployment behind LiteLLM, so
|
||||||
|
# deleting it outright would have silently degraded every large-tier request
|
||||||
|
# to the local 4B model via the fallbacks below. Repointed at the codex-backed
|
||||||
|
# adolf-llm wrapper (:8010, OpenAI-compatible, model id "adolf") instead of
|
||||||
|
# standing up a third CLI container with its own login.
|
||||||
|
- model_name: codex-agent
|
||||||
litellm_params:
|
litellm_params:
|
||||||
model: openai/kimi-agent
|
model: openai/adolf
|
||||||
api_base: http://kimi-agent:8000/v1
|
api_base: http://adolf-llm:8010/v1
|
||||||
api_key: dummy
|
api_key: dummy
|
||||||
|
|
||||||
# ── raw model exposure ─────────────────────────────────────────────────
|
# ── raw model exposure ─────────────────────────────────────────────────
|
||||||
@@ -134,9 +140,9 @@ model_list:
|
|||||||
# target = constraint-set ("any large model"), not a specific backbone.
|
# target = constraint-set ("any large model"), not a specific backbone.
|
||||||
# Two litellm_params entries sharing one model_name = a LiteLLM deployment
|
# Two litellm_params entries sharing one model_name = a LiteLLM deployment
|
||||||
# group; the router load-balances/fails-over across them. tier-large lists
|
# group; the router load-balances/fails-over across them. tier-large lists
|
||||||
# kimi-agent FIRST so it's preferred, with local-small as the in-group
|
# codex-agent FIRST so it's preferred, with local-small as the in-group
|
||||||
# failover partner -- this is also what the fallbacks: block below promotes
|
# failover partner -- this is also what the fallbacks: block below promotes
|
||||||
# to an explicit, auditable Kimi-429-degrades-to-local path (design §2
|
# to an explicit, auditable quota-429-degrades-to-local path (design §2
|
||||||
# theorem 2: quota-gated a(t)=0 -> park/degrade, never fail).
|
# theorem 2: quota-gated a(t)=0 -> park/degrade, never fail).
|
||||||
# tier-small mirrors model-registry.yaml's routing.tiers.small = [local-small].
|
# tier-small mirrors model-registry.yaml's routing.tiers.small = [local-small].
|
||||||
- model_name: tier-small
|
- model_name: tier-small
|
||||||
@@ -146,8 +152,8 @@ model_list:
|
|||||||
|
|
||||||
- model_name: tier-large
|
- model_name: tier-large
|
||||||
litellm_params:
|
litellm_params:
|
||||||
model: openai/kimi-agent
|
model: openai/adolf
|
||||||
api_base: http://kimi-agent:8000/v1
|
api_base: http://adolf-llm:8010/v1
|
||||||
api_key: dummy
|
api_key: dummy
|
||||||
|
|
||||||
# ── kb#128: Auto Router v2 -- embedding-based classification on the LOCAL
|
# ── kb#128: Auto Router v2 -- embedding-based classification on the LOCAL
|
||||||
@@ -180,7 +186,7 @@ model_list:
|
|||||||
{"name": "ollama/gemma3:4b", "description": "Simple, short, low-stakes requests -- greetings, quick factual lookups, formatting, one-line questions.",
|
{"name": "ollama/gemma3:4b", "description": "Simple, short, low-stakes requests -- greetings, quick factual lookups, formatting, one-line questions.",
|
||||||
"utterances": ["hi", "hello", "what time is it", "what's the weather", "thanks", "what does this word mean", "summarize this in one sentence", "give me a quick yes or no", "format this as a list", "what is 2 plus 2"],
|
"utterances": ["hi", "hello", "what time is it", "what's the weather", "thanks", "what does this word mean", "summarize this in one sentence", "give me a quick yes or no", "format this as a list", "what is 2 plus 2"],
|
||||||
"score_threshold": 0.5},
|
"score_threshold": 0.5},
|
||||||
{"name": "kimi-agent", "description": "Complex reasoning, multi-step planning, coding, or anything needing tool use and deep context.",
|
{"name": "codex-agent", "description": "Complex reasoning, multi-step planning, coding, or anything needing tool use and deep context.",
|
||||||
"utterances": ["write a function that parses this log file and extracts errors", "refactor this class to use dependency injection", "think through the tradeoffs of these two architectures step by step", "debug why this docker container keeps crashing", "plan out the migration from cognee to hindsight across five tasks", "analyze this design document and find inconsistencies", "write a SQL query that joins these three tables and aggregates by month", "review this pull request for security issues"],
|
"utterances": ["write a function that parses this log file and extracts errors", "refactor this class to use dependency injection", "think through the tradeoffs of these two architectures step by step", "debug why this docker container keeps crashing", "plan out the migration from cognee to hindsight across five tasks", "analyze this design document and find inconsistencies", "write a SQL query that joins these three tables and aggregates by month", "review this pull request for security issues"],
|
||||||
"score_threshold": 0.5}
|
"score_threshold": 0.5}
|
||||||
]}
|
]}
|
||||||
@@ -210,16 +216,31 @@ litellm_settings:
|
|||||||
success_callback: ["langfuse"]
|
success_callback: ["langfuse"]
|
||||||
failure_callback: ["langfuse"]
|
failure_callback: ["langfuse"]
|
||||||
drop_params: true
|
drop_params: true
|
||||||
|
# kb#148 (A2A-16): per-agent attribution + KB-task granularity in Langfuse.
|
||||||
|
# `user_api_key_alias` is populated automatically by LiteLLM from the
|
||||||
|
# calling virtual key (kb#128 provisioned one per agent with key_alias set
|
||||||
|
# to the agent id -- adolf/claude-coder/torgash/researcher), so every
|
||||||
|
# trace is tagged with its agent for free as soon as callers use their
|
||||||
|
# per-agent key. `agent`, `task-id` and `queue` are NOT auto-populated --
|
||||||
|
# callers must pass them explicitly as
|
||||||
|
# `extra_body={"metadata": {"agent": "...", "task-id": "...", "queue": "..."}}`
|
||||||
|
# (OpenAI-SDK-style) or the LiteLLM-native `metadata` field on the request;
|
||||||
|
# LiteLLM copies matching keys straight onto the Langfuse trace as tags.
|
||||||
|
# Wiring individual callers (adolf-llm, kimi-agent wrapper, thin workers)
|
||||||
|
# to actually send that metadata is separate follow-up work, out of this
|
||||||
|
# task's declared scope (docker-compose.yml + litellm-config.yaml only) --
|
||||||
|
# flagged in the kb#148 report as adjacent work.
|
||||||
|
langfuse_default_tags: ["agent", "task-id", "queue", "user_api_key_alias"]
|
||||||
fallbacks:
|
fallbacks:
|
||||||
- deepseek/deepseek-r1:free: ["ollama/qwen3.5:4b"]
|
- deepseek/deepseek-r1:free: ["ollama/qwen3.5:4b"]
|
||||||
# kb#128 acceptance: "a forced Kimi 429 degrades cleanly". kimi-agent is
|
# kb#128 acceptance: "a forced 429 degrades cleanly". codex-agent is the
|
||||||
# the only Kimi deployment actually routed through LiteLLM today (the
|
# only large deployment routed through LiteLLM today (the `codex`
|
||||||
# `kimi` model-registry id is called directly via the adolf-llm/
|
# model-registry id is also called directly via the adolf-llm/
|
||||||
# hindsight-llm wrappers, outside LiteLLM by design -- see model-
|
# hindsight-llm wrappers, outside LiteLLM by design -- see model-
|
||||||
# registry.yaml's kimi entry). Both the raw deployment and the tier-large
|
# registry.yaml's codex entry). Both the raw deployment and the tier-large
|
||||||
# pool degrade to the free local-small model on 429/quota-exhaustion
|
# pool degrade to the free local-small model on 429/quota-exhaustion
|
||||||
# rather than failing the caller.
|
# rather than failing the caller.
|
||||||
- kimi-agent: ["ollama/gemma3:4b"]
|
- codex-agent: ["ollama/gemma3:4b"]
|
||||||
- tier-large: ["tier-small"]
|
- tier-large: ["tier-small"]
|
||||||
# auto_router's embedding path is the one with the open bug report
|
# auto_router's embedding path is the one with the open bug report
|
||||||
# (design §3a) -- if it errors, fail over to the zero-API-call heuristic
|
# (design §3a) -- if it errors, fail over to the zero-API-call heuristic
|
||||||
151
ai/migrate-adolf-state.sh
Executable file
151
ai/migrate-adolf-state.sh
Executable file
@@ -0,0 +1,151 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Migration script for kb#219 — move Adolf's runtime state off the named
|
||||||
|
# Docker volume (openai_adolf-state) onto a host bind mount at
|
||||||
|
# /mnt/ssd/dbs/adolf/state, matching the convention every other Agap
|
||||||
|
# service already follows (hindsight, litellm, qdrant, langfuse, ...).
|
||||||
|
#
|
||||||
|
# SAFETY MODEL:
|
||||||
|
# - COPY ONLY. Never touches or deletes the source volume. The volume
|
||||||
|
# stays intact and usable as a rollback source until a human explicitly
|
||||||
|
# removes it (see rollback section in the compose-diff writeup /
|
||||||
|
# kb#219 report), long after this script has run and the container has
|
||||||
|
# been soak-tested on the new mount.
|
||||||
|
# - Dry-run by default. Pass --apply to actually copy.
|
||||||
|
# - Idempotent. Safe to re-run; re-copying onto an already-populated
|
||||||
|
# destination just refreshes it (cp -a overwrite-in-place). It will
|
||||||
|
# NOT delete files at the destination that were removed from the
|
||||||
|
# source between runs -- if that matters, wipe the dest dir yourself
|
||||||
|
# before re-running.
|
||||||
|
# - Verifies file counts + a sha256 manifest diff between source and
|
||||||
|
# destination before declaring success. Non-zero exit if they disagree.
|
||||||
|
# - Uses only `docker run` (alvis is in the `docker` group -- no `sudo`
|
||||||
|
# needed for container operations) to read the volume; never reads
|
||||||
|
# /var/lib/docker/volumes directly (root-only, 0700).
|
||||||
|
# - Does NOT create /mnt/ssd/dbs/adolf itself. That directory tree is
|
||||||
|
# root-owned (/mnt/ssd/dbs is 0755 root:root, same as every other
|
||||||
|
# service dir under it) and must be created + chowned by a human with
|
||||||
|
# sudo first -- see the paste-ready root block in the kb#219 report.
|
||||||
|
# This script aborts early with a clear message if the destination
|
||||||
|
# parent doesn't exist or isn't writable.
|
||||||
|
#
|
||||||
|
# USAGE:
|
||||||
|
# ./migrate-adolf-state.sh # dry run (default), prints plan
|
||||||
|
# ./migrate-adolf-state.sh --apply # actually copies + verifies
|
||||||
|
# ./migrate-adolf-state.sh --apply --dest /path/to/scratch --volume some-test-volume
|
||||||
|
# # point at a throwaway volume/dest for a trial run
|
||||||
|
#
|
||||||
|
# This script is NOT executed against live state as part of kb#219 prep.
|
||||||
|
# It has been dry-run tested and trial-run tested against a throwaway
|
||||||
|
# volume with a handful of files (see kb#219 report for the transcript).
|
||||||
|
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
SRC_VOLUME="openai_adolf-state"
|
||||||
|
DEST_DIR="/mnt/ssd/dbs/adolf/state"
|
||||||
|
APPLY=0
|
||||||
|
|
||||||
|
while [ $# -gt 0 ]; do
|
||||||
|
case "$1" in
|
||||||
|
--apply) APPLY=1; shift ;;
|
||||||
|
--dest) DEST_DIR="$2"; shift 2 ;;
|
||||||
|
--volume) SRC_VOLUME="$2"; shift 2 ;;
|
||||||
|
-h|--help)
|
||||||
|
grep '^#' "$0" | sed 's/^#//'
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
echo "Unknown argument: $1" >&2
|
||||||
|
exit 2
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
echo "== kb#219 adolf-state migration =="
|
||||||
|
echo "Source volume : $SRC_VOLUME"
|
||||||
|
echo "Dest dir : $DEST_DIR"
|
||||||
|
echo "Mode : $([ "$APPLY" -eq 1 ] && echo APPLY || echo DRY-RUN)"
|
||||||
|
echo
|
||||||
|
|
||||||
|
# --- 0. sanity: source volume exists ---
|
||||||
|
if ! docker volume inspect "$SRC_VOLUME" >/dev/null 2>&1; then
|
||||||
|
echo "ERROR: source volume '$SRC_VOLUME' does not exist." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# --- 1. sanity: destination parent exists and is writable ---
|
||||||
|
DEST_PARENT="$(dirname "$DEST_DIR")"
|
||||||
|
if [ ! -d "$DEST_PARENT" ]; then
|
||||||
|
cat >&2 <<EOF
|
||||||
|
ERROR: $DEST_PARENT does not exist.
|
||||||
|
|
||||||
|
/mnt/ssd/dbs is root-owned; this directory must be created by a human
|
||||||
|
with sudo before this script can run. See the paste-ready root block in
|
||||||
|
the kb#219 report (creates /mnt/ssd/dbs/adolf/{state,config,personas},
|
||||||
|
chowned 1000:1000 to match the adolf container's node user).
|
||||||
|
EOF
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
if [ ! -w "$DEST_PARENT" ]; then
|
||||||
|
echo "ERROR: $DEST_PARENT exists but is not writable by $(whoami). Check ownership (should be chowned to your uid, or 1000:1000)." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
mkdir -p "$DEST_DIR"
|
||||||
|
|
||||||
|
# --- 2. source manifest (counts + sha256, computed inside a container) ---
|
||||||
|
echo "-- Computing source manifest (read-only mount of $SRC_VOLUME) --"
|
||||||
|
SRC_COUNT=$(docker run --rm -v "$SRC_VOLUME":/from:ro alpine sh -c "find /from -type f | wc -l")
|
||||||
|
echo "Source file count: $SRC_COUNT"
|
||||||
|
|
||||||
|
if [ "$APPLY" -eq 0 ]; then
|
||||||
|
echo
|
||||||
|
echo "[DRY RUN] Would copy $SRC_COUNT files from volume '$SRC_VOLUME' into $DEST_DIR,"
|
||||||
|
echo "[DRY RUN] then verify file count + sha256 manifest match."
|
||||||
|
echo "[DRY RUN] Re-run with --apply to actually copy."
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
# --- 3. copy (tar stream preserves ownership/perms across the boundary) ---
|
||||||
|
echo "-- Copying (tar stream, preserves perms/ownership) --"
|
||||||
|
docker run --rm \
|
||||||
|
-v "$SRC_VOLUME":/from:ro \
|
||||||
|
-v "$DEST_DIR":/to \
|
||||||
|
alpine sh -c "cd /from && tar cf - . | (cd /to && tar xf -)"
|
||||||
|
|
||||||
|
# --- 4. verify: file count ---
|
||||||
|
DEST_COUNT=$(docker run --rm -v "$DEST_DIR":/to:ro alpine sh -c "find /to -type f | wc -l")
|
||||||
|
echo "Dest file count: $DEST_COUNT"
|
||||||
|
if [ "$SRC_COUNT" != "$DEST_COUNT" ]; then
|
||||||
|
echo "ERROR: file count mismatch (source=$SRC_COUNT dest=$DEST_COUNT). NOT declaring success." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# --- 5. verify: sha256 manifest diff ---
|
||||||
|
echo "-- Verifying sha256 manifests match --"
|
||||||
|
SRC_MANIFEST=$(mktemp)
|
||||||
|
DEST_MANIFEST=$(mktemp)
|
||||||
|
trap 'rm -f "$SRC_MANIFEST" "$DEST_MANIFEST"' EXIT
|
||||||
|
|
||||||
|
docker run --rm -v "$SRC_VOLUME":/from:ro alpine sh -c \
|
||||||
|
"cd /from && find . -type f -exec sha256sum {} \; | sort -k2" > "$SRC_MANIFEST"
|
||||||
|
docker run --rm -v "$DEST_DIR":/to:ro alpine sh -c \
|
||||||
|
"cd /to && find . -type f -exec sha256sum {} \; | sort -k2" > "$DEST_MANIFEST"
|
||||||
|
|
||||||
|
if diff -u "$SRC_MANIFEST" "$DEST_MANIFEST" > /tmp/adolf-state-migration.diff; then
|
||||||
|
echo "OK: manifests match byte-for-byte ($SRC_COUNT files)."
|
||||||
|
else
|
||||||
|
echo "ERROR: manifest mismatch, see /tmp/adolf-state-migration.diff" >&2
|
||||||
|
cat /tmp/adolf-state-migration.diff >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo
|
||||||
|
echo "== Migration copy verified OK =="
|
||||||
|
echo "Source volume '$SRC_VOLUME' left untouched (not deleted, not modified)."
|
||||||
|
echo "Next steps (NOT done by this script -- human-supervised, see kb#219 report):"
|
||||||
|
echo " 1. Apply the docker-compose.yml bind-mount diff for the 'adolf' service."
|
||||||
|
echo " 2. docker compose -f ai/docker-compose.yml config -q # validate"
|
||||||
|
echo " 3. docker compose -f ai/docker-compose.yml up -d adolf # recreates container on new mount"
|
||||||
|
echo " 4. Verify: docker inspect adolf shows /mnt/ssd/dbs/adolf/state, not the volume;"
|
||||||
|
echo " Matrix session survives (no re-login), memory/config/persona intact."
|
||||||
|
echo " 5. Only after a soak period: docker volume rm $SRC_VOLUME"
|
||||||
@@ -28,7 +28,8 @@
|
|||||||
# consolidation/reflect all route here as of 2026-07-26)
|
# consolidation/reflect all route here as of 2026-07-26)
|
||||||
# - judge -> id: paid-fallback (metered; see kb#164 for the fact that
|
# - judge -> id: paid-fallback (metered; see kb#164 for the fact that
|
||||||
# the no-metered-API constraint has no runtime enforcement yet)
|
# the no-metered-API constraint has no runtime enforcement yet)
|
||||||
# - kimi-agent -> id: kimi-agent (own container, live; see below)
|
# - codex-agent -> id: codex-agent (LiteLLM-routed large deployment; see
|
||||||
|
# below. Was kimi-agent + its own container until the 2026-08-01 purge)
|
||||||
# - bge-m3 -> id: bge-m3 (kb#164, 2026-07-26: wired into litellm-config
|
# - bge-m3 -> id: bge-m3 (kb#164, 2026-07-26: wired into litellm-config
|
||||||
# .yaml pointing at ollama on 11436, the real embedder/routing
|
# .yaml pointing at ollama on 11436, the real embedder/routing
|
||||||
# classifier; litellm_model_name below updated from null to "bge-m3")
|
# classifier; litellm_model_name below updated from null to "bge-m3")
|
||||||
@@ -64,62 +65,71 @@
|
|||||||
schema_version: 1
|
schema_version: 1
|
||||||
|
|
||||||
models:
|
models:
|
||||||
# ── kimi — main reasoning ──────────────────────────────────────────────
|
# ── codex — main reasoning ─────────────────────────────────────────────
|
||||||
# Flat Moonshot/Kimi subscription via `kimi login`, wrapped by two
|
# Flat ChatGPT subscription via `codex login`, wrapped by two independent
|
||||||
# independent Kimi-CLI containers (own OAuth creds volume each). Not
|
# Codex-CLI containers (own creds volume each). Not behind LiteLLM today —
|
||||||
# behind LiteLLM today — callers hit the wrapper HTTP endpoints directly.
|
# callers hit the wrapper HTTP endpoints directly.
|
||||||
- id: kimi
|
#
|
||||||
role: "main reasoning (adolf-llm / hindsight-llm Kimi-CLI wrappers)"
|
# Migrated from Kimi CLI 2026-07-31 for cost: this retires the separate
|
||||||
|
# Moonshot subscription in favour of the already-paid ChatGPT plan. The id
|
||||||
|
# changed `kimi` -> `codex`; routing.tiers below refers to it by id.
|
||||||
|
- id: codex
|
||||||
|
role: "main reasoning (adolf-llm / hindsight-llm Codex-CLI wrappers)"
|
||||||
litellm_model_name: null
|
litellm_model_name: null
|
||||||
endpoints:
|
endpoints:
|
||||||
- name: adolf-llm
|
- name: adolf-llm
|
||||||
purpose: "Adolf's conversational backbone"
|
purpose: "Adolf's conversational backbone"
|
||||||
url: "http://adolf-llm:8010"
|
url: "http://adolf-llm:8010"
|
||||||
|
# Route retained but returns HTTP 501 since the Codex migration —
|
||||||
|
# no machine-readable quota on this backend. See quota: below.
|
||||||
usage_url: "http://localhost:8010/usage"
|
usage_url: "http://localhost:8010/usage"
|
||||||
- name: hindsight-llm
|
- name: hindsight-llm
|
||||||
purpose: "Hindsight's structured-extraction LLM (HINDSIGHT_API_LLM_MODEL)"
|
purpose: "Hindsight's structured-extraction LLM (HINDSIGHT_API_LLM_MODEL)"
|
||||||
url: "http://hindsight-llm:8012/v1"
|
url: "http://hindsight-llm:8012/v1"
|
||||||
model_name: "openai/hindsight-llm"
|
model_name: "openai/hindsight-llm"
|
||||||
tier: large
|
tier: large
|
||||||
context_tokens: 200000 # Moonshot Kimi K2 context window; re-verify if the CLI's pinned model changes
|
context_tokens: 400000 # GPT-5-Codex context window; re-verify if the CLI's pinned model changes
|
||||||
tool_use_quality: high
|
tool_use_quality: high
|
||||||
lifecycle: quota-gated
|
lifecycle: quota-gated
|
||||||
quota:
|
quota:
|
||||||
probe_command: ["kimi-usage", "--compact"]
|
# UNRESOLVED at the Codex migration (2026-07-31): the Kimi backend had a
|
||||||
windows:
|
# machine-readable managed-usage API that adolf-llm's /usage route
|
||||||
- name: 5h
|
# normalized into these fields. Codex exposes no equivalent endpoint, so
|
||||||
field: "window_5h.pct" # adolf-llm server.js normalizeKimiUsage() field name
|
# /usage now returns HTTP 501 and there is currently NO quota probe for
|
||||||
approx_limit: "~60 msgs/5h"
|
# this model. Governor quota-gating on `codex` is therefore blind — it
|
||||||
- name: weekly
|
# will not see the ChatGPT plan's rate limits until a signal is found.
|
||||||
field: "weekly.pct"
|
probe_command: null
|
||||||
approx_limit: "~300 msgs/wk"
|
windows: []
|
||||||
threshold_pct: 95
|
threshold_pct: 95
|
||||||
gpu_residency: null
|
gpu_residency: null
|
||||||
cost_class: subscription # flat-rate, not metered — quota is the constraint, not spend
|
cost_class: subscription # flat-rate, not metered — quota is the constraint, not spend
|
||||||
metered: false
|
metered: false
|
||||||
opt_in_required: false
|
opt_in_required: false
|
||||||
|
|
||||||
# ── kimi-agent — own container, oO-adjacent Kimi CLI wrapper ───────────
|
# ── codex-agent — the LiteLLM-routed large deployment ──────────────────
|
||||||
# Distinct from `kimi` above: this is a third Kimi-CLI container
|
# Replaces the retired `kimi-agent` container (2026-08-01 Kimi purge). That
|
||||||
# (openai/kimi-agent/, own Moonshot/Kimi subscription via `kimi login`,
|
# container was the ONLY large-tier deployment behind LiteLLM, backing
|
||||||
# own docker-compose service `kimi-agent`) that IS routed through
|
# `tier-large`, the auto_router's complex-reasoning route and their
|
||||||
# LiteLLM today (litellm-config.yaml model_name: kimi-agent ->
|
# fallbacks — removing it without a replacement would have silently degraded
|
||||||
# openai/kimi-agent -> http://kimi-agent:8000/v1). Documented here per
|
# every large-tier request to the local 4B model. Rather than stand up a
|
||||||
# kb#195 coverage audit; deliberately NOT added to routing.tiers in this
|
# third CLI container with its own login, this now points at the existing
|
||||||
# pass (that would change litellm_key_spec() grants, out of scope for a
|
# codex-backed adolf-llm wrapper (:8010, model id "adolf").
|
||||||
# docs-alignment task) — no agent is currently opted into it.
|
#
|
||||||
- id: kimi-agent
|
# Same underlying ChatGPT subscription as `codex` above — the two ids differ
|
||||||
role: "Kimi-CLI wrapper, own container (openai/kimi-agent/) — purpose/consumer not yet documented outside this registry"
|
# only in call path (this one via LiteLLM, `codex` direct to the wrappers),
|
||||||
litellm_model_name: "kimi-agent" # openai/litellm-config.yaml model_list entry
|
# so their quota is shared and neither has a probe.
|
||||||
|
- id: codex-agent
|
||||||
|
role: "LiteLLM-routed large deployment; proxies to the codex-backed adolf-llm wrapper"
|
||||||
|
litellm_model_name: "codex-agent" # openai/litellm-config.yaml model_list entry
|
||||||
endpoints:
|
endpoints:
|
||||||
- name: kimi-agent
|
- name: adolf-llm
|
||||||
url: "http://kimi-agent:8000/v1"
|
url: "http://adolf-llm:8010/v1"
|
||||||
tier: large
|
tier: large
|
||||||
context_tokens: 200000 # same Moonshot Kimi K2 CLI as `kimi`; re-verify if the CLI's pinned model changes
|
context_tokens: 400000 # GPT-5-Codex context window; re-verify if the CLI's pinned model changes
|
||||||
tool_use_quality: high
|
tool_use_quality: high
|
||||||
lifecycle: quota-gated
|
lifecycle: quota-gated
|
||||||
quota:
|
quota:
|
||||||
probe_command: null # not yet wired to a probe; own subscription, same caveat as `kimi`
|
probe_command: null # no machine-readable quota on the Codex backend — see `codex`
|
||||||
windows: []
|
windows: []
|
||||||
threshold_pct: null
|
threshold_pct: null
|
||||||
gpu_residency: null
|
gpu_residency: null
|
||||||
@@ -256,10 +266,14 @@ gpu_residency_policy:
|
|||||||
routing:
|
routing:
|
||||||
tiers:
|
tiers:
|
||||||
small: [local-small]
|
small: [local-small]
|
||||||
# paid-fallback listed as a large-tier candidate AFTER kimi so resolve()
|
# paid-fallback listed as a large-tier candidate AFTER codex so resolve()
|
||||||
# can fail over to it when kimi's a(t)=0 (quota parked) — but only for a
|
# can fail over to it when codex's a(t)=0 (quota parked) — but only for a
|
||||||
# caller that both passes allow_metered=True AND appears in
|
# caller that both passes allow_metered=True AND appears in
|
||||||
# metered_opt_in below. With metered_opt_in empty (the shipped default)
|
# metered_opt_in below. With metered_opt_in empty (the shipped default)
|
||||||
# resolve() skips it unconditionally, so it stays unreachable.
|
# resolve() skips it unconditionally, so it stays unreachable.
|
||||||
large: [kimi, paid-fallback]
|
#
|
||||||
|
# NB: with the Codex migration there is no quota probe (see the `codex`
|
||||||
|
# entry), so a(t) never reads as parked and this failover cannot trigger
|
||||||
|
# on quota today.
|
||||||
|
large: [codex, paid-fallback]
|
||||||
metered_opt_in: [] # e.g. ["agent:torgash"] once a human explicitly opts a specific virtual key in
|
metered_opt_in: [] # e.g. ["agent:torgash"] once a human explicitly opts a specific virtual key in
|
||||||
@@ -25,8 +25,8 @@ callers two things instead:
|
|||||||
Usage (library):
|
Usage (library):
|
||||||
from model_registry import load_registry, resolve, to_probe_config, preload_check
|
from model_registry import load_registry, resolve, to_probe_config, preload_check
|
||||||
reg = load_registry()
|
reg = load_registry()
|
||||||
model = resolve(reg, tier="large") # -> the "kimi" entry
|
model = resolve(reg, tier="large") # -> the "codex" entry
|
||||||
cfg = to_probe_config(reg, "kimi") # -> kb_worker probe config dict
|
cfg = to_probe_config(reg, "codex") # -> kb_worker probe config dict
|
||||||
ok, reason = preload_check(reg, "local-small", headroom_mb=1900)
|
ok, reason = preload_check(reg, "local-small", headroom_mb=1900)
|
||||||
|
|
||||||
Usage (CLI, for manual verification):
|
Usage (CLI, for manual verification):
|
||||||
@@ -37,7 +37,7 @@ const FETCH_TIMEOUT_MS = 15000; // bge-m3 embed + Todoist create can take a few
|
|||||||
// no longer an open REST endpoint (it never should have been: it reaches
|
// no longer an open REST endpoint (it never should have been: it reaches
|
||||||
// Todoist writes from any LAN peer). This plugin runs inside the adolf
|
// Todoist writes from any LAN peer). This plugin runs inside the adolf
|
||||||
// container, so it presents Adolf's own agap-mcp bearer token, injected as
|
// container, so it presents Adolf's own agap-mcp bearer token, injected as
|
||||||
// AGAP_MCP_TOKEN by openai/docker-compose.yml from .env (never inlined
|
// AGAP_MCP_TOKEN by ai/docker-compose.yml from .env (never inlined
|
||||||
// here). If the var is unset the request goes out unauthenticated and
|
// here). If the var is unset the request goes out unauthenticated and
|
||||||
// agap-mcp answers 401 — a visible failure of /idea, not a silent one.
|
// agap-mcp answers 401 — a visible failure of /idea, not a silent one.
|
||||||
const AGAP_MCP_TOKEN = process.env.AGAP_MCP_TOKEN || "";
|
const AGAP_MCP_TOKEN = process.env.AGAP_MCP_TOKEN || "";
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
#!/bin/bash
|
#!/bin/bash
|
||||||
# Kanboard backup — tier-0 hardening (kb#158, A2A-26, DESIGN-a2a-agents.md v2.1 §6c).
|
# Kanboard backup — tier-0 hardening (kb#158, A2A-26, DESIGN-a2a-agents.md v2.1 §6c).
|
||||||
# Mirrors the vaultwarden backup.sh pattern (same repo, ~/agap_git/vaultwarden/backup.sh):
|
# Mirrors the vaultwarden backup.sh pattern (same repo, ~/agap_git/vaultwarden/backup.sh):
|
||||||
# scheduled dump -> /mnt/backups, retention of last 5, Zabbix freshness trapper.
|
# scheduled dump -> /mnt/backups, retention of last 5. Backup-freshness monitored via .age items.
|
||||||
#
|
#
|
||||||
# Runs every 3 days via alvis's user crontab (NOT root crontab like vaultwarden's --
|
# Runs every 3 days via alvis's user crontab (NOT root crontab like vaultwarden's --
|
||||||
# /mnt/backups/kanboard was bootstrapped chown'd to alvis specifically so this backup,
|
# /mnt/backups/kanboard was bootstrapped chown'd to alvis specifically so this backup,
|
||||||
@@ -17,9 +17,6 @@
|
|||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
BACKUP_DIR="/mnt/backups/kanboard"
|
BACKUP_DIR="/mnt/backups/kanboard"
|
||||||
ZABBIX_TOKEN_FILE="/home/alvis/.zabbix_token"
|
|
||||||
ZABBIX_URL="http://192.168.1.4:81/api_jsonrpc.php"
|
|
||||||
ZABBIX_ITEM_ID="70605" # kanboard.backup.ts on host AgapHost (10776)
|
|
||||||
|
|
||||||
DATE=$(date '+%Y%m%d-%H%M')
|
DATE=$(date '+%Y%m%d-%H%M')
|
||||||
DEST="$BACKUP_DIR/$DATE"
|
DEST="$BACKUP_DIR/$DATE"
|
||||||
@@ -48,22 +45,9 @@ docker run --rm --user 1000:1000 -v kanboard_plugins:/plugins:ro -v "$DEST":/des
|
|||||||
|
|
||||||
echo "$(date): Backup complete: $DEST"
|
echo "$(date): Backup complete: $DEST"
|
||||||
ls -la "$DEST/"
|
ls -la "$DEST/"
|
||||||
|
# Backup-freshness monitoring is now done via .age items (calculated fields showing
|
||||||
# Notify Zabbix (trapper item kanboard.backup.ts, unixtime) -- pushes a real epoch
|
# age of the backup). The .ts (timestamp) trappers were unreliable (history.push not
|
||||||
# timestamp, unlike vaultwarden.backup.ts which (kb#158 finding) pushes a formatted
|
# landing); removed in kb#189 in favor of .age overdue triggers.
|
||||||
# date STRING into a numeric item and has therefore never recorded a valid value.
|
|
||||||
if [[ -f "$ZABBIX_TOKEN_FILE" ]]; then
|
|
||||||
ZABBIX_TOKEN=$(cat "$ZABBIX_TOKEN_FILE")
|
|
||||||
NOW_EPOCH=$(date '+%s')
|
|
||||||
env -u HTTPS_PROXY -u HTTP_PROXY -u ALL_PROXY -u https_proxy -u http_proxy -u all_proxy \
|
|
||||||
curl -s -X POST "$ZABBIX_URL" \
|
|
||||||
-H "Content-Type: application/json" \
|
|
||||||
-H "Authorization: Bearer $ZABBIX_TOKEN" \
|
|
||||||
-d "{\"jsonrpc\":\"2.0\",\"method\":\"history.push\",\"id\":1,\"params\":{\"itemid\":\"$ZABBIX_ITEM_ID\",\"value\":$NOW_EPOCH}}" > /dev/null \
|
|
||||||
&& echo "Zabbix notified (kanboard.backup.ts=$NOW_EPOCH)."
|
|
||||||
else
|
|
||||||
echo "WARNING: $ZABBIX_TOKEN_FILE not found -- skipped Zabbix freshness push." >&2
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Rotate: keep last 5 backups
|
# Rotate: keep last 5 backups
|
||||||
ls -1dt "$BACKUP_DIR"/[0-9]*-[0-9]* 2>/dev/null | tail -n +6 | xargs -r rm -rf
|
ls -1dt "$BACKUP_DIR"/[0-9]*-[0-9]* 2>/dev/null | tail -n +6 | xargs -r rm -rf
|
||||||
|
|||||||
82
kanboard/restore.sh
Executable file
82
kanboard/restore.sh
Executable file
@@ -0,0 +1,82 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Kanboard restore — companion to backup.sh (kb#192).
|
||||||
|
#
|
||||||
|
# Restores a snapshot produced by backup.sh (db.sqlite + optional plugins.tar.gz)
|
||||||
|
# into a running Kanboard container. Defaults to the live "kanboard" container/
|
||||||
|
# volumes, but every target is overridable via env vars so the exact same script
|
||||||
|
# can be pointed at a disposable/throwaway container for a dry-run restore test
|
||||||
|
# (see kb#192 runbook for the recommended throwaway-container recipe).
|
||||||
|
#
|
||||||
|
# Usage:
|
||||||
|
# ./restore.sh /mnt/backups/kanboard/<snapshot-dir>
|
||||||
|
#
|
||||||
|
# Env overrides (defaults = live service):
|
||||||
|
# CONTAINER=kanboard # target container name
|
||||||
|
# DATA_PATH=/var/www/app/data # data dir inside the container
|
||||||
|
# PLUGINS_PATH=/var/www/app/plugins # plugins dir inside the container
|
||||||
|
#
|
||||||
|
# WARNING: this overwrites the target container's live database. Never run
|
||||||
|
# against the "kanboard" container name unless you intend a real disaster
|
||||||
|
# recovery — for testing, point CONTAINER at a throwaway container instead.
|
||||||
|
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
CONTAINER="${CONTAINER:-kanboard}"
|
||||||
|
DATA_PATH="${DATA_PATH:-/var/www/app/data}"
|
||||||
|
PLUGINS_PATH="${PLUGINS_PATH:-/var/www/app/plugins}"
|
||||||
|
|
||||||
|
if [ $# -lt 1 ]; then
|
||||||
|
echo "Usage: $0 <path-to-backup-snapshot-dir>" >&2
|
||||||
|
echo " e.g. $0 /mnt/backups/kanboard/20260728-0300" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
SRC="$(realpath "$1")"
|
||||||
|
DB_FILE="$SRC/db.sqlite"
|
||||||
|
PLUGINS_FILE="$SRC/plugins.tar.gz"
|
||||||
|
|
||||||
|
if [ ! -f "$DB_FILE" ]; then
|
||||||
|
echo "Error: $DB_FILE not found" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if ! docker inspect "$CONTAINER" > /dev/null 2>&1; then
|
||||||
|
echo "Error: container '$CONTAINER' does not exist" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "Restoring into container '$CONTAINER' from $SRC"
|
||||||
|
|
||||||
|
# Stop the app so the sqlite file isn't being written to concurrently.
|
||||||
|
docker stop "$CONTAINER" > /dev/null
|
||||||
|
|
||||||
|
# Replace the database file.
|
||||||
|
docker cp "$DB_FILE" "$CONTAINER:$DATA_PATH/db.sqlite"
|
||||||
|
|
||||||
|
# Restore plugins, if present in the snapshot.
|
||||||
|
if [ -f "$PLUGINS_FILE" ]; then
|
||||||
|
docker cp "$PLUGINS_FILE" "$CONTAINER:/tmp/plugins.tar.gz"
|
||||||
|
docker start "$CONTAINER" > /dev/null
|
||||||
|
# Extract inside the container so ownership matches what the app expects.
|
||||||
|
docker exec "$CONTAINER" sh -c "rm -rf '$PLUGINS_PATH'/* && tar -xzf /tmp/plugins.tar.gz -C '$PLUGINS_PATH' && rm -f /tmp/plugins.tar.gz"
|
||||||
|
else
|
||||||
|
echo "Note: no plugins.tar.gz in snapshot, skipping plugin restore"
|
||||||
|
docker start "$CONTAINER" > /dev/null
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "Waiting for Kanboard to come up..."
|
||||||
|
for i in $(seq 1 30); do
|
||||||
|
if docker exec "$CONTAINER" php -r 'exit(file_exists("'"$DATA_PATH"'/db.sqlite") ? 0 : 1);' > /dev/null 2>&1; then
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
sleep 1
|
||||||
|
done
|
||||||
|
|
||||||
|
echo "Verifying restored database..."
|
||||||
|
docker exec "$CONTAINER" php -r '
|
||||||
|
$db = new PDO("sqlite:'"$DATA_PATH"'/db.sqlite");
|
||||||
|
$count = $db->query("SELECT COUNT(*) FROM tasks")->fetchColumn();
|
||||||
|
echo "tasks table row count: $count\n";
|
||||||
|
'
|
||||||
|
|
||||||
|
echo "Restore complete: $CONTAINER"
|
||||||
@@ -1,11 +0,0 @@
|
|||||||
FROM node:22-slim
|
|
||||||
|
|
||||||
RUN npm install -g @moonshot-ai/kimi-code
|
|
||||||
|
|
||||||
WORKDIR /workspace
|
|
||||||
|
|
||||||
COPY server.js /app/server.js
|
|
||||||
|
|
||||||
EXPOSE 8010
|
|
||||||
|
|
||||||
ENTRYPOINT ["node", "/app/server.js"]
|
|
||||||
@@ -1,20 +0,0 @@
|
|||||||
# adolf-llm — conversational Kimi-CLI wrapper (P2, :8010). OpenAI-compatible,
|
|
||||||
# model id "adolf". Real streaming, chat_id session mapping (1:1 kimi -r resume),
|
|
||||||
# media persistence, shared .mcp.json per session. cognee/MCP wiring is stubbed
|
|
||||||
# until P4/P5. Needs `kimi login` credentials seeded into its own home volume.
|
|
||||||
# Orchestrator: merge this `adolf-llm` service + the two named volumes into
|
|
||||||
# openai/docker-compose.yml (do NOT edit that file here).
|
|
||||||
services:
|
|
||||||
adolf-llm:
|
|
||||||
build: ./adolf-llm
|
|
||||||
container_name: adolf-llm
|
|
||||||
ports:
|
|
||||||
- "8010:8010"
|
|
||||||
volumes:
|
|
||||||
- adolf-llm-workspace:/workspace
|
|
||||||
- adolf-llm-home:/root/.kimi-code
|
|
||||||
restart: unless-stopped
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
adolf-llm-workspace:
|
|
||||||
adolf-llm-home:
|
|
||||||
@@ -1,149 +0,0 @@
|
|||||||
# Adolf P4 — Cognee memory service config (mounted at /app/.env in the
|
|
||||||
# `cognee` container; matches upstream's own docker-compose `.env` pattern).
|
|
||||||
# cognee-mcp does NOT need this file — it runs in API mode (see
|
|
||||||
# service-block.yml) and only ever talks HTTP to `cognee`, never touching
|
|
||||||
# these DBs directly.
|
|
||||||
|
|
||||||
ENV=local
|
|
||||||
DEBUG=false
|
|
||||||
LOG_LEVEL=INFO
|
|
||||||
CORS_ALLOWED_ORIGINS=*
|
|
||||||
|
|
||||||
###############################################################################
|
|
||||||
# LLM — cognee runs on the Kimi subscription via the `cognee-llm` wrapper
|
|
||||||
# (:8011, built in P3). This is the intended backbone: the whole reason
|
|
||||||
# cognee-llm exists is to be cognee's LLM on the flat Kimi subscription (no
|
|
||||||
# per-token cost), consistent with adolf-llm doing the same for the assistant.
|
|
||||||
#
|
|
||||||
# Tradeoff (SPIKE-FINDINGS gate 5, accepted): the agentic CLI adds latency
|
|
||||||
# (~5s floor + ~22-24s/structured call) and runs on a single-seat subscription,
|
|
||||||
# so batch cognify is slower than a raw API. cognee-llm bounds concurrency
|
|
||||||
# (MAX_CONCURRENCY=3) to protect the account. If cognify throughput ever
|
|
||||||
# becomes a problem, the LiteLLM route below is the documented fallback.
|
|
||||||
#
|
|
||||||
# Requires: `kimi login` seeded into the `cognee-llm-home` volume (same as
|
|
||||||
# adolf-llm/kimi-agent).
|
|
||||||
###############################################################################
|
|
||||||
LLM_PROVIDER=openai
|
|
||||||
LLM_MODEL=openai/cognee-llm
|
|
||||||
# Must include /v1 — cognee's OpenAI-compatible LLM adapter passes this
|
|
||||||
# straight through to litellm as api_base and litellm appends
|
|
||||||
# "/chat/completions" verbatim (no path normalization). Without /v1 this hits
|
|
||||||
# http://cognee-llm:8011/chat/completions, which 404s (cognee-llm only serves
|
|
||||||
# /v1/chat/completions and /v1/models) — confirmed 2026-07-05 during the P4
|
|
||||||
# smoke test (litellm.NotFoundError: Error code 404 - 'not found').
|
|
||||||
LLM_ENDPOINT=http://cognee-llm:8011/v1
|
|
||||||
LLM_API_KEY=sk-cognee-llm-local
|
|
||||||
|
|
||||||
# Force instructor's plain JSON-in-content mode instead of its default
|
|
||||||
# tool-calling mode. cognee-llm's Kimi CLI wrapper is a text-only pass-through
|
|
||||||
# (no real OpenAI function/tool-calling support — it just returns
|
|
||||||
# {"content": "..."}), so instructor's default mode for the "openai" provider
|
|
||||||
# (tool-calling, since no explicit LLM_INSTRUCTOR_MODE means it never applies
|
|
||||||
# json_schema_mode either) fails with "Instructor does not support multiple
|
|
||||||
# tool calls, use List[Model] instead" — confirmed 2026-07-05 during the P4
|
|
||||||
# smoke test. json_mode matches cognee-llm's own documented behavior
|
|
||||||
# (STRUCTURED_SYSTEM_PREAMBLE: "When asked for JSON, output raw JSON only").
|
|
||||||
LLM_INSTRUCTOR_MODE=json_mode
|
|
||||||
|
|
||||||
# Fallback only (NOT the default) — route cognify's LLM to a LiteLLM model if
|
|
||||||
# the Kimi CLI path is ever too slow under batch load. Requires a working
|
|
||||||
# LiteLLM general model (fix judge's ANTHROPIC_API_KEY or a local qwen's port):
|
|
||||||
#LLM_MODEL=openai/judge
|
|
||||||
#LLM_ENDPOINT=http://litellm:4000
|
|
||||||
|
|
||||||
###############################################################################
|
|
||||||
# Embeddings — ollama directly (P4 blocker #1 resolution, per orchestrator:
|
|
||||||
# "use ollama directly"). LiteLLM's `embedder` route was dead (port bug), so
|
|
||||||
# rather than fix that indirection we go straight to ollama's own dedicated
|
|
||||||
# embedding-engine implementation (OllamaEmbeddingEngine, verified present in
|
|
||||||
# cognee 1.2.2's infra/databases/vector/embeddings/).
|
|
||||||
#
|
|
||||||
# Ollama lives in a SEPARATE compose project (not on this `openai` network),
|
|
||||||
# reachable from containers only via host.docker.internal — hence
|
|
||||||
# extra_hosts: host.docker.internal:host-gateway on the cognee service in
|
|
||||||
# docker-compose.yml. Verified 2026-07-05: `curl host.docker.internal:11436`
|
|
||||||
# from a throwaway container with that extra_hosts entry returns 200.
|
|
||||||
#
|
|
||||||
# EMBEDDING_ENDPOINT must be the FULL endpoint URL including path —
|
|
||||||
# OllamaEmbeddingEngine POSTs directly to whatever EMBEDDING_ENDPOINT is (its
|
|
||||||
# own default is "http://localhost:11434/api/embed"), unlike the
|
|
||||||
# openai_compatible engine which appends its own path onto a base URL. Ollama's
|
|
||||||
# native /api/embed (batch endpoint, not the singular /api/embeddings) returns
|
|
||||||
# {"embeddings": [[...]]}; the engine handles that key.
|
|
||||||
#
|
|
||||||
# Swapped nomic-embed-text (768-d) -> bge-m3 (1024-d, multilingual, GPU-served)
|
|
||||||
# 2026-07-06 [Adolf kb#60]. bge-m3 pulled into the same :11436 ollama; tested
|
|
||||||
# directly against :11436 -> 1024-dim vector, confirmed working. cognee's
|
|
||||||
# Qdrant collections were all still 768-d (a handful of P4 smoke-test points
|
|
||||||
# only — "pineapple-7742"/"p4 deployment smoke test" fixtures, no real
|
|
||||||
# conversation data; adolf-llm's cogneeSearch/cogneeAdd are still stubs and
|
|
||||||
# have never actually written to cognee), so the stale 768-d collections were
|
|
||||||
# dropped rather than migrated — cognee recreates them at the new dimension
|
|
||||||
# on first write.
|
|
||||||
###############################################################################
|
|
||||||
EMBEDDING_PROVIDER=ollama
|
|
||||||
EMBEDDING_MODEL=bge-m3
|
|
||||||
EMBEDDING_ENDPOINT=http://host.docker.internal:11436/api/embed
|
|
||||||
EMBEDDING_DIMENSIONS=1024
|
|
||||||
HUGGINGFACE_TOKENIZER=BAAI/bge-m3
|
|
||||||
|
|
||||||
###############################################################################
|
|
||||||
# Graph store — SPIKE-FINDINGS gate 4: Kuzu embedded, not Neo4j.
|
|
||||||
# This is cognee's own default; listed explicitly for clarity.
|
|
||||||
###############################################################################
|
|
||||||
GRAPH_DATABASE_PROVIDER=kuzu
|
|
||||||
GRAPH_DATASET_DATABASE_HANDLER=kuzu
|
|
||||||
|
|
||||||
###############################################################################
|
|
||||||
# Vector store — Qdrant (existing infra, :6333). Community adapter installed
|
|
||||||
# via the custom Dockerfile in this directory (see comments there).
|
|
||||||
###############################################################################
|
|
||||||
VECTOR_DB_PROVIDER=qdrant
|
|
||||||
VECTOR_DB_URL=http://qdrant:6333
|
|
||||||
VECTOR_DB_KEY=
|
|
||||||
VECTOR_DATASET_DATABASE_HANDLER=qdrant
|
|
||||||
|
|
||||||
###############################################################################
|
|
||||||
# Relational metadata DB (cognee's own bookkeeping, not the memory graph).
|
|
||||||
###############################################################################
|
|
||||||
DB_PROVIDER=sqlite
|
|
||||||
DB_NAME=cognee_db
|
|
||||||
|
|
||||||
###############################################################################
|
|
||||||
# Storage paths — persisted under /mnt/ssd/dbs/cognee/ on the host (see
|
|
||||||
# service-block.yml volume mounts to /data and /system).
|
|
||||||
###############################################################################
|
|
||||||
DATA_ROOT_DIRECTORY=/data
|
|
||||||
SYSTEM_ROOT_DIRECTORY=/system
|
|
||||||
|
|
||||||
###############################################################################
|
|
||||||
# Single-user/single-agent posture. Adolf is one Matrix bot (SPIKE-FINDINGS
|
|
||||||
# gate 4's own reasoning: no multi-tenant/concurrent-writer need at this
|
|
||||||
# scale). Scoping happens at the *dataset* level (one dataset per OpenClaw
|
|
||||||
# chat_id — see P4 report), not via cognee's own per-user auth/isolation
|
|
||||||
# machinery, so we skip that machinery rather than bootstrap a default user
|
|
||||||
# just to satisfy it.
|
|
||||||
#
|
|
||||||
# ENABLE_BACKEND_ACCESS_CONTROL=true (cognee's own default) would give each
|
|
||||||
# (user, dataset) pair a fully isolated Kuzu+vector store, but *requires*
|
|
||||||
# authentication (REQUIRE_AUTHENTICATION=false is ignored when this is true)
|
|
||||||
# - extra machinery (default user bootstrap, token plumbing into cognee-mcp)
|
|
||||||
# for no real benefit in a single-owner home deployment. With it off, all
|
|
||||||
# datasets share one graph/vector backend; dataset_name/datasets filters on
|
|
||||||
# remember/recall/forget still scope top-level data points per conversation,
|
|
||||||
# with one documented caveat: GRAPH_COMPLETION search can traverse into
|
|
||||||
# nodes from other datasets. Acceptable for one person's own conversation
|
|
||||||
# threads; revisit (flip this flag + bootstrap a default user) if that
|
|
||||||
# leakage ever matters.
|
|
||||||
###############################################################################
|
|
||||||
ENABLE_BACKEND_ACCESS_CONTROL=False
|
|
||||||
REQUIRE_AUTHENTICATION=False
|
|
||||||
|
|
||||||
# Only exercised if the above is ever flipped to true.
|
|
||||||
FASTAPI_USERS_JWT_SECRET=059bd0fdd9cecc46d055cf589d4275bd34c0fb73543f286beff09da2c2d27b65
|
|
||||||
FASTAPI_USERS_VERIFICATION_TOKEN_SECRET=7246494bb622c9c89417fbe0b94de6d7718f1338eb40dd370fb072873f921832
|
|
||||||
FASTAPI_USERS_RESET_PASSWORD_TOKEN_SECRET=18ad75671edf003f0142aad124276268fa766e702ab6bdb71a75d1c71a688beb
|
|
||||||
|
|
||||||
TOKENIZERS_PARALLELISM=false
|
|
||||||
LITELLM_LOG=ERROR
|
|
||||||
@@ -1,11 +0,0 @@
|
|||||||
FROM node:22-slim
|
|
||||||
|
|
||||||
RUN npm install -g @moonshot-ai/kimi-code
|
|
||||||
|
|
||||||
WORKDIR /workspace
|
|
||||||
|
|
||||||
COPY server.js /app/server.js
|
|
||||||
|
|
||||||
EXPOSE 8012
|
|
||||||
|
|
||||||
ENTRYPOINT ["node", "/app/server.js"]
|
|
||||||
@@ -1,11 +0,0 @@
|
|||||||
FROM node:22-slim
|
|
||||||
|
|
||||||
RUN npm install -g @moonshot-ai/kimi-code
|
|
||||||
|
|
||||||
WORKDIR /workspace
|
|
||||||
|
|
||||||
COPY server.js /app/server.js
|
|
||||||
|
|
||||||
EXPOSE 8000
|
|
||||||
|
|
||||||
ENTRYPOINT ["node", "/app/server.js"]
|
|
||||||
@@ -1,236 +0,0 @@
|
|||||||
const http = require('http');
|
|
||||||
const fs = require('fs');
|
|
||||||
const path = require('path');
|
|
||||||
const crypto = require('crypto');
|
|
||||||
const { spawn } = require('child_process');
|
|
||||||
|
|
||||||
const PORT = 8000;
|
|
||||||
const MODEL_ID = 'kimi-agent';
|
|
||||||
const TIMEOUT_MS = 15 * 60 * 1000;
|
|
||||||
|
|
||||||
const WORKSPACE = '/workspace';
|
|
||||||
const CONV_ROOT = path.join(WORKSPACE, 'conversations');
|
|
||||||
const STATE_DIR = path.join(WORKSPACE, '.kimi-agent');
|
|
||||||
const MAP_FILE = path.join(STATE_DIR, 'sessions.json');
|
|
||||||
const MAX_ENTRIES = 1000; // prune oldest beyond this
|
|
||||||
|
|
||||||
fs.mkdirSync(CONV_ROOT, { recursive: true });
|
|
||||||
fs.mkdirSync(STATE_DIR, { recursive: true });
|
|
||||||
|
|
||||||
// --- persistent conversation -> session map ---------------------------------
|
|
||||||
// key = hash(history-so-far) -> { convId, sessionId, dir, ts }
|
|
||||||
let sessionMap = {};
|
|
||||||
try {
|
|
||||||
sessionMap = JSON.parse(fs.readFileSync(MAP_FILE, 'utf8'));
|
|
||||||
} catch {
|
|
||||||
sessionMap = {};
|
|
||||||
}
|
|
||||||
|
|
||||||
let writeQueue = Promise.resolve();
|
|
||||||
function persistMap() {
|
|
||||||
// prune to the MAX_ENTRIES most-recently-used before writing
|
|
||||||
const keys = Object.keys(sessionMap);
|
|
||||||
if (keys.length > MAX_ENTRIES) {
|
|
||||||
keys.sort((a, b) => (sessionMap[a].ts || 0) - (sessionMap[b].ts || 0));
|
|
||||||
for (const k of keys.slice(0, keys.length - MAX_ENTRIES)) delete sessionMap[k];
|
|
||||||
}
|
|
||||||
const snapshot = JSON.stringify(sessionMap);
|
|
||||||
writeQueue = writeQueue.then(
|
|
||||||
() => fs.promises.writeFile(MAP_FILE, snapshot),
|
|
||||||
() => fs.promises.writeFile(MAP_FILE, snapshot),
|
|
||||||
);
|
|
||||||
return writeQueue;
|
|
||||||
}
|
|
||||||
|
|
||||||
// --- message helpers --------------------------------------------------------
|
|
||||||
function textOf(msg) {
|
|
||||||
const c = msg.content;
|
|
||||||
if (Array.isArray(c)) return c.map(p => p.text || '').join('\n');
|
|
||||||
return c == null ? '' : String(c);
|
|
||||||
}
|
|
||||||
|
|
||||||
// only user/assistant turns define conversation identity (system is constant)
|
|
||||||
function convTurns(messages) {
|
|
||||||
return messages.filter(m => m.role === 'user' || m.role === 'assistant');
|
|
||||||
}
|
|
||||||
|
|
||||||
function historyKey(turns) {
|
|
||||||
const norm = turns.map(m => ({ role: m.role, text: textOf(m).trim() }));
|
|
||||||
return crypto.createHash('sha256').update(JSON.stringify(norm)).digest('hex');
|
|
||||||
}
|
|
||||||
|
|
||||||
function renderTranscript(turns) {
|
|
||||||
return turns
|
|
||||||
.map(m => `${m.role === 'assistant' ? 'Assistant' : 'User'}: ${textOf(m)}`)
|
|
||||||
.join('\n\n');
|
|
||||||
}
|
|
||||||
|
|
||||||
// --- kimi invocation --------------------------------------------------------
|
|
||||||
// Returns { text, sessionId }. Parses --output-format stream-json:
|
|
||||||
// {"role":"assistant","content":"..."}
|
|
||||||
// {"role":"meta","type":"session.resume_hint","session_id":"session_..."}
|
|
||||||
function runKimi({ prompt, cwd, resumeId }) {
|
|
||||||
return new Promise((resolve, reject) => {
|
|
||||||
const args = [];
|
|
||||||
if (resumeId) args.push('-r', resumeId);
|
|
||||||
args.push('-p', prompt, '--output-format', 'stream-json');
|
|
||||||
|
|
||||||
const child = spawn('kimi', args, { cwd, timeout: TIMEOUT_MS });
|
|
||||||
|
|
||||||
let stdout = '';
|
|
||||||
let stderr = '';
|
|
||||||
child.stdout.on('data', d => { stdout += d; });
|
|
||||||
child.stderr.on('data', d => { stderr += d; });
|
|
||||||
|
|
||||||
child.on('error', reject);
|
|
||||||
child.on('close', code => {
|
|
||||||
const parts = [];
|
|
||||||
let sessionId = null;
|
|
||||||
for (const line of stdout.split('\n')) {
|
|
||||||
const t = line.trim();
|
|
||||||
if (!t) continue;
|
|
||||||
let obj;
|
|
||||||
try { obj = JSON.parse(t); } catch { continue; }
|
|
||||||
if (obj.role === 'assistant' && obj.content) parts.push(obj.content);
|
|
||||||
if (obj.type === 'session.resume_hint' && obj.session_id) sessionId = obj.session_id;
|
|
||||||
}
|
|
||||||
const text = parts.join('').trim();
|
|
||||||
if (!text && code !== 0) {
|
|
||||||
reject(new Error(`kimi exited ${code}: ${stderr.slice(0, 2000)}`));
|
|
||||||
} else {
|
|
||||||
resolve({ text, sessionId });
|
|
||||||
}
|
|
||||||
});
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
// Decide session/dir, run kimi, and record the forward mapping.
|
|
||||||
async function handleTurn(messages) {
|
|
||||||
const turns = convTurns(messages);
|
|
||||||
// find the last user turn = the new prompt; everything before it is prior history
|
|
||||||
let lastUserIdx = -1;
|
|
||||||
for (let i = turns.length - 1; i >= 0; i--) {
|
|
||||||
if (turns[i].role === 'user') { lastUserIdx = i; break; }
|
|
||||||
}
|
|
||||||
if (lastUserIdx === -1) throw new Error('no user message found');
|
|
||||||
|
|
||||||
const newPrompt = textOf(turns[lastUserIdx]);
|
|
||||||
const prior = turns.slice(0, lastUserIdx);
|
|
||||||
|
|
||||||
let convId;
|
|
||||||
let dir;
|
|
||||||
let resumeId = null;
|
|
||||||
let prompt = newPrompt;
|
|
||||||
|
|
||||||
if (prior.length === 0) {
|
|
||||||
// brand-new conversation
|
|
||||||
convId = crypto.randomUUID();
|
|
||||||
dir = path.join(CONV_ROOT, convId);
|
|
||||||
} else {
|
|
||||||
const entry = sessionMap[historyKey(prior)];
|
|
||||||
if (entry) {
|
|
||||||
// known conversation -> resume the same kimi session in its own dir
|
|
||||||
convId = entry.convId;
|
|
||||||
dir = entry.dir;
|
|
||||||
resumeId = entry.sessionId;
|
|
||||||
} else {
|
|
||||||
// lost mapping (restart / edited history): reseed a fresh session with
|
|
||||||
// the full transcript so continuity is preserved
|
|
||||||
convId = crypto.randomUUID();
|
|
||||||
dir = path.join(CONV_ROOT, convId);
|
|
||||||
prompt = renderTranscript(turns.slice(0, lastUserIdx + 1));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fs.mkdirSync(dir, { recursive: true });
|
|
||||||
const { text, sessionId } = await runKimi({ prompt, cwd: dir, resumeId });
|
|
||||||
|
|
||||||
// store forward mapping: next request's prior history == these turns + reply
|
|
||||||
const forward = turns.slice(0, lastUserIdx + 1).concat([{ role: 'assistant', content: text }]);
|
|
||||||
sessionMap[historyKey(forward)] = {
|
|
||||||
convId,
|
|
||||||
sessionId: sessionId || resumeId,
|
|
||||||
dir,
|
|
||||||
ts: Date.now(),
|
|
||||||
};
|
|
||||||
persistMap();
|
|
||||||
|
|
||||||
return text;
|
|
||||||
}
|
|
||||||
|
|
||||||
// --- OpenAI-compatible HTTP surface ----------------------------------------
|
|
||||||
function completionBody(text) {
|
|
||||||
return {
|
|
||||||
id: `chatcmpl-${Date.now()}`,
|
|
||||||
object: 'chat.completion',
|
|
||||||
created: Math.floor(Date.now() / 1000),
|
|
||||||
model: MODEL_ID,
|
|
||||||
choices: [{
|
|
||||||
index: 0,
|
|
||||||
message: { role: 'assistant', content: text },
|
|
||||||
finish_reason: 'stop',
|
|
||||||
}],
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
const server = http.createServer((req, res) => {
|
|
||||||
if (req.method === 'GET' && req.url === '/v1/models') {
|
|
||||||
res.writeHead(200, { 'Content-Type': 'application/json' });
|
|
||||||
res.end(JSON.stringify({
|
|
||||||
object: 'list',
|
|
||||||
data: [{ id: MODEL_ID, object: 'model', owned_by: 'moonshot' }],
|
|
||||||
}));
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
if (req.method === 'POST' && req.url === '/v1/chat/completions') {
|
|
||||||
let body = '';
|
|
||||||
req.on('data', d => { body += d; });
|
|
||||||
req.on('end', async () => {
|
|
||||||
let parsed;
|
|
||||||
try {
|
|
||||||
parsed = JSON.parse(body);
|
|
||||||
} catch {
|
|
||||||
res.writeHead(400, { 'Content-Type': 'application/json' });
|
|
||||||
res.end(JSON.stringify({ error: 'invalid JSON body' }));
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
try {
|
|
||||||
const text = await handleTurn(parsed.messages || []);
|
|
||||||
|
|
||||||
if (parsed.stream) {
|
|
||||||
res.writeHead(200, {
|
|
||||||
'Content-Type': 'text/event-stream',
|
|
||||||
'Cache-Control': 'no-cache',
|
|
||||||
Connection: 'keep-alive',
|
|
||||||
});
|
|
||||||
const id = `chatcmpl-${Date.now()}`;
|
|
||||||
const created = Math.floor(Date.now() / 1000);
|
|
||||||
res.write(`data: ${JSON.stringify({
|
|
||||||
id, object: 'chat.completion.chunk', created, model: MODEL_ID,
|
|
||||||
choices: [{ index: 0, delta: { role: 'assistant', content: text }, finish_reason: null }],
|
|
||||||
})}\n\n`);
|
|
||||||
res.write(`data: ${JSON.stringify({
|
|
||||||
id, object: 'chat.completion.chunk', created, model: MODEL_ID,
|
|
||||||
choices: [{ index: 0, delta: {}, finish_reason: 'stop' }],
|
|
||||||
})}\n\n`);
|
|
||||||
res.write('data: [DONE]\n\n');
|
|
||||||
res.end();
|
|
||||||
} else {
|
|
||||||
res.writeHead(200, { 'Content-Type': 'application/json' });
|
|
||||||
res.end(JSON.stringify(completionBody(text)));
|
|
||||||
}
|
|
||||||
} catch (err) {
|
|
||||||
res.writeHead(500, { 'Content-Type': 'application/json' });
|
|
||||||
res.end(JSON.stringify({ error: String(err.message || err) }));
|
|
||||||
}
|
|
||||||
});
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
res.writeHead(404, { 'Content-Type': 'application/json' });
|
|
||||||
res.end(JSON.stringify({ error: 'not found' }));
|
|
||||||
});
|
|
||||||
|
|
||||||
server.listen(PORT, () => console.log(`kimi-agent wrapper listening on :${PORT}`));
|
|
||||||
@@ -2,7 +2,7 @@
|
|||||||
# Seafile backup script.
|
# Seafile backup script.
|
||||||
# Backs up MySQL databases and seafile data directory.
|
# Backs up MySQL databases and seafile data directory.
|
||||||
# Runs every 3 days via root crontab. Keeps last 5 backups.
|
# Runs every 3 days via root crontab. Keeps last 5 backups.
|
||||||
# Notifies Zabbix (item seafile.backup.ts, id 70369 on AgapHost) after success.
|
# Backup-freshness monitored via .age items.
|
||||||
|
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
@@ -29,18 +29,9 @@ rsync -a --delete \
|
|||||||
|
|
||||||
echo "$(date): Backup complete: $DEST"
|
echo "$(date): Backup complete: $DEST"
|
||||||
ls "$DEST/"
|
ls "$DEST/"
|
||||||
|
# Backup-freshness monitoring is now done via .age items (calculated fields showing
|
||||||
# Notify Zabbix
|
# age of the backup). The .ts (timestamp) trappers were unreliable (history.push not
|
||||||
if [[ -f /root/.zabbix_token ]]; then
|
# landing); removed in kb#189 in favor of .age overdue triggers.
|
||||||
ZABBIX_TOKEN=$(cat /root/.zabbix_token)
|
|
||||||
NOW_EPOCH=$(date '+%s')
|
|
||||||
env -u HTTPS_PROXY -u HTTP_PROXY -u ALL_PROXY -u https_proxy -u http_proxy -u all_proxy \
|
|
||||||
curl -s -X POST http://192.168.1.4:81/api_jsonrpc.php \
|
|
||||||
-H "Content-Type: application/json" \
|
|
||||||
-H "Authorization: Bearer $ZABBIX_TOKEN" \
|
|
||||||
-d "{\"jsonrpc\":\"2.0\",\"method\":\"history.push\",\"id\":1,\"params\":{\"itemid\":\"70369\",\"value\":$NOW_EPOCH}}" > /dev/null \
|
|
||||||
&& echo "Zabbix notified (seafile.backup.ts=$NOW_EPOCH)."
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Rotate: keep last 5 backups
|
# Rotate: keep last 5 backups
|
||||||
ls -1dt "$BACKUP_DIR"/[0-9]*-[0-9]* 2>/dev/null | tail -n +6 | xargs -r rm -rf
|
ls -1dt "$BACKUP_DIR"/[0-9]*-[0-9]* 2>/dev/null | tail -n +6 | xargs -r rm -rf
|
||||||
|
|||||||
100
seafile/restore.sh
Executable file
100
seafile/restore.sh
Executable file
@@ -0,0 +1,100 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Seafile restore — companion to backup.sh (kb#192).
|
||||||
|
#
|
||||||
|
# Restores a snapshot produced by backup.sh (ccnet_db.sql, seafile_db.sql,
|
||||||
|
# seahub_db.sql, and a data/ directory tree) into a running MariaDB
|
||||||
|
# container plus a data directory. Defaults to the live containers/paths,
|
||||||
|
# but every target is overridable via env vars so the same script can be
|
||||||
|
# pointed at throwaway containers + a scratch data dir for a dry-run
|
||||||
|
# restore test.
|
||||||
|
#
|
||||||
|
# Usage:
|
||||||
|
# ./restore.sh /mnt/backups/seafile/<snapshot-dir>
|
||||||
|
#
|
||||||
|
# Env overrides (defaults = live service):
|
||||||
|
# MYSQL_CONTAINER=seafile-mysql
|
||||||
|
# SEAFILE_CONTAINER=seafile # stopped/started around the data-dir copy; set to "" to skip
|
||||||
|
# DATA_DIR=/mnt/misc/seafile # host path the seafile-net containers bind-mount
|
||||||
|
# MYSQL_USER=seafile
|
||||||
|
# MYSQL_PASSWORD=<from agap_git/seafile/.env SEAFILE_MYSQL_DB_PASSWORD, falls back to backup.sh's literal>
|
||||||
|
#
|
||||||
|
# WARNING: this drops and reloads the target's live ccnet/seafile/seahub
|
||||||
|
# databases and overwrites its data directory. Never run against the live
|
||||||
|
# "seafile-mysql"/"seafile" containers or /mnt/misc/seafile unless you
|
||||||
|
# intend a real disaster recovery — for testing, point the *_CONTAINER
|
||||||
|
# and DATA_DIR vars at throwaway equivalents instead.
|
||||||
|
#
|
||||||
|
# NOTE: restoring only the three databases (no data/ directory, e.g.
|
||||||
|
# MYSQL_CONTAINER set but SEAFILE_CONTAINER="" and no data/ present in the
|
||||||
|
# snapshot) is a valid partial restore for verifying DB integrity —
|
||||||
|
# the script skips the data-dir step automatically if snapshot has no data/.
|
||||||
|
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||||
|
[ -f "$SCRIPT_DIR/.env" ] && source "$SCRIPT_DIR/.env"
|
||||||
|
|
||||||
|
MYSQL_CONTAINER="${MYSQL_CONTAINER:-seafile-mysql}"
|
||||||
|
# NOTE: use ${VAR-default} (no colon) for SEAFILE_CONTAINER so that an
|
||||||
|
# explicit empty string (SEAFILE_CONTAINER="") disables the data-dir step,
|
||||||
|
# as documented above — ${VAR:-default} would treat "" as unset and fall
|
||||||
|
# back to the live container name, which is not what an operator asking
|
||||||
|
# to skip that step intends.
|
||||||
|
SEAFILE_CONTAINER="${SEAFILE_CONTAINER-seafile}"
|
||||||
|
DATA_DIR="${DATA_DIR:-/mnt/misc/seafile}"
|
||||||
|
MYSQL_USER="${MYSQL_USER:-seafile}"
|
||||||
|
MYSQL_PASSWORD="${MYSQL_PASSWORD:-${SEAFILE_MYSQL_DB_PASSWORD:-FWsYYeZa15ro6x}}"
|
||||||
|
|
||||||
|
if [ $# -lt 1 ]; then
|
||||||
|
echo "Usage: $0 <path-to-backup-snapshot-dir>" >&2
|
||||||
|
echo " e.g. $0 /mnt/backups/seafile/20260728-0200" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
SRC="$(realpath "$1")"
|
||||||
|
|
||||||
|
for DB in ccnet_db seafile_db seahub_db; do
|
||||||
|
if [ ! -s "$SRC/${DB}.sql" ]; then
|
||||||
|
echo "Error: $SRC/${DB}.sql is missing or empty — refusing to restore from a broken snapshot" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
if ! docker inspect "$MYSQL_CONTAINER" > /dev/null 2>&1; then
|
||||||
|
echo "Error: mysql container '$MYSQL_CONTAINER' does not exist" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "Restoring databases into '$MYSQL_CONTAINER' from $SRC"
|
||||||
|
|
||||||
|
for DB in ccnet_db seafile_db seahub_db; do
|
||||||
|
echo "Restoring $DB..."
|
||||||
|
docker exec -i "$MYSQL_CONTAINER" mysql -u "$MYSQL_USER" -p"$MYSQL_PASSWORD" \
|
||||||
|
-e "DROP DATABASE IF EXISTS \`$DB\`; CREATE DATABASE \`$DB\` CHARACTER SET utf8mb4;"
|
||||||
|
docker exec -i "$MYSQL_CONTAINER" mysql -u "$MYSQL_USER" -p"$MYSQL_PASSWORD" "$DB" < "$SRC/${DB}.sql"
|
||||||
|
echo "Restored: $DB"
|
||||||
|
done
|
||||||
|
|
||||||
|
echo "Verifying restored databases..."
|
||||||
|
for DB in ccnet_db seafile_db seahub_db; do
|
||||||
|
TABLES=$(docker exec "$MYSQL_CONTAINER" mysql -u "$MYSQL_USER" -p"$MYSQL_PASSWORD" -N -e "SHOW TABLES FROM \`$DB\`;" | wc -l)
|
||||||
|
echo "$DB: $TABLES tables"
|
||||||
|
done
|
||||||
|
|
||||||
|
if [ -d "$SRC/data" ] && [ -n "$SEAFILE_CONTAINER" ]; then
|
||||||
|
echo "Restoring data directory into $DATA_DIR..."
|
||||||
|
if docker inspect "$SEAFILE_CONTAINER" > /dev/null 2>&1; then
|
||||||
|
docker stop "$SEAFILE_CONTAINER" > /dev/null
|
||||||
|
fi
|
||||||
|
rsync -a --delete \
|
||||||
|
--exclude='seafile-mysql/' \
|
||||||
|
--exclude='seafile-caddy/' \
|
||||||
|
"$SRC/data/" "$DATA_DIR/"
|
||||||
|
if docker inspect "$SEAFILE_CONTAINER" > /dev/null 2>&1; then
|
||||||
|
docker start "$SEAFILE_CONTAINER" > /dev/null
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
echo "Note: no data/ dir in snapshot or SEAFILE_CONTAINER unset — skipping data-dir restore (DB-only restore)"
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "Restore complete."
|
||||||
@@ -1,7 +1,6 @@
|
|||||||
#!/bin/bash
|
#!/bin/bash
|
||||||
# Backup /mnt/misc/alvis and /mnt/misc/liza to /mnt/backups/users/
|
# Backup /mnt/misc/alvis and /mnt/misc/liza to /mnt/backups/users/
|
||||||
# Runs every 3 days via root crontab.
|
# Runs every 3 days via root crontab. Backup-freshness monitored via .age items.
|
||||||
# Notifies Zabbix (item users.backup.ts, id 70379 on AgapHost) after success.
|
|
||||||
|
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
@@ -13,13 +12,6 @@ rsync -a --delete /mnt/misc/alvis/ "$DEST/alvis/"
|
|||||||
rsync -a --delete /mnt/misc/liza/ "$DEST/liza/"
|
rsync -a --delete /mnt/misc/liza/ "$DEST/liza/"
|
||||||
|
|
||||||
echo "$(date): Backup complete."
|
echo "$(date): Backup complete."
|
||||||
|
# Backup-freshness monitoring is now done via .age items (calculated fields showing
|
||||||
# Notify Zabbix (token stored in /root/.zabbix_token)
|
# age of the backup). The .ts (timestamp) trappers were unreliable (history.push not
|
||||||
if [[ -f /root/.zabbix_token ]]; then
|
# landing); removed in kb#189 in favor of .age overdue triggers.
|
||||||
ZABBIX_TOKEN=$(cat /root/.zabbix_token)
|
|
||||||
curl -s -X POST http://localhost:81/api_jsonrpc.php \
|
|
||||||
-H "Content-Type: application/json" \
|
|
||||||
-H "Authorization: Bearer $ZABBIX_TOKEN" \
|
|
||||||
-d "{\"jsonrpc\":\"2.0\",\"method\":\"history.push\",\"id\":1,\"params\":{\"itemid\":\"70379\",\"value\":\"$(date '+%Y-%m-%d %H:%M')\"}}" > /dev/null \
|
|
||||||
&& echo "Zabbix notified."
|
|
||||||
fi
|
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
#!/bin/bash
|
#!/bin/bash
|
||||||
# Vaultwarden backup — uses built-in container backup command (safe with live DB).
|
# Vaultwarden backup — uses built-in container backup command (safe with live DB).
|
||||||
# Runs every 3 days via root crontab. Keeps last 5 backups.
|
# Runs every 3 days via root crontab. Keeps last 5 backups.
|
||||||
# Notifies Zabbix (item vaultwarden.backup.ts, id 70368 on AgapHost) after success.
|
# Backup-freshness monitored via .age items.
|
||||||
|
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
@@ -26,18 +26,9 @@ cp "$DATA_DIR"/rsa_key* "$DEST/"
|
|||||||
|
|
||||||
echo "$(date): Backup complete: $DEST"
|
echo "$(date): Backup complete: $DEST"
|
||||||
ls "$DEST/"
|
ls "$DEST/"
|
||||||
|
# Backup-freshness monitoring is now done via .age items (calculated fields showing
|
||||||
# Notify Zabbix (token stored in /root/.zabbix_token)
|
# age of the backup). The .ts (timestamp) trappers were unreliable (history.push not
|
||||||
if [[ -f /root/.zabbix_token ]]; then
|
# landing); removed in kb#189 in favor of .age overdue triggers.
|
||||||
ZABBIX_TOKEN=$(cat /root/.zabbix_token)
|
|
||||||
NOW_EPOCH=$(date '+%s')
|
|
||||||
env -u HTTPS_PROXY -u HTTP_PROXY -u ALL_PROXY -u https_proxy -u http_proxy -u all_proxy \
|
|
||||||
curl -s -X POST http://192.168.1.4:81/api_jsonrpc.php \
|
|
||||||
-H "Content-Type: application/json" \
|
|
||||||
-H "Authorization: Bearer $ZABBIX_TOKEN" \
|
|
||||||
-d "{\"jsonrpc\":\"2.0\",\"method\":\"history.push\",\"id\":1,\"params\":{\"itemid\":\"70368\",\"value\":$NOW_EPOCH}}" > /dev/null \
|
|
||||||
&& echo "Zabbix notified (vaultwarden.backup.ts=$NOW_EPOCH)."
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Rotate: keep last 5 backups
|
# Rotate: keep last 5 backups
|
||||||
ls -1dt "$BACKUP_DIR"/[0-9]*-[0-9]* 2>/dev/null | tail -n +6 | xargs -r rm -rf
|
ls -1dt "$BACKUP_DIR"/[0-9]*-[0-9]* 2>/dev/null | tail -n +6 | xargs -r rm -rf
|
||||||
|
|||||||
72
vaultwarden/restore.sh
Executable file
72
vaultwarden/restore.sh
Executable file
@@ -0,0 +1,72 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Vaultwarden restore — companion to backup.sh (kb#192).
|
||||||
|
#
|
||||||
|
# Restores a snapshot produced by backup.sh (db_*.sqlite3, config.json,
|
||||||
|
# rsa_key*, attachments/, sends/) into a Vaultwarden data directory.
|
||||||
|
# Defaults to the live container/data dir, but every target is overridable
|
||||||
|
# via env vars so the same script can be pointed at a throwaway
|
||||||
|
# container + scratch data dir for a dry-run restore test.
|
||||||
|
#
|
||||||
|
# Usage:
|
||||||
|
# ./restore.sh /mnt/backups/vaultwarden/<snapshot-dir>
|
||||||
|
#
|
||||||
|
# Env overrides (defaults = live service):
|
||||||
|
# CONTAINER=vaultwarden
|
||||||
|
# DATA_DIR=/mnt/ssd/dbs/vw-data
|
||||||
|
#
|
||||||
|
# WARNING: this overwrites the target's live vault database. Never run
|
||||||
|
# against the "vaultwarden" container / /mnt/ssd/dbs/vw-data unless you
|
||||||
|
# intend a real disaster recovery — for testing, point CONTAINER/DATA_DIR
|
||||||
|
# at a throwaway container and a scratch directory instead.
|
||||||
|
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
CONTAINER="${CONTAINER:-vaultwarden}"
|
||||||
|
DATA_DIR="${DATA_DIR:-/mnt/ssd/dbs/vw-data}"
|
||||||
|
|
||||||
|
if [ $# -lt 1 ]; then
|
||||||
|
echo "Usage: $0 <path-to-backup-snapshot-dir>" >&2
|
||||||
|
echo " e.g. $0 /mnt/backups/vaultwarden/20260728-0200" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
SRC="$(realpath "$1")"
|
||||||
|
DB_FILE="$(find "$SRC" -maxdepth 1 -name 'db_*.sqlite3' | head -n1)"
|
||||||
|
|
||||||
|
if [ -z "$DB_FILE" ] || [ ! -f "$DB_FILE" ]; then
|
||||||
|
echo "Error: no db_*.sqlite3 file found in $SRC" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "Restoring into '$CONTAINER' (data dir: $DATA_DIR) from $SRC"
|
||||||
|
|
||||||
|
docker stop "$CONTAINER" > /dev/null
|
||||||
|
|
||||||
|
mkdir -p "$DATA_DIR"
|
||||||
|
cp "$DB_FILE" "$DATA_DIR/db.sqlite3"
|
||||||
|
[ -f "$SRC/config.json" ] && cp "$SRC/config.json" "$DATA_DIR/"
|
||||||
|
[ -f "$SRC/rsa_key.pem" ] && cp "$SRC"/rsa_key* "$DATA_DIR/" 2>/dev/null || true
|
||||||
|
[ -d "$SRC/attachments" ] && rsync -a --delete "$SRC/attachments/" "$DATA_DIR/attachments/"
|
||||||
|
[ -d "$SRC/sends" ] && rsync -a --delete "$SRC/sends/" "$DATA_DIR/sends/"
|
||||||
|
|
||||||
|
docker start "$CONTAINER" > /dev/null
|
||||||
|
|
||||||
|
echo "Waiting for Vaultwarden to come up..."
|
||||||
|
for i in $(seq 1 30); do
|
||||||
|
if docker exec "$CONTAINER" test -f /data/db.sqlite3 > /dev/null 2>&1; then
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
sleep 1
|
||||||
|
done
|
||||||
|
|
||||||
|
echo "Verifying restored database..."
|
||||||
|
docker exec "$CONTAINER" sh -c '
|
||||||
|
if command -v sqlite3 >/dev/null 2>&1; then
|
||||||
|
echo "users row count: $(sqlite3 /data/db.sqlite3 "SELECT COUNT(*) FROM users;")"
|
||||||
|
else
|
||||||
|
echo "(sqlite3 CLI not present in image; file size check only)"
|
||||||
|
ls -la /data/db.sqlite3
|
||||||
|
fi
|
||||||
|
'
|
||||||
|
|
||||||
|
echo "Restore complete: $CONTAINER"
|
||||||
Reference in New Issue
Block a user