Compare commits
42 Commits
e04f9059ae
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 4a9ae75912 | |||
| e5438ace79 | |||
| d37801806d | |||
| 41f3f15d27 | |||
| f67a5bee67 | |||
| b27d31b3ca | |||
| a5c625b9b6 | |||
| fc4e1c75ed | |||
| 5b649123a8 | |||
| 59af13f1fe | |||
| 0f9d83f3db | |||
| b548a8f345 | |||
| 288cb6b36a | |||
| b126e7aaed | |||
| b5aaceb65a | |||
| d9668928c0 | |||
| ebb7fb8af3 | |||
| 442bc377bf | |||
| e74d28a808 | |||
| e049eef81a | |||
| 4ac595a3a9 | |||
| 544637c073 | |||
| 4dfc870dd1 | |||
| cf4d57ea16 | |||
| 9ab6b7dfed | |||
| 1e66d3dcb5 | |||
| fb93655636 | |||
| eadda808d2 | |||
| f5c14efb37 | |||
| 546d3b9438 | |||
|
|
73ebe6408d | ||
| 6869e4ea09 | |||
|
|
fa1bddf537 | ||
|
|
406083310f | ||
|
|
995d639b60 | ||
|
|
b8efe4732d | ||
|
|
4363130163 | ||
|
|
808c3ee254 | ||
|
|
7ef3ffa00e | ||
|
|
85033136d8 | ||
|
|
52190b63b8 | ||
|
|
b7c503499a |
16
.gitignore
vendored
16
.gitignore
vendored
@@ -1,2 +1,18 @@
|
|||||||
|
# Secrets — per-service env files. Never commit; real values live in
|
||||||
|
# Vaultwarden and are injected via docker-compose ${VAR} substitution.
|
||||||
adolf/.env
|
adolf/.env
|
||||||
seafile/.env
|
seafile/.env
|
||||||
|
openai/.env
|
||||||
|
agap-mcp/.env
|
||||||
|
.env
|
||||||
|
|
||||||
|
# Machine-generated noise (added 2026-07-30). These had accumulated untracked
|
||||||
|
# for weeks; they churn on every run and only produce noisy diffs.
|
||||||
|
__pycache__/
|
||||||
|
*.pyc
|
||||||
|
*.log
|
||||||
|
|
||||||
|
# Timestamped backup snapshots left behind by edit scripts
|
||||||
|
# (e.g. docker-compose.yml.bak-20260704-141509, CLAUDE.md.bak-kb).
|
||||||
|
*.bak
|
||||||
|
*.bak-*
|
||||||
|
|||||||
29
CLAUDE.md
29
CLAUDE.md
@@ -4,16 +4,27 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
|
|||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
This repository manages Docker Compose configurations for the **Agap** self-hosted home server. It is not a software project — it is infrastructure-as-config for several independent services.
|
This repository is the **server CONFIGURATION** repo for **Agap**, the self-hosted
|
||||||
|
home server. It is not a software project — it holds `docker-compose` + host-level
|
||||||
|
config for every service running on Agap. Convention: Dockerfiles/application source
|
||||||
|
live in the service's own Gitea repo; `agap_git` keeps only the compose + config
|
||||||
|
that runs it. See [README.md](./README.md) for the full service list, the
|
||||||
|
self-developed-services table, and a known gap (matrixbot/moodtracker/marketplace-mcp/oO
|
||||||
|
currently have no compose here — see README for details).
|
||||||
|
|
||||||
## Services
|
## Services
|
||||||
|
|
||||||
|
Selected services with notes below; see [README.md](./README.md) for the complete list.
|
||||||
|
|
||||||
| Directory | Service | Port | Notes |
|
| Directory | Service | Port | Notes |
|
||||||
|-----------|---------|------|-------|
|
|-----------|---------|------|-------|
|
||||||
| `immich-app/` | Immich (photo management) | 2283 | Main compose via root `docker-compose.yml` |
|
| `immich-app/` | Immich (photo management) | 2283 | Main compose via root `docker-compose.yml` |
|
||||||
| `gitea/` | Gitea (git hosting) + Postgres | 3000, 222 | Standalone compose |
|
| `gitea/` | Gitea (git hosting) + Postgres | 3000, 222 | Standalone compose |
|
||||||
| `openai/` | Open WebUI + Ollama (AI chat) | 3125 | Requires NVIDIA GPU |
|
| `openai/` | Adolf (OpenClaw gateway) + LiteLLM + Hindsight + Qdrant + Langfuse | see `openai/docker-compose.yml` | Requires NVIDIA GPU |
|
||||||
|
| `openwebui/` | Open WebUI (AI chat) | 3125 | Standalone compose |
|
||||||
|
| `ollama/` | Ollama (local LLM runtime) | 11436 | Requires NVIDIA GPU |
|
||||||
| `vaultwarden/` | Vaultwarden (password manager) | 8041 | Backup script in `vaultwarden/backup.sh` |
|
| `vaultwarden/` | Vaultwarden (password manager) | 8041 | Backup script in `vaultwarden/backup.sh` |
|
||||||
|
| `kanboard/` | Kanboard (kanban board) | 4800 | Tasks assignable to the `claude` bot user — see `kanboard/CLAUDE.md` |
|
||||||
|
|
||||||
## Common Commands
|
## Common Commands
|
||||||
|
|
||||||
@@ -88,7 +99,7 @@ When changes are made to infrastructure (services, config, setup), update the re
|
|||||||
| Home | Index — links to all pages |
|
| Home | Index — links to all pages |
|
||||||
| Network | Netplan bridge setup, Caddy reverse proxy |
|
| Network | Netplan bridge setup, Caddy reverse proxy |
|
||||||
| Storage | LVM setup and commands |
|
| Storage | LVM setup and commands |
|
||||||
| Home-Assistant | KVM-based Home Assistant setup |
|
| Home-Assistant | Home Assistant (runs on lizacer, `192.168.1.4`, Docker) |
|
||||||
| 3X-UI | VPN proxy panel |
|
| 3X-UI | VPN proxy panel |
|
||||||
| Gitea | Git hosting Docker service |
|
| Gitea | Git hosting Docker service |
|
||||||
| Vaultwarden | Password manager, CLI setup, backup |
|
| Vaultwarden | Password manager, CLI setup, backup |
|
||||||
@@ -200,27 +211,29 @@ Home Assistant automations push alerts to Zabbix via `history.push` API (Zabbix
|
|||||||
|
|
||||||
## Zabbix API
|
## Zabbix API
|
||||||
|
|
||||||
**Instance**: `http://localhost:81` (local), `https://zb.alogins.net` (external)
|
**Zabbix does not run on Agap — it lives on lizacer (`192.168.1.4`).** Config for the stack moved to the `lizacer` Gitea repo (kb#81).
|
||||||
**Endpoint**: `http://localhost:81/api_jsonrpc.php`
|
|
||||||
|
**Instance**: `http://192.168.1.4:81` (local), `https://zb.alogins.net` (external, Caddy on Agap → `192.168.1.4:81`)
|
||||||
|
**Endpoint**: `http://192.168.1.4:81/api_jsonrpc.php`
|
||||||
**Token**: Read from `$ZABBIX_TOKEN` environment variable — never hardcode it
|
**Token**: Read from `$ZABBIX_TOKEN` environment variable — never hardcode it
|
||||||
**Auth header**: `Authorization: Bearer <token>`
|
**Auth header**: `Authorization: Bearer <token>`
|
||||||
|
|
||||||
### Common Requests
|
### Common Requests
|
||||||
```bash
|
```bash
|
||||||
# Check API version
|
# Check API version
|
||||||
curl -s -X POST http://localhost:81/api_jsonrpc.php \
|
curl -s -X POST http://192.168.1.4:81/api_jsonrpc.php \
|
||||||
-H "Content-Type: application/json" \
|
-H "Content-Type: application/json" \
|
||||||
-H "Authorization: Bearer $ZABBIX_TOKEN" \
|
-H "Authorization: Bearer $ZABBIX_TOKEN" \
|
||||||
-d '{"jsonrpc":"2.0","method":"apiinfo.version","params":{},"id":1}'
|
-d '{"jsonrpc":"2.0","method":"apiinfo.version","params":{},"id":1}'
|
||||||
|
|
||||||
# Get all hosts
|
# Get all hosts
|
||||||
curl -s -X POST http://localhost:81/api_jsonrpc.php \
|
curl -s -X POST http://192.168.1.4:81/api_jsonrpc.php \
|
||||||
-H "Content-Type: application/json" \
|
-H "Content-Type: application/json" \
|
||||||
-H "Authorization: Bearer $ZABBIX_TOKEN" \
|
-H "Authorization: Bearer $ZABBIX_TOKEN" \
|
||||||
-d '{"jsonrpc":"2.0","method":"host.get","params":{"output":"extend"},"id":1}'
|
-d '{"jsonrpc":"2.0","method":"host.get","params":{"output":"extend"},"id":1}'
|
||||||
|
|
||||||
# Get problems/issues
|
# Get problems/issues
|
||||||
curl -s -X POST http://localhost:81/api_jsonrpc.php \
|
curl -s -X POST http://192.168.1.4:81/api_jsonrpc.php \
|
||||||
-H "Content-Type: application/json" \
|
-H "Content-Type: application/json" \
|
||||||
-H "Authorization: Bearer $ZABBIX_TOKEN" \
|
-H "Authorization: Bearer $ZABBIX_TOKEN" \
|
||||||
-d '{"jsonrpc":"2.0","method":"problem.get","params":{"output":"extend"},"id":1}'
|
-d '{"jsonrpc":"2.0","method":"problem.get","params":{"output":"extend"},"id":1}'
|
||||||
|
|||||||
109
Caddyfile
109
Caddyfile
@@ -1,5 +1,11 @@
|
|||||||
|
{
|
||||||
|
servers {
|
||||||
|
protocols h1 h2
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
haos.alogins.net {
|
haos.alogins.net {
|
||||||
reverse_proxy http://192.168.1.141:8123 {
|
reverse_proxy http://192.168.1.4:8123 {
|
||||||
|
|
||||||
header_up X-Forwarded-For {remote_host}
|
header_up X-Forwarded-For {remote_host}
|
||||||
header_up X-Forwarded-Proto {scheme}
|
header_up X-Forwarded-Proto {scheme}
|
||||||
@@ -16,7 +22,7 @@ doc.alogins.net {
|
|||||||
}
|
}
|
||||||
|
|
||||||
zb.alogins.net {
|
zb.alogins.net {
|
||||||
reverse_proxy localhost:81
|
reverse_proxy 192.168.1.4:81
|
||||||
}
|
}
|
||||||
|
|
||||||
wiki.alogins.net {
|
wiki.alogins.net {
|
||||||
@@ -27,6 +33,10 @@ wiki.alogins.net {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
lt.alogins.net {
|
||||||
|
reverse_proxy localhost:4321
|
||||||
|
}
|
||||||
|
|
||||||
nn.alogins.net {
|
nn.alogins.net {
|
||||||
reverse_proxy localhost:5678
|
reverse_proxy localhost:5678
|
||||||
}
|
}
|
||||||
@@ -43,12 +53,41 @@ ai.alogins.net {
|
|||||||
reverse_proxy localhost:3125
|
reverse_proxy localhost:3125
|
||||||
}
|
}
|
||||||
|
|
||||||
|
news.alogins.net {
|
||||||
|
reverse_proxy localhost:8091
|
||||||
|
}
|
||||||
|
|
||||||
|
family.alogins.net {
|
||||||
|
reverse_proxy localhost:8099
|
||||||
|
}
|
||||||
|
|
||||||
|
lw.alogins.net {
|
||||||
|
reverse_proxy localhost:3012
|
||||||
|
}
|
||||||
|
|
||||||
|
todo.alogins.net {
|
||||||
|
reverse_proxy localhost:3457
|
||||||
|
}
|
||||||
|
|
||||||
|
ttt.alogins.net {
|
||||||
|
reverse_proxy localhost:3003
|
||||||
|
}
|
||||||
|
|
||||||
openpi.alogins.net {
|
openpi.alogins.net {
|
||||||
root * /home/alvis/tmp/files/pi05_droid
|
root * /home/alvis/tmp/files/pi05_droid
|
||||||
file_server browse
|
file_server browse
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
dl.alogins.net {
|
||||||
|
@chiefx17 path /chief-x17.zip
|
||||||
|
handle @chiefx17 {
|
||||||
|
root * /mnt/misc/qbittorrent/downloads
|
||||||
|
file_server
|
||||||
|
}
|
||||||
|
respond 404
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
vui3.alogins.net {
|
vui3.alogins.net {
|
||||||
@xhttp {
|
@xhttp {
|
||||||
@@ -69,6 +108,24 @@ vui3.alogins.net {
|
|||||||
respond 401
|
respond 401
|
||||||
}
|
}
|
||||||
|
|
||||||
|
o.alogins.net {
|
||||||
|
handle /api/* {
|
||||||
|
reverse_proxy localhost:3078
|
||||||
|
}
|
||||||
|
handle /admin* {
|
||||||
|
reverse_proxy localhost:3080
|
||||||
|
}
|
||||||
|
handle /mlflow* {
|
||||||
|
reverse_proxy localhost:5000
|
||||||
|
}
|
||||||
|
handle /airflow* {
|
||||||
|
reverse_proxy localhost:8080
|
||||||
|
}
|
||||||
|
handle {
|
||||||
|
reverse_proxy localhost:3079
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
vui4.alogins.net {
|
vui4.alogins.net {
|
||||||
reverse_proxy localhost:58959
|
reverse_proxy localhost:58959
|
||||||
}
|
}
|
||||||
@@ -116,6 +173,54 @@ lk.alogins.net {
|
|||||||
reverse_proxy localhost:7880
|
reverse_proxy localhost:7880
|
||||||
}
|
}
|
||||||
|
|
||||||
|
lf.alogins.net {
|
||||||
|
reverse_proxy localhost:3200
|
||||||
|
}
|
||||||
|
|
||||||
|
llm.alogins.net {
|
||||||
|
reverse_proxy localhost:4000
|
||||||
|
}
|
||||||
|
|
||||||
|
overleaf.alogins.net {
|
||||||
|
reverse_proxy localhost:8089
|
||||||
|
}
|
||||||
|
|
||||||
|
voice.alogins.net {
|
||||||
|
reverse_proxy localhost:8882
|
||||||
|
}
|
||||||
|
|
||||||
|
iperf.alogins.net {
|
||||||
|
reverse_proxy localhost:8095
|
||||||
|
}
|
||||||
|
|
||||||
|
anki.alogins.net {
|
||||||
|
reverse_proxy localhost:8180
|
||||||
|
}
|
||||||
|
|
||||||
|
kb.alogins.net {
|
||||||
|
reverse_proxy localhost:4800
|
||||||
|
}
|
||||||
|
|
||||||
|
mood.alogins.net {
|
||||||
|
reverse_proxy localhost:5177
|
||||||
|
}
|
||||||
|
|
||||||
|
sync.alogins.net {
|
||||||
|
reverse_proxy localhost:8384
|
||||||
|
}
|
||||||
|
|
||||||
|
dav.alogins.net {
|
||||||
|
reverse_proxy localhost:5232
|
||||||
|
}
|
||||||
|
|
||||||
|
tor.alogins.net {
|
||||||
|
reverse_proxy localhost:8085
|
||||||
|
}
|
||||||
|
|
||||||
|
win.alogins.net {
|
||||||
|
reverse_proxy localhost:8006
|
||||||
|
}
|
||||||
|
|
||||||
localhost:8042 {
|
localhost:8042 {
|
||||||
reverse_proxy localhost:8041
|
reverse_proxy localhost:8041
|
||||||
tls internal
|
tls internal
|
||||||
|
|||||||
120
README.md
120
README.md
@@ -1,60 +1,108 @@
|
|||||||
# Agap Home Server
|
# agap_git — Agap Server Configuration
|
||||||
|
|
||||||
Docker Compose configurations for the Agap self-hosted home server infrastructure.
|
`agap_git` is the **configuration repo** for Agap, the self-hosted home server. It holds:
|
||||||
|
|
||||||
## Services
|
- `docker-compose` files for services running on Agap
|
||||||
|
- host-level config: Caddy (`Caddyfile`), backup/install scripts, `.env` files
|
||||||
|
|
||||||
- **Immich** (`immich-app/`) — Photo management and backup (port 2283)
|
**Convention:** a Dockerfile, application source, or anything you'd `build:` from an
|
||||||
- **Gitea** (`gitea/`) — Self-hosted Git server with web UI (port 3000, SSH 222)
|
image belongs in the *service's own Gitea repo* — not here. `agap_git` keeps the
|
||||||
- **Open WebUI** (`openai/`) — AI chat interface with Ollama, GPU-accelerated (port 3125)
|
compose file that runs the resulting image, plus whatever config the compose needs.
|
||||||
|
Third-party services (pulling a public image) live here in full, since there's no
|
||||||
|
source to split out.
|
||||||
|
|
||||||
|
## Self-developed services
|
||||||
|
|
||||||
|
Services with source written in-house are meant to be dual: source lives in their
|
||||||
|
own Gitea repo, and the compose that runs them is described here.
|
||||||
|
|
||||||
|
| Service | Own repo | Compose in agap_git |
|
||||||
|
|---|---|---|
|
||||||
|
| matrixbot (Matrix bot, Adolf channel adapter) | [`alvis/matrixbot`](http://localhost:3000/alvis/matrixbot) | **not present** — runs from `~/matrixbot/docker-compose.yml` in its own repo |
|
||||||
|
| moodtracker | [`alvis/moodtracker`](http://localhost:3000/alvis/moodtracker) | **not present** — runs from `~/moodtracker/docker-compose.yml` |
|
||||||
|
| marketplace-mcp | [`alvis/marketplace-mcp`](http://localhost:3000/alvis/marketplace-mcp) | **not present** — runs from `~/marketplace-mcp/docker-compose.yml` |
|
||||||
|
| oO | [`alvis/oO`](http://localhost:3000/alvis/oO) | **not present** — runs from `oO/infra/docker/docker-compose.yml`; oO was already a fully separate project before this convention existed |
|
||||||
|
| mood (mood-archive) | none yet | fully vendored here (`mood/`, source + compose) — Kanboard **#209** will extract it to its own repo |
|
||||||
|
|
||||||
|
**Known gap:** the convention calls for compose to be described in `agap_git` *and*
|
||||||
|
source to live in its own repo. For matrixbot / moodtracker / marketplace-mcp / oO,
|
||||||
|
extraction (kb#78) moved the compose file along with the source into each new repo,
|
||||||
|
so `agap_git` currently has zero footprint for these four — no compose, no config.
|
||||||
|
Not fixed in this doc-only pass; flagged for a follow-up decision: either copy each
|
||||||
|
compose back into `agap_git`, or formally amend the convention to let self-developed
|
||||||
|
services keep compose in their own repo.
|
||||||
|
|
||||||
|
## Third-party services (compose lives here in full)
|
||||||
|
|
||||||
|
| Directory | Service | Port |
|
||||||
|
|---|---|---|
|
||||||
|
| `immich-app/` | Immich — photo management | 2283 |
|
||||||
|
| `gitea/` | Gitea — git hosting + Postgres | 3000, 222 |
|
||||||
|
| `openai/` | Adolf (OpenClaw gateway) + LiteLLM + Hindsight + Qdrant + Langfuse + Whisper/TTS stack | see `openai/docker-compose.yml` (adolf-llm on 8010) |
|
||||||
|
| `adolf/` | Adolf config only (`openclaw.json`, design docs) — the `adolf` compose service lives in `openai/docker-compose.yml`; the OpenClaw fork source is a separate tree at `~/adolf` | — |
|
||||||
|
| `vaultwarden/` | Password manager | 8041 |
|
||||||
|
| `kanboard/` | Kanban board, task orchestration for the `claude` bot | 4800 |
|
||||||
|
| `seafile/` | File sync, OnlyOffice, WebDAV (multiple compose files) | 8078 (server), 8077 (Caddy) |
|
||||||
|
| `matrix/` | Synapse homeserver + coturn + LiveKit (not the `matrixbot` bot — see above) | — |
|
||||||
|
| `overleaf/` | LaTeX editor (ShareLaTeX/Overleaf CE) | — |
|
||||||
|
| `linkwarden/` | Bookmark manager | 3012 |
|
||||||
|
| `vikunja/` | Task manager | 3457 |
|
||||||
|
| `radicale/` | CalDAV/CardDAV server | 5232 |
|
||||||
|
| `ntfy/` | Push notifications | 8840 |
|
||||||
|
| `syncthing/` | File sync | 8384 |
|
||||||
|
| `qbittorrent/` | Torrent client | 8085 |
|
||||||
|
| `searxng/` | Metasearch engine | 11437 |
|
||||||
|
| `ollama/` | Local LLM runtime | 11436 |
|
||||||
|
| `openwebui/` | AI chat UI | 3125 |
|
||||||
|
| `anki/` | Anki sync server | 8180 |
|
||||||
|
| `family/` | MediaWiki (family wiki) | 8099 |
|
||||||
|
| `freshrss/` | RSS reader | 8091 |
|
||||||
|
| `iperf3/` | Network throughput testing | 8095 |
|
||||||
|
| `otter/` | OtterWiki | 8083 |
|
||||||
|
| `agap-mcp/` | MCP tool server for Claude (Node/Express) — vendored source, no separate repo | 3100 |
|
||||||
|
| `personal-sensing/` | SQLite store + schema for HA/Health Connect data (no compose; library used by an ETL, see kb#207) | — |
|
||||||
|
|
||||||
|
## Removed / moved out
|
||||||
|
|
||||||
|
- `windows/` — removed (kb#82): no VM, no container, dead config.
|
||||||
|
- `zabbix/`, `haos/` — moved to [`alvis/lizacer`](http://localhost:3000/alvis/lizacer) (kb#81). Zabbix runs on the **lizacer** server (`192.168.1.4:81`), not Agap.
|
||||||
|
|
||||||
|
## Top-level files
|
||||||
|
|
||||||
|
- `Caddyfile` — reverse proxy / TLS routing for all services
|
||||||
|
- `nvidia-docker-install.sh`, `install-cuda.sh` — GPU host setup
|
||||||
|
- `users-backup.sh` — host user backup
|
||||||
|
|
||||||
## Quick Start
|
## Quick Start
|
||||||
|
|
||||||
### Start Immich (main service)
|
Each service is standalone; from its directory:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker compose up -d
|
docker compose up -d
|
||||||
|
docker compose restart
|
||||||
|
docker compose logs -f
|
||||||
|
docker compose pull
|
||||||
```
|
```
|
||||||
|
|
||||||
### Start Gitea (from gitea/ directory)
|
The root `docker-compose.yml` is an alias that includes `immich-app/docker-compose.yml`.
|
||||||
|
|
||||||
```bash
|
|
||||||
cd gitea
|
|
||||||
docker compose up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
### Start Open WebUI (from openai/ directory)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd openai
|
|
||||||
docker compose up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
## Configuration
|
|
||||||
|
|
||||||
Environment variables are in the root `.env` file for Immich:
|
|
||||||
- `UPLOAD_LOCATION` — where photo originals are stored
|
|
||||||
- `THUMB_LOCATION` — thumbnail cache directory
|
|
||||||
- `ENCODED_VIDEO_LOCATION` — transcoded video cache
|
|
||||||
- `DB_DATA_LOCATION` — Postgres database directory
|
|
||||||
- `DB_PASSWORD` — Postgres password
|
|
||||||
|
|
||||||
## Storage
|
## Storage
|
||||||
|
|
||||||
Media is stored on:
|
|
||||||
- `/mnt/media/upload` — Immich originals
|
- `/mnt/media/upload` — Immich originals
|
||||||
- `/mnt/ssd1/media/` — Immich thumbnails, encoded video, and Postgres database
|
- `/mnt/ssd1/media/` — Immich thumbnails, encoded video, Postgres
|
||||||
- `/mnt/misc/gitea` — Gitea repositories and data
|
- `/mnt/misc/gitea` — Gitea repositories and data
|
||||||
|
|
||||||
## GPU Support
|
## GPU Support
|
||||||
|
|
||||||
For GPU acceleration (Open WebUI/Ollama, Immich ML):
|
For GPU acceleration (Ollama/Open WebUI, Immich ML):
|
||||||
|
|
||||||
1. Install NVIDIA Docker runtime: `sudo ./nvidia-docker-install.sh`
|
1. `sudo ./nvidia-docker-install.sh` — Docker + NVIDIA Container Toolkit
|
||||||
2. Install CUDA toolkit: `./install-cuda.sh`
|
2. `./install-cuda.sh` — CUDA toolkit
|
||||||
|
|
||||||
## Documentation
|
## Documentation
|
||||||
|
|
||||||
See [CLAUDE.md](./CLAUDE.md) for detailed developer instructions and Gitea wiki integration guidelines.
|
See [CLAUDE.md](./CLAUDE.md) for developer instructions, credentials handling, and
|
||||||
|
Gitea/Zabbix/Home Assistant API integration details.
|
||||||
|
|
||||||
See the [Gitea wiki](http://localhost:3000/alvis/AgapHost/wiki) for infrastructure documentation (storage, network, services setup).
|
See the [Gitea wiki](http://localhost:3000/alvis/AgapHost/wiki) for infrastructure
|
||||||
|
documentation (storage, network, per-service setup).
|
||||||
|
|||||||
161
SWAP_EXHAUSTION_ANALYSIS_20260726.md
Normal file
161
SWAP_EXHAUSTION_ANALYSIS_20260726.md
Normal file
@@ -0,0 +1,161 @@
|
|||||||
|
# Swap Exhaustion Analysis — 2026-07-26
|
||||||
|
|
||||||
|
**Status:** CRITICAL — Swap 4.0Gi/4.0Gi exhausted (8.0Ki free)
|
||||||
|
**Alert Status:** Zabbix "High swap space usage" FIRING on AgapHost since 2026-07-26 04:42
|
||||||
|
**Memory Pressure:** 12Gi/15Gi RAM used (458Mi free, 3.5Gi available with cache)
|
||||||
|
|
||||||
|
## Current Measurements (2026-07-26 11:46 UTC+3)
|
||||||
|
|
||||||
|
```
|
||||||
|
RAM: 12Gi/15Gi (80% used, 458Mi free, 3.5Gi cache)
|
||||||
|
Swap: 4.0Gi/4.0Gi (100% EXHAUSTED, 8.0Ki free)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Top Swap Consumers
|
||||||
|
|
||||||
|
### 1. Claude Code Processes (Host) — 265 MB swap total
|
||||||
|
|
||||||
|
These are interactive development sessions running on the host, not containers:
|
||||||
|
|
||||||
|
| PID | Process | Swap | RSS | Description |
|
||||||
|
|-----|---------|------|-----|-------------|
|
||||||
|
| 4036111 | claude 2.1.220 main | 71.7 MB | 266 MB | Active main session (opus model) |
|
||||||
|
| 4036089 | claude bg-pty-host | 62.6 MB | 42.7 MB | Background PTY host |
|
||||||
|
| 4034268 | claude bg-pty-host | 63.9 MB | 38.6 MB | Background PTY host |
|
||||||
|
| 4034247 | claude | 42.6 MB | 102 MB | Claude process |
|
||||||
|
| 4034281 | claude bg-spare | 29.8 MB | 79.7 MB | Spare background process |
|
||||||
|
|
||||||
|
**Finding:** Multiple interactive Claude Code sessions are consuming ~500 MB combined RSS and paging ~265 MB to swap due to RAM pressure.
|
||||||
|
|
||||||
|
### 2. Docker Containers (Top 4 by Memory)
|
||||||
|
|
||||||
|
| Container | Image | Memory | Swap | Status |
|
||||||
|
|-----------|-------|--------|------|--------|
|
||||||
|
| hindsight | ghcr.io/vectorize-io/hindsight | 834.3 MiB | 11.7 MB | Memory-intensive but stable |
|
||||||
|
| sharelatex | sharelatex/sharelatex:6.1.2 | 612.8 MiB | <1 MB | Large footprint |
|
||||||
|
| tei-reranker | openai-tei-reranker | 363.9 MiB | ~1 MB | Minimal swap |
|
||||||
|
| adolf (matrixbot) | adolf:local | 478.2 MiB | 73.6 MB | Modest swap usage |
|
||||||
|
|
||||||
|
### 3. Other Notable Processes
|
||||||
|
|
||||||
|
- `hindsight-api` (PID 996): 11.7 MB swap, 717 MB RSS
|
||||||
|
- `qbittorrent-nox`: 7.4 MB swap, 53.8 MB RSS
|
||||||
|
- `syncthing` (2 instances): 2.2 MB swap, 82 MB RSS combined
|
||||||
|
- `postgres`: <1 MB swap per process
|
||||||
|
|
||||||
|
## Root Cause Analysis
|
||||||
|
|
||||||
|
**Primary driver:** Multiple interactive Claude Code sessions consuming ~500 MB combined memory, with 265 MB swapped out due to low available RAM.
|
||||||
|
|
||||||
|
**Secondary pressure:** Hindsight (834 MiB) and ShareLatex (612 MiB) are large but mostly RSS; they don't cause the swap explosion directly, but contribute to overall memory pressure that forces smaller processes into swap.
|
||||||
|
|
||||||
|
**System state:** With only 458 Mi RAM free and cache being reclaimed, any process trying to allocate memory gets swapped, including the interactive Claude sessions.
|
||||||
|
|
||||||
|
## Mitigation Options (Staged)
|
||||||
|
|
||||||
|
### Stage 1: Kill Idle Claude Sessions (IMMEDIATE, ZERO RISK)
|
||||||
|
|
||||||
|
**Action:** Terminate idle/background Claude Code sessions, keep only essential active session(s).
|
||||||
|
|
||||||
|
**Impact:** Frees ~200–300 MB swap (5–7% relief), swap would drop to ~3.7 Gi.
|
||||||
|
|
||||||
|
**Risk:** None — these are human-driven interactive sessions, not persistent services.
|
||||||
|
|
||||||
|
**Commands:**
|
||||||
|
```bash
|
||||||
|
# Kill all background Claude processes except the main session
|
||||||
|
pkill -f "claude.*bg-pty-host"
|
||||||
|
pkill -f "claude.*bg-spare"
|
||||||
|
# Or selectively: kill 4034268 4036089 4034281
|
||||||
|
```
|
||||||
|
|
||||||
|
**Expected result:** Immediate swap relief; Zabbix alert will clear once usage drops below 80%.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Stage 2: Evaluate ShareLatex (SHORT TERM, IF NEEDED)
|
||||||
|
|
||||||
|
**Action:** If ShareLatex is not actively used, remove it.
|
||||||
|
|
||||||
|
**Impact:** Frees ~612 MB RAM; would bring total free RAM to ~1 Gi.
|
||||||
|
|
||||||
|
**Risk:** Low if ShareLatex is idle; medium if it's required.
|
||||||
|
|
||||||
|
**Commands:**
|
||||||
|
```bash
|
||||||
|
docker compose stop sharelatex
|
||||||
|
docker compose rm sharelatex
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Stage 3: Add Memory Limits to Containers (MEDIUM TERM, REQUIRES RESTART)
|
||||||
|
|
||||||
|
**Action:** Add explicit memory limits to docker-compose.yml for hindsight and other memory-heavy services.
|
||||||
|
|
||||||
|
**Example for hindsight:**
|
||||||
|
```yaml
|
||||||
|
services:
|
||||||
|
hindsight:
|
||||||
|
image: ghcr.io/vectorize-io/hindsight:latest
|
||||||
|
deploy:
|
||||||
|
resources:
|
||||||
|
limits:
|
||||||
|
memory: 512M # or 768M depending on headroom needed
|
||||||
|
```
|
||||||
|
|
||||||
|
**Impact:** Prevents services from consuming unbounded memory; predictable resource allocation.
|
||||||
|
|
||||||
|
**Risk:** Medium — requires container restart; if limit is too low, hindsight may OOM.
|
||||||
|
|
||||||
|
**Recommendation:** Test at 768M first; monitor for OOM events. Hindsight's memory is cache-heavy (RSS 834 MiB suggests it might stabilize lower).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Stage 4: Increase Swap (TEMPORARY, LOW PRIORITY)
|
||||||
|
|
||||||
|
**Action:** Add more swap (6–8 Gi).
|
||||||
|
|
||||||
|
**Impact:** Symptom relief only; doesn't address root cause (working set is larger than available RAM).
|
||||||
|
|
||||||
|
**Risk:** Low operational risk, but performance would degrade under paging load.
|
||||||
|
|
||||||
|
**Not recommended** as primary fix; use only if Stages 1–3 are insufficient.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Zabbix Alert Status
|
||||||
|
|
||||||
|
**Trigger:** `Linux: High swap space usage` on AgapHost
|
||||||
|
**Value:** 1 (FIRING)
|
||||||
|
**Last change:** 2026-07-26 04:42:29 UTC+3
|
||||||
|
**Condition:** Swap usage > 80%
|
||||||
|
|
||||||
|
The alert will **automatically clear** once swap drops below the threshold (typically when used ≤ 3.2 Gi / 4.0 Gi).
|
||||||
|
|
||||||
|
## Long-Term Recommendations
|
||||||
|
|
||||||
|
1. **Docker Compose Memory Limits:** Add `deploy.resources.limits.memory` to all services in docker-compose.yml. This prevents runaway memory consumption and makes the system predictable.
|
||||||
|
|
||||||
|
2. **Monitor Claude Code Sessions:** Interactive development tools are memory-intensive. Consider:
|
||||||
|
- Limiting the number of concurrent sessions
|
||||||
|
- Restarting sessions periodically if they grow unbounded
|
||||||
|
- Monitoring growth patterns
|
||||||
|
|
||||||
|
3. **Hindsight Configuration:** Check if hindsight can reduce its cache footprint via environment variables or config (e.g., max memory, cache size limits).
|
||||||
|
|
||||||
|
4. **Periodic Audits:** Add task to quarterly review top memory consumers and adjust limits as needed.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Acceptance Criteria Status
|
||||||
|
|
||||||
|
| Criterion | Status | Notes |
|
||||||
|
|-----------|--------|-------|
|
||||||
|
| Swap free >1 GB sustained | ❌ Pending mitigation | Currently 8 Ki free; Stage 1 would bring to ~700 MiB |
|
||||||
|
| Zabbix 'High swap' clears | ❌ Pending mitigation | Alert will clear once swap < 80% (~3.2 Gi) |
|
||||||
|
| Note on dominant consumer + mitigation | ✅ Complete | This document |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Next step:** Execute Stage 1 (kill idle Claude sessions) for immediate relief, then monitor Zabbix alert status.
|
||||||
345
adolf/DESIGN-proactive-prioritization.md
Normal file
345
adolf/DESIGN-proactive-prioritization.md
Normal file
@@ -0,0 +1,345 @@
|
|||||||
|
# DESIGN — Proactive action impact/cost prioritization (kb#123)
|
||||||
|
|
||||||
|
Status: **draft, for review** — written 2026-07-22
|
||||||
|
Owner: alvis · Written with Claude
|
||||||
|
Scope: kanboard Adolf task **#123**. This is a design/ruleset only — no wiring
|
||||||
|
into `openclaw.json`, no code. Claude implements it in a follow-up task.
|
||||||
|
|
||||||
|
Related, not duplicated here:
|
||||||
|
- **#122** (utility/ROI metrics via Langfuse) — that's the after-the-fact
|
||||||
|
"was Adolf worth it" readout across all of Adolf's spend. This doc is the
|
||||||
|
**before-the-fact gate** on one specific class of spend: proactive sends.
|
||||||
|
#122's Langfuse data is a candidate future input to the acceptance-rate term
|
||||||
|
below (§3.3), but this design does not depend on #122 landing first.
|
||||||
|
- **#124** (proactive cadence / cron schedule) — decides **when Adolf looks**
|
||||||
|
(daily/weekly/monthly audit cadence, adapted to quota). This design decides
|
||||||
|
**whether a specific candidate action fires** once #124 (or an ad-hoc
|
||||||
|
trigger, e.g. reacting to a calendar change) has already produced one.
|
||||||
|
#124 is the scheduler; this is the gate every candidate passes through
|
||||||
|
regardless of what triggered it.
|
||||||
|
- **#125** (feedback loop) — this design's acceptance-probability term (§3.3)
|
||||||
|
is a **consumer** of #125's feedback log. #125 is not built yet; §5 below
|
||||||
|
specifies exactly what it needs to log, as a dependency, not an assumption.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Problem
|
||||||
|
|
||||||
|
Adolf can generate a proactive message (reminder, nudge, digest item) from
|
||||||
|
several places: the cadence jobs in #124, a reactive trigger (calendar event
|
||||||
|
changed, HA sensor fired, a task went overdue), or background reasoning
|
||||||
|
noticing something. Not every candidate should be sent — some are low-value,
|
||||||
|
some are redundant with something already said, some cost real Kimi quota to
|
||||||
|
formulate and the window is nearly exhausted.
|
||||||
|
|
||||||
|
This design is a **gate function**: given a candidate proactive action, decide
|
||||||
|
fire / suppress / defer, using a score computed from four inputs the task
|
||||||
|
description names, plus a threshold that scales with remaining quota.
|
||||||
|
|
||||||
|
## 2. Where the gate sits
|
||||||
|
|
||||||
|
```
|
||||||
|
[trigger: #124 cadence job | reactive event | background noticing]
|
||||||
|
|
|
||||||
|
v
|
||||||
|
candidate proactive action
|
||||||
|
(draft content + metadata)
|
||||||
|
|
|
||||||
|
v
|
||||||
|
┌─────────────────────────┐
|
||||||
|
│ IMPACT/COST GATE │ <-- this design
|
||||||
|
│ (score, threshold) │
|
||||||
|
└─────────────────────────┘
|
||||||
|
| |
|
||||||
|
fire suppress / defer
|
||||||
|
| |
|
||||||
|
send via Matrix log decision + reason
|
||||||
|
log outcome slot (no send; feedback loop
|
||||||
|
for #125 has nothing to attach to)
|
||||||
|
```
|
||||||
|
|
||||||
|
The gate is a pure function of the candidate + current state. It does not
|
||||||
|
decide *what* to consider sending (#124's job) or *how* to learn from
|
||||||
|
responses (#125's job) — only whether a given candidate clears the bar right
|
||||||
|
now.
|
||||||
|
|
||||||
|
## 3. Scoring model
|
||||||
|
|
||||||
|
For each candidate action `a`, compute:
|
||||||
|
|
||||||
|
```
|
||||||
|
score(a) = (benefit(a) * accept_prob(a) * urgency(a)) / cost(a)
|
||||||
|
```
|
||||||
|
|
||||||
|
Ratio form, not a weighted sum: cost is a genuine denominator (token cost is
|
||||||
|
literally what you're trading against benefit), and the three numerator terms
|
||||||
|
are gates on each other, not additive alternatives — a high-benefit,
|
||||||
|
low-acceptance-probability action *should* be suppressed even if urgency is
|
||||||
|
high, not partially compensated the way a sum would allow. All four terms are
|
||||||
|
normalized to comparable ranges as defined below so no single term dominates
|
||||||
|
by scale alone.
|
||||||
|
|
||||||
|
### 3.1 Benefit — `benefit(a) ∈ [0, 1]`
|
||||||
|
|
||||||
|
"How important is this to the user, if accepted." Estimated by Adolf itself
|
||||||
|
(the model doing the drafting) using a small fixed rubric — this is a
|
||||||
|
judgment call, not a measurement, so keep the rubric coarse enough to be
|
||||||
|
stable across repeated runs:
|
||||||
|
|
||||||
|
| Band | Value | Examples |
|
||||||
|
|---|---|---|
|
||||||
|
| Critical | 1.0 | Hard deadline today/tomorrow, safety/health-adjacent, financial penalty if missed |
|
||||||
|
| High | 0.7 | Real deadline this week, blocks another person, irreversible if missed |
|
||||||
|
| Medium | 0.4 | Useful reminder, no hard deadline, low cost if ignored |
|
||||||
|
| Low | 0.15 | "Might be nice to know", trivia-adjacent, no consequence |
|
||||||
|
|
||||||
|
Adolf assigns the band as part of drafting the candidate (one extra field in
|
||||||
|
the same generation pass — no separate LLM call). This is inherently noisy;
|
||||||
|
it is corrected over time by the acceptance-probability term (§3.3), which is
|
||||||
|
grounded in actual logged outcomes rather than self-assessment.
|
||||||
|
|
||||||
|
### 3.2 Cost — `cost(a) ∈ (0, 1]`, token cost normalized
|
||||||
|
|
||||||
|
Raw cost is estimable *before* sending: `tokens_estimate(a)` = prompt tokens
|
||||||
|
to formulate (system + context already loaded for the turn, since it's
|
||||||
|
piggy-backing on an existing generation) + estimated output tokens for the
|
||||||
|
message itself. For a message that requires its **own** dedicated Kimi call
|
||||||
|
(true incremental cost) vs. one riding along inside an already-scheduled
|
||||||
|
cadence turn (near-zero marginal cost), these are very different costs — the
|
||||||
|
estimate must distinguish "marginal call I wouldn't otherwise make" from
|
||||||
|
"free byproduct of a call happening anyway":
|
||||||
|
|
||||||
|
```
|
||||||
|
cost_tokens(a) = marginal_prompt_tokens(a) + marginal_output_tokens(a)
|
||||||
|
```
|
||||||
|
|
||||||
|
where `marginal_*` is 0 (or near-0, e.g. a few output tokens) if the action
|
||||||
|
rides inside a scheduled #124 audit turn that would run regardless, and the
|
||||||
|
full call cost if it requires spinning up a fresh Kimi turn.
|
||||||
|
|
||||||
|
Normalize against a reference ceiling (a "typical expensive proactive send",
|
||||||
|
empirically ~2-3K tokens per the #122 baseline measurements of ~32.8K for a
|
||||||
|
full reply turn — a standalone proactive nudge should be far cheaper than a
|
||||||
|
full conversational turn, since it's one-directional with no back-and-forth):
|
||||||
|
|
||||||
|
```
|
||||||
|
cost(a) = clamp(cost_tokens(a) / COST_CEILING, floor=0.05, cap=1.0)
|
||||||
|
```
|
||||||
|
|
||||||
|
`COST_CEILING` = 3000 tokens (tunable constant, revisit once #122 gives real
|
||||||
|
distributions). The 0.05 floor stops a literally-free riding action from
|
||||||
|
dividing by ~0 and producing a runaway score — even "free" actions carry some
|
||||||
|
opportunity cost (attention, message-count against the 60/5h ceiling, not
|
||||||
|
just tokens).
|
||||||
|
|
||||||
|
### 3.3 Acceptance probability — `accept_prob(a) ∈ [0, 1]`
|
||||||
|
|
||||||
|
**This term has no data source yet.** It depends entirely on #125 (feedback
|
||||||
|
loop) being built and logging outcomes. Until then, use a flat prior:
|
||||||
|
|
||||||
|
```
|
||||||
|
accept_prob(a) = 0.5 # uninformative prior, pending #125
|
||||||
|
```
|
||||||
|
|
||||||
|
Once #125 logs `(action_class, outcome)` pairs (see §5's exact schema
|
||||||
|
requirement — this design does not invent history, it specifies what must
|
||||||
|
exist), compute a per-class empirical rate with Laplace smoothing so a class
|
||||||
|
with zero or few samples doesn't overfit to noise:
|
||||||
|
|
||||||
|
```
|
||||||
|
accept_prob(class) = (accepted_count(class) + 1) / (total_count(class) + 2)
|
||||||
|
```
|
||||||
|
|
||||||
|
`class` is a coarse bucket, not per-message: e.g. `{calendar_reminder,
|
||||||
|
task_overdue, ha_anomaly, family_wiki_gap, digest_item, ...}` — one row per
|
||||||
|
class, not per exact message text, since exact-text history rarely repeats
|
||||||
|
but the class does. A candidate's class is assigned at draft time (same pass
|
||||||
|
as §3.1's benefit band).
|
||||||
|
|
||||||
|
Recency matters more than total count — a user who started dismissing
|
||||||
|
`calendar_reminder` last week should pull that class's rate down faster than
|
||||||
|
five-year-old acceptances prop it up. Use a decayed count (e.g. half-life of
|
||||||
|
30 days, or simply windowing to the trailing N=50 outcomes per class) rather
|
||||||
|
than an all-time average, once enough volume exists to make decay
|
||||||
|
meaningful.
|
||||||
|
|
||||||
|
### 3.4 Urgency — `urgency(a) ∈ [0, 1]`
|
||||||
|
|
||||||
|
Distinct from benefit: benefit is "how much it matters", urgency is "how soon
|
||||||
|
it stops being actionable". A time-decay curve against the nearest relevant
|
||||||
|
deadline (`due_at`) known for the candidate (calendar event start, task
|
||||||
|
`date_due`, HA-derived risk window):
|
||||||
|
|
||||||
|
```
|
||||||
|
hours_to_deadline = (due_at - now) in hours
|
||||||
|
|
||||||
|
urgency(a) =
|
||||||
|
1.0 if hours_to_deadline <= 1
|
||||||
|
1.0 - 0.6 * (h - 1) / 23 if 1 < h <= 24 (1.0 -> 0.4 over the day)
|
||||||
|
0.4 * exp(-(h - 24) / 168) if h > 24 (decays over the following week)
|
||||||
|
0.2 if no deadline (informational-only action)
|
||||||
|
```
|
||||||
|
|
||||||
|
Concretely: something due within the hour scores 1.0, something due
|
||||||
|
tomorrow ~0.4-1.0 depending on how close, something a week out trails off
|
||||||
|
toward the 0.2 floor for undated nudges. This is a simple monotonic decay, not
|
||||||
|
a precise model — tune the constants once real cadence data exists (#124).
|
||||||
|
|
||||||
|
### 3.5 Putting it together
|
||||||
|
|
||||||
|
```
|
||||||
|
score(a) = (benefit(a) * accept_prob(a) * urgency(a)) / cost(a)
|
||||||
|
```
|
||||||
|
|
||||||
|
Range: numerator ∈ [0, 1], denominator ∈ [0.05, 1], so
|
||||||
|
`score(a) ∈ [0, 20]` in the degenerate cheapest/most-urgent/most-beneficial
|
||||||
|
case. In practice, typical scores cluster well below that ceiling — the
|
||||||
|
threshold (§4) is calibrated empirically against observed scores, not derived
|
||||||
|
analytically from the range.
|
||||||
|
|
||||||
|
## 4. Firing rule
|
||||||
|
|
||||||
|
```
|
||||||
|
fire(a) iff score(a) >= threshold(current_quota_state)
|
||||||
|
else suppress(a) # or defer(a), see below
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4.1 Quota signal
|
||||||
|
|
||||||
|
Read `adolf-llm:8010/usage` (confirmed live shape, sampled 2026-07-22):
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"weekly": {"pct": 32, "used": 32, "limit": 100, "remaining": 68, "resets": "..."},
|
||||||
|
"window_5h":{"pct": 79, "used": 79, "limit": 100, "remaining": 21, "resets": "..."},
|
||||||
|
"stale": false
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Use the **tighter** of the two windows — whichever pct is higher is the
|
||||||
|
binding constraint right now:
|
||||||
|
|
||||||
|
```
|
||||||
|
quota_pressure = max(weekly.pct, window_5h.pct) / 100 # ∈ [0, 1]
|
||||||
|
```
|
||||||
|
|
||||||
|
If `stale: true` (Kimi login/session broken, per the adolf-llm fallback
|
||||||
|
behavior), treat as `quota_pressure = 1.0` (most conservative) — an unknown
|
||||||
|
quota state should suppress non-critical sends, not fire them.
|
||||||
|
|
||||||
|
### 4.2 Threshold as a function of quota pressure
|
||||||
|
|
||||||
|
```
|
||||||
|
threshold(quota_pressure) = T_BASE + (T_MAX - T_BASE) * quota_pressure^2
|
||||||
|
```
|
||||||
|
|
||||||
|
- `T_BASE` = 0.3 — threshold when quota is abundant (pressure ~0): let most
|
||||||
|
medium-benefit things through.
|
||||||
|
- `T_MAX` = 3.0 — threshold when quota is nearly exhausted (pressure ~1):
|
||||||
|
only near-maximal score (critical benefit, high acceptance history, urgent,
|
||||||
|
cheap) still fires.
|
||||||
|
- Squaring `quota_pressure` keeps the threshold flat and permissive through
|
||||||
|
low-to-mid pressure (nothing changes until quota actually gets tight) and
|
||||||
|
then rises steeply as the window approaches exhaustion — matching the
|
||||||
|
actual failure mode (403 usage-limit) which is a cliff, not a slope.
|
||||||
|
|
||||||
|
This gives a single tunable curve with two constants, both revisitable once
|
||||||
|
#122 supplies real score distributions and false-negative/positive rates.
|
||||||
|
|
||||||
|
### 4.3 Fire / suppress / defer
|
||||||
|
|
||||||
|
- **fire**: send now.
|
||||||
|
- **suppress**: below threshold and no deadline pressure — drop it. Log the
|
||||||
|
decision (§5) but do not re-surface it later; if it's still relevant, the
|
||||||
|
next cadence pass (#124) will regenerate it as a fresh candidate with
|
||||||
|
updated urgency.
|
||||||
|
- **defer**: below threshold *only because of quota pressure*, but
|
||||||
|
`urgency(a) >= 0.8` (i.e., something time-critical got starved by a quota
|
||||||
|
cliff, not by low benefit). Requeue for immediate re-evaluation once
|
||||||
|
`quota_pressure` drops (next window reset, per `resets` timestamp in the
|
||||||
|
usage payload) rather than silently dropping it. This is the one exception
|
||||||
|
to "gate is stateless" — a deferred item carries state (its own candidate
|
||||||
|
record) until it either fires or its deadline passes, at which point it is
|
||||||
|
logged as a missed/expired suppression, not silently lost.
|
||||||
|
|
||||||
|
## 5. Dependency: what #125's feedback log must contain
|
||||||
|
|
||||||
|
This design's accept_prob term (§3.3) is inert without it. #125 owns building
|
||||||
|
the collection mechanism (reactions, "+/-/неактуально" replies); this design
|
||||||
|
only specifies the **shape** the gate needs to consume, so the two tasks
|
||||||
|
don't diverge on schema:
|
||||||
|
|
||||||
|
```
|
||||||
|
proactive_outcome {
|
||||||
|
action_class: string # matches the class taxonomy in §3.3, e.g. "calendar_reminder"
|
||||||
|
sent_at: timestamp
|
||||||
|
benefit_band: float # the benefit(a) value used at send time, for later calibration
|
||||||
|
cost_tokens: int # actual cost, for calibrating COST_CEILING
|
||||||
|
urgency_at_send: float
|
||||||
|
outcome: enum { accepted, dismissed, ignored, irrelevant }
|
||||||
|
responded_at: timestamp | null
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`ignored` (no response within some window, e.g. 24h) must be distinguished
|
||||||
|
from `dismissed` (explicit "−") — an ignored item is weaker negative signal
|
||||||
|
than an explicit rejection and should decay the acceptance rate less
|
||||||
|
aggressively. Without this distinction the Laplace-smoothed rate in §3.3
|
||||||
|
conflates "user didn't care" with "user was just busy."
|
||||||
|
|
||||||
|
Every **suppressed** and **deferred** candidate should also be logged (not
|
||||||
|
just fired ones) with `outcome: not_sent` — this is what lets a later audit
|
||||||
|
(#122) compute false-suppression rate (was a suppressed item actually needed?
|
||||||
|
only knowable in hindsight, e.g. if the same underlying deadline later caused
|
||||||
|
a problem) as well as false-fire rate.
|
||||||
|
|
||||||
|
## 6. Worked example
|
||||||
|
|
||||||
|
Candidate: "reminder that the Seafile SSL cert renews in 3 days" (from a
|
||||||
|
#124 daily cadence audit, riding along inside that scheduled call).
|
||||||
|
|
||||||
|
- `benefit`: Medium band → 0.4 (annoying if missed, not critical — auto-renew
|
||||||
|
likely already configured, this is a check not a fire drill).
|
||||||
|
- `cost`: marginal — rides inside the already-running daily audit call, say
|
||||||
|
~150 marginal output tokens → `150/3000 = 0.05` → floored at 0.05.
|
||||||
|
- `accept_prob`: no #125 data yet → flat prior 0.5.
|
||||||
|
- `urgency`: `hours_to_deadline` = 72h → falls in the `h > 24` branch:
|
||||||
|
`0.4 * exp(-(72-24)/168) = 0.4 * exp(-0.286) ≈ 0.4 * 0.751 ≈ 0.30`.
|
||||||
|
|
||||||
|
`score = (0.4 * 0.5 * 0.30) / 0.05 = 0.06 / 0.05 = 1.2`
|
||||||
|
|
||||||
|
At `quota_pressure = 0` (abundant quota), `threshold = 0.3` → **1.2 ≥ 0.3,
|
||||||
|
fires.** At `quota_pressure = 1` (window nearly exhausted, matching the
|
||||||
|
measured 79% 5h-window sample above, rounding up toward the cliff),
|
||||||
|
`threshold = 3.0` → **1.2 < 3.0, suppressed** (not deferred: urgency 0.30 is
|
||||||
|
well under the 0.8 defer bar) — correctly deprioritized under quota pressure
|
||||||
|
in favor of anything more urgent or already proven to land well.
|
||||||
|
|
||||||
|
## 7. Open parameters to tune post-implementation
|
||||||
|
|
||||||
|
Everything with a concrete numeric constant above (`COST_CEILING`, `T_BASE`,
|
||||||
|
`T_MAX`, the urgency decay constants, the defer bar) is a starting guess
|
||||||
|
consistent with the measurements already on hand (#122's token baseline, the
|
||||||
|
live `/usage` sample). None of it is load-bearing on the *shape* of the
|
||||||
|
model — only on where the dial sits. Revisit once:
|
||||||
|
|
||||||
|
- #125 supplies real `accept_prob` data (replacing the flat 0.5 prior is the
|
||||||
|
single highest-value follow-up — everything else is a reasonable guess,
|
||||||
|
this term is currently a placeholder).
|
||||||
|
- #122's Langfuse integration supplies real per-action token costs to
|
||||||
|
recalibrate `COST_CEILING`.
|
||||||
|
- A few weeks of fire/suppress/defer logs (§5) exist to check the threshold
|
||||||
|
isn't systematically over- or under-firing.
|
||||||
|
|
||||||
|
## 8. Acceptance check against kb#123
|
||||||
|
|
||||||
|
- Scoring formula with each term defined, ranged, and its estimation method
|
||||||
|
stated: §3.
|
||||||
|
- Firing rule (fire only if impact/cost exceeds a threshold): §4.
|
||||||
|
- Threshold tunable to current quota, against the real `adolf-llm:8010/usage`
|
||||||
|
signal: §4.1-4.2.
|
||||||
|
- Acceptance-probability term flagged as dependent on unbuilt history
|
||||||
|
(#125), with the exact log schema it needs specified rather than
|
||||||
|
fabricated: §3.3, §5.
|
||||||
|
- Cross-references to sibling tasks #122, #124, #125 without duplicating
|
||||||
|
their scope: header + inline.
|
||||||
237
adolf/DESIGN-todoist-capture.md
Normal file
237
adolf/DESIGN-todoist-capture.md
Normal file
@@ -0,0 +1,237 @@
|
|||||||
|
# DESIGN — Todoist capture + AI classification (kb#170)
|
||||||
|
|
||||||
|
Status: **v1, components 1/2 built + proven; components 3/4 designed, not
|
||||||
|
built** — written 2026-07-23
|
||||||
|
Owner: alvis · Written with Claude
|
||||||
|
Scope: kanboard Adolf task **#170**. Related, not duplicated: Welfare
|
||||||
|
#102 (proactive-secretary umbrella), #105 (idea capture, pre-Todoist
|
||||||
|
version of the same need), #106 (people/events reminders).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Why Todoist, and why encoder-only classification
|
||||||
|
|
||||||
|
Todoist becomes Adolf's **inbox for ideas and quick tasks** — the place a
|
||||||
|
thought gets captured immediately, before it's clear whether it's a
|
||||||
|
one-liner or a project. Two things must stay true per
|
||||||
|
`DESIGN-a2a-agents.md` v2.1:
|
||||||
|
|
||||||
|
- **No metered API by default** (§3a): classification must not spend a
|
||||||
|
Kimi/gemma turn per capture. The stack already keeps **bge-m3** resident
|
||||||
|
(Hindsight's embedder, never-evict, `model-registry.yaml`) — reusing it
|
||||||
|
for classification is ~0 marginal cost, same reasoning §3a already
|
||||||
|
applies to LiteLLM's Auto Router.
|
||||||
|
- **Native commands run before the agent** (proven pattern:
|
||||||
|
`quota-command-openclaw-plugin`, kb#62): a `/idea` command that never
|
||||||
|
invokes Kimi at all keeps the entire capture path — not just the
|
||||||
|
classification step — off the metered/quota-gated path.
|
||||||
|
|
||||||
|
So: **nearest-centroid classification over bge-m3 embeddings**, not a
|
||||||
|
classifier LLM call and not hard tag rules. This is genuinely an
|
||||||
|
encoder-only model in the literal sense (bge-m3 is an encoder, not a
|
||||||
|
generative LLM) — no fine-tuning, no training loop, because **no labelled
|
||||||
|
dataset exists** (inventing one would be guessing scope that wasn't
|
||||||
|
asked for). The "training data" is a small, git-editable exemplar list per
|
||||||
|
class (`agap-mcp/src/classifier.js`) — extending accuracy later means
|
||||||
|
adding exemplars, not retraining.
|
||||||
|
|
||||||
|
## 2. Architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
Matrix "/idea <текст>" Adolf prose ("запомни идею...")
|
||||||
|
| |
|
||||||
|
v v
|
||||||
|
todoist-capture-plugin Adolf (Kimi) -> MCP tool call
|
||||||
|
(native command, 0 Kimi calls) todoist_capture_idea (agap-mcp)
|
||||||
|
| |
|
||||||
|
+--------------------+-------------------+
|
||||||
|
v
|
||||||
|
POST /capture-idea (agap-mcp, plain REST)
|
||||||
|
|
|
||||||
|
v
|
||||||
|
classifier.js: embed(text) via bge-m3
|
||||||
|
(1 embedding call, reused for all 3 axes)
|
||||||
|
|
|
||||||
|
+----------------+----------------+
|
||||||
|
v v v
|
||||||
|
area centroid urgency centroid decompose centroid
|
||||||
|
(5 classes) (3 classes) (2 classes)
|
||||||
|
| | |
|
||||||
|
+----------------+------------------+
|
||||||
|
v
|
||||||
|
capture.js: label + priority mapping
|
||||||
|
|
|
||||||
|
v
|
||||||
|
todoistCreateTask() -> real Todoist task
|
||||||
|
labels: area-*, urgency-*, [decompose], [area-uncertain]
|
||||||
|
```
|
||||||
|
|
||||||
|
Two entry points converge on one pipeline (`capture.js`'s
|
||||||
|
`todoistCaptureIdea`), reachable either as an MCP tool
|
||||||
|
(`todoist_capture_idea`, for Adolf's/Claude's model-driven path — "запомни
|
||||||
|
идею: ...") or as a plain REST route (`POST /capture-idea`, for the native
|
||||||
|
`/idea` command, which cannot speak MCP JSON-RPC). Both call the exact
|
||||||
|
same function — no duplicated classification/creation logic.
|
||||||
|
|
||||||
|
## 3. Component 2 — AI classification (built, proven)
|
||||||
|
|
||||||
|
Three independent axes per idea, one bge-m3 embedding shared across all
|
||||||
|
three:
|
||||||
|
|
||||||
|
| Axis | Classes | Source of exemplars |
|
||||||
|
|---|---|---|
|
||||||
|
| **area** | `adolf`, `welfare`, `дом`, `семья`, `здоровье` (kb#170 spec, verbatim) | `AREA_EXEMPLARS` |
|
||||||
|
| **urgency** | `high`, `medium`, `low` | `URGENCY_EXEMPLARS` |
|
||||||
|
| **decompose** | `simple-task`, `needs-decomposition` | `DECOMPOSE_EXEMPLARS` |
|
||||||
|
|
||||||
|
Classification = cosine similarity of the idea's embedding against each
|
||||||
|
class's centroid (mean of that class's exemplar embeddings), argmax per
|
||||||
|
axis. Each result also carries a **margin** (gap between the top two
|
||||||
|
scores) and an `ambiguous: true` flag when the margin is small
|
||||||
|
(< 0.03, an empirical starting threshold — same "tune later" posture as
|
||||||
|
`DESIGN-proactive-prioritization.md`'s constants). Ambiguous area
|
||||||
|
classifications get an extra `area-uncertain` label instead of being
|
||||||
|
silently forced — component 4 (periodic review) is where a human
|
||||||
|
resolves them, not an auto-retry on a bigger model (consistent with
|
||||||
|
`DESIGN-a2a-agents.md` §5's always-ask escalation policy, scaled down:
|
||||||
|
this isn't a costly/irreversible action, so the "escalation" here is just
|
||||||
|
a label, not a blocking gate to alvis's inbox).
|
||||||
|
|
||||||
|
**Proven** (`agap-mcp/src/classifier.test.mjs`, run against the real,
|
||||||
|
live bge-m3 at `:11436` — 7/7 pass): area/urgency/decompose all resolve
|
||||||
|
sensibly on hand-written Russian idea text spanning all 5 areas, both
|
||||||
|
urgency bands, and both decompose classes. `capture.test.mjs` (6/6 pass)
|
||||||
|
proves the label/priority/project mapping with a **stubbed** Todoist
|
||||||
|
client — no test data was written to the real Todoist account while
|
||||||
|
proving this out.
|
||||||
|
|
||||||
|
### 3.1 Project vs. label mapping — a decision made, not guessed
|
||||||
|
|
||||||
|
Todoist's real, live projects today (`todoist_list_projects`, confirmed
|
||||||
|
2026-07-23): `Inbox`, `One-Off`, `Family`, `Planning`, `Pending`. These do
|
||||||
|
**not** line up with the 5 kb#170 areas except `семья` ≈ `Family`.
|
||||||
|
Creating four new Todoist projects (`Adolf`, `Welfare`, `дом`,
|
||||||
|
`здоровье`) to match would be a structural change to the user's real
|
||||||
|
Todoist account — **not done here without sign-off** (see §6, open
|
||||||
|
question 1). Instead, v1 uses **labels** (`area-*`, `urgency-*`,
|
||||||
|
`decompose`, `area-uncertain`) for every axis — purely additive and
|
||||||
|
reversible (Todoist auto-creates labels on first use; deleting a label
|
||||||
|
loses no task data) — and auto-routes to an existing project only for the
|
||||||
|
one unambiguous match (`семья` → `Family`), never inventing a project
|
||||||
|
selection the classifier merely guessed at.
|
||||||
|
|
||||||
|
## 4. Component 1 — capture command (built, not activated)
|
||||||
|
|
||||||
|
`openai/todoist-capture-plugin/` — same shape as `quota-command-openclaw-
|
||||||
|
plugin` (kb#62): `definePluginEntry` + `api.registerCommand({ name:
|
||||||
|
"idea", acceptsArgs: true, requireAuth: true, handler })`. Verified
|
||||||
|
against the real `PluginCommandHandler`/`PluginCommandContext`/
|
||||||
|
`OpenClawPluginCommandDefinition` types in the OpenClaw source
|
||||||
|
(`/home/alvis/adolf/src/plugins/types.ts`) — `ctx.args` is the raw string
|
||||||
|
after `/idea`, handler returns `{ text, suppressReply }`.
|
||||||
|
|
||||||
|
`requireAuth: true` (default) keeps it behind the same Matrix DM allowlist
|
||||||
|
(`channels.matrix.dm.allowFrom`) gating every other Adolf interaction — no
|
||||||
|
new privilege tier, since creating a Todoist task in the operator's own
|
||||||
|
inbox isn't a privileged/destructive action.
|
||||||
|
|
||||||
|
**Wired but not live**: `docker-compose.yml` gets the read-only bind mount
|
||||||
|
(same pattern as `quota-command`/`hindsight-memory`/`kimi-quota-footer`),
|
||||||
|
`openclaw.json` gets `plugins.entries.todoist-capture.enabled: true`, and
|
||||||
|
`agap-mcp/src/server.js` gets the `POST /capture-idea` route the plugin
|
||||||
|
calls. All three are plain config/code edits, proven end-to-end with a
|
||||||
|
stub Todoist client (see kb#170 report) — but **none of this takes effect
|
||||||
|
until the adolf container is restarted** (same activation gate every prior
|
||||||
|
plugin in this repo has hit: bind-mounts and `plugins.entries` are read at
|
||||||
|
process start). That restart is the kb#170 handoff — see report.
|
||||||
|
|
||||||
|
## 5. Component 3 — sync with services (designed, not built)
|
||||||
|
|
||||||
|
kb#170's spec: "идеи из Todoist синхронизируются с Kanboard, календарём,
|
||||||
|
проектами." This is underspecified enough that building a concrete
|
||||||
|
bidirectional sync now would be guessing scope (direction? conflict
|
||||||
|
resolution? which Todoist state maps to which Kanboard column?) rather
|
||||||
|
than following it. **v1 proposal, one-way, human-gated — not built yet:**
|
||||||
|
|
||||||
|
- Todoist is the **source of truth for the idea itself** (text, labels,
|
||||||
|
done/not-done). Kanboard is the source of truth for **anything that
|
||||||
|
became real, tracked work**.
|
||||||
|
- Sync fires only from component 4's periodic review (§6), not on a
|
||||||
|
schedule or webhook: when Adolf proposes "this idea is ready to become
|
||||||
|
work" and the human agrees, Adolf creates **one** Kanboard task whose
|
||||||
|
description contains a `context ref` back to the Todoist task id (per
|
||||||
|
`DESIGN-a2a-agents.md` §2's "context travels by reference" rule — no
|
||||||
|
content duplication) and the Todoist task gets a `kanboard-<id>` label
|
||||||
|
and stays open until the Kanboard task closes.
|
||||||
|
- **No calendar sync is proposed in v1.** A Todoist due-date does not
|
||||||
|
imply a calendar event (most captured ideas won't have a real due
|
||||||
|
time), and the reverse (creating calendar entries from arbitrary idea
|
||||||
|
due-dates) risks cluttering Radicale with noise. Calendar involvement
|
||||||
|
belongs with Welfare #106's people/events reminders design, not
|
||||||
|
invented here.
|
||||||
|
- **No two-way Kanboard→Todoist sync.** Completing the Kanboard task does
|
||||||
|
not need to close the Todoist item automatically for v1 — a human
|
||||||
|
glancing at Todoist can see the `kanboard-<id>` label and close it
|
||||||
|
manually; automating that closure is a small, safe follow-up once the
|
||||||
|
one-way direction above is live and observed, not blocking v1.
|
||||||
|
|
||||||
|
This keeps sync a **consequence of the human-gated review** (§6), never
|
||||||
|
an autonomous background writer to three services at once — consistent
|
||||||
|
with `DESIGN-a2a-agents.md` §5's always-ask posture for anything crossing
|
||||||
|
a service boundary. Building this is out of scope for this pass; flagged
|
||||||
|
in the kb#170 report as a natural follow-up task once components 1/2 are
|
||||||
|
live and real capture data exists to review.
|
||||||
|
|
||||||
|
## 6. Component 4 — periodic review (designed, not built)
|
||||||
|
|
||||||
|
"Adolf предлагает, какие идеи созрели для превращения в задачи" — this is
|
||||||
|
explicitly Adolf proposing, not auto-converting; matches
|
||||||
|
`DESIGN-a2a-agents.md` §5's always-ask escalation policy exactly (a
|
||||||
|
decision task to alvis's inbox, not a silent action). Proposed shape,
|
||||||
|
**not implemented**:
|
||||||
|
|
||||||
|
- A low-priority proactive cadence job (same shape as Welfare #124's
|
||||||
|
cadence design, once that lands) that runs `todoist_list_tasks` scoped
|
||||||
|
to labels `decompose` or `area-uncertain`, drafts a short proposal per
|
||||||
|
candidate ("эта идея выглядит готовой к декомпозиции — завести задачу
|
||||||
|
в Kanboard?"), and sends it via Matrix — same impact/cost gate as
|
||||||
|
`DESIGN-proactive-prioritization.md` (kb#123) should apply here too,
|
||||||
|
once that gate exists, rather than inventing a second one.
|
||||||
|
- On accept: component 3's one-way sync (§5) fires for that one idea.
|
||||||
|
- On dismiss: the idea's `decompose`/`area-uncertain` label is cleared so
|
||||||
|
the same candidate doesn't re-surface every cadence run.
|
||||||
|
|
||||||
|
Not built now because it depends on #124 (cadence) and, for a well-
|
||||||
|
calibrated gate, #123 (impact/cost gate) — both cited as dependencies
|
||||||
|
rather than duplicated, same posture `DESIGN-proactive-prioritization.md`
|
||||||
|
itself takes toward its own siblings.
|
||||||
|
|
||||||
|
## 7. What's built vs. handed off
|
||||||
|
|
||||||
|
| Piece | State |
|
||||||
|
|---|---|
|
||||||
|
| `agap-mcp/src/classifier.js` + test | Built, proven live against bge-m3 |
|
||||||
|
| `agap-mcp/src/capture.js` + test | Built, proven with a stubbed Todoist client (no real writes) |
|
||||||
|
| `agap-mcp/src/server.js`: `todoist_capture_idea` MCP tool + `POST /capture-idea` | Built; **not live** — needs an agap-mcp rebuild/restart (already true of the whole Todoist tool surface per kb#170 orchestrator note) |
|
||||||
|
| `adolf/openclaw.json`, `openai/shared-mcp.json`, `openai/agent-registry.yaml` | Edited, `validate_capability_grants.py` passes clean at both layers |
|
||||||
|
| `openai/todoist-capture-plugin/` (`/idea` command) | Built, HTTP contract proven with a local harness; **not live** — needs the adolf container restarted (bind mount + `plugins.entries` already wired) |
|
||||||
|
| Component 3 (sync) | Designed (§5), not built — genuine scope decisions flagged, not guessed |
|
||||||
|
| Component 4 (periodic review) | Designed (§6), not built — depends on Welfare #123/#124 |
|
||||||
|
|
||||||
|
## 8. Open questions for alvis (not guessed)
|
||||||
|
|
||||||
|
1. **Todoist project structure**: keep the label-only v1 (§3.1), or
|
||||||
|
create dedicated Todoist projects per area? The latter is a real,
|
||||||
|
visible change to the account structure and needs explicit sign-off.
|
||||||
|
2. **Activation**: rebuilding agap-mcp (its own repo/compose,
|
||||||
|
`agap_git/agap-mcp/docker-compose.yml`, `build: .` — the `Dockerfile`
|
||||||
|
`COPY`s `src/` into the image, no bind mount, so a code change needs a
|
||||||
|
rebuild, not just a restart) and restarting the adolf container (its
|
||||||
|
compose is `agap_git/openai/docker-compose.yml`, to re-read
|
||||||
|
`openclaw.json` and pick up the new plugin bind mount) are the two
|
||||||
|
outward-facing steps this task deliberately did not take. Exact
|
||||||
|
commands, once approved:
|
||||||
|
```
|
||||||
|
cd /home/alvis/agap_git/agap-mcp && docker compose build && docker compose up -d
|
||||||
|
cd /home/alvis/agap_git/openai && docker compose up -d adolf
|
||||||
|
```
|
||||||
253
adolf/HINDSIGHT-MIGRATION.md
Normal file
253
adolf/HINDSIGHT-MIGRATION.md
Normal file
@@ -0,0 +1,253 @@
|
|||||||
|
# Adolf memory migration — Cognee → Hindsight
|
||||||
|
|
||||||
|
**Status:** Planned · **Date:** 2026-07-13 · **Owner:** alvis
|
||||||
|
|
||||||
|
This is the authoritative design + plan for replacing Adolf's long-term memory
|
||||||
|
subsystem (**Cognee**) with **Hindsight** (Vectorize, MIT, self-hosted). It
|
||||||
|
supersedes the Cognee-specific parts of `docs/ARCHITECTURE.md` and gates 4–5 of
|
||||||
|
`docs/SPIKE-FINDINGS.md` in the OpenClaw fork (`/home/alvis/adolf`).
|
||||||
|
|
||||||
|
Memory stays integrated into Adolf **exactly the two ways Cognee was** — as a
|
||||||
|
**tool** (MCP) and as **forced hooks** (an OpenClaw memory plugin) — so no
|
||||||
|
behaviour the user sees is lost; only the backend changes.
|
||||||
|
|
||||||
|
> Scope note: this document + the kanboard **Ready** tasks (H1–H5) are the
|
||||||
|
> migration. No live service, compose file, `openclaw.json`, or plugin code has
|
||||||
|
> been changed yet — those edits are the H-tasks.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Why migrate
|
||||||
|
|
||||||
|
Cognee works, but its Agap deployment carries three structural costs, all
|
||||||
|
documented in the (now-retired) kanboard cognee tasks:
|
||||||
|
|
||||||
|
- **Three bespoke services** to keep the memory stack alive: `cognee` (FastAPI +
|
||||||
|
embedded Kuzu graph + Qdrant vectors), `cognee-mcp` (a patched MCP→HTTP proxy,
|
||||||
|
local build overlay from kb#70), and `cognee-llm` (a stateless Kimi-CLI wrapper
|
||||||
|
that exists *only* to give Cognee an LLM on the flat subscription).
|
||||||
|
- **The cognify pipeline is fragile and expensive.** "Cognify" (turning raw
|
||||||
|
turns into a graph) is an LLM step. On the Kimi CLI it runs ~5–24 s per call
|
||||||
|
and drains the single-seat subscription quota (SPIKE gate 5 recommended
|
||||||
|
LiteLLM instead). The plugin's async "cognify sweep" also silently stalled
|
||||||
|
twice (kb#69) because of OpenClaw plugin-lifecycle edge cases, so freshly told
|
||||||
|
facts weren't retrievable cross-session until the sweep was re-armed.
|
||||||
|
- **Scoping is best-effort.** Under `ENABLE_BACKEND_ACCESS_CONTROL=False` all
|
||||||
|
datasets share one graph/vector backend, so per-chat isolation leaks (kb#59).
|
||||||
|
|
||||||
|
## 2. What Hindsight gives us
|
||||||
|
|
||||||
|
- **One container.** `ghcr.io/vectorize-io/hindsight:latest` — REST API on
|
||||||
|
**:8888**, web UI on **:9999**, built-in PostgreSQL (`pg0`, persisted under
|
||||||
|
`/home/hindsight/.pg0`). It owns its own vector + graph + temporal
|
||||||
|
representation internally (the "four-network" model), so **no external Qdrant
|
||||||
|
or Kuzu** is needed for memory.
|
||||||
|
- **Built-in MCP server** mounted at `/mcp` on the same port — ~30 tools
|
||||||
|
including `retain` / `recall` / `reflect`. This **removes the need for a
|
||||||
|
separate `cognee-mcp` proxy container entirely**.
|
||||||
|
- **Retain learns on its own.** `retain` runs Hindsight's extraction/reflection
|
||||||
|
pipeline internally (optionally `async`), so there is **no separate "cognify
|
||||||
|
sweep" to arm, throttle, or watch** — the whole class of kb#69 bugs disappears.
|
||||||
|
- **Memory banks** are first-class isolation units, scoped by URL path
|
||||||
|
(`/v1/{tenant}/banks/{bank_id}/…`), so per-chat / per-user scoping is real, not
|
||||||
|
best-effort.
|
||||||
|
- **Bring-your-own LLM/embeddings** — OpenAI-compatible, Anthropic, Ollama,
|
||||||
|
LMStudio, etc. We point it at the infra we already run (LiteLLM `:4000` and/or
|
||||||
|
Ollama), so we can **delete `cognee-llm`** rather than port it.
|
||||||
|
|
||||||
|
Net: **3 services → 1**, plus we drop the cognify-sweep machinery.
|
||||||
|
|
||||||
|
## 3. Target architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
Matrix ⇄ OpenClaw ("Adolf" gateway)
|
||||||
|
│ provider adolf-llm (Kimi wrapper, :8010) — unchanged
|
||||||
|
▼
|
||||||
|
adolf-llm ── kimi CLI (agent)
|
||||||
|
│
|
||||||
|
┌─────────┴───────────────── memory is a boundary concern ─────────────┐
|
||||||
|
│ │
|
||||||
|
│ (a) FORCED HOOKS — hindsight-memory OpenClaw plugin │
|
||||||
|
│ before_prompt_build → recall → inject as prependContext │
|
||||||
|
│ agent_end → retain (async:true) the turn │
|
||||||
|
│ │
|
||||||
|
│ (b) TOOL — Hindsight built-in MCP in openclaw.json mcp.servers │
|
||||||
|
│ http://hindsight:8888/mcp/adolf/ → retain/recall/reflect/… │
|
||||||
|
└───────────────────────────────┬───────────────────────────────────────┘
|
||||||
|
▼
|
||||||
|
hindsight (ONE container)
|
||||||
|
:8888 REST + /mcp · :9999 UI
|
||||||
|
built-in Postgres (pg0)
|
||||||
|
LLM → per-stage routing [resolved, §10]
|
||||||
|
embeddings → ollama bge-m3 (GPU) [resolved, §10]
|
||||||
|
```
|
||||||
|
|
||||||
|
Reused infra: **LiteLLM `:4000`** and/or **Ollama** for Hindsight's model calls.
|
||||||
|
**Retired:** `cognee`, `cognee-mcp`, `cognee-llm`, Qdrant-for-cognee, Kuzu, the
|
||||||
|
`cognee-openclaw-plugin`, and all `openclaw.json` cognee references.
|
||||||
|
|
||||||
|
### 3.1 Surface (a) — forced hooks (the `hindsight-memory` plugin)
|
||||||
|
|
||||||
|
A new OpenClaw memory plugin replacing `cognee-openclaw-plugin`, modelled on the
|
||||||
|
same Honcho touchpoints Cognee used, so the plugin shape is familiar:
|
||||||
|
|
||||||
|
| Cognee plugin (`cognee-memory`) | Hindsight plugin (`hindsight-memory`) |
|
||||||
|
|-----------------------------------------------------|---------------------------------------------------------------|
|
||||||
|
| `before_prompt_build` → LLM-free graph recall inject | `before_prompt_build` → `recall` → inject `prependContext` |
|
||||||
|
| `agent_end` → raw `/add` (no inline cognify) | `agent_end` → `retain` (`async:true`) the user+assistant turn |
|
||||||
|
| throttled **cognify sweep** (dirty-tracker, timers) | **removed** — retain does extraction/learning internally |
|
||||||
|
| `cognee_recall` registered tool | `hindsight_recall` (+ optional `hindsight_reflect`) tool |
|
||||||
|
|
||||||
|
- Activated via `plugins.entries.hindsight-memory` in `openclaw.json` with the
|
||||||
|
same hook grants Cognee needed: `hooks.allowConversationAccess: true` and
|
||||||
|
`allowPromptInjection: true` (external plugins must opt in).
|
||||||
|
- **Recall stays off the hot LLM path.** Hindsight `recall` is retrieval
|
||||||
|
(semantic + BM25 + graph + temporal) with evidence scoring — no generative
|
||||||
|
synthesis — so it's the direct analogue of Cognee's LLM-free `onlyContext`
|
||||||
|
recall. The LLM-backed synthesis path is `reflect`, exposed as a deliberate
|
||||||
|
tool, not run per-turn.
|
||||||
|
- **No freshness dial / sweep.** Retain with `async:true` returns fast and lets
|
||||||
|
Hindsight extract/consolidate in the background; there is no plugin-owned timer
|
||||||
|
to stall.
|
||||||
|
- Same "untrusted metadata" framing on the injected block; same cleaning of
|
||||||
|
OpenClaw's `Conversation info (untrusted metadata):` and the memory block out
|
||||||
|
of stored/queried text.
|
||||||
|
|
||||||
|
### 3.2 Surface (b) — tool (built-in MCP)
|
||||||
|
|
||||||
|
Replace the `cognee` entry in `openclaw.json` `mcp.servers` with:
|
||||||
|
|
||||||
|
```jsonc
|
||||||
|
mcp: {
|
||||||
|
servers: {
|
||||||
|
hindsight: {
|
||||||
|
type: "http",
|
||||||
|
url: "http://hindsight:8888/mcp/adolf/", // bank-scoped by URL path
|
||||||
|
},
|
||||||
|
// openclaw-tools, kanboard, marketplace — unchanged
|
||||||
|
},
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The single bank in the path (`adolf`, or a per-chat bank) selects isolation; the
|
||||||
|
built-in MCP then exposes `retain`, `recall`, `reflect`, plus mental-model /
|
||||||
|
directive / memory-browse tools. This replaces cognee-mcp's `remember` / `recall`
|
||||||
|
/ `forget` — and gives deliberate delete via the memory-management tools instead
|
||||||
|
of the hand-patched `forget(data_id)` from kb#70.
|
||||||
|
|
||||||
|
## 4. REST / MCP API mapping
|
||||||
|
|
||||||
|
Base path: `http://hindsight:8888/v1/default` (tenant `default`). Bank id is a
|
||||||
|
**path** parameter.
|
||||||
|
|
||||||
|
| Operation | Cognee (old) | Hindsight (new) |
|
||||||
|
|------------------|------------------------------------------------|-----------------------------------------------------------------------|
|
||||||
|
| store a turn | `POST /api/v1/add` (+ later `/cognify`) | `POST /banks/{bank}/memories` body `{items:[{content,context,tags,timestamp}], async:true}` |
|
||||||
|
| recall (no LLM) | `POST /api/v1/search` `GRAPH_COMPLETION` `onlyContext:true` | `POST /banks/{bank}/memories/recall` body `{query, budget, max_tokens, tags}` |
|
||||||
|
| deep answer (LLM)| cognee-mcp `recall` (GRAPH_COMPLETION) | `POST /banks/{bank}/reflect` body `{query, budget, max_tokens, response_schema?}` |
|
||||||
|
| delete an entry | patched `forget(dataset, data_id)` (kb#70) | memory-management endpoints / MCP (`delete`, `clear_memories`) |
|
||||||
|
| list / inspect | dataset status polling | `GET /banks/{bank}/memories/list`, `GET /banks` |
|
||||||
|
| create bank | dataset created implicitly on add | `PUT /banks/{bank}` |
|
||||||
|
|
||||||
|
Built-in MCP tools live at `http://hindsight:8888/mcp/{bank}/` (HTTP transport;
|
||||||
|
bank via URL path, `X-Bank-Id` header, or `HINDSIGHT_MCP_BANK_ID` default).
|
||||||
|
|
||||||
|
> Exact request-body field names and any auth headers must be confirmed against
|
||||||
|
> the running instance's OpenAPI (`http://localhost:8888/docs`) and the Hindsight
|
||||||
|
> configuration docs during H1 — treat the bodies above as the shape, not gospel.
|
||||||
|
|
||||||
|
## 5. Bank scoping
|
||||||
|
|
||||||
|
Mirror Cognee's per-conversation `chat_<chatId>` dataset with a per-conversation
|
||||||
|
**bank**: derive `bank_id` from OpenClaw's `chat_id` (the
|
||||||
|
`Conversation info (untrusted metadata):` block; see SPIKE gate 2), sanitized to
|
||||||
|
`chat_<slug>`. A single shared `adolf` bank is the simpler alternative if
|
||||||
|
cross-chat recall is actually wanted — decide in H3. Banks are hard isolation in
|
||||||
|
Hindsight, so per-chat is now safe (unlike Cognee's leaky datasets).
|
||||||
|
|
||||||
|
## 6. Open decisions (resolve in H1)
|
||||||
|
|
||||||
|
1. **LLM backend for retain/reflect.** SPIKE gate 5 already concluded the
|
||||||
|
extraction workload should *not* sit on the Kimi CLI (latency + single-seat
|
||||||
|
quota). Recommendation: point Hindsight's LLM at **LiteLLM `:4000`** (or a
|
||||||
|
local **Ollama** model for zero marginal cost). This is why `cognee-llm` is
|
||||||
|
deleted, not ported. Confirm Hindsight's provider env-var names on the image.
|
||||||
|
2. **Embeddings.** Prefer the local **Ollama** embedder already available
|
||||||
|
(`nomic-embed` / `bge-m3` at `host.docker.internal:11436`) or Hindsight's
|
||||||
|
built-in, to keep embeddings off any paid path.
|
||||||
|
3. **Storage path.** Persist `pg0` under `/mnt/ssd/dbs/hindsight/` to match the
|
||||||
|
Agap storage layout (replaces `/mnt/ssd/dbs/cognee/`).
|
||||||
|
4. **UI exposure.** Whether to reverse-proxy the `:9999` UI (Caddy) or keep it
|
||||||
|
internal-only.
|
||||||
|
5. **Auth.** Open by default; enable the tenant API-key extension
|
||||||
|
(`HINDSIGHT_API_TENANT_API_KEY`, `Authorization: Bearer`) if the service is
|
||||||
|
reachable beyond the compose network.
|
||||||
|
|
||||||
|
## 7. Data migration
|
||||||
|
|
||||||
|
Cognee's Kuzu graph is **not** portable into Hindsight's store. The memory corpus
|
||||||
|
is low-value conversational history, so **start Hindsight empty** rather than
|
||||||
|
building an exporter. Optionally replay a handful of durable facts by calling
|
||||||
|
`retain` once at cutover. The two throwaway datasets left in Cognee
|
||||||
|
(`chat_verify`, `chat_webchat`) are discarded with the stack.
|
||||||
|
|
||||||
|
## 8. Migration phases (kanboard **Ready**, project *Adolf*)
|
||||||
|
|
||||||
|
- **H1 · Deploy Hindsight service** — add the `hindsight` container to
|
||||||
|
`openai/docker-compose.yml` (image, ports 8888/9999, `pg0` volume, LLM +
|
||||||
|
embedding provider env → LiteLLM/Ollama), bring it up, confirm `/docs` + a
|
||||||
|
round-trip `retain`→`recall`. Resolves §6 decisions.
|
||||||
|
- **H2 · Wire built-in MCP as an Adolf tool** — swap `mcp.servers.cognee` →
|
||||||
|
`mcp.servers.hindsight` (`/mcp/{bank}/`) in `openclaw.json`; verify Adolf can
|
||||||
|
call `retain`/`recall`/`reflect` as tools.
|
||||||
|
- **H3 · `hindsight-memory` OpenClaw plugin (forced hooks)** — build the plugin
|
||||||
|
replacing `cognee-openclaw-plugin`: `before_prompt_build`→recall inject,
|
||||||
|
`agent_end`→`retain(async)`, `hindsight_recall`/`hindsight_reflect` tools, bank
|
||||||
|
scoping; activate in `openclaw.json`. Delete the cognify-sweep machinery.
|
||||||
|
- **H4 · Decommission Cognee** — remove `cognee`, `cognee-mcp`, `cognee-llm`
|
||||||
|
services + volumes, the `cognee-openclaw-plugin`, and all `openclaw.json`/
|
||||||
|
`shared-mcp.json` cognee references. Free `/mnt/ssd/dbs/cognee`.
|
||||||
|
- **H5 · End-to-end verification** — state a fact → fresh session → recalled via
|
||||||
|
injected memory (no LLM on the recall path); measure recall latency; confirm
|
||||||
|
per-bank isolation; confirm no sweep/timer exists to stall.
|
||||||
|
|
||||||
|
## 9. What is unchanged
|
||||||
|
|
||||||
|
The Kimi/OpenClaw/Matrix substrate is untouched: `adolf` gateway, `adolf-llm`
|
||||||
|
(:8010) provider, the SSE-heartbeat/idle-watchdog fix (kb#71), the
|
||||||
|
`openclaw-tools` bridge, `kanboard`/`marketplace` MCP servers, Matrix allow-list
|
||||||
|
and E2EE. Only the memory backend and its two integration surfaces change.
|
||||||
|
|
||||||
|
## 10. LLM provider strategy — per-stage routing (kb#84, kb#88)
|
||||||
|
|
||||||
|
Hindsight calls an LLM in several distinct stages, and they have very different
|
||||||
|
cost/quality profiles. Hindsight supports a **separate provider per stage**
|
||||||
|
(`HINDSIGHT_API_<STAGE>_LLM_{PROVIDER,BASE_URL,MODEL,API_KEY}`, falling back to
|
||||||
|
the global `HINDSIGHT_API_LLM_*` when unset), so each is routed to the model that
|
||||||
|
fits it. Config lives in `openai/docker-compose.yml`, `hindsight` service.
|
||||||
|
|
||||||
|
| Stage | Model | Where | Why this model |
|
||||||
|
|-------|-------|-------|----------------|
|
||||||
|
| **RETAIN / extraction** | Kimi — `hindsight-llm:8012` | flat-rate subscription | Pulls atomic facts + entities out of raw **Russian** conversation. Quality-critical and user-visible (bad extraction ⇒ bad memory), and **low volume** — roughly one pass per turn. Worth the strong model. |
|
||||||
|
| **CONSOLIDATION** | `ollama/gemma3:4b` via LiteLLM `:4000` | local GPU (free) | Merges / dedups / reconciles stored memories. **Very high volume** — a background reconcile loop + per-retain triggers: ~900 calls / 3 h at ~190 memories. Mostly mechanical; a small model is good enough. |
|
||||||
|
| **REFLECT / mental-models** | `ollama/gemma3:4b` via LiteLLM `:4000` | local GPU (free) | Periodic synthesis over the bank and the `reflect` tool. Background, frequent (~170 calls / 3 h), never on the reply path. |
|
||||||
|
| Embeddings | `bge-m3` (ollama, GPU) | local | multilingual, GPU-served — see §3 / kb#84. |
|
||||||
|
| Reranker | `jina-reranker-v2-base-multilingual` | local (CPU) | multilingual; kb#84. |
|
||||||
|
|
||||||
|
**Why the split matters — the kb#88 incident.** Originally (H1b) the *entire*
|
||||||
|
Hindsight LLM was pointed at Kimi via `HINDSIGHT_API_LLM_PROVIDER` alone. The
|
||||||
|
background consolidation + mental-model jobs then hammered Kimi ~1100 calls / 3 h
|
||||||
|
and **maxed the Kimi 5-hour rate window (100 %)** with *no* chat traffic at all —
|
||||||
|
because Kimi is a **rate-limited flat subscription**, not a per-token API, and
|
||||||
|
these jobs run continuously regardless of user activity. Moving the
|
||||||
|
high-volume / low-stakes stages to a free local GPU model dropped Kimi 5 h usage
|
||||||
|
100 % → ~38 % while keeping fact extraction on the good model.
|
||||||
|
|
||||||
|
**Rule of thumb:** *frequent, background, mechanical* stages (consolidation,
|
||||||
|
reflect, mental-models) → **cheap local model**; *user-facing, quality-critical,
|
||||||
|
low-volume* work (extraction, and the assistant's own replies) → **Kimi**.
|
||||||
|
|
||||||
|
Levers if the local box gets loaded or quality is off:
|
||||||
|
`HINDSIGHT_API_CONSOLIDATION_RECONCILE_INTERVAL_SECONDS` (raise to run less
|
||||||
|
often), `_MAX_MEMORIES_PER_ROUND` (lower). If `gemma3:4b`'s Russian consolidation
|
||||||
|
quality is too weak, `qwen3.5:9b` (same ollama box, `:11436`) is the next step up.
|
||||||
150
adolf/README.md
Normal file
150
adolf/README.md
Normal file
@@ -0,0 +1,150 @@
|
|||||||
|
# Adolf — OpenClaw gateway deployment
|
||||||
|
|
||||||
|
Adolf is the self-hosted OpenClaw fork that runs as the `adolf` container
|
||||||
|
(Matrix-first personal assistant). This directory holds its **version-controlled
|
||||||
|
gateway configuration**.
|
||||||
|
|
||||||
|
- Source tree (the OpenClaw fork being built): `/home/alvis/adolf`
|
||||||
|
- Compose service `adolf` lives in: `openai/docker-compose.yml`
|
||||||
|
- This config directory lives at the repo root (`agap_git/adolf/`), not
|
||||||
|
nested inside `openai/`, since it is shared config rather than part of
|
||||||
|
the `openai` compose project's own tree.
|
||||||
|
- Model backend: `adolf-llm` container (Kimi-CLI wrapper) on `:8010`
|
||||||
|
|
||||||
|
## Memory
|
||||||
|
|
||||||
|
Adolf's long-term memory is being **migrated from Cognee to Hindsight** (a single
|
||||||
|
self-hosted container, `:8888` REST + built-in MCP, `:9999` UI). It stays wired in
|
||||||
|
the same two ways: as a **tool** (Hindsight's built-in MCP in `openclaw.json`
|
||||||
|
`mcp.servers.hindsight`) and as **forced hooks** (the `hindsight-memory` OpenClaw
|
||||||
|
plugin: `before_prompt_build`→recall inject, `agent_end`→retain). Authoritative
|
||||||
|
plan and target architecture: **[`HINDSIGHT-MIGRATION.md`](./HINDSIGHT-MIGRATION.md)**
|
||||||
|
(kanboard *Adolf* tasks H1–H5). Until those land, the running stack is still Cognee.
|
||||||
|
|
||||||
|
## Config source of truth
|
||||||
|
|
||||||
|
The gateway config is **`openclaw.json` in this directory**. It is bind-mounted
|
||||||
|
**read-only** over the `adolf-state` volume:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
volumes:
|
||||||
|
- adolf-state:/home/node/.openclaw # runtime state only
|
||||||
|
- ../adolf/openclaw.json:/home/node/.openclaw/openclaw.json:ro # tracked config
|
||||||
|
```
|
||||||
|
|
||||||
|
Previously this file was a hand-edited copy inside the `adolf-state` Docker
|
||||||
|
volume (edited via `docker cp` into the running container). It is now tracked
|
||||||
|
in git and seeded into the container by the mount, so **git is the single
|
||||||
|
source of truth**.
|
||||||
|
|
||||||
|
- The file is **JSONC** (comments + unquoted keys allowed).
|
||||||
|
- **No secrets live here.** Every credential is a `${VAR}` reference resolved
|
||||||
|
from the container's environment, which is sourced from `openai/.env`
|
||||||
|
(gitignored, never committed): `OPENCLAW_GATEWAY_TOKEN`, `ADOLF_KEY`,
|
||||||
|
`MATRIX_*`, `MARKETPLACE_MCP_TOKEN`.
|
||||||
|
- The gateway reads this file and writes its own `openclaw.json.last-good`
|
||||||
|
and `openclaw.json.rejected.*` snapshots into the volume dir (writable). It
|
||||||
|
does **not** rewrite this file, so the read-only mount is safe.
|
||||||
|
|
||||||
|
### Changing the config
|
||||||
|
|
||||||
|
1. Edit `agap_git/adolf/openclaw.json`.
|
||||||
|
2. Restart the container:
|
||||||
|
```bash
|
||||||
|
env -u HTTPS_PROXY -u HTTP_PROXY -u ALL_PROXY -u https_proxy -u http_proxy -u all_proxy \
|
||||||
|
docker compose -f /home/alvis/agap_git/openai/docker-compose.yml up -d adolf
|
||||||
|
```
|
||||||
|
3. Verify it came up healthy and the config was accepted:
|
||||||
|
```bash
|
||||||
|
env -u HTTPS_PROXY -u HTTP_PROXY -u ALL_PROXY -u https_proxy -u http_proxy -u all_proxy \
|
||||||
|
docker ps --filter name=adolf --format '{{.Names}} {{.Status}}'
|
||||||
|
env -u HTTPS_PROXY -u HTTP_PROXY -u ALL_PROXY -u https_proxy -u http_proxy -u all_proxy \
|
||||||
|
docker logs --tail 40 adolf
|
||||||
|
```
|
||||||
|
A fresh `openclaw.json.rejected.*` file in `/home/node/.openclaw` means the
|
||||||
|
edit failed validation and the previous `.last-good` is still in use.
|
||||||
|
|
||||||
|
Because the mount is read-only, editing config through the gateway UI/API is
|
||||||
|
intentionally disabled — all changes go through git.
|
||||||
|
|
||||||
|
## Granting a Matrix user access to Adolf
|
||||||
|
|
||||||
|
Adolf only responds to Matrix users on an **allow-list**. This is the
|
||||||
|
`channels.matrix.dm` block in `openclaw.json`:
|
||||||
|
|
||||||
|
```jsonc
|
||||||
|
channels: {
|
||||||
|
matrix: {
|
||||||
|
enabled: true,
|
||||||
|
encryption: true,
|
||||||
|
dm: {
|
||||||
|
policy: "allowlist",
|
||||||
|
allowFrom: [
|
||||||
|
"@admin:mtx.alogins.net",
|
||||||
|
"@elizaveta:mtx.alogins.net",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
groupPolicy: "disabled", // no group-room handling yet
|
||||||
|
autoJoin: "always",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- **`dm.policy: "allowlist"`** — only users whose full Matrix ID appears in
|
||||||
|
`allowFrom` can DM the bot. Everyone else is ignored.
|
||||||
|
(`policy: "pairing"` is the alternative: the owner must approve each unknown
|
||||||
|
sender interactively. `allowlist` is stricter and declarative.)
|
||||||
|
- **`dm.allowFrom`** — the list of authorized Matrix user IDs.
|
||||||
|
- **`groupPolicy: "disabled"`** — Adolf does not act in group rooms; DM only.
|
||||||
|
|
||||||
|
### To grant a new user access
|
||||||
|
|
||||||
|
1. Get the user's **full Matrix ID**, e.g. `@ivan:mtx.alogins.net`.
|
||||||
|
2. Add it to `allowFrom` in `agap_git/adolf/openclaw.json`:
|
||||||
|
```jsonc
|
||||||
|
allowFrom: [
|
||||||
|
"@admin:mtx.alogins.net",
|
||||||
|
"@elizaveta:mtx.alogins.net",
|
||||||
|
"@ivan:mtx.alogins.net",
|
||||||
|
],
|
||||||
|
```
|
||||||
|
3. Restart adolf (see "Changing the config" above).
|
||||||
|
4. The user can now start a DM with Adolf's Matrix account
|
||||||
|
(`MATRIX_USER_ID` in `openai/.env`). `autoJoin: "always"` means Adolf
|
||||||
|
auto-accepts the DM invite; conversation is end-to-end encrypted
|
||||||
|
(`encryption: true`).
|
||||||
|
|
||||||
|
To **revoke** access, remove the ID from `allowFrom` and restart.
|
||||||
|
|
||||||
|
> Matrix accounts themselves are created on the Synapse homeserver
|
||||||
|
> (`mtx.alogins.net`) — see the AgapHost wiki **Matrix** page. The allow-list
|
||||||
|
> here only controls which existing Matrix users Adolf will talk to.
|
||||||
|
|
||||||
|
## Matrix device identity (kb#67)
|
||||||
|
|
||||||
|
Adolf's Matrix login used **password auth with no pinned `device_id`**
|
||||||
|
(`MATRIX_PASSWORD` in `openai/.env`). Every time OpenClaw's own credential
|
||||||
|
cache (in the `adolf-state` volume) was missing — first boot, a lost/rebuilt
|
||||||
|
volume — a fresh password login minted a **brand-new Matrix device** with no
|
||||||
|
cross-signing, leaving dead ghost devices behind and risking new encrypted
|
||||||
|
DMs getting keys shared to a device that no longer exists.
|
||||||
|
|
||||||
|
Fix: `openai/.env` now also pins `MATRIX_ACCESS_TOKEN` + `MATRIX_DEVICE_ID` to
|
||||||
|
Adolf's current live device (`TIANDTKUZJ`, cross-signed; token in Vaultwarden
|
||||||
|
as `MATRIX_ADOLF_GATEWAY_TOKEN`). OpenClaw's matrix extension prefers a
|
||||||
|
configured access token over password login
|
||||||
|
(`extensions/matrix/src/matrix/client/config.ts` `resolveMatrixAuth`), so as
|
||||||
|
long as that token stays valid, restarts — even after a volume loss — reuse
|
||||||
|
the same device instead of minting a new one. `MATRIX_PASSWORD` stays set as
|
||||||
|
a manual-recovery fallback only (unset `MATRIX_ACCESS_TOKEN` to force a fresh
|
||||||
|
password login if the token is ever revoked).
|
||||||
|
|
||||||
|
Cross-signing for `@bot` is already bootstrapped automatically by OpenClaw's
|
||||||
|
matrix extension (`extensions/matrix/src/matrix/sdk/crypto-bootstrap.ts`) —
|
||||||
|
no separate setup needed.
|
||||||
|
|
||||||
|
If the pinned token is ever revoked/rotated, get a fresh one bound to the
|
||||||
|
*same* device by logging in with `device_id` explicitly set to `TIANDTKUZJ`
|
||||||
|
(Synapse reuses an existing device when its id is given in `/login`, instead
|
||||||
|
of creating a new one), then update `MATRIX_ACCESS_TOKEN` in `openai/.env`
|
||||||
|
and Vaultwarden's `MATRIX_ADOLF_GATEWAY_TOKEN`.
|
||||||
393
adolf/openclaw.json
Normal file
393
adolf/openclaw.json
Normal file
@@ -0,0 +1,393 @@
|
|||||||
|
{
|
||||||
|
// Adolf P6 — OpenClaw gateway config for the "adolf" container.
|
||||||
|
// Lives in the adolf-state VOLUME (mounted at /home/node/.openclaw), not
|
||||||
|
// in the openai/ git repo. Secrets referenced below (${VAR}) are resolved
|
||||||
|
// from this container's process env, itself sourced from openai/.env
|
||||||
|
// (gitignored) via docker-compose.yml — never inlined here.
|
||||||
|
|
||||||
|
gateway: {
|
||||||
|
mode: "local",
|
||||||
|
auth: {
|
||||||
|
// Compose already binds "lan" (0.0.0.0) and publishes 18789/18790 to
|
||||||
|
// the host, so this is a non-loopback bind and auth is mandatory.
|
||||||
|
// Shared-secret token auth also gives the openclaw-tools bridge
|
||||||
|
// (which calls POST /tools/invoke with the same token) full
|
||||||
|
// trusted-operator scope, which is what lets gateway.tools.allow
|
||||||
|
// below actually unlock cron/nodes for it.
|
||||||
|
mode: "token",
|
||||||
|
token: "${OPENCLAW_GATEWAY_TOKEN}",
|
||||||
|
},
|
||||||
|
tools: {
|
||||||
|
// cron and nodes are owner-only and hard-denied on the HTTP
|
||||||
|
// /tools/invoke surface by default. The openclaw-tools MCP bridge
|
||||||
|
// (P5) calls that surface for cron_create/cron_list/nodes_invoke, so
|
||||||
|
// without this allow-list those tools 404 even with a valid token.
|
||||||
|
// "browser" added (kb#64 follow-up) so Adolf can drive the OpenClaw
|
||||||
|
// browser via the openclaw-tools bridge for authenticated web access
|
||||||
|
// (e.g. the family wiki login form).
|
||||||
|
allow: ["cron", "nodes", "browser"],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
|
||||||
|
// Audio understanding (kb#175, STT source 2: "local STT on Agap").
|
||||||
|
// OpenClaw's bundled media-understanding pipeline (docs/nodes/media-
|
||||||
|
// understanding.md) auto-transcribes inbound audio attachments before the
|
||||||
|
// agent turn runs -- no plugin code needed, this is pure config. A voice
|
||||||
|
// note sent to Adolf over the existing Matrix DM (source 1, see below)
|
||||||
|
// gets transcribed by the `openai`-shaped entry below, which is redirected
|
||||||
|
// via baseUrl/apiKey to the LOCAL faster-whisper server (openai/docker-
|
||||||
|
// compose.yml's `faster-whisper` service, same compose project as this
|
||||||
|
// container, reachable by service name) instead of hosted OpenAI --
|
||||||
|
// confirmed supported via src/media-understanding/runner.entries.ts's
|
||||||
|
// per-entry baseUrl override (docs/gateway/config-tools.md "Tools and
|
||||||
|
// custom providers"). apiKey is a dummy: faster-whisper-server has no
|
||||||
|
// auth, but OpenClaw's schema requires a non-empty value.
|
||||||
|
//
|
||||||
|
// ⚠️ NOT ACTIVATED YET (2026-07-26): the `faster-whisper` container does
|
||||||
|
// not exist (never started -- `docker ps -a` shows no such container).
|
||||||
|
// Starting it opens a 4th tenant on the single 8GB GTX 1070 already at
|
||||||
|
// ~6.2GB with bge-m3 + gemma3:4b + tei-reranker (~1.8GB headroom) and
|
||||||
|
// could evict tei-reranker, silently breaking Hindsight recall (see
|
||||||
|
// DESIGN-a2a-agents.md §3b and kb#191, which is the unimplemented
|
||||||
|
// residency-guard/VRAM-alert task -- still in Backlog). This config
|
||||||
|
// block is deliberately inert until #191 lands or alvis explicitly
|
||||||
|
// accepts the co-residency risk; `enabled: false` below.
|
||||||
|
// SCHEMA FIX 2026-07-30 (kb#180 restart). This block previously carried a
|
||||||
|
// per-entry `apiKey: "not-needed"`, which the schema REJECTS:
|
||||||
|
// tools.media.audio.models.0: Invalid input
|
||||||
|
// and an invalid config makes the gateway refuse to start outright -- adolf
|
||||||
|
// crash-looped on the first restart after the block was added. Two lessons
|
||||||
|
// encoded here: `enabled: false` does NOT exempt an entry from schema
|
||||||
|
// validation, and a config edit is not "done" until a real restart boots
|
||||||
|
// healthy (this block sat invalid but latent because the running gateway
|
||||||
|
// still held an older loaded config).
|
||||||
|
//
|
||||||
|
// The old comment claimed the schema "requires a non-empty apiKey" -- it is
|
||||||
|
// the opposite: `apiKey` is not a valid per-entry key at all. Verified with
|
||||||
|
// `openclaw config validate` against this image (2026.6.11): entries with
|
||||||
|
// {provider, model} and {provider, model, baseUrl} validate; adding
|
||||||
|
// `apiKey` is the sole cause of the failure. Per docs/nodes/audio.md,
|
||||||
|
// provider auth follows the normal model auth order (auth profiles, env
|
||||||
|
// vars, models.providers.*.apiKey) -- and faster-whisper-server has no auth
|
||||||
|
// to satisfy anyway, so no key belongs here. `baseUrl` is kept: that is the
|
||||||
|
// per-entry override that redirects the `openai`-shaped provider to the
|
||||||
|
// LOCAL faster-whisper server.
|
||||||
|
tools: {
|
||||||
|
media: {
|
||||||
|
audio: {
|
||||||
|
// Still deliberately inert: the `faster-whisper` container does not
|
||||||
|
// exist, and starting it opens a 4th tenant on the 8GB GTX 1070 (see
|
||||||
|
// the kb#191 residency-guard note above). This block is now merely
|
||||||
|
// SCHEMA-VALID rather than boot-breaking; flipping this to true is a
|
||||||
|
// separate decision that still belongs to kb#175/#191.
|
||||||
|
enabled: false,
|
||||||
|
echoTranscript: true, // let the sender see what Adolf heard before it acts
|
||||||
|
echoFormat: '📝 "{transcript}"',
|
||||||
|
models: [
|
||||||
|
{
|
||||||
|
provider: "openai",
|
||||||
|
model: "deepdml/faster-whisper-large-v3-turbo-ct2", // must match WHISPER__MODEL in openai/docker-compose.yml
|
||||||
|
baseUrl: "http://faster-whisper:8000/v1",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
|
||||||
|
// Browser tool — bundled plugin, off by default. Enables a dedicated,
|
||||||
|
// agent-only headless Chromium profile ("openclaw") driven through the
|
||||||
|
// gateway's loopback control service. Chromium is already in the image
|
||||||
|
// (playwright chromium-1228). Needs both this browser.enabled=true and the
|
||||||
|
// "browser" entry in gateway.tools.allow above.
|
||||||
|
browser: {
|
||||||
|
enabled: true,
|
||||||
|
// Chromium's setuid sandbox can't initialize inside this container (no
|
||||||
|
// unprivileged user namespaces), so run with --no-sandbox. Safe here: the
|
||||||
|
// browser profile is agent-only and isolated, and the container already
|
||||||
|
// drops NET_RAW/NET_ADMIN. Without this, `browser start` fails with
|
||||||
|
// "No usable sandbox".
|
||||||
|
noSandbox: true,
|
||||||
|
// The local *.alogins.net services resolve to the host gateway (private
|
||||||
|
// 172.17.0.1 via extra_hosts), so the browser's SSRF guard blocks them by
|
||||||
|
// default ("navigation blocked by policy"). Opt in for this trusted,
|
||||||
|
// self-owned network — same decision as channels.matrix.network above.
|
||||||
|
ssrfPolicy: {
|
||||||
|
dangerouslyAllowPrivateNetwork: true,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
|
||||||
|
// Model provider: adolf-llm (P2/P4), the Kimi-CLI OpenAI-compatible
|
||||||
|
// wrapper on :8010. Its HTTP server (openai/adolf-llm/server.js) performs
|
||||||
|
// NO api-key/Authorization validation at all -- ADOLF_KEY's value is
|
||||||
|
// functionally irrelevant to adolf-llm itself. It's still wired through
|
||||||
|
// env (not hardcoded) because OpenClaw's custom-provider schema requires
|
||||||
|
// a non-empty apiKey field and ${VAR} substitution fails closed on an
|
||||||
|
// empty/missing var.
|
||||||
|
models: {
|
||||||
|
mode: "merge",
|
||||||
|
providers: {
|
||||||
|
"adolf-llm": {
|
||||||
|
baseUrl: "http://adolf-llm:8010/v1",
|
||||||
|
apiKey: "${ADOLF_KEY}",
|
||||||
|
api: "openai-completions",
|
||||||
|
// Margin above the server.js SSE heartbeat cadence (empty-content
|
||||||
|
// keepalive delta every ~25s once idle) so the idle watchdog never
|
||||||
|
// fires on long thinking/tool/MCP phases even if a heartbeat tick
|
||||||
|
// is delayed (kb #71). Raised to 10min: long agentic turns (Cognee
|
||||||
|
// tool-loops / recalls up to 150s each) were producing no *content*
|
||||||
|
// progress for >300s, tripping "no response from model" and surfacing
|
||||||
|
// an error before the agent finished. The wrapper now also kills the
|
||||||
|
// kimi child on disconnect, so an over-timeout turn no longer orphans.
|
||||||
|
timeoutSeconds: 600,
|
||||||
|
models: [
|
||||||
|
{ id: "adolf", name: "Adolf", input: ["text", "image"] },
|
||||||
|
],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
|
||||||
|
agents: {
|
||||||
|
defaults: {
|
||||||
|
model: "adolf-llm/adolf",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
|
||||||
|
ui: {
|
||||||
|
assistant: { name: "Adolf" },
|
||||||
|
},
|
||||||
|
|
||||||
|
// Matrix channel (P6). Deliberately no accessToken/userId/password here:
|
||||||
|
// MATRIX_HOMESERVER / MATRIX_USER_ID / MATRIX_PASSWORD / MATRIX_DEVICE_NAME
|
||||||
|
// are config-key-backed env vars OpenClaw reads automatically when the
|
||||||
|
// matching config key is unset, so real creds never touch this file or
|
||||||
|
// git. Password auth (not the raw access token) mints Adolf its own fresh
|
||||||
|
// Matrix device instead of reusing the existing matrixbot (Python/nio)
|
||||||
|
// device -- see the P6 report for why that separation matters.
|
||||||
|
//
|
||||||
|
// Conservative defaults on purpose: dm "pairing" (owner must approve
|
||||||
|
// unknown senders) and groupPolicy "disabled" (no room handling yet).
|
||||||
|
// Revisit once the legacy matrixbot adapter is retired.
|
||||||
|
channels: {
|
||||||
|
matrix: {
|
||||||
|
enabled: true,
|
||||||
|
encryption: true,
|
||||||
|
dm: { policy: "allowlist", allowFrom: ["@admin:mtx.alogins.net", "@elizaveta:mtx.alogins.net"] },
|
||||||
|
groupPolicy: "disabled",
|
||||||
|
autoJoin: "always",
|
||||||
|
// mtx.alogins.net is deliberately mapped to the host-gateway (private)
|
||||||
|
// IP via extra_hosts in docker-compose.yml to dodge a hairpin-NAT dead
|
||||||
|
// end on the public route -- not an actual SSRF exposure, so opt out
|
||||||
|
// of the private-network block for this trusted, self-owned homeserver.
|
||||||
|
network: { dangerouslyAllowPrivateNetwork: true },
|
||||||
|
},
|
||||||
|
},
|
||||||
|
|
||||||
|
// MCP registry (P6) -- same servers as openai/shared-mcp.json,
|
||||||
|
// expressed in OpenClaw's own mcp.servers schema. `type: "http"` is
|
||||||
|
// OpenClaw's documented CLI-native alias for transport: "streamable-http".
|
||||||
|
mcp: {
|
||||||
|
servers: {
|
||||||
|
// kb#144 (A2A-12): toolFilter.include is OpenClaw's OWN per-server
|
||||||
|
// tool-scoping mechanism (mcp.servers.*.toolFilter, zod-validated;
|
||||||
|
// applied in agent-bundle-mcp-materialize.js when THIS container's own
|
||||||
|
// tool bundle is built). CORRECTION (kb#144 second pass, 2026-07-22):
|
||||||
|
// this does NOT reach the model on Adolf's kimi backbone -- Kimi CLI
|
||||||
|
// (inside the separate adolf-llm container) reads its own
|
||||||
|
// project-root .mcp.json, seeded from openai/shared-mcp.json, and
|
||||||
|
// applies ITS OWN enabledTools/disabledTools (McpServerCommonFields,
|
||||||
|
// computeEnabledNames). Live wire.jsonl verification (restart + one
|
||||||
|
// real turn) proved OpenClaw's toolFilter alone left Kimi's actual
|
||||||
|
// tool counts unchanged. This block is still correct for OpenClaw's
|
||||||
|
// own MCP client surface -- see openai/shared-mcp.json for the layer
|
||||||
|
// that actually scopes what the model sees.
|
||||||
|
//
|
||||||
|
// Scoped to Adolf's CORE memory ops: recall/retain/reflect (the
|
||||||
|
// hooks below do IMPLICIT recall/retain automatically; these MCP
|
||||||
|
// tools cover explicit "remember this" / "what do you recall about
|
||||||
|
// X" turns) plus single-memory read/update and directive CRUD
|
||||||
|
// (standing instructions). Cuts 20 rarely-used/admin tools: the
|
||||||
|
// mental-model CRUD (7), document CRUD (3), operation-tracking (3),
|
||||||
|
// tags (1), bank admin (4), sync_retain, invalidate_memory -- none
|
||||||
|
// of which Adolf's Matrix persona drives turn-to-turn; reach the
|
||||||
|
// hindsight MCP directly (unscoped) for that admin work instead of
|
||||||
|
// paying for it on every Adolf turn. 29 tools -> 9.
|
||||||
|
//
|
||||||
|
// kb#169: this raw MCP surface previously pointed at /mcp/adolf/ --
|
||||||
|
// the single unpartitioned bank with content from every human's
|
||||||
|
// conversations. #153 scopes the hindsight-memory PLUGIN's
|
||||||
|
// recall/retain hooks by interlocutor, but this MCP tool surface is a
|
||||||
|
// second, independent path to memory that #153 does not touch: a
|
||||||
|
// model call to e.g. `recall` here bypassed interlocutor scoping
|
||||||
|
// entirely. Repointed to /mcp/adolf-shared/ (option 2 of #169) --
|
||||||
|
// the household-shared bank (0 facts as of 2026-07-26, pre-existing
|
||||||
|
// per agent-registry.yaml's memory.banks target list). This surface
|
||||||
|
// can now only ever read/write the shared bank, never a private one,
|
||||||
|
// regardless of which human is talking to Adolf -- safe by
|
||||||
|
// construction, no dependency on #153 landing first. Option 1 (drop
|
||||||
|
// entirely) was not chosen because, until #153's scoped plugin tools
|
||||||
|
// land, this is still Adolf's only path for explicit "remember
|
||||||
|
// this"/"what do you recall" turns; option 3 (dynamic per-interlocutor
|
||||||
|
// bank selection) is not supported -- this config's url is a single
|
||||||
|
// static bank_id per MCP server entry, not a per-request parameter.
|
||||||
|
hindsight: {
|
||||||
|
type: "http",
|
||||||
|
url: "http://hindsight:8888/mcp/adolf-shared/",
|
||||||
|
toolFilter: {
|
||||||
|
include: ["recall", "retain", "reflect", "list_memories", "get_memory", "update_memory", "list_directives", "create_directive", "delete_directive"],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
// Already minimal (5 tools -- message_send/cron_create/cron_list/
|
||||||
|
// nodes_invoke/browser_invoke, all core to Adolf's matrix-chat +
|
||||||
|
// cron-scheduling + browser capabilities) -- deliberately NO
|
||||||
|
// toolFilter here, not an oversight.
|
||||||
|
"openclaw-tools": {
|
||||||
|
type: "http",
|
||||||
|
url: "http://openclaw-tools:8020/mcp",
|
||||||
|
},
|
||||||
|
// kanboard-mcp-adolf (kb task #58) -- standalone kanboard-mcp image
|
||||||
|
// (source /home/alvis/kanboard/mcp), second instance on :3104,
|
||||||
|
// authenticated as the Kanboard "adolf" user via its own personal API
|
||||||
|
// access token (KANBOARD_AUTH_USER=adolf in that instance's
|
||||||
|
// .env.adolf) -- a genuinely distinct credential from the "claude"
|
||||||
|
// instance on :3103 (app-wide jsonrpc token). Not part of the openai
|
||||||
|
// compose network, so reached via host.docker.internal (already
|
||||||
|
// extra_hosts-mapped for this container) rather than a service name.
|
||||||
|
// kb#144: scoped to task-triage/proactive-monitoring core --
|
||||||
|
// list/read/search/create/update/move/status/assign/comment/
|
||||||
|
// activity. Cuts subtask CRUD, task-link CRUD, tag CRUD, and
|
||||||
|
// destructive remove_task/remove_comment (9 tools) -- deep
|
||||||
|
// task-graph management is claude-coder's job (the actual task
|
||||||
|
// worker), not Adolf's lighter triage/reporting role. 23 -> 14.
|
||||||
|
kanboard: {
|
||||||
|
type: "http",
|
||||||
|
url: "http://host.docker.internal:3104/mcp",
|
||||||
|
toolFilter: {
|
||||||
|
include: ["kanboard_list_projects", "kanboard_get_project", "kanboard_list_tasks", "kanboard_my_tasks", "kanboard_get_task", "kanboard_search_tasks", "kanboard_list_users", "kanboard_project_activity", "kanboard_create_task", "kanboard_update_task", "kanboard_move_task", "kanboard_change_task_status", "kanboard_assign_task", "kanboard_add_comment"],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
// marketplace-mcp (kb task #61) -- the SAME shared marketplace-mcp
|
||||||
|
// instance Claude Code and OpenWebUI use (single service, port 3101,
|
||||||
|
// network_mode: host on the Agap host), not a second copy. It can
|
||||||
|
// place real orders on live marketplace accounts, so it's gated by a
|
||||||
|
// shared bearer token (MARKETPLACE_MCP_TOKEN in Vaultwarden / this
|
||||||
|
// container's env, injected via openai/.env -> docker-compose.yml).
|
||||||
|
// Reached via host.docker.internal, same reasoning as kanboard above.
|
||||||
|
// kb#144: scoped to READ-ONLY discovery (find_best/search/product/
|
||||||
|
// recommendations/reviews/compare/status). Cuts the checkout
|
||||||
|
// surface -- add_to_cart, get_cart, login, open_vnc, save_session,
|
||||||
|
// submit_sms (6 tools) -- real-money/session-auth actions Adolf
|
||||||
|
// shouldn't take unattended from background Matrix chat. This
|
||||||
|
// whole server is also the FIRST candidate to move OUT to Torgash
|
||||||
|
// (kb#130's marketplace-analyst persona, not yet built) per
|
||||||
|
// design §2/§8; scoping now both cuts tokens and removes purchase
|
||||||
|
// risk in the meantime rather than waiting on Torgash to exist.
|
||||||
|
// 13 -> 7.
|
||||||
|
marketplace: {
|
||||||
|
type: "http",
|
||||||
|
url: "http://host.docker.internal:3101/mcp",
|
||||||
|
headers: {
|
||||||
|
Authorization: "Bearer ${MARKETPLACE_MCP_TOKEN}",
|
||||||
|
},
|
||||||
|
toolFilter: {
|
||||||
|
include: ["marketplace_find_best", "marketplace_search", "marketplace_get_product", "marketplace_get_recommendations", "marketplace_get_reviews", "marketplace_compare_prices", "marketplace_status"],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
// agap-mcp (kb#64) -- the SAME shared agap-mcp instance Claude Code uses
|
||||||
|
// (network_mode: host, :3100, bearer-authenticated since kb#180). Grants Adolf
|
||||||
|
// the same access as Claude: vault (vw_* for credential fetching) plus
|
||||||
|
// gitea/ha/zabbix/radicale. Reached via host.docker.internal like
|
||||||
|
// kanboard/marketplace above.
|
||||||
|
// kb#144: scoped to Adolf's proactive-auditor/personal-assistant
|
||||||
|
// core -- vault read/write (trusted, credential help), HA
|
||||||
|
// (smart-home monitoring), Zabbix (the literal "proactive
|
||||||
|
// auditor" job), calendar read/write (minus admin
|
||||||
|
// create/delete_calendar), Todoist. Cuts all 6 gitea_* tools
|
||||||
|
// (repo/wiki/issue management is claude-coder's infra-ops domain,
|
||||||
|
// not Adolf's Matrix persona) plus radicale_create_calendar/
|
||||||
|
// delete_calendar (rare calendar-admin ops) -- 8 tools. 32 -> 24.
|
||||||
|
// kb#95: added wiki_search/wiki_read/wiki_edit (family MediaWiki,
|
||||||
|
// family.alogins.net / РодоВики -- source of truth for relatives,
|
||||||
|
// dates, events) -- squarely Adolf's persona domain. 35 -> 27.
|
||||||
|
agap: {
|
||||||
|
type: "http",
|
||||||
|
url: "http://host.docker.internal:3100/mcp",
|
||||||
|
// kb#180: agap-mcp's listener is authenticated now (DESIGN §4 --
|
||||||
|
// :3100 is host-networked and the LAN carries VPN peers, so an
|
||||||
|
// open JSON-RPC listener handed ha_call_service/wiki_edit/todoist
|
||||||
|
// writes to anyone). Same pattern as marketplace above:
|
||||||
|
// AGAP_MCP_TOKEN lives in Vaultwarden, is injected into this
|
||||||
|
// container via openai/.env -> docker-compose.yml, and is only
|
||||||
|
// substituted here -- never inlined. The token maps to agent id
|
||||||
|
// `adolf` in AGAP_MCP_AGENT_TOKENS, which is also what the kb#147
|
||||||
|
// vault gate reads to allow vw_* (adolf = trust_class trusted).
|
||||||
|
headers: {
|
||||||
|
Authorization: "Bearer ${AGAP_MCP_TOKEN}",
|
||||||
|
},
|
||||||
|
toolFilter: {
|
||||||
|
// kb#170: added todoist_capture_idea (classify + create in one
|
||||||
|
// call — see agap-mcp/src/capture.js) so Adolf's proactive-
|
||||||
|
// secretary persona can capture+tag an idea in one tool call
|
||||||
|
// instead of list_projects+create_task+manual tagging.
|
||||||
|
include: ["vw_get_password", "vw_get_item", "vw_list_items", "vw_create_login", "vw_update_password", "ha_get_state", "ha_list_entities", "ha_call_service", "ha_get_history", "zabbix_get_problems", "zabbix_get_hosts", "zabbix_get_items", "zabbix_get_triggers", "radicale_list_calendars", "radicale_list_events", "radicale_get_event", "radicale_put_event", "radicale_delete_event", "radicale_move_event", "todoist_list_tasks", "todoist_list_projects", "todoist_create_task", "todoist_update_task", "todoist_complete_task", "todoist_capture_idea", "wiki_search", "wiki_read", "wiki_edit"],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
|
||||||
|
// Memory plugins (P8). Activation entry is required for the gateway to
|
||||||
|
// load a plugin at startup (discovery alone is not enough).
|
||||||
|
plugins: {
|
||||||
|
entries: {
|
||||||
|
// Hindsight memory plugin (kb #75, H3) — installed external plugin
|
||||||
|
// under .openclaw/extensions/hindsight-memory, bind-mounted read-only
|
||||||
|
// from openai/hindsight-openclaw-plugin (see that project's
|
||||||
|
// docker-compose.yml adolf.volumes). Structural successor to
|
||||||
|
// cognee-memory above: LLM-free recall inject (before_prompt_build) +
|
||||||
|
// async retain (agent_end) against the hindsight service, bank
|
||||||
|
// "adolf" (same bank the mcp.servers.hindsight tool surface above
|
||||||
|
// uses, so hook-based and tool-based memory stay one consistent
|
||||||
|
// store). No cognify/sweep config here — Hindsight's retain does
|
||||||
|
// extraction/consolidation server-side, so that whole class of
|
||||||
|
// config (sweepIntervalMs etc. above) doesn't apply.
|
||||||
|
"hindsight-memory": {
|
||||||
|
enabled: true,
|
||||||
|
// Same opt-in requirement as cognee-memory above: before_prompt_build
|
||||||
|
// => allowPromptInjection; agent_end => allowConversationAccess.
|
||||||
|
hooks: { allowConversationAccess: true, allowPromptInjection: true },
|
||||||
|
},
|
||||||
|
// Kimi quota readout (kb #62) — installed external plugin, bind-mounted
|
||||||
|
// read-only from openai/quota-command-openclaw-plugin (see that
|
||||||
|
// project's docker-compose.yml adolf.volumes) onto
|
||||||
|
// .openclaw/extensions/quota-command. Registers a `/quota` native
|
||||||
|
// command; no hooks, so no allowConversationAccess/allowPromptInjection
|
||||||
|
// opt-in needed.
|
||||||
|
"quota-command": {
|
||||||
|
enabled: true,
|
||||||
|
},
|
||||||
|
// Kimi quota footer (kb #85) — installed external plugin, bind-mounted
|
||||||
|
// read-only from openai/kimi-quota-footer-plugin (see that project's
|
||||||
|
// docker-compose.yml adolf.volumes) onto
|
||||||
|
// .openclaw/extensions/kimi-quota-footer. Appends a compact Kimi
|
||||||
|
// usage line to every outgoing reply via the reply_payload_sending
|
||||||
|
// hook (not a raw conversation hook, so no allowConversationAccess/
|
||||||
|
// allowPromptInjection opt-in needed), reusing the same LLM-free
|
||||||
|
// adolf-llm:8010/usage route as quota-command above. Verified this
|
||||||
|
// hook fires on Adolf's actual send path (sendDurableMessageBatch ->
|
||||||
|
// deliverOutboundPayloadsInternal) as long as channels.matrix.streaming
|
||||||
|
// stays unset/"off" as it is today — see the plugin's index.js header
|
||||||
|
// comment for the streaming caveat if that ever changes.
|
||||||
|
"kimi-quota-footer": {
|
||||||
|
enabled: true,
|
||||||
|
},
|
||||||
|
// Todoist idea capture (kb#170 component 1) — installed external
|
||||||
|
// plugin, bind-mounted read-only from openai/todoist-capture-plugin
|
||||||
|
// (see that project's docker-compose.yml adolf.volumes) onto
|
||||||
|
// .openclaw/extensions/todoist-capture. Registers a `/idea` native
|
||||||
|
// command; no hooks (no allowConversationAccess/allowPromptInjection
|
||||||
|
// needed) — it POSTs straight to agap-mcp's /capture-idea route.
|
||||||
|
"todoist-capture": {
|
||||||
|
enabled: true,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
28
adolf/vw-mcp/.env.example
Normal file
28
adolf/vw-mcp/.env.example
Normal file
@@ -0,0 +1,28 @@
|
|||||||
|
# Copy to .env (git-ignored) and fill in real values before `docker compose up`.
|
||||||
|
# This server is intentionally narrow: read-only vw_* tools, a dedicated bot
|
||||||
|
# vault identity, and its own bearer token. See CLAUDE.md / kb task #64 for
|
||||||
|
# the full architecture and the sensitive setup steps (creating the bot user
|
||||||
|
# and the "Adolf" collection) that are NOT done by this scaffolding.
|
||||||
|
|
||||||
|
# Port this server listens on. 3100=agap-mcp, 3101=marketplace-mcp,
|
||||||
|
# 3103=kanboard-mcp, 3104=kanboard-mcp-adolf — 3105 verified free at write time.
|
||||||
|
PORT=3105
|
||||||
|
|
||||||
|
# Local Vaultwarden instance (NOT bitwarden.com). Unlike agap-mcp/
|
||||||
|
# marketplace-mcp, this service owns a fresh BITWARDENCLI_APPDATA_DIR volume
|
||||||
|
# with no pre-existing `bw config server`, so vaultwarden.js sets it on every
|
||||||
|
# boot from this var.
|
||||||
|
VW_URL=http://localhost:8041
|
||||||
|
|
||||||
|
# Dedicated bot identity — NEVER the master allogn@gmail.com account.
|
||||||
|
# Create this user in Vaultwarden first (sensitive step, reserved for the
|
||||||
|
# orchestrator — see report). Password: generate one and store it in
|
||||||
|
# Vaultwarden as item "ADOLF_VW_PASSWORD" (also a sensitive step).
|
||||||
|
BW_EMAIL=adolf-vault@auth.local
|
||||||
|
BW_PASSWORD=
|
||||||
|
|
||||||
|
# Bearer token gating /mcp, /sse, /messages (same pattern as
|
||||||
|
# marketplace-mcp). Generate with e.g. `openssl rand -hex 32`, store it in
|
||||||
|
# Vaultwarden as its own item (e.g. "VW_MCP_ADOLF_TOKEN"), and put the real
|
||||||
|
# value here — the line below is a PLACEHOLDER, not a usable secret.
|
||||||
|
VW_MCP_TOKEN=replace-with-output-of-openssl-rand--hex-32
|
||||||
2
adolf/vw-mcp/.gitignore
vendored
Normal file
2
adolf/vw-mcp/.gitignore
vendored
Normal file
@@ -0,0 +1,2 @@
|
|||||||
|
.env
|
||||||
|
node_modules/
|
||||||
10
adolf/vw-mcp/Dockerfile
Normal file
10
adolf/vw-mcp/Dockerfile
Normal file
@@ -0,0 +1,10 @@
|
|||||||
|
FROM node:22-slim
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
RUN npm install -g @bitwarden/cli
|
||||||
|
COPY package.json ./
|
||||||
|
RUN npm install --production
|
||||||
|
COPY vaultwarden.js server.js ./
|
||||||
|
COPY start.sh ./
|
||||||
|
RUN chmod +x start.sh
|
||||||
|
CMD ["./start.sh"]
|
||||||
27
adolf/vw-mcp/docker-compose.yml
Normal file
27
adolf/vw-mcp/docker-compose.yml
Normal file
@@ -0,0 +1,27 @@
|
|||||||
|
name: vw-mcp-adolf
|
||||||
|
services:
|
||||||
|
vw-mcp-adolf:
|
||||||
|
build: .
|
||||||
|
container_name: vw-mcp-adolf
|
||||||
|
restart: unless-stopped
|
||||||
|
network_mode: host
|
||||||
|
env_file:
|
||||||
|
- .env
|
||||||
|
environment:
|
||||||
|
- BITWARDENCLI_APPDATA_DIR=/bw-data
|
||||||
|
- NODE_TLS_REJECT_UNAUTHORIZED=0
|
||||||
|
- HTTPS_PROXY=
|
||||||
|
- HTTP_PROXY=
|
||||||
|
- ALL_PROXY=
|
||||||
|
- https_proxy=
|
||||||
|
- http_proxy=
|
||||||
|
- all_proxy=
|
||||||
|
volumes:
|
||||||
|
# Dedicated, OWN volume — deliberately NOT the host bind mount
|
||||||
|
# (`/home/alvis/.config/Bitwarden CLI`) that agap-mcp/marketplace-mcp
|
||||||
|
# share, and NOT any other bw data dir. This bot's login/session state
|
||||||
|
# must never mix with the master account's or any other bot's.
|
||||||
|
- vw-mcp-adolf_bw-data:/bw-data
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
vw-mcp-adolf_bw-data:
|
||||||
11
adolf/vw-mcp/package.json
Normal file
11
adolf/vw-mcp/package.json
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
{
|
||||||
|
"name": "vw-mcp-adolf",
|
||||||
|
"version": "1.0.0",
|
||||||
|
"type": "module",
|
||||||
|
"description": "Standalone, read-only MCP server giving Adolf a narrow slice of Vaultwarden (vw_get_password, vw_get_item, vw_list_items only), split out of agap-mcp per kb task #64",
|
||||||
|
"dependencies": {
|
||||||
|
"@modelcontextprotocol/sdk": "^1.29.0",
|
||||||
|
"express": "^4.19.0",
|
||||||
|
"zod": "^3.23.0"
|
||||||
|
}
|
||||||
|
}
|
||||||
137
adolf/vw-mcp/server.js
Normal file
137
adolf/vw-mcp/server.js
Normal file
@@ -0,0 +1,137 @@
|
|||||||
|
// vw-mcp-adolf — standalone, read-only Vaultwarden MCP server for Adolf (kb
|
||||||
|
// task #64).
|
||||||
|
//
|
||||||
|
// Gives Adolf a narrow, scoped slice of Vaultwarden WITHOUT exposing the
|
||||||
|
// master vault:
|
||||||
|
// - Only 3 read-only tools: vw_get_password, vw_get_item, vw_list_items.
|
||||||
|
// No write tools (vw_create_login / vw_update_password) exist here at
|
||||||
|
// all — omitted, not just unregistered, so there is no code path that
|
||||||
|
// could ever write to the vault.
|
||||||
|
// - Authenticates to Vaultwarden as a DEDICATED bot user
|
||||||
|
// (adolf-vault@auth.local), never the master account.
|
||||||
|
// - Server-side scoping is the real fence: that bot user is granted
|
||||||
|
// read-only access to a narrow "Adolf" collection only.
|
||||||
|
// - Every MCP transport requires `Authorization: Bearer $VW_MCP_TOKEN`,
|
||||||
|
// same pattern as marketplace-mcp (src/server.js) — refuses to start if
|
||||||
|
// VW_MCP_TOKEN is unset, so it can never silently run open. /health stays
|
||||||
|
// unauthenticated (no sensitive data, used for liveness checks).
|
||||||
|
import express from 'express';
|
||||||
|
import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
|
||||||
|
import { SSEServerTransport } from '@modelcontextprotocol/sdk/server/sse.js';
|
||||||
|
import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js';
|
||||||
|
import { z } from 'zod';
|
||||||
|
|
||||||
|
import { initVaultwarden, vwGetPassword, vwGetItem, vwListItems, vwListOrgItems } from './vaultwarden.js';
|
||||||
|
|
||||||
|
const PORT = parseInt(process.env.PORT || '3105');
|
||||||
|
|
||||||
|
// --- Init ---
|
||||||
|
async function init() {
|
||||||
|
await initVaultwarden();
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- MCP server factory (one per session — McpServer can't share transports) ---
|
||||||
|
function ok(text) {
|
||||||
|
return { content: [{ type: 'text', text: typeof text === 'string' ? text : JSON.stringify(text, null, 2) }] };
|
||||||
|
}
|
||||||
|
|
||||||
|
function err(e) {
|
||||||
|
return { content: [{ type: 'text', text: `Error: ${e.message}` }], isError: true };
|
||||||
|
}
|
||||||
|
|
||||||
|
function createServer() {
|
||||||
|
const server = new McpServer({ name: 'vw-mcp-adolf', version: '1.0.0' });
|
||||||
|
|
||||||
|
server.tool('vw_get_password', 'Get password for a Vaultwarden item by name (read-only; scoped to the Adolf collection)', { name: z.string() },
|
||||||
|
async ({ name }) => {
|
||||||
|
try { return ok(vwGetPassword(name)); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('vw_get_item', 'Get full details of a Vaultwarden item (name, username, password, url, notes; read-only; scoped to the Adolf collection)', { name: z.string() },
|
||||||
|
async ({ name }) => {
|
||||||
|
try {
|
||||||
|
const item = vwGetItem(name);
|
||||||
|
return ok({ name: item.name, username: item.login?.username, password: item.login?.password, url: item.login?.uris?.[0]?.uri, notes: item.notes });
|
||||||
|
} catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('vw_list_items', 'List Vaultwarden items visible to the Adolf bot user (read-only). Searches its personal vault by default; set org=true to search the org (only the Adolf collection is actually visible)', {
|
||||||
|
search: z.string().optional(),
|
||||||
|
org: z.boolean().optional(),
|
||||||
|
}, async ({ search, org }) => {
|
||||||
|
try {
|
||||||
|
const items = org ? vwListOrgItems(search) : vwListItems(search);
|
||||||
|
return ok(items.map(i => ({ id: i.id, name: i.name, username: i.login?.username, url: i.login?.uris?.[0]?.uri })));
|
||||||
|
} catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
return server;
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- Auth gate ---
|
||||||
|
// This server holds real credentials (a narrow slice, but real), so every
|
||||||
|
// MCP transport requires a bearer token. VW_MCP_TOKEN lives in Vaultwarden
|
||||||
|
// (create it as its own item once the server is live) and is injected via
|
||||||
|
// docker-compose env — never hardcode it here. /health stays open (no
|
||||||
|
// sensitive data, used for liveness checks). If VW_MCP_TOKEN is unset the
|
||||||
|
// server refuses to start, so this can never silently run open.
|
||||||
|
const AUTH_TOKEN = process.env.VW_MCP_TOKEN;
|
||||||
|
if (!AUTH_TOKEN) {
|
||||||
|
console.error('VW_MCP_TOKEN env var is required (see docker-compose.yml / .env)');
|
||||||
|
process.exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
function requireAuth(req, res, next) {
|
||||||
|
const header = req.get('authorization') || '';
|
||||||
|
const token = header.startsWith('Bearer ') ? header.slice(7) : null;
|
||||||
|
if (token !== AUTH_TOKEN) {
|
||||||
|
return res.status(401).json({ jsonrpc: '2.0', error: { code: -32001, message: 'Unauthorized' }, id: null });
|
||||||
|
}
|
||||||
|
next();
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- HTTP server (Streamable HTTP + legacy SSE) ---
|
||||||
|
const app = express();
|
||||||
|
app.use(express.json());
|
||||||
|
|
||||||
|
const sseTransports = new Map();
|
||||||
|
|
||||||
|
// Streamable HTTP — stateless: fresh server per request, survives container restarts
|
||||||
|
app.all('/mcp', requireAuth, async (req, res) => {
|
||||||
|
try {
|
||||||
|
const transport = new StreamableHTTPServerTransport({ sessionIdGenerator: undefined });
|
||||||
|
res.on('close', () => transport.close());
|
||||||
|
await createServer().connect(transport);
|
||||||
|
await transport.handleRequest(req, res, req.body);
|
||||||
|
} catch (e) {
|
||||||
|
console.error('MCP request error:', e.message);
|
||||||
|
if (!res.headersSent) {
|
||||||
|
res.status(500).json({ jsonrpc: '2.0', error: { code: -32603, message: e.message }, id: null });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
// Legacy SSE — kept for backward compatibility, same pattern as agap-mcp/kanboard-mcp
|
||||||
|
app.get('/sse', requireAuth, async (req, res) => {
|
||||||
|
const transport = new SSEServerTransport('/messages', res);
|
||||||
|
sseTransports.set(transport.sessionId, transport);
|
||||||
|
res.on('close', () => sseTransports.delete(transport.sessionId));
|
||||||
|
await createServer().connect(transport);
|
||||||
|
});
|
||||||
|
|
||||||
|
app.post('/messages', requireAuth, async (req, res) => {
|
||||||
|
const transport = sseTransports.get(req.query.sessionId);
|
||||||
|
if (!transport) return res.status(400).send('Unknown session');
|
||||||
|
await transport.handlePostMessage(req, res);
|
||||||
|
});
|
||||||
|
|
||||||
|
app.get('/health', (_, res) => res.json({ status: 'ok', tools: 3 }));
|
||||||
|
|
||||||
|
init()
|
||||||
|
.then(() => {
|
||||||
|
app.listen(PORT, () => console.log(`vw-mcp-adolf listening on :${PORT}`));
|
||||||
|
})
|
||||||
|
.catch(e => {
|
||||||
|
console.error('Init failed:', e.message);
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
2
adolf/vw-mcp/start.sh
Executable file
2
adolf/vw-mcp/start.sh
Executable file
@@ -0,0 +1,2 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
exec node server.js
|
||||||
100
adolf/vw-mcp/vaultwarden.js
Normal file
100
adolf/vw-mcp/vaultwarden.js
Normal file
@@ -0,0 +1,100 @@
|
|||||||
|
// Trimmed, read-only copy of agap-mcp/src/vaultwarden.js (kb task #64).
|
||||||
|
//
|
||||||
|
// Deliberate differences from the master copy:
|
||||||
|
// - Only the 3 read tools are implemented: get password, get item, list
|
||||||
|
// items. vwCreateLogin / vwUpdatePassword (and AI_COLLECTION, which only
|
||||||
|
// those write paths needed) are NOT here — this server must never be able
|
||||||
|
// to write to the vault, even in principle.
|
||||||
|
// - Authenticates as a DEDICATED bot user (BW_EMAIL=adolf-vault@auth.local),
|
||||||
|
// never the master allogn@gmail.com account. No default email/password —
|
||||||
|
// both must be explicit in .env so this can never silently fall back to
|
||||||
|
// the master identity.
|
||||||
|
// - BITWARDENCLI_APPDATA_DIR (see docker-compose.yml) points at this
|
||||||
|
// service's OWN volume, separate from the agap-mcp/marketplace-mcp host
|
||||||
|
// bind mount (`/home/alvis/.config/Bitwarden CLI`) — the bot's bw
|
||||||
|
// login/session state must never share a directory with the master's.
|
||||||
|
//
|
||||||
|
// The real fence is server-side: the bot user is granted read-only access to
|
||||||
|
// a narrow "Adolf" collection only (not the whole "AI" collection). This
|
||||||
|
// client code does not filter by collection — Vaultwarden itself only
|
||||||
|
// returns items the bot user's permissions allow, whatever org-wide ORG_ID
|
||||||
|
// is passed.
|
||||||
|
import { execFileSync } from 'child_process';
|
||||||
|
|
||||||
|
const BW = 'bw';
|
||||||
|
const ORG_ID = '4bd75130-b4d3-48d4-a4cb-e52b70295a51';
|
||||||
|
|
||||||
|
let _session = null;
|
||||||
|
|
||||||
|
function bwEnv() {
|
||||||
|
const env = { ...process.env };
|
||||||
|
for (const k of ['HTTPS_PROXY', 'HTTP_PROXY', 'ALL_PROXY', 'https_proxy', 'http_proxy', 'all_proxy'])
|
||||||
|
delete env[k];
|
||||||
|
env.NODE_TLS_REJECT_UNAUTHORIZED = '0';
|
||||||
|
return env;
|
||||||
|
}
|
||||||
|
|
||||||
|
function run(args, input) {
|
||||||
|
return execFileSync(BW, args, {
|
||||||
|
env: bwEnv(),
|
||||||
|
encoding: 'utf8',
|
||||||
|
input,
|
||||||
|
stdio: input ? ['pipe', 'pipe', 'pipe'] : ['ignore', 'pipe', 'pipe'],
|
||||||
|
}).trim();
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function initVaultwarden() {
|
||||||
|
const email = process.env.BW_EMAIL;
|
||||||
|
const password = process.env.BW_PASSWORD;
|
||||||
|
const server = process.env.VW_URL;
|
||||||
|
if (!email || !password) {
|
||||||
|
throw new Error('BW_EMAIL and BW_PASSWORD env vars are required — the dedicated adolf-vault@auth.local bot credentials, never the master account (see .env.example)');
|
||||||
|
}
|
||||||
|
if (!server) {
|
||||||
|
throw new Error('VW_URL env var is required — this service owns a FRESH BITWARDENCLI_APPDATA_DIR volume (unlike agap-mcp/marketplace-mcp, which reuse the host dir that already has `bw config server` set), so it must configure the server itself on every boot (see .env.example)');
|
||||||
|
}
|
||||||
|
|
||||||
|
// Idempotent — safe to call on every boot, including against an
|
||||||
|
// already-configured appdata dir.
|
||||||
|
run(['config', 'server', server]);
|
||||||
|
|
||||||
|
let status = 'unauthenticated';
|
||||||
|
try {
|
||||||
|
status = JSON.parse(run(['status'])).status;
|
||||||
|
} catch {}
|
||||||
|
|
||||||
|
if (status === 'unauthenticated') {
|
||||||
|
run(['login', email, password, '--raw']);
|
||||||
|
}
|
||||||
|
|
||||||
|
_session = run(['unlock', password, '--raw']);
|
||||||
|
run(['sync', '--session', _session]);
|
||||||
|
console.log('Vaultwarden: ready (adolf-vault bot identity)');
|
||||||
|
}
|
||||||
|
|
||||||
|
function session() {
|
||||||
|
if (!_session) throw new Error('Vaultwarden not initialized');
|
||||||
|
return _session;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function vwGetPassword(name) {
|
||||||
|
return run(['get', 'password', name, '--session', session()]);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function vwGetItem(name) {
|
||||||
|
return JSON.parse(run(['get', 'item', name, '--session', session()]));
|
||||||
|
}
|
||||||
|
|
||||||
|
export function vwListItems(search) {
|
||||||
|
const args = ['list', 'items', '--session', session()];
|
||||||
|
if (search) args.push('--search', search);
|
||||||
|
return JSON.parse(run(args));
|
||||||
|
}
|
||||||
|
|
||||||
|
export function vwListOrgItems(search) {
|
||||||
|
// Server-side collection permissions (not this code) decide what actually
|
||||||
|
// comes back — the bot user only sees the narrow "Adolf" collection.
|
||||||
|
const args = ['list', 'items', '--organizationid', ORG_ID, '--session', session()];
|
||||||
|
if (search) args.push('--search', search);
|
||||||
|
return JSON.parse(run(args));
|
||||||
|
}
|
||||||
2
agap-mcp/.gitignore
vendored
Normal file
2
agap-mcp/.gitignore
vendored
Normal file
@@ -0,0 +1,2 @@
|
|||||||
|
.env
|
||||||
|
node_modules/
|
||||||
21
agap-mcp/Dockerfile
Normal file
21
agap-mcp/Dockerfile
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
FROM node:22-slim
|
||||||
|
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
|
git ca-certificates \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
# PINNED — do not float this back to `@bitwarden/cli` (kb#180, 2026-07-30).
|
||||||
|
# An unpinned rebuild pulled 2026.7.0, whose WASM cipher deserializer rejects
|
||||||
|
# any stored login object with `"uri": null` ("invalid type: JsValue(Object({...})),
|
||||||
|
# expected a string") — the MATRIX_ADOLF_GATEWAY_TOKEN item in this vault has
|
||||||
|
# exactly that shape. `bw list` then exits 1, server.js's init fails, and the
|
||||||
|
# container crash-loops on restart. 2026.2.0 parses that item fine and is the
|
||||||
|
# version the host CLI runs. Re-test against the live vault before bumping.
|
||||||
|
RUN npm install -g @bitwarden/cli@2026.2.0
|
||||||
|
COPY package.json ./
|
||||||
|
RUN npm install --production
|
||||||
|
COPY src/ ./src/
|
||||||
|
COPY start.sh ./
|
||||||
|
RUN chmod +x start.sh
|
||||||
|
CMD ["./start.sh"]
|
||||||
75
agap-mcp/docker-compose.yml
Normal file
75
agap-mcp/docker-compose.yml
Normal file
@@ -0,0 +1,75 @@
|
|||||||
|
services:
|
||||||
|
agap-mcp:
|
||||||
|
build: .
|
||||||
|
restart: unless-stopped
|
||||||
|
network_mode: host
|
||||||
|
environment:
|
||||||
|
- PORT=3100
|
||||||
|
- NODE_TLS_REJECT_UNAUTHORIZED=0
|
||||||
|
- HTTPS_PROXY=
|
||||||
|
- HTTP_PROXY=
|
||||||
|
- ALL_PROXY=
|
||||||
|
- https_proxy=
|
||||||
|
- http_proxy=
|
||||||
|
- all_proxy=
|
||||||
|
- BITWARDENCLI_APPDATA_DIR=/bw-data
|
||||||
|
- BW_EMAIL=adolf46@proton.me
|
||||||
|
- BW_PASSWORD=${BW_PASSWORD}
|
||||||
|
- VAULTWARDEN_URL=http://localhost:8041
|
||||||
|
- GITEA_URL=http://localhost:3000
|
||||||
|
- HA_URL=http://192.168.1.4:8123
|
||||||
|
- ZABBIX_URL=http://192.168.1.4:81
|
||||||
|
- RADICALE_URL=http://localhost:5232
|
||||||
|
- RADICALE_USER=alvis
|
||||||
|
# kb#147 (A2A-15) — vault trust gate, OFF by default (0). Flipping this
|
||||||
|
# to 1 requires a container restart AND real values for
|
||||||
|
# AGENT_REGISTRY_PATH/AGAP_MCP_AGENT_TOKENS below to be populated first
|
||||||
|
# (see src/trust-gate.js) — that restart is the deliberate handover
|
||||||
|
# step this task does NOT perform (never restart the live agap-mcp
|
||||||
|
# service unattended). Until both are set, vw_* tools behave exactly
|
||||||
|
# as before this change.
|
||||||
|
- AGAP_MCP_ENFORCE_VAULT_TRUST=0
|
||||||
|
- AGENT_REGISTRY_PATH=/agent-registry.yaml
|
||||||
|
# JSON map {"<bearer-token>": "<agent-id>"}. Real per-agent tokens must
|
||||||
|
# be generated, stored in Vaultwarden (e.g. AGAP_MCP_TOKEN_ADOLF,
|
||||||
|
# AGAP_MCP_TOKEN_CLAUDE_CODER), and referenced here via .env — never
|
||||||
|
# inlined in this committed file. Empty object = no caller resolves to
|
||||||
|
# any agent, i.e. fail-closed once ENFORCE is turned on.
|
||||||
|
- AGAP_MCP_AGENT_TOKENS=${AGAP_MCP_AGENT_TOKENS:-{}}
|
||||||
|
# kb#180 (DESIGN-a2a-agents.md §4) — authentication of the LISTENER
|
||||||
|
# itself, a strictly larger gate than the vw_*-only one above. With
|
||||||
|
# this on (the default in code), /mcp, /sse, /messages and
|
||||||
|
# /capture-idea all require `Authorization: Bearer <token>` resolving
|
||||||
|
# to an agent id in AGAP_MCP_AGENT_TOKENS; /health stays open for this
|
||||||
|
# healthcheck. :3100 is bound on every interface (network_mode: host)
|
||||||
|
# and the LAN carries VPN-terminated peers, so an open listener means
|
||||||
|
# any peer can call ha_call_service / gitea_wiki_write / wiki_edit /
|
||||||
|
# todoist writes.
|
||||||
|
#
|
||||||
|
# ACTIVATION IS NOT AUTOMATIC-SAFE: with auth on and
|
||||||
|
# AGAP_MCP_AGENT_TOKENS empty, the process REFUSES TO START (loud
|
||||||
|
# crash instead of denying every caller while /health says ok). So
|
||||||
|
# AGAP_MCP_AGENT_TOKENS must be populated in this directory's .env
|
||||||
|
# BEFORE the next restart of this service, and every caller
|
||||||
|
# (Adolf/shared-mcp.json, Claude Code .claude.json, the
|
||||||
|
# todoist-capture-plugin) must be given its token — see the kb#180
|
||||||
|
# migration list. Set AGAP_MCP_REQUIRE_AUTH=0 in .env only as a
|
||||||
|
# deliberate emergency rollback to the old open listener.
|
||||||
|
- AGAP_MCP_REQUIRE_AUTH=${AGAP_MCP_REQUIRE_AUTH:-1}
|
||||||
|
volumes:
|
||||||
|
- /home/alvis/.config/Bitwarden CLI:/bw-data
|
||||||
|
# Read-only: agent-registry.yaml is the version-controlled source of
|
||||||
|
# truth for trust classes (kb#134/kb#147) — mounted, never copied, so
|
||||||
|
# a registry edit takes effect on container restart with no rebuild.
|
||||||
|
- /home/alvis/agap_git/openai/agent-registry.yaml:/agent-registry.yaml:ro
|
||||||
|
# kb#190: /health responds 200 with no auth/side effects (confirmed).
|
||||||
|
# This is a SEPARATE compose project from openai/docker-compose.yml
|
||||||
|
# (network_mode: host, reached from adolf-llm etc. via
|
||||||
|
# host.docker.internal), so it cannot be wired into that file's
|
||||||
|
# depends_on/condition chain -- this only gives it its own status.
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "node -e \"fetch('http://localhost:3100/health').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))\""]
|
||||||
|
interval: 15s
|
||||||
|
timeout: 10s
|
||||||
|
retries: 5
|
||||||
|
start_period: 20s
|
||||||
1611
agap-mcp/package-lock.json
generated
Normal file
1611
agap-mcp/package-lock.json
generated
Normal file
File diff suppressed because it is too large
Load Diff
11
agap-mcp/package.json
Normal file
11
agap-mcp/package.json
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
{
|
||||||
|
"name": "agap-mcp",
|
||||||
|
"version": "1.0.0",
|
||||||
|
"type": "module",
|
||||||
|
"dependencies": {
|
||||||
|
"@modelcontextprotocol/sdk": "^1.29.0",
|
||||||
|
"express": "^4.19.0",
|
||||||
|
"js-yaml": "^4.1.0",
|
||||||
|
"zod": "^3.23.0"
|
||||||
|
}
|
||||||
|
}
|
||||||
58
agap-mcp/src/capture.js
Normal file
58
agap-mcp/src/capture.js
Normal file
@@ -0,0 +1,58 @@
|
|||||||
|
// Idea capture pipeline (kb#170): classify -> Todoist task.
|
||||||
|
//
|
||||||
|
// Deliberately does NOT create/move anything in Kanboard, Radicale, or
|
||||||
|
// gitea — kb#170 component 4 ("Adolf предлагает, какие идеи созрели")
|
||||||
|
// is a human-in-the-loop periodic review, not an automatic conversion.
|
||||||
|
// This module only tags the idea so that later review has something to
|
||||||
|
// act on (labels: area-*, urgency-*, and "decompose" when flagged).
|
||||||
|
//
|
||||||
|
// Project/label mapping is a deliberate v1 decision, not a guess left
|
||||||
|
// unstated: Todoist's real projects today (Inbox, One-Off, Family,
|
||||||
|
// Planning, Pending — confirmed live via todoist_list_projects) don't
|
||||||
|
// line up with the 5 kb#170 areas except "семья" ~= "Family". Creating
|
||||||
|
// four new Todoist projects to match Adolf/Welfare/дом/здоровье is a
|
||||||
|
// structural change to the user's real Todoist account, so it is NOT done
|
||||||
|
// here without explicit sign-off — see kb#170 report. Instead every idea
|
||||||
|
// keeps its default project (Inbox, unless the caller passes one) and
|
||||||
|
// gets an `area-*` label, which is purely additive/reversible (Todoist
|
||||||
|
// auto-creates labels on first use, and any label can be deleted later
|
||||||
|
// with zero data loss).
|
||||||
|
import { classifyIdea } from './classifier.js';
|
||||||
|
import { todoistCreateTask } from './todoist.js';
|
||||||
|
|
||||||
|
const FAMILY_PROJECT_NAME = 'Family';
|
||||||
|
|
||||||
|
// urgency label -> Todoist API priority (1=normal..4=urgent, inverse of
|
||||||
|
// the Todoist UI's p1..p4 — see todoist.js header comment).
|
||||||
|
const URGENCY_TO_PRIORITY = { high: 4, medium: 2, low: 1 };
|
||||||
|
|
||||||
|
export async function todoistCaptureIdea({ text, project_id } = {}, { listProjects = null, createTask = todoistCreateTask } = {}) {
|
||||||
|
if (!text || !text.trim()) throw new Error('text is required');
|
||||||
|
|
||||||
|
const classification = await classifyIdea(text.trim());
|
||||||
|
|
||||||
|
const labels = [`area-${classification.area.label}`, `urgency-${classification.urgency.label}`];
|
||||||
|
if (classification.decompose.label === 'needs-decomposition') labels.push('decompose');
|
||||||
|
if (classification.area.ambiguous) labels.push('area-uncertain');
|
||||||
|
|
||||||
|
// Only auto-route to an existing project when it's an unambiguous, exact
|
||||||
|
// match (семья -> Family) — never invent/select a project the classifier
|
||||||
|
// merely guessed at, and never override a project_id the caller passed
|
||||||
|
// explicitly.
|
||||||
|
let resolvedProjectId = project_id;
|
||||||
|
if (!resolvedProjectId && classification.area.label === 'семья' && !classification.area.ambiguous && typeof listProjects === 'function') {
|
||||||
|
const projects = await listProjects();
|
||||||
|
const family = projects.find((p) => p.name === FAMILY_PROJECT_NAME);
|
||||||
|
if (family) resolvedProjectId = family.id;
|
||||||
|
}
|
||||||
|
|
||||||
|
const task = await createTask({
|
||||||
|
content: text.trim(),
|
||||||
|
description: `Захвачено через AI-классификацию (kb#170): area=${classification.area.label} (${classification.area.score}), urgency=${classification.urgency.label} (${classification.urgency.score}), decompose=${classification.decompose.label} (${classification.decompose.score}).`,
|
||||||
|
priority: URGENCY_TO_PRIORITY[classification.urgency.label],
|
||||||
|
project_id: resolvedProjectId,
|
||||||
|
labels,
|
||||||
|
});
|
||||||
|
|
||||||
|
return { task, classification };
|
||||||
|
}
|
||||||
84
agap-mcp/src/capture.test.mjs
Normal file
84
agap-mcp/src/capture.test.mjs
Normal file
@@ -0,0 +1,84 @@
|
|||||||
|
// Proof for kb#170 capture pipeline (classify -> Todoist task shape),
|
||||||
|
// run with:
|
||||||
|
// BGE_M3_URL=http://localhost:11436/v1/embeddings node src/capture.test.mjs
|
||||||
|
//
|
||||||
|
// Deliberately stubs createTask/listProjects instead of calling the real
|
||||||
|
// Todoist API — this proves the classify -> label/priority/project mapping
|
||||||
|
// logic without writing test data into the user's live Todoist account
|
||||||
|
// (kb#170 report: no live Todoist writes were made while proving this out).
|
||||||
|
|
||||||
|
import assert from 'node:assert/strict';
|
||||||
|
import { todoistCaptureIdea } from './capture.js';
|
||||||
|
|
||||||
|
let passed = 0;
|
||||||
|
function check(label, fn) {
|
||||||
|
fn();
|
||||||
|
passed++;
|
||||||
|
console.log(`ok - ${label}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
const projects = [
|
||||||
|
{ id: '6CrfPQ8FXxf5ghrx', name: 'Inbox', is_inbox: true },
|
||||||
|
{ id: '6cg4j8CX3vj7H9rJ', name: 'Family' },
|
||||||
|
];
|
||||||
|
|
||||||
|
function makeStubCreateTask() {
|
||||||
|
const calls = [];
|
||||||
|
const createTask = async (args) => {
|
||||||
|
calls.push(args);
|
||||||
|
return { id: 'stub-1', content: args.content, project_id: args.project_id, priority: args.priority, labels: args.labels };
|
||||||
|
};
|
||||||
|
return { createTask, calls };
|
||||||
|
}
|
||||||
|
|
||||||
|
const { createTask: createTaskFamily, calls: callsFamily } = makeStubCreateTask();
|
||||||
|
const familyResult = await todoistCaptureIdea(
|
||||||
|
{ text: 'позвонить маме поздравить с днём рождения' },
|
||||||
|
{ listProjects: async () => projects, createTask: createTaskFamily }
|
||||||
|
);
|
||||||
|
|
||||||
|
check('семья idea auto-routes to the existing Family project', () => {
|
||||||
|
assert.equal(familyResult.classification.area.label, 'семья');
|
||||||
|
assert.equal(callsFamily[0].project_id, '6cg4j8CX3vj7H9rJ');
|
||||||
|
});
|
||||||
|
|
||||||
|
check('семья idea is labelled area-семья + urgency-*', () => {
|
||||||
|
assert.ok(callsFamily[0].labels.includes('area-семья'));
|
||||||
|
assert.ok(callsFamily[0].labels.some((l) => l.startsWith('urgency-')));
|
||||||
|
});
|
||||||
|
|
||||||
|
const { createTask: createTaskUrgent, calls: callsUrgent } = makeStubCreateTask();
|
||||||
|
await todoistCaptureIdea(
|
||||||
|
{ text: 'починить квоту Kimi у Adolf, срочно сегодня' },
|
||||||
|
{ listProjects: async () => projects, createTask: createTaskUrgent }
|
||||||
|
);
|
||||||
|
|
||||||
|
check('high-urgency idea gets Todoist priority 4 (urgent)', () => {
|
||||||
|
assert.equal(callsUrgent[0].priority, 4);
|
||||||
|
});
|
||||||
|
|
||||||
|
check('adolf-area idea is NOT auto-routed to a project (no matching project exists)', () => {
|
||||||
|
assert.equal(callsUrgent[0].project_id, undefined);
|
||||||
|
});
|
||||||
|
|
||||||
|
const { createTask: createTaskProject, calls: callsProject } = makeStubCreateTask();
|
||||||
|
await todoistCaptureIdea(
|
||||||
|
{ text: 'купить новый пылесос для дома', project_id: 'explicit-override' },
|
||||||
|
{ listProjects: async () => projects, createTask: createTaskProject }
|
||||||
|
);
|
||||||
|
|
||||||
|
check('an explicit project_id always wins over auto-routing', () => {
|
||||||
|
assert.equal(callsProject[0].project_id, 'explicit-override');
|
||||||
|
});
|
||||||
|
|
||||||
|
const { createTask: createTaskDecompose, calls: callsDecompose } = makeStubCreateTask();
|
||||||
|
await todoistCaptureIdea(
|
||||||
|
{ text: 'спроектировать и запустить proactive-секретаря на Agap' },
|
||||||
|
{ listProjects: async () => projects, createTask: createTaskDecompose }
|
||||||
|
);
|
||||||
|
|
||||||
|
check('a multi-step idea gets the "decompose" label', () => {
|
||||||
|
assert.ok(callsDecompose[0].labels.includes('decompose'));
|
||||||
|
});
|
||||||
|
|
||||||
|
console.log(`\n${passed} passed`);
|
||||||
228
agap-mcp/src/classifier.js
Normal file
228
agap-mcp/src/classifier.js
Normal file
@@ -0,0 +1,228 @@
|
|||||||
|
// Todoist idea classifier (kb#170, component 2) — encoder-only, NOT a
|
||||||
|
// classifier LLM call. Per DESIGN-a2a-agents.md v2.1 §3a/§3.1/theorem 24:
|
||||||
|
// "routing classification is embedding-based on the local bge-m3 ... no
|
||||||
|
// classifier LLM, no API spend". This module applies that same idea to
|
||||||
|
// Todoist-capture classification: embed the idea text with bge-m3 (already
|
||||||
|
// GPU-resident, never-evict per model-registry.yaml) and classify by
|
||||||
|
// nearest-centroid against a small hand-labelled exemplar set — no Kimi/
|
||||||
|
// gemma call, ~0 marginal cost, no metered API.
|
||||||
|
//
|
||||||
|
// Three independent classification axes (each idea gets one label per axis,
|
||||||
|
// not a single combined class):
|
||||||
|
// area — which part of life the idea belongs to (kb#170 spec)
|
||||||
|
// urgency — how soon it stops being actionable
|
||||||
|
// decompose — is this a single atomic action, or a multi-step project
|
||||||
|
// that should eventually become a Kanboard task graph
|
||||||
|
//
|
||||||
|
// This is deliberately a NEAREST-CENTROID classifier, not a trained model:
|
||||||
|
// no labelled training set exists (kb#170 orchestrator note — inventing one
|
||||||
|
// would be guessing), so the "training data" IS the exemplar list below,
|
||||||
|
// reviewed/editable in code (git-controlled, per DESIGN-a2a-agents.md
|
||||||
|
// "Personas and Cards are code"). Extending accuracy later means adding
|
||||||
|
// exemplars here, not retraining a model.
|
||||||
|
|
||||||
|
const DEFAULT_BGE_URL = process.env.BGE_M3_URL || 'http://host.docker.internal:11436/v1/embeddings';
|
||||||
|
|
||||||
|
// --- Exemplars -------------------------------------------------------------
|
||||||
|
// Kept short and idiomatic (the kind of one-line idea a person actually
|
||||||
|
// captures), Russian-first since that's the capture language (kb#170 desc).
|
||||||
|
// Centroids are the mean of these exemplars' embeddings — adding more
|
||||||
|
// exemplars per class only requires appending strings here.
|
||||||
|
|
||||||
|
const AREA_EXEMPLARS = {
|
||||||
|
adolf: [
|
||||||
|
'починить квоту Kimi у Adolf',
|
||||||
|
'настроить cron задачу в Kanboard',
|
||||||
|
'добавить новую MCP команду',
|
||||||
|
'проверить логи agap-mcp контейнера',
|
||||||
|
'написать воркер для очереди задач',
|
||||||
|
'обновить конфиг openclaw.json',
|
||||||
|
],
|
||||||
|
welfare: [
|
||||||
|
'продумать еженедельный ревью задач',
|
||||||
|
'настроить трекер настроения и энергии',
|
||||||
|
'сделать ежедневный брифинг по утрам',
|
||||||
|
'завести журнал решений',
|
||||||
|
'придумать систему напоминаний о важных вещах',
|
||||||
|
'разобраться с личной продуктивностью',
|
||||||
|
'спроектировать proactive-секретаря для себя',
|
||||||
|
'построить систему, которая сама напоминает и планирует',
|
||||||
|
'придумать, как автоматизировать личный распорядок дня',
|
||||||
|
],
|
||||||
|
'дом': [
|
||||||
|
'купить новый пылесос',
|
||||||
|
'почистить фильтр кондиционера',
|
||||||
|
'вызвать сантехника починить кран',
|
||||||
|
'заказать доставку воды',
|
||||||
|
'разобрать кладовку',
|
||||||
|
'поменять лампочку в коридоре',
|
||||||
|
'оплатить счёт за квартиру',
|
||||||
|
'оплатить интернет и коммуналку',
|
||||||
|
],
|
||||||
|
'семья': [
|
||||||
|
'позвонить маме',
|
||||||
|
'поздравить сестру с днём рождения',
|
||||||
|
'купить подарок жене',
|
||||||
|
'спланировать поездку с семьёй',
|
||||||
|
'написать бабушке',
|
||||||
|
'забрать детей из школы',
|
||||||
|
],
|
||||||
|
'здоровье': [
|
||||||
|
'записаться к врачу',
|
||||||
|
'сдать анализы крови',
|
||||||
|
'начать бегать по утрам',
|
||||||
|
'купить витамины',
|
||||||
|
'сходить к стоматологу',
|
||||||
|
'записаться на массаж',
|
||||||
|
],
|
||||||
|
};
|
||||||
|
|
||||||
|
const URGENCY_EXEMPLARS = {
|
||||||
|
high: [
|
||||||
|
'сделать это сегодня, срочно',
|
||||||
|
'дедлайн завтра утром',
|
||||||
|
'оплатить штраф до пятницы, иначе пени',
|
||||||
|
'нужно решить прямо сейчас',
|
||||||
|
],
|
||||||
|
medium: [
|
||||||
|
'сделать на этой неделе',
|
||||||
|
'стоит сделать в ближайшие дни',
|
||||||
|
'через пару дней надо разобраться',
|
||||||
|
'неплохо бы успеть до конца месяца',
|
||||||
|
],
|
||||||
|
low: [
|
||||||
|
'когда-нибудь было бы неплохо',
|
||||||
|
'не к спеху, просто идея на будущее',
|
||||||
|
'если будет время',
|
||||||
|
'мысль про потом, без срока',
|
||||||
|
],
|
||||||
|
};
|
||||||
|
|
||||||
|
const DECOMPOSE_EXEMPLARS = {
|
||||||
|
'needs-decomposition': [
|
||||||
|
'организовать переезд на новую квартиру',
|
||||||
|
'спроектировать и запустить новый сервис на сервере',
|
||||||
|
'спланировать отпуск в другую страну',
|
||||||
|
'построить систему проактивного секретаря',
|
||||||
|
'провести ремонт в квартире',
|
||||||
|
'подготовить и провести презентацию проекта',
|
||||||
|
],
|
||||||
|
'simple-task': [
|
||||||
|
'позвонить маме',
|
||||||
|
'купить хлеб',
|
||||||
|
'оплатить счёт за интернет',
|
||||||
|
'отправить один email',
|
||||||
|
'поставить будильник',
|
||||||
|
'записать одну мысль в заметки',
|
||||||
|
],
|
||||||
|
};
|
||||||
|
|
||||||
|
// --- Embeddings + cosine similarity -----------------------------------------
|
||||||
|
|
||||||
|
async function embed(text, bgeUrl = DEFAULT_BGE_URL) {
|
||||||
|
const res = await fetch(bgeUrl, {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({ model: 'bge-m3', input: text }),
|
||||||
|
});
|
||||||
|
if (!res.ok) {
|
||||||
|
throw new Error(`bge-m3 embeddings ${res.status}: ${(await res.text()).slice(0, 300)}`);
|
||||||
|
}
|
||||||
|
const body = await res.json();
|
||||||
|
const vec = body?.data?.[0]?.embedding;
|
||||||
|
if (!Array.isArray(vec)) throw new Error('bge-m3 embeddings: no vector in response');
|
||||||
|
return vec;
|
||||||
|
}
|
||||||
|
|
||||||
|
function dot(a, b) {
|
||||||
|
let s = 0;
|
||||||
|
for (let i = 0; i < a.length; i++) s += a[i] * b[i];
|
||||||
|
return s;
|
||||||
|
}
|
||||||
|
|
||||||
|
function norm(a) {
|
||||||
|
return Math.sqrt(dot(a, a));
|
||||||
|
}
|
||||||
|
|
||||||
|
function normalize(a) {
|
||||||
|
const n = norm(a) || 1;
|
||||||
|
return a.map((x) => x / n);
|
||||||
|
}
|
||||||
|
|
||||||
|
function mean(vectors) {
|
||||||
|
const dim = vectors[0].length;
|
||||||
|
const out = new Array(dim).fill(0);
|
||||||
|
for (const v of vectors) for (let i = 0; i < dim; i++) out[i] += v[i];
|
||||||
|
return out.map((x) => x / vectors.length);
|
||||||
|
}
|
||||||
|
|
||||||
|
function cosine(a, b) {
|
||||||
|
return dot(a, b) / ((norm(a) || 1) * (norm(b) || 1));
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- Centroid cache ----------------------------------------------------------
|
||||||
|
// Computed once per process (exemplars are static, embedding a few dozen
|
||||||
|
// short strings at startup is cheap and happens lazily on first classify()
|
||||||
|
// call, not at import time — keeps agap-mcp's init() path unaffected).
|
||||||
|
|
||||||
|
let _centroidsPromise = null;
|
||||||
|
|
||||||
|
async function buildCentroidSet(exemplarMap, bgeUrl) {
|
||||||
|
const labels = Object.keys(exemplarMap);
|
||||||
|
const centroids = {};
|
||||||
|
for (const label of labels) {
|
||||||
|
const vectors = await Promise.all(exemplarMap[label].map((t) => embed(t, bgeUrl).then(normalize)));
|
||||||
|
centroids[label] = normalize(mean(vectors));
|
||||||
|
}
|
||||||
|
return centroids;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function getCentroids(bgeUrl = DEFAULT_BGE_URL) {
|
||||||
|
if (!_centroidsPromise) {
|
||||||
|
_centroidsPromise = Promise.all([
|
||||||
|
buildCentroidSet(AREA_EXEMPLARS, bgeUrl),
|
||||||
|
buildCentroidSet(URGENCY_EXEMPLARS, bgeUrl),
|
||||||
|
buildCentroidSet(DECOMPOSE_EXEMPLARS, bgeUrl),
|
||||||
|
]).then(([area, urgency, decompose]) => ({ area, urgency, decompose }));
|
||||||
|
}
|
||||||
|
return _centroidsPromise;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Test-only: let tests reset the cache (e.g. to inject a different BGE_URL).
|
||||||
|
export function _resetCentroidCacheForTests() {
|
||||||
|
_centroidsPromise = null;
|
||||||
|
}
|
||||||
|
|
||||||
|
// nearestLabel: pick argmax cosine similarity; also report the runner-up
|
||||||
|
// and the margin between them. A small margin means the idea sits between
|
||||||
|
// two classes — surfaced as `ambiguous: true` rather than silently forced,
|
||||||
|
// so the periodic-review pass (kb#170 component 4) can have Adolf confirm
|
||||||
|
// instead of trusting a low-confidence auto-tag.
|
||||||
|
function nearestLabel(vec, centroidMap) {
|
||||||
|
const scored = Object.entries(centroidMap)
|
||||||
|
.map(([label, centroid]) => ({ label, score: cosine(vec, centroid) }))
|
||||||
|
.sort((a, b) => b.score - a.score);
|
||||||
|
const [top, second] = scored;
|
||||||
|
const margin = second ? top.score - second.score : 1;
|
||||||
|
return {
|
||||||
|
label: top.label,
|
||||||
|
score: Number(top.score.toFixed(4)),
|
||||||
|
margin: Number(margin.toFixed(4)),
|
||||||
|
ambiguous: margin < 0.03, // empirical starting threshold — revisit once real captures accumulate (same posture as DESIGN-proactive-prioritization.md's tunable constants)
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
// classify: the one entry point. Embeds the idea text ONCE, reuses it
|
||||||
|
// across all three axes (one bge-m3 call, not three) — consistent with
|
||||||
|
// the "no metered/needless calls" cost discipline in DESIGN-a2a-agents.md.
|
||||||
|
export async function classifyIdea(text, { bgeUrl = DEFAULT_BGE_URL } = {}) {
|
||||||
|
if (!text || !text.trim()) throw new Error('text is required');
|
||||||
|
const [vec, centroids] = await Promise.all([embed(text, bgeUrl).then(normalize), getCentroids(bgeUrl)]);
|
||||||
|
return {
|
||||||
|
area: nearestLabel(vec, centroids.area),
|
||||||
|
urgency: nearestLabel(vec, centroids.urgency),
|
||||||
|
decompose: nearestLabel(vec, centroids.decompose),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export const _internal = { AREA_EXEMPLARS, URGENCY_EXEMPLARS, DECOMPOSE_EXEMPLARS, cosine, embed, getCentroids };
|
||||||
45
agap-mcp/src/classifier.test.mjs
Normal file
45
agap-mcp/src/classifier.test.mjs
Normal file
@@ -0,0 +1,45 @@
|
|||||||
|
// Proof for kb#170 component 2 — run with:
|
||||||
|
// BGE_M3_URL=http://localhost:11436/v1/embeddings node src/classifier.test.mjs
|
||||||
|
// (default BGE_M3_URL assumes host.docker.internal, which only resolves
|
||||||
|
// inside a container; override to localhost when running on the Agap host
|
||||||
|
// directly, same pattern as the bge-m3 curl checks elsewhere in this repo).
|
||||||
|
//
|
||||||
|
// This is a LIVE test against the real bge-m3 embedder (no mock) — the
|
||||||
|
// point of an encoder-only classifier is that it's cheap enough to just
|
||||||
|
// call for real (~30 short strings embedded once, then one embedding per
|
||||||
|
// test case). It does not touch Todoist, Kanboard, or any other live
|
||||||
|
// service.
|
||||||
|
|
||||||
|
import assert from 'node:assert/strict';
|
||||||
|
import { classifyIdea } from './classifier.js';
|
||||||
|
|
||||||
|
const cases = [
|
||||||
|
{ text: 'позвонить маме поздравить с днём рождения', expectArea: 'семья' },
|
||||||
|
{ text: 'купить новый пылесос для дома', expectArea: 'дом' },
|
||||||
|
{ text: 'записаться на приём к стоматологу', expectArea: 'здоровье' },
|
||||||
|
{ text: 'починить квоту Kimi у Adolf, срочно сегодня', expectArea: 'adolf', expectUrgency: 'high' },
|
||||||
|
{ text: 'спроектировать и запустить proactive-секретаря на Agap', expectArea: 'welfare', expectDecompose: 'needs-decomposition' },
|
||||||
|
{ text: 'оплатить счёт за интернет', expectDecompose: 'simple-task' },
|
||||||
|
{ text: 'организовать переезд на новую квартиру, когда-нибудь', expectDecompose: 'needs-decomposition', expectUrgency: 'low' },
|
||||||
|
];
|
||||||
|
|
||||||
|
let passed = 0;
|
||||||
|
let failed = 0;
|
||||||
|
|
||||||
|
for (const c of cases) {
|
||||||
|
const result = await classifyIdea(c.text);
|
||||||
|
const row = `"${c.text}" -> area=${result.area.label}(${result.area.score}) urgency=${result.urgency.label}(${result.urgency.score}) decompose=${result.decompose.label}(${result.decompose.score})`;
|
||||||
|
try {
|
||||||
|
if (c.expectArea) assert.equal(result.area.label, c.expectArea, `area mismatch for "${c.text}"`);
|
||||||
|
if (c.expectUrgency) assert.equal(result.urgency.label, c.expectUrgency, `urgency mismatch for "${c.text}"`);
|
||||||
|
if (c.expectDecompose) assert.equal(result.decompose.label, c.expectDecompose, `decompose mismatch for "${c.text}"`);
|
||||||
|
console.log(`ok - ${row}`);
|
||||||
|
passed++;
|
||||||
|
} catch (e) {
|
||||||
|
console.log(`FAIL - ${row}\n ${e.message}`);
|
||||||
|
failed++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n${passed} passed, ${failed} failed`);
|
||||||
|
if (failed > 0) process.exit(1);
|
||||||
97
agap-mcp/src/gitea.js
Normal file
97
agap-mcp/src/gitea.js
Normal file
@@ -0,0 +1,97 @@
|
|||||||
|
import { execSync } from 'child_process';
|
||||||
|
import { writeFileSync, mkdirSync } from 'fs';
|
||||||
|
import { tmpdir } from 'os';
|
||||||
|
import { join } from 'path';
|
||||||
|
|
||||||
|
// Askpass helper: git invokes this script (path is what shows up in ps/args),
|
||||||
|
// and it reads the actual token from an env var — never from argv or the URL.
|
||||||
|
// This keeps the token out of the process table and out of any git error text.
|
||||||
|
let _askpassPath = null;
|
||||||
|
function askpassScript() {
|
||||||
|
if (_askpassPath) return _askpassPath;
|
||||||
|
const dir = join(tmpdir(), 'agap-mcp-wiki');
|
||||||
|
mkdirSync(dir, { recursive: true });
|
||||||
|
const scriptPath = join(dir, 'git-askpass.sh');
|
||||||
|
writeFileSync(scriptPath, '#!/bin/sh\nprintf %s "$GITEA_ASKPASS_TOKEN"\n', { mode: 0o700 });
|
||||||
|
_askpassPath = scriptPath;
|
||||||
|
return scriptPath;
|
||||||
|
}
|
||||||
|
|
||||||
|
const BASE = () => process.env.GITEA_URL || 'http://localhost:3000';
|
||||||
|
let _token = null;
|
||||||
|
|
||||||
|
export function initGitea(token) {
|
||||||
|
_token = token;
|
||||||
|
console.log('Gitea: ready');
|
||||||
|
}
|
||||||
|
|
||||||
|
function token() {
|
||||||
|
if (!_token) throw new Error('Gitea not initialized');
|
||||||
|
return _token;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function api(path, opts = {}) {
|
||||||
|
const res = await fetch(`${BASE()}/api/v1${path}`, {
|
||||||
|
...opts,
|
||||||
|
headers: { Authorization: `token ${token()}`, 'Content-Type': 'application/json', ...opts.headers },
|
||||||
|
});
|
||||||
|
if (!res.ok) throw new Error(`Gitea API ${path}: ${res.status} ${await res.text()}`);
|
||||||
|
return res.json();
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function giteaListRepos() {
|
||||||
|
return api('/repos/search?limit=50').then(r => r.data.map(r => ({
|
||||||
|
name: r.full_name, description: r.description, stars: r.stars_count,
|
||||||
|
})));
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function giteaReadFile(repo, path, ref = 'HEAD') {
|
||||||
|
const data = await api(`/repos/${repo}/contents/${path}?ref=${ref}`);
|
||||||
|
return Buffer.from(data.content, 'base64').toString('utf8');
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function giteaWikiList(repo = 'alvis/AgapHost') {
|
||||||
|
const data = await api(`/repos/${repo}/wiki/pages?limit=50`);
|
||||||
|
return data.map(p => ({ name: p.title, updated: p.last_commit?.created }));
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function giteaWikiRead(page, repo = 'alvis/AgapHost') {
|
||||||
|
const data = await api(`/repos/${repo}/wiki/page/${encodeURIComponent(page)}`);
|
||||||
|
return Buffer.from(data.content_base64, 'base64').toString('utf8');
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function giteaWikiWrite(page, content, message, repo = 'alvis/AgapHost') {
|
||||||
|
const dir = join(tmpdir(), 'agap-mcp-wiki');
|
||||||
|
// Username in the URL is not secret; the password/token is supplied out-of-band
|
||||||
|
// via GIT_ASKPASS + GITEA_ASKPASS_TOKEN, so it never appears in the URL, the
|
||||||
|
// execSync command string, ps/process args, or surfaced git error output.
|
||||||
|
const wikiUrl = `${BASE().replace('http://', 'http://alvis@')}/alvis/AgapHost.wiki.git`;
|
||||||
|
|
||||||
|
const gitEnv = {
|
||||||
|
...process.env,
|
||||||
|
GIT_AUTHOR_NAME: 'agap-mcp', GIT_AUTHOR_EMAIL: 'allogn@gmail.com',
|
||||||
|
GIT_COMMITTER_NAME: 'agap-mcp', GIT_COMMITTER_EMAIL: 'allogn@gmail.com',
|
||||||
|
GIT_ASKPASS: askpassScript(),
|
||||||
|
GIT_TERMINAL_PROMPT: '0',
|
||||||
|
GITEA_ASKPASS_TOKEN: token(),
|
||||||
|
};
|
||||||
|
|
||||||
|
try {
|
||||||
|
execSync(`git -C ${dir} pull ${wikiUrl} main`, { env: gitEnv, stdio: 'pipe' });
|
||||||
|
} catch {
|
||||||
|
execSync(`git clone ${wikiUrl} ${dir}`, { env: gitEnv, stdio: 'pipe' });
|
||||||
|
}
|
||||||
|
|
||||||
|
const file = join(dir, `${page}.md`);
|
||||||
|
writeFileSync(file, content);
|
||||||
|
execSync(`git -C ${dir} add "${page}.md"`, { env: gitEnv, stdio: 'pipe' });
|
||||||
|
execSync(`git -C ${dir} commit -m "${message || `Update ${page}`}"`, { env: gitEnv, stdio: 'pipe' });
|
||||||
|
execSync(`git -C ${dir} push ${wikiUrl} main`, { env: gitEnv, stdio: 'pipe' });
|
||||||
|
return `${page} updated`;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function giteaListIssues(repo, state = 'open') {
|
||||||
|
return api(`/repos/${repo}/issues?state=${state}&type=issues&limit=50`).then(issues =>
|
||||||
|
issues.map(i => ({ number: i.number, title: i.title, state: i.state, labels: i.labels.map(l => l.name) }))
|
||||||
|
);
|
||||||
|
}
|
||||||
45
agap-mcp/src/homeassistant.js
Normal file
45
agap-mcp/src/homeassistant.js
Normal file
@@ -0,0 +1,45 @@
|
|||||||
|
const BASE = () => process.env.HA_URL || 'http://192.168.1.4:8123';
|
||||||
|
let _token = null;
|
||||||
|
|
||||||
|
export function initHA(token) {
|
||||||
|
_token = token;
|
||||||
|
console.log('Home Assistant: ready');
|
||||||
|
}
|
||||||
|
|
||||||
|
function token() {
|
||||||
|
if (!_token) throw new Error('HA not initialized');
|
||||||
|
return _token;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function api(path, opts = {}) {
|
||||||
|
const res = await fetch(`${BASE()}/api${path}`, {
|
||||||
|
...opts,
|
||||||
|
headers: { Authorization: `Bearer ${token()}`, 'Content-Type': 'application/json', ...opts.headers },
|
||||||
|
});
|
||||||
|
if (!res.ok) throw new Error(`HA API ${path}: ${res.status}`);
|
||||||
|
return res.json();
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function haGetState(entityId) {
|
||||||
|
const s = await api(`/states/${entityId}`);
|
||||||
|
return { entity_id: s.entity_id, state: s.state, attributes: s.attributes, last_changed: s.last_changed };
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function haListEntities(domain) {
|
||||||
|
const states = await api('/states');
|
||||||
|
const filtered = domain ? states.filter(s => s.entity_id.startsWith(`${domain}.`)) : states;
|
||||||
|
return filtered.map(s => ({ entity_id: s.entity_id, state: s.state, friendly_name: s.attributes.friendly_name }));
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function haCallService(domain, service, data = {}) {
|
||||||
|
return api(`/services/${domain}/${service}`, {
|
||||||
|
method: 'POST',
|
||||||
|
body: JSON.stringify(data),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function haGetHistory(entityId, hours = 24) {
|
||||||
|
const start = new Date(Date.now() - hours * 3600 * 1000).toISOString();
|
||||||
|
const data = await api(`/history/period/${start}?filter_entity_id=${entityId}&minimal_response=true`);
|
||||||
|
return data[0] || [];
|
||||||
|
}
|
||||||
147
agap-mcp/src/listener-auth.js
Normal file
147
agap-mcp/src/listener-auth.js
Normal file
@@ -0,0 +1,147 @@
|
|||||||
|
// listener-auth — kb#180: authenticate the agap-mcp :3100 listener itself,
|
||||||
|
// not just the vault tools.
|
||||||
|
//
|
||||||
|
// WHY THIS EXISTS
|
||||||
|
// agap-mcp runs `network_mode: host` and binds :3100 on every interface. The
|
||||||
|
// LAN is explicitly NOT a trust boundary here — DESIGN-a2a-agents.md v2.1 §4
|
||||||
|
// ("Auth is mandatory on every A2A surface. The LAN is not trusted — the
|
||||||
|
// xray/3x-ui VPN terminates other people's peers on it. No unauthenticated
|
||||||
|
// JSON-RPC listener, ever: shared tokens minimum, mTLS preferred.").
|
||||||
|
//
|
||||||
|
// Before this module, the ONLY gate in the process was requireVaultAccess()
|
||||||
|
// (kb#147), which covers vw_* tools alone and only at ENFORCE=1. Everything
|
||||||
|
// else — ha_call_service, gitea_wiki_write, wiki_edit, radicale_*/todoist_*
|
||||||
|
// writes, and the plain-REST POST /capture-idea — was callable by any LAN
|
||||||
|
// peer with curl. This module closes that: the transport itself now requires
|
||||||
|
// a bearer token that resolves to a known agent id.
|
||||||
|
//
|
||||||
|
// TWO DISTINCT GATES, DELIBERATELY LAYERED
|
||||||
|
// 1. listener auth (this file) — "are you *an* agent at all?" → any id in
|
||||||
|
// AGAP_MCP_AGENT_TOKENS passes; unknown/absent token = 401.
|
||||||
|
// 2. vault trust gate (trust-gate.js / requireVaultAccess in server.js)
|
||||||
|
// — "are you trust_class >= trusted?" → only then may vw_* run.
|
||||||
|
// Both read the SAME token map, so one token per agent covers both. Gate 2
|
||||||
|
// stays independently switchable (AGAP_MCP_ENFORCE_VAULT_TRUST) exactly as
|
||||||
|
// kb#147 shipped it; turning on gate 1 does not turn on gate 2.
|
||||||
|
//
|
||||||
|
// FAIL-FAST, NOT FAIL-SILENT
|
||||||
|
// Auth is ON by default (AGAP_MCP_REQUIRE_AUTH != '0'). If it is on and the
|
||||||
|
// token map is empty, assertListenerAuthConfig() throws at boot rather than
|
||||||
|
// letting the process serve 401 to literally everyone while /health says
|
||||||
|
// "ok" — a missing .env value must look like a broken restart, not like a
|
||||||
|
// quietly dead integration. See server.js boot path.
|
||||||
|
//
|
||||||
|
// SSE SESSION BINDING
|
||||||
|
// The legacy SSE transport hands out a sessionId at GET /sse and accepts JSON-RPC
|
||||||
|
// on POST /messages?sessionId=... . Previously /messages trusted ANY sessionId
|
||||||
|
// with no credential — a session-id guess/leak was full tool access (hijack).
|
||||||
|
// bindSseSession()/authorizeSseSession() below pin the caller identity captured
|
||||||
|
// at handshake to the session, and /messages must present the same agent's
|
||||||
|
// token or it is rejected.
|
||||||
|
|
||||||
|
import { resolveCallerAgent, authHeaderToken } from './trust-gate.js';
|
||||||
|
|
||||||
|
// ON unless explicitly disabled. The opposite default from kb#147's vault gate
|
||||||
|
// on purpose: an unauthenticated JSON-RPC listener is the thing §4 forbids
|
||||||
|
// outright, so "off" has to be a deliberate, visible opt-out.
|
||||||
|
export function requireAuthEnabled(env = process.env) {
|
||||||
|
return env.AGAP_MCP_REQUIRE_AUTH !== '0';
|
||||||
|
}
|
||||||
|
|
||||||
|
// Routes that stay open even with auth on:
|
||||||
|
// /health — the compose healthcheck calls it with no credential, it has no
|
||||||
|
// side effects, and it returns only counts/booleans (no secrets, no tool
|
||||||
|
// surface). Keeping it open is what lets a misconfigured token map still be
|
||||||
|
// diagnosable from outside the container.
|
||||||
|
export const PUBLIC_PATHS = new Set(['/health']);
|
||||||
|
|
||||||
|
export class ListenerAuthConfigError extends Error {}
|
||||||
|
|
||||||
|
// Called once at boot from server.js. Throws (crash loudly) instead of
|
||||||
|
// booting an all-callers-denied service.
|
||||||
|
export function assertListenerAuthConfig(tokenMap, env = process.env) {
|
||||||
|
if (!requireAuthEnabled(env)) {
|
||||||
|
console.error(
|
||||||
|
'\n*** agap-mcp WARNING: AGAP_MCP_REQUIRE_AUTH=0 — the :3100 MCP listener is ' +
|
||||||
|
'UNAUTHENTICATED. Every tool (ha_call_service, gitea_wiki_write, wiki_edit, ' +
|
||||||
|
'radicale/todoist writes, POST /capture-idea) is callable by any LAN peer, and ' +
|
||||||
|
'the LAN carries VPN-terminated peers. This violates DESIGN-a2a-agents.md §4 ' +
|
||||||
|
'and is only acceptable as a temporary, deliberate rollback (kb#180). ***\n'
|
||||||
|
);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (Object.keys(tokenMap || {}).length === 0) {
|
||||||
|
throw new ListenerAuthConfigError(
|
||||||
|
'AGAP_MCP_REQUIRE_AUTH is on (default) but AGAP_MCP_AGENT_TOKENS is empty/unset, ' +
|
||||||
|
'so no caller could ever authenticate. Populate AGAP_MCP_AGENT_TOKENS in this ' +
|
||||||
|
"container's .env with a JSON map {\"<bearer-token>\":\"<agent-id>\"} (agent ids " +
|
||||||
|
'must exist in agent-registry.yaml), or set AGAP_MCP_REQUIRE_AUTH=0 to ' +
|
||||||
|
'deliberately run the listener unauthenticated (kb#180, DESIGN-a2a-agents.md §4).'
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Express middleware factory. On success sets req.callerAgentId (string) and
|
||||||
|
// req.callerToken, which the /mcp, /sse and /messages routes consume.
|
||||||
|
// With auth disabled it sets req.callerAgentId from the token if one happens
|
||||||
|
// to be present (so the vault gate keeps working) and lets the request through.
|
||||||
|
export function listenerAuth(tokenMap, env = process.env) {
|
||||||
|
const enabled = requireAuthEnabled(env);
|
||||||
|
return function listenerAuthMiddleware(req, res, next) {
|
||||||
|
const token = authHeaderToken(req);
|
||||||
|
const agentId = resolveCallerAgent(token, tokenMap);
|
||||||
|
req.callerToken = token;
|
||||||
|
req.callerAgentId = agentId;
|
||||||
|
|
||||||
|
if (!enabled) return next();
|
||||||
|
if (PUBLIC_PATHS.has(req.path)) return next();
|
||||||
|
if (agentId) return next();
|
||||||
|
|
||||||
|
return denyUnauthenticated(req, res, token ? 'unknown-token' : 'no-credential');
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
// 401 body shape: JSON-RPC error for MCP routes (so an MCP client surfaces a
|
||||||
|
// real protocol error rather than a parse failure), plain JSON elsewhere.
|
||||||
|
export function denyUnauthenticated(req, res, reason) {
|
||||||
|
const message =
|
||||||
|
`unauthenticated: this endpoint requires an Authorization: Bearer <token> header ` +
|
||||||
|
`resolving to a known agent (kb#180, DESIGN-a2a-agents.md §4) [${reason}]`;
|
||||||
|
res.set('WWW-Authenticate', 'Bearer realm="agap-mcp"');
|
||||||
|
if (isJsonRpcPath(req.path)) {
|
||||||
|
return res.status(401).json({ jsonrpc: '2.0', error: { code: -32001, message }, id: null });
|
||||||
|
}
|
||||||
|
return res.status(401).json({ error: message });
|
||||||
|
}
|
||||||
|
|
||||||
|
export function isJsonRpcPath(path) {
|
||||||
|
return path === '/mcp' || path === '/messages' || path === '/sse';
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- SSE session binding -------------------------------------------------
|
||||||
|
// sessions: Map<sessionId, { transport, agentId, token }>
|
||||||
|
|
||||||
|
export function bindSseSession(sessions, sessionId, transport, req) {
|
||||||
|
sessions.set(sessionId, {
|
||||||
|
transport,
|
||||||
|
agentId: req.callerAgentId || null,
|
||||||
|
token: req.callerToken || null,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// Returns { ok: true, transport } or { ok: false, status, reason }.
|
||||||
|
// A /messages POST must (a) name a live session and (b) carry the SAME
|
||||||
|
// caller identity that opened it. Comparing the agent id (not just "is
|
||||||
|
// authenticated") is what stops agent B from driving agent A's session; the
|
||||||
|
// token is compared too so two tokens mapped to the same agent id are still
|
||||||
|
// treated as distinct sessions.
|
||||||
|
export function authorizeSseSession(sessions, sessionId, req, env = process.env) {
|
||||||
|
const entry = sessions.get(sessionId);
|
||||||
|
if (!entry) return { ok: false, status: 400, reason: 'unknown-session' };
|
||||||
|
if (!requireAuthEnabled(env)) return { ok: true, transport: entry.transport };
|
||||||
|
if (!req.callerAgentId) return { ok: false, status: 401, reason: 'no-credential' };
|
||||||
|
if (entry.agentId !== req.callerAgentId || entry.token !== req.callerToken) {
|
||||||
|
return { ok: false, status: 403, reason: 'session-caller-mismatch' };
|
||||||
|
}
|
||||||
|
return { ok: true, transport: entry.transport };
|
||||||
|
}
|
||||||
160
agap-mcp/src/mediawiki.js
Normal file
160
agap-mcp/src/mediawiki.js
Normal file
@@ -0,0 +1,160 @@
|
|||||||
|
// MediaWiki (РодоВики family wiki) tools for agap-mcp — kb#95.
|
||||||
|
// Target is family.alogins.net (МediaWiki), NOT Gitea. Follows the exact
|
||||||
|
// login->edit flow documented in the `rodowiki` skill: login token ->
|
||||||
|
// action=login -> session cookie -> CSRF token -> read/search/edit. Read
|
||||||
|
// operations also require login ($wgGroupPermissions['*']['read'] = false).
|
||||||
|
//
|
||||||
|
// Credentials: Vaultwarden org item "family.alogins.net" (username field
|
||||||
|
// holds the wiki account name "Claude", password field holds its password),
|
||||||
|
// fetched once at init like every other agap-mcp integration.
|
||||||
|
//
|
||||||
|
// Session handling: Node's fetch does not auto-manage cookies like curl's
|
||||||
|
// --cookie-jar, so a tiny module-level cookie jar (Map) is kept here and
|
||||||
|
// reused across calls -- this module is a singleton within the process, so
|
||||||
|
// one login session serves every MCP request for the process lifetime.
|
||||||
|
// withAuth() retries once on an auth-shaped failure (expired session) by
|
||||||
|
// forcing a fresh login, mirroring how a human re-running the skill's shell
|
||||||
|
// snippet would just log in again.
|
||||||
|
|
||||||
|
const BASE = () => (process.env.MEDIAWIKI_URL || 'http://localhost:8099').replace(/\/$/, '');
|
||||||
|
|
||||||
|
let _username = null;
|
||||||
|
let _password = null;
|
||||||
|
let _cookies = new Map();
|
||||||
|
let _loggedIn = false;
|
||||||
|
|
||||||
|
export function initMediaWiki(username, password) {
|
||||||
|
_username = username;
|
||||||
|
_password = password;
|
||||||
|
console.log('MediaWiki: ready');
|
||||||
|
}
|
||||||
|
|
||||||
|
function cookieHeader() {
|
||||||
|
return [..._cookies.entries()].map(([k, v]) => `${k}=${v}`).join('; ');
|
||||||
|
}
|
||||||
|
|
||||||
|
function storeCookies(res) {
|
||||||
|
const setCookies =
|
||||||
|
typeof res.headers.getSetCookie === 'function'
|
||||||
|
? res.headers.getSetCookie()
|
||||||
|
: res.headers.get('set-cookie')
|
||||||
|
? [res.headers.get('set-cookie')]
|
||||||
|
: [];
|
||||||
|
for (const sc of setCookies) {
|
||||||
|
const pair = sc.split(';')[0];
|
||||||
|
const idx = pair.indexOf('=');
|
||||||
|
if (idx > -1) _cookies.set(pair.slice(0, idx).trim(), pair.slice(idx + 1).trim());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function api(params, { method = 'GET' } = {}) {
|
||||||
|
const url = new URL(`${BASE()}/api.php`);
|
||||||
|
const headers = { Cookie: cookieHeader() };
|
||||||
|
let body;
|
||||||
|
if (method === 'GET') {
|
||||||
|
for (const [k, v] of Object.entries(params)) url.searchParams.set(k, v);
|
||||||
|
} else {
|
||||||
|
body = new URLSearchParams(params);
|
||||||
|
headers['Content-Type'] = 'application/x-www-form-urlencoded';
|
||||||
|
}
|
||||||
|
const res = await fetch(url, { method, headers, body });
|
||||||
|
storeCookies(res);
|
||||||
|
const data = await res.json();
|
||||||
|
if (data.error) throw new Error(`MediaWiki API error: ${data.error.info || JSON.stringify(data.error)}`);
|
||||||
|
return data;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function login() {
|
||||||
|
if (!_username || !_password) throw new Error('MediaWiki not initialized');
|
||||||
|
const tokenData = await api({ action: 'query', meta: 'tokens', type: 'login', format: 'json' });
|
||||||
|
const logintoken = tokenData.query.tokens.logintoken;
|
||||||
|
const loginData = await api(
|
||||||
|
{ action: 'login', lgname: _username, lgpassword: _password, lgtoken: logintoken, format: 'json' },
|
||||||
|
{ method: 'POST' }
|
||||||
|
);
|
||||||
|
if (loginData.login?.result !== 'Success') {
|
||||||
|
throw new Error(`MediaWiki login failed: ${loginData.login?.result || JSON.stringify(loginData)}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function withAuth(fn) {
|
||||||
|
if (!_loggedIn) {
|
||||||
|
await login();
|
||||||
|
_loggedIn = true;
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
return await fn();
|
||||||
|
} catch (e) {
|
||||||
|
// Session likely expired mid-process: force a fresh login and retry once.
|
||||||
|
if (/notloggedin|readapidenied|permissiondenied|mustbeloggedin/i.test(e.message)) {
|
||||||
|
_loggedIn = false;
|
||||||
|
await login();
|
||||||
|
_loggedIn = true;
|
||||||
|
return await fn();
|
||||||
|
}
|
||||||
|
throw e;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function stripHtml(s) {
|
||||||
|
return (s || '').replace(/<[^>]+>/g, '');
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function wikiSearch(query, limit = 20) {
|
||||||
|
if (!query || !query.trim()) throw new Error('query is required');
|
||||||
|
return withAuth(async () => {
|
||||||
|
const data = await api({
|
||||||
|
action: 'query',
|
||||||
|
list: 'search',
|
||||||
|
srsearch: query,
|
||||||
|
srlimit: String(limit),
|
||||||
|
format: 'json',
|
||||||
|
});
|
||||||
|
return data.query.search.map((p) => ({
|
||||||
|
title: p.title,
|
||||||
|
snippet: stripHtml(p.snippet),
|
||||||
|
wordcount: p.wordcount,
|
||||||
|
}));
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function wikiRead(title) {
|
||||||
|
if (!title || !title.trim()) throw new Error('title is required');
|
||||||
|
return withAuth(async () => {
|
||||||
|
const data = await api({
|
||||||
|
action: 'query',
|
||||||
|
titles: title,
|
||||||
|
prop: 'revisions',
|
||||||
|
rvprop: 'content',
|
||||||
|
rvslots: 'main',
|
||||||
|
format: 'json',
|
||||||
|
});
|
||||||
|
const page = Object.values(data.query.pages)[0];
|
||||||
|
if (page.missing !== undefined) throw new Error(`Page not found: ${title}`);
|
||||||
|
return page.revisions?.[0]?.slots?.main?.['*'] ?? '';
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function wikiEdit(title, text, summary) {
|
||||||
|
if (!title || !title.trim()) throw new Error('title is required');
|
||||||
|
if (text === undefined || text === null) throw new Error('text is required');
|
||||||
|
return withAuth(async () => {
|
||||||
|
const tokenData = await api({ action: 'query', meta: 'tokens', format: 'json' });
|
||||||
|
const csrftoken = tokenData.query.tokens.csrftoken;
|
||||||
|
const editData = await api(
|
||||||
|
{
|
||||||
|
action: 'edit',
|
||||||
|
title,
|
||||||
|
text,
|
||||||
|
summary: summary || `Update ${title}`,
|
||||||
|
token: csrftoken,
|
||||||
|
format: 'json',
|
||||||
|
},
|
||||||
|
{ method: 'POST' }
|
||||||
|
);
|
||||||
|
if (editData.edit?.result !== 'Success') {
|
||||||
|
throw new Error(`MediaWiki edit failed: ${JSON.stringify(editData.edit || editData)}`);
|
||||||
|
}
|
||||||
|
return { title, result: editData.edit.result, newrevid: editData.edit.newrevid, oldrevid: editData.edit.oldrevid };
|
||||||
|
});
|
||||||
|
}
|
||||||
198
agap-mcp/src/radicale.js
Normal file
198
agap-mcp/src/radicale.js
Normal file
@@ -0,0 +1,198 @@
|
|||||||
|
import { randomUUID } from 'crypto';
|
||||||
|
|
||||||
|
const BASE = () => (process.env.RADICALE_URL || 'http://localhost:5232').replace(/\/$/, '');
|
||||||
|
const DEFAULT_USER = () => process.env.RADICALE_USER || 'alvis';
|
||||||
|
|
||||||
|
let _password = null;
|
||||||
|
|
||||||
|
export function initRadicale(password, _user) {
|
||||||
|
_password = password;
|
||||||
|
console.log('Radicale: ready');
|
||||||
|
}
|
||||||
|
|
||||||
|
function authHeader(user) {
|
||||||
|
if (!_password) throw new Error('Radicale not initialized');
|
||||||
|
const u = user || DEFAULT_USER();
|
||||||
|
return 'Basic ' + Buffer.from(`${u}:${_password}`).toString('base64');
|
||||||
|
}
|
||||||
|
|
||||||
|
async function dav(method, path, { user, headers = {}, body } = {}) {
|
||||||
|
const res = await fetch(`${BASE()}${path}`, {
|
||||||
|
method,
|
||||||
|
headers: { Authorization: authHeader(user), ...headers },
|
||||||
|
body,
|
||||||
|
});
|
||||||
|
const text = await res.text();
|
||||||
|
if (!res.ok && res.status !== 207) {
|
||||||
|
throw new Error(`Radicale ${method} ${path}: ${res.status} ${text}`);
|
||||||
|
}
|
||||||
|
return { status: res.status, text, headers: res.headers };
|
||||||
|
}
|
||||||
|
|
||||||
|
function userPath(user) {
|
||||||
|
return `/${encodeURIComponent(user || DEFAULT_USER())}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Extract <response> blocks; for each, pull href + displayname + resourcetype components
|
||||||
|
function parseMultistatus(xml) {
|
||||||
|
const responses = [];
|
||||||
|
const re = /<(?:[a-zA-Z0-9]+:)?response\b[^>]*>([\s\S]*?)<\/(?:[a-zA-Z0-9]+:)?response>/g;
|
||||||
|
let m;
|
||||||
|
while ((m = re.exec(xml)) !== null) {
|
||||||
|
const block = m[1];
|
||||||
|
const href = (block.match(/<(?:[a-zA-Z0-9]+:)?href\b[^>]*>([\s\S]*?)<\/(?:[a-zA-Z0-9]+:)?href>/) || [])[1];
|
||||||
|
const displayname = (block.match(/<(?:[a-zA-Z0-9]+:)?displayname\b[^>]*>([\s\S]*?)<\/(?:[a-zA-Z0-9]+:)?displayname>/) || [])[1];
|
||||||
|
const isCalendar = /<(?:[a-zA-Z0-9]+:)?calendar\b/.test(block);
|
||||||
|
const isAddressbook = /<(?:[a-zA-Z0-9]+:)?addressbook\b/.test(block);
|
||||||
|
const isCollection = /<(?:[a-zA-Z0-9]+:)?collection\b/.test(block);
|
||||||
|
const getetag = (block.match(/<(?:[a-zA-Z0-9]+:)?getetag\b[^>]*>([\s\S]*?)<\/(?:[a-zA-Z0-9]+:)?getetag>/) || [])[1];
|
||||||
|
const componentSet = [];
|
||||||
|
const csRe = /<(?:[a-zA-Z0-9]+:)?comp\b[^>]*\sname="([^"]+)"/g;
|
||||||
|
let cm;
|
||||||
|
while ((cm = csRe.exec(block)) !== null) componentSet.push(cm[1]);
|
||||||
|
responses.push({ href, displayname, isCalendar, isAddressbook, isCollection, getetag, componentSet });
|
||||||
|
}
|
||||||
|
return responses;
|
||||||
|
}
|
||||||
|
|
||||||
|
const PROPFIND_COLLECTIONS = `<?xml version="1.0" encoding="utf-8"?>
|
||||||
|
<d:propfind xmlns:d="DAV:" xmlns:c="urn:ietf:params:xml:ns:caldav" xmlns:ic="http://apple.com/ns/ical/">
|
||||||
|
<d:prop>
|
||||||
|
<d:displayname/>
|
||||||
|
<d:resourcetype/>
|
||||||
|
<c:supported-calendar-component-set/>
|
||||||
|
<ic:calendar-color/>
|
||||||
|
</d:prop>
|
||||||
|
</d:propfind>`;
|
||||||
|
|
||||||
|
const PROPFIND_EVENTS = `<?xml version="1.0" encoding="utf-8"?>
|
||||||
|
<d:propfind xmlns:d="DAV:">
|
||||||
|
<d:prop>
|
||||||
|
<d:getetag/>
|
||||||
|
<d:resourcetype/>
|
||||||
|
</d:prop>
|
||||||
|
</d:propfind>`;
|
||||||
|
|
||||||
|
export async function radicaleListCalendars(user) {
|
||||||
|
const { text } = await dav('PROPFIND', `${userPath(user)}/`, {
|
||||||
|
user,
|
||||||
|
headers: { Depth: '1', 'Content-Type': 'application/xml' },
|
||||||
|
body: PROPFIND_COLLECTIONS,
|
||||||
|
});
|
||||||
|
const items = parseMultistatus(text);
|
||||||
|
const out = [];
|
||||||
|
for (const r of items) {
|
||||||
|
if (!r.href || !r.isCalendar) continue;
|
||||||
|
// skip the parent (the user principal itself, which usually has no displayname)
|
||||||
|
const id = decodeURIComponent(r.href.replace(/\/$/, '').split('/').pop());
|
||||||
|
if (!id || id === (user || DEFAULT_USER())) continue;
|
||||||
|
out.push({
|
||||||
|
id,
|
||||||
|
displayname: r.displayname || null,
|
||||||
|
components: r.componentSet,
|
||||||
|
href: r.href,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseIcsField(ics, field, { component = 'VEVENT' } = {}) {
|
||||||
|
// restrict search to the named component block (default VEVENT)
|
||||||
|
const block = (() => {
|
||||||
|
if (!component) return ics;
|
||||||
|
const re = new RegExp(`BEGIN:${component}([\\s\\S]*?)END:${component}`);
|
||||||
|
const m = ics.match(re);
|
||||||
|
return m ? m[1] : ics;
|
||||||
|
})();
|
||||||
|
const re = new RegExp(`(?:^|\\n)${field}(?:;[^:\\n]*)?:([^\\n\\r]*)`);
|
||||||
|
const m = block.match(re);
|
||||||
|
return m ? m[1].trim() : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function radicaleListEvents(calendarId, user) {
|
||||||
|
const path = `${userPath(user)}/${encodeURIComponent(calendarId)}/`;
|
||||||
|
const { text } = await dav('PROPFIND', path, {
|
||||||
|
user,
|
||||||
|
headers: { Depth: '1', 'Content-Type': 'application/xml' },
|
||||||
|
body: PROPFIND_EVENTS,
|
||||||
|
});
|
||||||
|
const items = parseMultistatus(text);
|
||||||
|
const out = [];
|
||||||
|
for (const r of items) {
|
||||||
|
if (!r.href || !r.href.endsWith('.ics')) continue;
|
||||||
|
const filename = decodeURIComponent(r.href.split('/').pop());
|
||||||
|
// GET event to extract summary + start time
|
||||||
|
const ev = await dav('GET', r.href, { user });
|
||||||
|
const summary = parseIcsField(ev.text, 'SUMMARY');
|
||||||
|
const dtstart = parseIcsField(ev.text, 'DTSTART');
|
||||||
|
const dtend = parseIcsField(ev.text, 'DTEND');
|
||||||
|
const uid = parseIcsField(ev.text, 'UID');
|
||||||
|
out.push({ filename, uid, summary, dtstart, dtend, etag: r.getetag, href: r.href });
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function radicaleGetEvent(calendarId, filename, user) {
|
||||||
|
const path = `${userPath(user)}/${encodeURIComponent(calendarId)}/${encodeURIComponent(filename)}`;
|
||||||
|
const { text } = await dav('GET', path, { user });
|
||||||
|
return text;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function radicaleCreateCalendar({ displayname, color, components = ['VEVENT'], user, id }) {
|
||||||
|
if (!displayname) throw new Error('displayname required');
|
||||||
|
const calId = id || randomUUID();
|
||||||
|
const compXml = components.map(c => `<c:comp name="${c}"/>`).join('');
|
||||||
|
const body = `<?xml version="1.0" encoding="utf-8"?>
|
||||||
|
<c:mkcalendar xmlns:d="DAV:" xmlns:c="urn:ietf:params:xml:ns:caldav" xmlns:ic="http://apple.com/ns/ical/">
|
||||||
|
<d:set>
|
||||||
|
<d:prop>
|
||||||
|
<d:displayname>${escapeXml(displayname)}</d:displayname>
|
||||||
|
<c:supported-calendar-component-set>${compXml}</c:supported-calendar-component-set>
|
||||||
|
${color ? `<ic:calendar-color>${escapeXml(color)}</ic:calendar-color>` : ''}
|
||||||
|
</d:prop>
|
||||||
|
</d:set>
|
||||||
|
</c:mkcalendar>`;
|
||||||
|
const path = `${userPath(user)}/${calId}/`;
|
||||||
|
await dav('MKCALENDAR', path, {
|
||||||
|
user,
|
||||||
|
headers: { 'Content-Type': 'application/xml' },
|
||||||
|
body,
|
||||||
|
});
|
||||||
|
return { id: calId, displayname, components };
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function radicaleDeleteCalendar(calendarId, user) {
|
||||||
|
await dav('DELETE', `${userPath(user)}/${encodeURIComponent(calendarId)}/`, { user });
|
||||||
|
return { deleted: calendarId };
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function radicalePutEvent(calendarId, filename, ics, user) {
|
||||||
|
const name = filename.endsWith('.ics') ? filename : `${filename}.ics`;
|
||||||
|
const path = `${userPath(user)}/${encodeURIComponent(calendarId)}/${encodeURIComponent(name)}`;
|
||||||
|
await dav('PUT', path, {
|
||||||
|
user,
|
||||||
|
headers: { 'Content-Type': 'text/calendar; charset=utf-8' },
|
||||||
|
body: ics,
|
||||||
|
});
|
||||||
|
return { put: name };
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function radicaleDeleteEvent(calendarId, filename, user) {
|
||||||
|
const path = `${userPath(user)}/${encodeURIComponent(calendarId)}/${encodeURIComponent(filename)}`;
|
||||||
|
await dav('DELETE', path, { user });
|
||||||
|
return { deleted: filename };
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function radicaleMoveEvent({ fromCalendar, toCalendar, filename, user }) {
|
||||||
|
if (!fromCalendar || !toCalendar || !filename) {
|
||||||
|
throw new Error('fromCalendar, toCalendar, filename required');
|
||||||
|
}
|
||||||
|
const ics = await radicaleGetEvent(fromCalendar, filename, user);
|
||||||
|
await radicalePutEvent(toCalendar, filename, ics, user);
|
||||||
|
await radicaleDeleteEvent(fromCalendar, filename, user);
|
||||||
|
return { moved: filename, from: fromCalendar, to: toCalendar };
|
||||||
|
}
|
||||||
|
|
||||||
|
function escapeXml(s) {
|
||||||
|
return String(s).replace(/[<>&'"]/g, c => ({ '<': '<', '>': '>', '&': '&', "'": ''', '"': '"' }[c]));
|
||||||
|
}
|
||||||
513
agap-mcp/src/server.js
Normal file
513
agap-mcp/src/server.js
Normal file
@@ -0,0 +1,513 @@
|
|||||||
|
import express from 'express';
|
||||||
|
import { randomUUID } from 'crypto';
|
||||||
|
import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
|
||||||
|
import { SSEServerTransport } from '@modelcontextprotocol/sdk/server/sse.js';
|
||||||
|
import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js';
|
||||||
|
import { isInitializeRequest } from '@modelcontextprotocol/sdk/types.js';
|
||||||
|
import { z } from 'zod';
|
||||||
|
|
||||||
|
import { initVaultwarden, vwGetPassword, vwGetItem, vwListItems, vwListOrgItems, vwCreateLogin, vwUpdatePassword } from './vaultwarden.js';
|
||||||
|
import { initGitea, giteaListRepos, giteaReadFile, giteaWikiList, giteaWikiRead, giteaWikiWrite, giteaListIssues } from './gitea.js';
|
||||||
|
import { initHA, haGetState, haListEntities, haCallService, haGetHistory } from './homeassistant.js';
|
||||||
|
import { initZabbix, zabbixGetProblems, zabbixGetHosts, zabbixGetItems, zabbixGetTriggers } from './zabbix.js';
|
||||||
|
import { initRadicale, radicaleListCalendars, radicaleListEvents, radicaleGetEvent, radicaleCreateCalendar, radicaleDeleteCalendar, radicalePutEvent, radicaleDeleteEvent, radicaleMoveEvent } from './radicale.js';
|
||||||
|
import { initTodoist, todoistListTasks, todoistListProjects, todoistCreateTask, todoistUpdateTask, todoistCompleteTask } from './todoist.js';
|
||||||
|
import { todoistCaptureIdea } from './capture.js';
|
||||||
|
import { initMediaWiki, wikiSearch, wikiRead, wikiEdit } from './mediawiki.js';
|
||||||
|
import { loadTokenMap, resolveCallerAgent, vaultAllowed, authHeaderToken } from './trust-gate.js';
|
||||||
|
import { listenerAuth, assertListenerAuthConfig, requireAuthEnabled, bindSseSession, authorizeSseSession } from './listener-auth.js';
|
||||||
|
|
||||||
|
const PORT = parseInt(process.env.PORT || '3100');
|
||||||
|
|
||||||
|
// --- kb#147: vault trust gate (A2A-15) ---------------------------------
|
||||||
|
// OFF by default (ENFORCE_VAULT_TRUST=false) so merging this file changes
|
||||||
|
// NOTHING about the live service's behavior until an operator deliberately
|
||||||
|
// sets AGAP_MCP_ENFORCE_VAULT_TRUST=1 AND populates AGAP_MCP_AGENT_TOKENS
|
||||||
|
// with real per-agent bearer tokens (stored in Vaultwarden, injected via
|
||||||
|
// this container's .env — never committed to git). That two-step
|
||||||
|
// activation is the kb#147 handover: it requires a docker-compose env
|
||||||
|
// change + container restart, which this task deliberately does not do
|
||||||
|
// (see DESIGN-a2a-agents.md v2.1 §5 — vault access = trusted only).
|
||||||
|
const ENFORCE_VAULT_TRUST = process.env.AGAP_MCP_ENFORCE_VAULT_TRUST === '1';
|
||||||
|
const AGENT_TOKENS = loadTokenMap();
|
||||||
|
const AGENT_TOKEN_COUNT = Object.keys(AGENT_TOKENS).length;
|
||||||
|
|
||||||
|
// kb#182: ENFORCE=1 with an empty token map is fail-closed by design (see
|
||||||
|
// requireVaultAccess below) but that means it silently denies EVERY caller,
|
||||||
|
// including Adolf itself — a self-inflicted vault brick with no signal
|
||||||
|
// unless someone is watching the logs. Make that state loud on boot.
|
||||||
|
if (ENFORCE_VAULT_TRUST && AGENT_TOKEN_COUNT === 0) {
|
||||||
|
console.error(
|
||||||
|
'\n*** agap-mcp WARNING: AGAP_MCP_ENFORCE_VAULT_TRUST=1 but AGAP_MCP_AGENT_TOKENS ' +
|
||||||
|
'is empty/unset. Every caller — including Adolf — will be denied vault access. ' +
|
||||||
|
'This is fail-closed, not a crash: the process will keep serving non-vault tools, ' +
|
||||||
|
'but ALL vw_* calls will error until AGAP_MCP_AGENT_TOKENS is populated with real ' +
|
||||||
|
'per-agent bearer tokens (kb#147/kb#182). Check /health for tokenMapSize. ***\n'
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
function requireVaultAccess(callerAgentId) {
|
||||||
|
if (!ENFORCE_VAULT_TRUST) return; // legacy behavior: unchanged until activated
|
||||||
|
if (!vaultAllowed(callerAgentId)) {
|
||||||
|
throw new Error(
|
||||||
|
`vault access denied: caller ${callerAgentId ? `'${callerAgentId}'` : '(unauthenticated)'} ` +
|
||||||
|
`is not trust_class >= trusted (kb#147, DESIGN-a2a-agents.md §5)`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- Init services ---
|
||||||
|
async function init() {
|
||||||
|
await initVaultwarden();
|
||||||
|
|
||||||
|
// Fetch all tokens from org to avoid "more than one result" on duplicates
|
||||||
|
const orgItems = vwListOrgItems();
|
||||||
|
const orgToken = (name) => {
|
||||||
|
const item = orgItems.find(i => i.name === name);
|
||||||
|
if (!item) throw new Error(`Token not found in org: ${name}`);
|
||||||
|
return item.login?.password;
|
||||||
|
};
|
||||||
|
|
||||||
|
const gitea_token = orgToken('GITEA_TOKEN');
|
||||||
|
const ha_token = orgToken('HA_TOKEN');
|
||||||
|
const zabbix_token = orgToken('ZABBIX_TOKEN');
|
||||||
|
const radicale_password = orgToken('RADICALE_PASSWORD');
|
||||||
|
const todoist_token = orgToken('TODOIST_TOKEN');
|
||||||
|
|
||||||
|
// family.alogins.net (РодоВики MediaWiki) — a login item, not a bare
|
||||||
|
// token: username + password both needed for the login flow.
|
||||||
|
const wikiItem = orgItems.find(i => i.name === 'family.alogins.net');
|
||||||
|
if (!wikiItem) throw new Error('Token not found in org: family.alogins.net');
|
||||||
|
|
||||||
|
initGitea(gitea_token);
|
||||||
|
initHA(ha_token);
|
||||||
|
initZabbix(zabbix_token);
|
||||||
|
initRadicale(radicale_password);
|
||||||
|
initTodoist(todoist_token);
|
||||||
|
initMediaWiki(wikiItem.login?.username, wikiItem.login?.password);
|
||||||
|
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- MCP Server factory (one per session — McpServer can't share transports) ---
|
||||||
|
function ok(text) {
|
||||||
|
return { content: [{ type: 'text', text: typeof text === 'string' ? text : JSON.stringify(text, null, 2) }] };
|
||||||
|
}
|
||||||
|
|
||||||
|
function err(e) {
|
||||||
|
return { content: [{ type: 'text', text: `Error: ${e.message}` }], isError: true };
|
||||||
|
}
|
||||||
|
|
||||||
|
// Track tool registrations for /health endpoint
|
||||||
|
let registeredToolCount = 0;
|
||||||
|
|
||||||
|
function createServer(callerAgentId = null) {
|
||||||
|
const server = new McpServer({ name: 'agap-mcp', version: '1.0.0' });
|
||||||
|
const serverToolCount = { count: 0 };
|
||||||
|
|
||||||
|
// Wrap server.tool() to count registrations
|
||||||
|
const originalTool = server.tool.bind(server);
|
||||||
|
server.tool = function(name, description, params, handler) {
|
||||||
|
serverToolCount.count++;
|
||||||
|
return originalTool(name, description, params, handler);
|
||||||
|
};
|
||||||
|
|
||||||
|
// --- Vaultwarden tools (kb#147: gated to trust_class >= trusted) ---
|
||||||
|
server.tool('vw_get_password', 'Get password for a Vaultwarden item by name', { name: z.string() },
|
||||||
|
async ({ name }) => {
|
||||||
|
try { requireVaultAccess(callerAgentId); return ok(vwGetPassword(name)); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('vw_get_item', 'Get full details of a Vaultwarden item (name, username, password, url, notes)', { name: z.string() },
|
||||||
|
async ({ name }) => {
|
||||||
|
try {
|
||||||
|
requireVaultAccess(callerAgentId);
|
||||||
|
const item = vwGetItem(name);
|
||||||
|
return ok({ name: item.name, username: item.login?.username, password: item.login?.password, url: item.login?.uris?.[0]?.uri, notes: item.notes });
|
||||||
|
} catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('vw_list_items', 'List Vaultwarden items. Searches personal vault by default; set org=true to search org/AI collection', {
|
||||||
|
search: z.string().optional(),
|
||||||
|
org: z.boolean().optional(),
|
||||||
|
}, async ({ search, org }) => {
|
||||||
|
try {
|
||||||
|
requireVaultAccess(callerAgentId);
|
||||||
|
const items = org ? vwListOrgItems(search) : vwListItems(search);
|
||||||
|
return ok(items.map(i => ({ id: i.id, name: i.name, username: i.login?.username, url: i.login?.uris?.[0]?.uri })));
|
||||||
|
} catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('vw_create_login', 'Create a new login item in Vaultwarden AI collection', {
|
||||||
|
name: z.string(),
|
||||||
|
username: z.string().optional(),
|
||||||
|
password: z.string(),
|
||||||
|
url: z.string().optional(),
|
||||||
|
notes: z.string().optional(),
|
||||||
|
}, async (args) => {
|
||||||
|
try { requireVaultAccess(callerAgentId); return ok(vwCreateLogin(args)); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('vw_update_password', 'Update the password of an existing Vaultwarden item', {
|
||||||
|
name: z.string().describe('Item name or ID'),
|
||||||
|
password: z.string(),
|
||||||
|
}, async ({ name, password }) => {
|
||||||
|
try { requireVaultAccess(callerAgentId); return ok(vwUpdatePassword(name, password)); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
// --- Gitea tools ---
|
||||||
|
server.tool('gitea_list_repos', 'List all Gitea repositories', {},
|
||||||
|
async () => {
|
||||||
|
try { return ok(await giteaListRepos()); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('gitea_read_file', 'Read a file from a Gitea repository', {
|
||||||
|
repo: z.string().describe('e.g. alvis/AgapHost'),
|
||||||
|
path: z.string().describe('file path in repo'),
|
||||||
|
ref: z.string().optional().describe('branch/tag/commit, default HEAD'),
|
||||||
|
}, async ({ repo, path, ref }) => {
|
||||||
|
try { return ok(await giteaReadFile(repo, path, ref)); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('gitea_wiki_list', 'List wiki pages', {
|
||||||
|
repo: z.string().optional().describe('default: alvis/AgapHost'),
|
||||||
|
}, async ({ repo }) => {
|
||||||
|
try { return ok(await giteaWikiList(repo)); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('gitea_wiki_read', 'Read a wiki page', {
|
||||||
|
page: z.string(),
|
||||||
|
repo: z.string().optional().describe('default: alvis/AgapHost'),
|
||||||
|
}, async ({ page, repo }) => {
|
||||||
|
try { return ok(await giteaWikiRead(page, repo)); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('gitea_wiki_write', 'Write/update a wiki page', {
|
||||||
|
page: z.string(),
|
||||||
|
content: z.string(),
|
||||||
|
message: z.string().optional().describe('commit message'),
|
||||||
|
repo: z.string().optional().describe('default: alvis/AgapHost'),
|
||||||
|
}, async ({ page, content, message, repo }) => {
|
||||||
|
try { return ok(await giteaWikiWrite(page, content, message, repo)); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('gitea_list_issues', 'List issues for a Gitea repo', {
|
||||||
|
repo: z.string().describe('e.g. alvis/AgapHost'),
|
||||||
|
state: z.enum(['open', 'closed', 'all']).optional(),
|
||||||
|
}, async ({ repo, state }) => {
|
||||||
|
try { return ok(await giteaListIssues(repo, state)); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
// --- Home Assistant tools ---
|
||||||
|
server.tool('ha_get_state', 'Get current state of a Home Assistant entity', {
|
||||||
|
entity_id: z.string().describe('e.g. light.living_room'),
|
||||||
|
}, async ({ entity_id }) => {
|
||||||
|
try { return ok(await haGetState(entity_id)); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('ha_list_entities', 'List Home Assistant entities, optionally filtered by domain', {
|
||||||
|
domain: z.string().optional().describe('e.g. light, switch, sensor, binary_sensor'),
|
||||||
|
}, async ({ domain }) => {
|
||||||
|
try { return ok(await haListEntities(domain)); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('ha_call_service', 'Call a Home Assistant service', {
|
||||||
|
domain: z.string().describe('e.g. light, switch, automation'),
|
||||||
|
service: z.string().describe('e.g. turn_on, turn_off, toggle'),
|
||||||
|
data: z.record(z.unknown()).optional().describe('service call data, e.g. {"entity_id": "light.x"}'),
|
||||||
|
}, async ({ domain, service, data }) => {
|
||||||
|
try { return ok(await haCallService(domain, service, data)); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('ha_get_history', 'Get state history for a Home Assistant entity', {
|
||||||
|
entity_id: z.string(),
|
||||||
|
hours: z.number().optional().describe('hours of history, default 24'),
|
||||||
|
}, async ({ entity_id, hours }) => {
|
||||||
|
try { return ok(await haGetHistory(entity_id, hours)); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
// --- Zabbix tools ---
|
||||||
|
server.tool('zabbix_get_problems', 'Get current active problems in Zabbix', {},
|
||||||
|
async () => {
|
||||||
|
try { return ok(await zabbixGetProblems()); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('zabbix_get_hosts', 'List all monitored hosts in Zabbix with availability status', {},
|
||||||
|
async () => {
|
||||||
|
try { return ok(await zabbixGetHosts()); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('zabbix_get_items', 'Get monitored items and their latest values for a Zabbix host', {
|
||||||
|
hostid: z.string().describe('Zabbix host ID'),
|
||||||
|
}, async ({ hostid }) => {
|
||||||
|
try { return ok(await zabbixGetItems(hostid)); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('zabbix_get_triggers', 'Get triggers for a Zabbix host or all hosts', {
|
||||||
|
hostid: z.string().optional().describe('Zabbix host ID, omit for all hosts'),
|
||||||
|
}, async ({ hostid }) => {
|
||||||
|
try { return ok(await zabbixGetTriggers(hostid)); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
// --- Radicale (CalDAV) tools ---
|
||||||
|
server.tool('radicale_list_calendars', 'List CalDAV calendars for a Radicale user (defaults to alvis)', {
|
||||||
|
user: z.string().optional(),
|
||||||
|
}, async ({ user }) => {
|
||||||
|
try { return ok(await radicaleListCalendars(user)); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('radicale_list_events', 'List events in a Radicale calendar (returns filename, uid, summary, dtstart, dtend)', {
|
||||||
|
calendar_id: z.string().describe('Calendar UUID from radicale_list_calendars'),
|
||||||
|
user: z.string().optional(),
|
||||||
|
}, async ({ calendar_id, user }) => {
|
||||||
|
try { return ok(await radicaleListEvents(calendar_id, user)); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('radicale_get_event', 'Get raw iCalendar text for a single event', {
|
||||||
|
calendar_id: z.string(),
|
||||||
|
filename: z.string().describe('e.g. ABCD-1234.ics'),
|
||||||
|
user: z.string().optional(),
|
||||||
|
}, async ({ calendar_id, filename, user }) => {
|
||||||
|
try { return ok(await radicaleGetEvent(calendar_id, filename, user)); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('radicale_create_calendar', 'Create a new CalDAV calendar (MKCALENDAR). Returns the new calendar id.', {
|
||||||
|
displayname: z.string(),
|
||||||
|
color: z.string().optional().describe('e.g. #33CC66ff'),
|
||||||
|
components: z.array(z.string()).optional().describe('default ["VEVENT"]'),
|
||||||
|
id: z.string().optional().describe('explicit collection id; otherwise UUID generated'),
|
||||||
|
user: z.string().optional(),
|
||||||
|
}, async (args) => {
|
||||||
|
try { return ok(await radicaleCreateCalendar(args)); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('radicale_delete_calendar', 'Delete a Radicale calendar collection', {
|
||||||
|
calendar_id: z.string(),
|
||||||
|
user: z.string().optional(),
|
||||||
|
}, async ({ calendar_id, user }) => {
|
||||||
|
try { return ok(await radicaleDeleteCalendar(calendar_id, user)); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('radicale_put_event', 'PUT an iCalendar event into a calendar (create or replace)', {
|
||||||
|
calendar_id: z.string(),
|
||||||
|
filename: z.string().describe('Event filename, with or without .ics'),
|
||||||
|
ics: z.string().describe('Full VCALENDAR body'),
|
||||||
|
user: z.string().optional(),
|
||||||
|
}, async ({ calendar_id, filename, ics, user }) => {
|
||||||
|
try { return ok(await radicalePutEvent(calendar_id, filename, ics, user)); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('radicale_delete_event', 'Delete an event from a calendar', {
|
||||||
|
calendar_id: z.string(),
|
||||||
|
filename: z.string(),
|
||||||
|
user: z.string().optional(),
|
||||||
|
}, async ({ calendar_id, filename, user }) => {
|
||||||
|
try { return ok(await radicaleDeleteEvent(calendar_id, filename, user)); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('radicale_move_event', 'Move an event between calendars (copy then delete original)', {
|
||||||
|
from_calendar: z.string(),
|
||||||
|
to_calendar: z.string(),
|
||||||
|
filename: z.string(),
|
||||||
|
user: z.string().optional(),
|
||||||
|
}, async ({ from_calendar, to_calendar, filename, user }) => {
|
||||||
|
try { return ok(await radicaleMoveEvent({ fromCalendar: from_calendar, toCalendar: to_calendar, filename, user })); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
// --- Todoist tools (Todoist API v1) ---
|
||||||
|
server.tool('todoist_list_tasks', 'List active Todoist tasks. Use `query` for a Todoist filter (e.g. "today", "overdue", "#Work & p1"), or `project_id` to scope to one project. Returns compact tasks.', {
|
||||||
|
query: z.string().optional().describe('Todoist filter query, e.g. "today", "overdue", "#Project & p1". Omit for all active tasks.'),
|
||||||
|
project_id: z.string().optional().describe('Restrict to a project id (from todoist_list_projects). Ignored when query is set.'),
|
||||||
|
}, async ({ query, project_id }) => {
|
||||||
|
try { return ok(await todoistListTasks({ query, project_id })); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('todoist_list_projects', 'List Todoist projects (id + name).', {}, async () => {
|
||||||
|
try { return ok(await todoistListProjects()); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('todoist_create_task', 'Create a Todoist task. `due_string` is natural language ("tomorrow 9am", "every monday"). `priority`: API 1=normal..4=urgent (note: Todoist UI p1 = API 4). Defaults to the Inbox project.', {
|
||||||
|
content: z.string().describe('Task title/content (required).'),
|
||||||
|
description: z.string().optional(),
|
||||||
|
due_string: z.string().optional().describe('Natural-language due date, e.g. "today", "tomorrow 9am", "next monday".'),
|
||||||
|
priority: z.number().int().min(1).max(4).optional().describe('API priority 1=normal .. 4=urgent.'),
|
||||||
|
project_id: z.string().optional().describe('Target project id (default Inbox).'),
|
||||||
|
labels: z.array(z.string()).optional(),
|
||||||
|
}, async (args) => {
|
||||||
|
try { return ok(await todoistCreateTask(args)); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('todoist_update_task', 'Update a Todoist task (reschedule/edit). Give the task id plus the fields to change.', {
|
||||||
|
id: z.string(),
|
||||||
|
content: z.string().optional(),
|
||||||
|
description: z.string().optional(),
|
||||||
|
due_string: z.string().optional(),
|
||||||
|
priority: z.number().int().min(1).max(4).optional(),
|
||||||
|
labels: z.array(z.string()).optional(),
|
||||||
|
}, async (args) => {
|
||||||
|
try { return ok(await todoistUpdateTask(args)); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('todoist_complete_task', 'Complete (close) a Todoist task by id.', {
|
||||||
|
id: z.string().describe('Task id from todoist_list_tasks.'),
|
||||||
|
}, async ({ id }) => {
|
||||||
|
try { return ok(await todoistCompleteTask({ id })); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
// kb#170: capture an idea/quick task with lightweight (encoder-only, no
|
||||||
|
// Kimi/gemma call) AI classification -- area/urgency/decompose-need --
|
||||||
|
// then create it in Todoist tagged with the result. See capture.js header
|
||||||
|
// for why this tags with labels rather than reassigning projects.
|
||||||
|
server.tool('todoist_capture_idea', 'Capture a free-text idea: classify it (area: adolf/welfare/дом/семья/здоровье, urgency, whether it needs Kanboard decomposition) using local bge-m3 embeddings, then create it as a labelled Todoist task. No LLM call.', {
|
||||||
|
text: z.string().describe('The idea, in free text (Russian or English).'),
|
||||||
|
project_id: z.string().optional().describe('Force a specific Todoist project id; otherwise auto-routed only for an unambiguous семья match, else Inbox.'),
|
||||||
|
}, async ({ text, project_id }) => {
|
||||||
|
try { return ok(await todoistCaptureIdea({ text, project_id }, { listProjects: todoistListProjects })); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
// --- MediaWiki (family wiki / РодоВики) tools ---
|
||||||
|
server.tool('wiki_search', 'Search the family wiki (РодоВики) for pages matching a query. Returns title + snippet.', {
|
||||||
|
query: z.string().describe('Search text, e.g. a person\'s name or event.'),
|
||||||
|
limit: z.number().int().min(1).max(50).optional().describe('Max results, default 20.'),
|
||||||
|
}, async ({ query, limit }) => {
|
||||||
|
try { return ok(await wikiSearch(query, limit)); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('wiki_read', 'Read the raw wikitext content of a family wiki page by exact title.', {
|
||||||
|
title: z.string().describe('Exact page title, e.g. "Антон Логинс".'),
|
||||||
|
}, async ({ title }) => {
|
||||||
|
try { return ok(await wikiRead(title)); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
server.tool('wiki_edit', 'Create or update a family wiki page. Overwrites the page with the given wikitext.', {
|
||||||
|
title: z.string().describe('Exact page title to create/update.'),
|
||||||
|
text: z.string().describe('Full wikitext content of the page.'),
|
||||||
|
summary: z.string().optional().describe('Edit summary, default "Update <title>".'),
|
||||||
|
}, async ({ title, text, summary }) => {
|
||||||
|
try { return ok(await wikiEdit(title, text, summary)); } catch (e) { return err(e); }
|
||||||
|
});
|
||||||
|
|
||||||
|
// Store tool count on the server for /health endpoint to access
|
||||||
|
server._toolCount = serverToolCount.count;
|
||||||
|
registeredToolCount = serverToolCount.count;
|
||||||
|
|
||||||
|
return server;
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- HTTP server (Streamable HTTP + legacy SSE) ---
|
||||||
|
const app = express();
|
||||||
|
app.use(express.json());
|
||||||
|
|
||||||
|
// kb#180: authenticate the LISTENER, not just vault tools. Mounted before
|
||||||
|
// every route below, so /mcp, /sse, /messages and /capture-idea all require
|
||||||
|
// a bearer token resolving to a known agent (/health stays open — see
|
||||||
|
// listener-auth.js PUBLIC_PATHS). This middleware also populates
|
||||||
|
// req.callerAgentId, which replaces the per-route resolveCallerAgent() call
|
||||||
|
// the /mcp and /sse handlers used to do inline; the kb#147 vault gate then
|
||||||
|
// consumes that same id, so one token per agent serves both gates.
|
||||||
|
app.use(listenerAuth(AGENT_TOKENS));
|
||||||
|
|
||||||
|
// sessionId -> { transport, agentId, token } (kb#180: the identity captured at
|
||||||
|
// the /sse handshake is pinned to the session so /messages can't be hijacked
|
||||||
|
// by anyone who merely learns/guesses the sessionId).
|
||||||
|
const sseTransports = new Map();
|
||||||
|
|
||||||
|
// Streamable HTTP — stateless: fresh server per request, survives container restarts
|
||||||
|
app.all('/mcp', async (req, res) => {
|
||||||
|
try {
|
||||||
|
const callerAgentId = req.callerAgentId ?? resolveCallerAgent(authHeaderToken(req), AGENT_TOKENS);
|
||||||
|
const transport = new StreamableHTTPServerTransport({ sessionIdGenerator: undefined });
|
||||||
|
res.on('close', () => transport.close());
|
||||||
|
await createServer(callerAgentId).connect(transport);
|
||||||
|
await transport.handleRequest(req, res, req.body);
|
||||||
|
} catch (e) {
|
||||||
|
console.error('MCP request error:', e.message);
|
||||||
|
if (!res.headersSent) {
|
||||||
|
res.status(500).json({ jsonrpc: '2.0', error: { code: -32603, message: e.message }, id: null });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
// Legacy SSE — kept for backward compatibility
|
||||||
|
app.get('/sse', async (req, res) => {
|
||||||
|
const callerAgentId = req.callerAgentId ?? resolveCallerAgent(authHeaderToken(req), AGENT_TOKENS);
|
||||||
|
const transport = new SSEServerTransport('/messages', res);
|
||||||
|
bindSseSession(sseTransports, transport.sessionId, transport, req);
|
||||||
|
res.on('close', () => sseTransports.delete(transport.sessionId));
|
||||||
|
await createServer(callerAgentId).connect(transport);
|
||||||
|
});
|
||||||
|
|
||||||
|
app.post('/messages', async (req, res) => {
|
||||||
|
// kb#180: a live sessionId is no longer sufficient — the POST must carry the
|
||||||
|
// same caller identity that opened the session at /sse.
|
||||||
|
const auth = authorizeSseSession(sseTransports, req.query.sessionId, req);
|
||||||
|
if (!auth.ok) return res.status(auth.status).json({ error: `/messages rejected: ${auth.reason} (kb#180)` });
|
||||||
|
await auth.transport.handlePostMessage(req, res);
|
||||||
|
});
|
||||||
|
|
||||||
|
app.get('/health', (_, res) => res.json({
|
||||||
|
status: 'ok',
|
||||||
|
tools: registeredToolCount,
|
||||||
|
vaultTrustEnforced: ENFORCE_VAULT_TRUST,
|
||||||
|
tokenMapSize: AGENT_TOKEN_COUNT,
|
||||||
|
// kb#182: surfaces the vault-brick footgun (ENFORCE=1 + no tokens = fail-closed
|
||||||
|
// for everyone, including Adolf) directly in /health instead of only at boot log.
|
||||||
|
vaultBrickRisk: ENFORCE_VAULT_TRUST && AGENT_TOKEN_COUNT === 0,
|
||||||
|
// kb#180: whether the listener itself (not just vw_*) requires a bearer
|
||||||
|
// token. false here means an unauthenticated JSON-RPC surface on the LAN.
|
||||||
|
listenerAuthEnabled: requireAuthEnabled(),
|
||||||
|
}));
|
||||||
|
|
||||||
|
// kb#170: plain-REST twin of the todoist_capture_idea MCP tool, added for
|
||||||
|
// the todoist-capture-plugin native `/idea` command (openai/
|
||||||
|
// todoist-capture-plugin) — a native-command handler doesn't speak MCP
|
||||||
|
// JSON-RPC, so it needs a plain JSON endpoint to reach the same
|
||||||
|
// classify+create logic (capture.js) the MCP tool already exposes to
|
||||||
|
// Adolf/Claude's model-driven path. No new trust boundary: same
|
||||||
|
// unauthenticated-on-localhost posture as every other route in this file
|
||||||
|
// today (see trust-gate.js header for the tracked gap).
|
||||||
|
app.post('/capture-idea', async (req, res) => {
|
||||||
|
try {
|
||||||
|
const { text, project_id } = req.body || {};
|
||||||
|
const result = await todoistCaptureIdea({ text, project_id }, { listProjects: todoistListProjects });
|
||||||
|
res.json(result);
|
||||||
|
} catch (e) {
|
||||||
|
res.status(400).json({ error: e.message });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
// kb#179: guard the real init()+listen() side effects so this module can be
|
||||||
|
// `import`-ed by tests (trust-gate-http.test.mjs) to exercise the real
|
||||||
|
// createServer()/requireVaultAccess()/app on a throwaway port WITHOUT
|
||||||
|
// touching Vaultwarden/Gitea/HA/Zabbix or the live :3100 container. Only run
|
||||||
|
// the side effects when server.js is executed directly (`node src/server.js`
|
||||||
|
// / the production container entrypoint), never on import.
|
||||||
|
const isMainModule = process.argv[1] && import.meta.url === `file://${process.argv[1]}`;
|
||||||
|
if (isMainModule) {
|
||||||
|
// kb#180: refuse to boot into an "authenticated but nobody can authenticate"
|
||||||
|
// state (auth on + empty token map). Crashing here makes a missing
|
||||||
|
// AGAP_MCP_AGENT_TOKENS look like a broken restart instead of a silently
|
||||||
|
// dead integration. Only reached when run directly, never on import.
|
||||||
|
try {
|
||||||
|
assertListenerAuthConfig(AGENT_TOKENS);
|
||||||
|
} catch (e) {
|
||||||
|
console.error(`agap-mcp refusing to start: ${e.message}`);
|
||||||
|
process.exit(1);
|
||||||
|
}
|
||||||
|
init()
|
||||||
|
.then(() => {
|
||||||
|
app.listen(PORT, () => console.log(`agap-mcp listening on :${PORT}`));
|
||||||
|
})
|
||||||
|
.catch(e => {
|
||||||
|
console.error('Init failed:', e.message);
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- kb#179 test-only exports -------------------------------------------
|
||||||
|
// Exposes the exact functions/objects the production /mcp route uses so
|
||||||
|
// integration tests can boot the real enforcement path (ENFORCE=1 + a
|
||||||
|
// synthetic token map/registry) over real HTTP, instead of re-implementing
|
||||||
|
// the gate inline. Importing these does not start the server or call init().
|
||||||
|
// kb#180 additionally exports the live sse session map so the HTTP tests can
|
||||||
|
// assert /messages session-binding without reaching into module internals.
|
||||||
|
export { app, createServer, requireVaultAccess, ENFORCE_VAULT_TRUST, AGENT_TOKENS, sseTransports };
|
||||||
101
agap-mcp/src/todoist.js
Normal file
101
agap-mcp/src/todoist.js
Normal file
@@ -0,0 +1,101 @@
|
|||||||
|
// Todoist tools for agap-mcp (Adolf + Claude). Uses the unified Todoist API v1
|
||||||
|
// (https://api.todoist.com/api/v1) — REST v2 / Sync v9 are deprecated (410).
|
||||||
|
// Bearer auth with a personal API token (Vaultwarden: TODOIST_TOKEN), injected
|
||||||
|
// at init like the other services. Deliberately a SMALL, curated tool surface:
|
||||||
|
// every tool schema here is re-sent to Kimi on every Adolf turn (kb#101), so we
|
||||||
|
// expose only the essentials and return compact task objects.
|
||||||
|
|
||||||
|
const BASE = 'https://api.todoist.com/api/v1';
|
||||||
|
|
||||||
|
let _token = null;
|
||||||
|
|
||||||
|
export function initTodoist(token) {
|
||||||
|
_token = token;
|
||||||
|
console.log('Todoist: ready');
|
||||||
|
}
|
||||||
|
|
||||||
|
async function api(method, path, body) {
|
||||||
|
if (!_token) throw new Error('Todoist not initialized');
|
||||||
|
const res = await fetch(`${BASE}${path}`, {
|
||||||
|
method,
|
||||||
|
headers: {
|
||||||
|
Authorization: `Bearer ${_token}`,
|
||||||
|
...(body ? { 'Content-Type': 'application/json' } : {}),
|
||||||
|
},
|
||||||
|
body: body ? JSON.stringify(body) : undefined,
|
||||||
|
});
|
||||||
|
const text = await res.text();
|
||||||
|
if (!res.ok) throw new Error(`Todoist ${method} ${path}: ${res.status} ${text.slice(0, 300)}`);
|
||||||
|
return text ? JSON.parse(text) : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Keep tool output (and thus Kimi context) small — return only fields an agent
|
||||||
|
// needs to reason about or act on, not the full Todoist task object.
|
||||||
|
function slimTask(t) {
|
||||||
|
if (!t || typeof t !== 'object') return t;
|
||||||
|
return {
|
||||||
|
id: t.id,
|
||||||
|
content: t.content,
|
||||||
|
project_id: t.project_id,
|
||||||
|
priority: t.priority, // API 1=normal .. 4=urgent (inverse of the Todoist UI's p1..p4)
|
||||||
|
due: t.due?.string || t.due?.date || null,
|
||||||
|
labels: t.labels && t.labels.length ? t.labels : undefined,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function results(data) {
|
||||||
|
if (Array.isArray(data?.results)) return data.results;
|
||||||
|
return Array.isArray(data) ? data : [];
|
||||||
|
}
|
||||||
|
|
||||||
|
// List tasks — either a Todoist filter query (e.g. "today", "overdue",
|
||||||
|
// "#Work & p1") via the /tasks/filter endpoint, or all active tasks optionally
|
||||||
|
// scoped to one project.
|
||||||
|
export async function todoistListTasks({ query, project_id } = {}) {
|
||||||
|
let path;
|
||||||
|
if (query && query.trim()) {
|
||||||
|
path = `/tasks/filter?query=${encodeURIComponent(query.trim())}`;
|
||||||
|
} else if (project_id) {
|
||||||
|
path = `/tasks?project_id=${encodeURIComponent(project_id)}`;
|
||||||
|
} else {
|
||||||
|
path = '/tasks';
|
||||||
|
}
|
||||||
|
return results(await api('GET', path)).map(slimTask);
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function todoistListProjects() {
|
||||||
|
return results(await api('GET', '/projects')).map((p) => ({
|
||||||
|
id: p.id,
|
||||||
|
name: p.name,
|
||||||
|
is_favorite: p.is_favorite || undefined,
|
||||||
|
is_inbox: p.inbox_project || p.is_inbox_project || undefined,
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function todoistCreateTask({ content, description, due_string, priority, project_id, labels } = {}) {
|
||||||
|
if (!content || !content.trim()) throw new Error('content is required');
|
||||||
|
const body = { content: content.trim() };
|
||||||
|
if (description) body.description = description;
|
||||||
|
if (due_string) body.due_string = due_string;
|
||||||
|
if (priority) body.priority = priority;
|
||||||
|
if (project_id) body.project_id = project_id;
|
||||||
|
if (Array.isArray(labels) && labels.length) body.labels = labels;
|
||||||
|
return slimTask(await api('POST', '/tasks', body));
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function todoistUpdateTask({ id, content, description, due_string, priority, labels } = {}) {
|
||||||
|
if (!id) throw new Error('id is required');
|
||||||
|
const body = {};
|
||||||
|
if (content !== undefined) body.content = content;
|
||||||
|
if (description !== undefined) body.description = description;
|
||||||
|
if (due_string !== undefined) body.due_string = due_string;
|
||||||
|
if (priority !== undefined) body.priority = priority;
|
||||||
|
if (labels !== undefined) body.labels = labels;
|
||||||
|
return slimTask(await api('POST', `/tasks/${encodeURIComponent(id)}`, body));
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function todoistCompleteTask({ id } = {}) {
|
||||||
|
if (!id) throw new Error('id is required');
|
||||||
|
await api('POST', `/tasks/${encodeURIComponent(id)}/close`);
|
||||||
|
return { id, completed: true };
|
||||||
|
}
|
||||||
363
agap-mcp/src/trust-gate-http.test.mjs
Normal file
363
agap-mcp/src/trust-gate-http.test.mjs
Normal file
@@ -0,0 +1,363 @@
|
|||||||
|
// kb#179 HTTP-layer proof: exercises the REAL server.js /mcp handler, not a
|
||||||
|
// reimplementation of it. Run with: node src/trust-gate-http.test.mjs
|
||||||
|
//
|
||||||
|
// kb#147's original version of this file re-implemented the gate inline
|
||||||
|
// (its own express app + its own copy of the "if vault tool and not
|
||||||
|
// allowed" check). That proved trust-gate.js's exported functions compose
|
||||||
|
// correctly, but never proved the shipped server.js actually wires
|
||||||
|
// requireVaultAccess() into every vw_* tool -- a vw_* tool registered
|
||||||
|
// without the gate would still pass that test.
|
||||||
|
//
|
||||||
|
// This version instead:
|
||||||
|
// 1. Sets AGAP_MCP_ENFORCE_VAULT_TRUST=1 and a synthetic
|
||||||
|
// AGAP_MCP_AGENT_TOKENS map BEFORE importing server.js (both are read
|
||||||
|
// once at module-load time), then dynamically imports server.js so it
|
||||||
|
// picks up ENFORCE=1 with a harness-only token map -- never the live
|
||||||
|
// container's tokens.
|
||||||
|
// 2. Overrides trust-gate's registry cache with a synthetic registry (no
|
||||||
|
// real agent-registry.yaml read) via _resetRegistryCacheForTests --
|
||||||
|
// the exact test hook trust-gate.js already exports for this purpose.
|
||||||
|
// 3. Boots server.js's real `app` (its actual app.all('/mcp', ...)
|
||||||
|
// handler, its real createServer()/requireVaultAccess()) on an
|
||||||
|
// ephemeral local port, and drives it over real HTTP using the MCP
|
||||||
|
// SDK's own Client + StreamableHTTPClientTransport -- real
|
||||||
|
// initialize + tools/call JSON-RPC round trips, not raw fetch().
|
||||||
|
// 4. Does NOT call init() (never touches Vaultwarden/Gitea/HA/Zabbix) and
|
||||||
|
// never touches the live :3100 container -- server.js's init()+
|
||||||
|
// app.listen() side effects are guarded behind an isMainModule check
|
||||||
|
// specifically so this file can import the module safely (kb#179).
|
||||||
|
//
|
||||||
|
// Because init() never runs, the underlying vw* functions (vaultwarden.js)
|
||||||
|
// are never given a bw session. For a TRUSTED caller the gate must let the
|
||||||
|
// call through to that downstream code -- which then fails for its own
|
||||||
|
// unrelated reason (no bw session) -- so "allowed" is asserted as "did NOT
|
||||||
|
// fail with the gate's specific denial message", not "the vault call
|
||||||
|
// succeeded". That's deliberate: it proves requireVaultAccess() did not
|
||||||
|
// block the call, without shelling out to a real `bw` session anywhere in
|
||||||
|
// this test.
|
||||||
|
|
||||||
|
import assert from 'node:assert/strict';
|
||||||
|
|
||||||
|
process.env.AGAP_MCP_ENFORCE_VAULT_TRUST = '1';
|
||||||
|
process.env.AGAP_MCP_AGENT_TOKENS = JSON.stringify({
|
||||||
|
'tok-adolf-e2e-test': 'adolf',
|
||||||
|
'tok-torgash-e2e-test': 'torgash',
|
||||||
|
});
|
||||||
|
|
||||||
|
const registry = {
|
||||||
|
trust_classes: {
|
||||||
|
trusted: { rank: 2 },
|
||||||
|
sandboxed: { rank: 1 },
|
||||||
|
untrusted: { rank: 0 },
|
||||||
|
},
|
||||||
|
agents: [
|
||||||
|
{ id: 'adolf', trust_class: 'trusted' },
|
||||||
|
{ id: 'torgash', trust_class: 'sandboxed' },
|
||||||
|
],
|
||||||
|
};
|
||||||
|
|
||||||
|
const { _resetRegistryCacheForTests } = await import('./trust-gate.js');
|
||||||
|
_resetRegistryCacheForTests(registry);
|
||||||
|
|
||||||
|
const { app, createServer, requireVaultAccess, ENFORCE_VAULT_TRUST, AGENT_TOKENS, sseTransports } = await import('./server.js');
|
||||||
|
const {
|
||||||
|
listenerAuth,
|
||||||
|
assertListenerAuthConfig,
|
||||||
|
ListenerAuthConfigError,
|
||||||
|
requireAuthEnabled,
|
||||||
|
bindSseSession,
|
||||||
|
authorizeSseSession,
|
||||||
|
} = await import('./listener-auth.js');
|
||||||
|
|
||||||
|
assert.equal(ENFORCE_VAULT_TRUST, true, 'sanity: server.js must have picked up AGAP_MCP_ENFORCE_VAULT_TRUST=1 at import time');
|
||||||
|
assert.equal(Object.keys(AGENT_TOKENS).length, 2, 'sanity: server.js must have picked up the synthetic AGAP_MCP_AGENT_TOKENS');
|
||||||
|
|
||||||
|
const VW_TOOLS = ['vw_get_password', 'vw_get_item', 'vw_list_items', 'vw_create_login', 'vw_update_password'];
|
||||||
|
const DENIED_RE = /vault access denied/;
|
||||||
|
|
||||||
|
// Bind explicitly to 127.0.0.1, with retries: `app.listen(0)` binds the IPv6
|
||||||
|
// wildcard `::` on this host and intermittently fails EADDRINUSE under
|
||||||
|
// ephemeral-port pressure, which made this harness flaky (~2 runs in 3)
|
||||||
|
// regardless of what it asserts. Loopback-only is also the right posture for
|
||||||
|
// a test that deliberately probes an unauthenticated endpoint.
|
||||||
|
async function listenOnFreeLoopbackPort(attempts = 10) {
|
||||||
|
for (let i = 0; i < attempts; i++) {
|
||||||
|
const s = app.listen(0, '127.0.0.1');
|
||||||
|
const outcome = await new Promise(resolve => {
|
||||||
|
s.once('listening', () => resolve('ok'));
|
||||||
|
s.once('error', e => resolve(e));
|
||||||
|
});
|
||||||
|
if (outcome === 'ok') return s;
|
||||||
|
if (outcome.code !== 'EADDRINUSE') throw outcome;
|
||||||
|
}
|
||||||
|
throw new Error('could not bind a free loopback port after multiple attempts');
|
||||||
|
}
|
||||||
|
|
||||||
|
const server = await listenOnFreeLoopbackPort();
|
||||||
|
const port = server.address().port;
|
||||||
|
const baseUrl = `http://127.0.0.1:${port}/mcp`;
|
||||||
|
|
||||||
|
const { Client } = await import('@modelcontextprotocol/sdk/client/index.js');
|
||||||
|
const { StreamableHTTPClientTransport } = await import('@modelcontextprotocol/sdk/client/streamableHttp.js');
|
||||||
|
|
||||||
|
// Minimal argument stubs matching each tool's zod schema -- the gate must
|
||||||
|
// fire before any of these are used for real (requireVaultAccess is the
|
||||||
|
// first statement in every vw_* handler in server.js), so their exact
|
||||||
|
// values don't matter for the deny path.
|
||||||
|
const ARGS = {
|
||||||
|
vw_get_password: { name: 'probe' },
|
||||||
|
vw_get_item: { name: 'probe' },
|
||||||
|
vw_list_items: {},
|
||||||
|
vw_create_login: { name: 'probe', password: 'x' },
|
||||||
|
vw_update_password: { name: 'probe', password: 'x' },
|
||||||
|
};
|
||||||
|
|
||||||
|
// callToolAsCaller: connects a fresh MCP client to the real /mcp route with
|
||||||
|
// the given bearer token (or none), calls `tool`, and returns the tool
|
||||||
|
// result. `undefined`/omitted mimics resolveCallerAgent's "no header at
|
||||||
|
// all" path; a token string that isn't in AGENT_TOKENS mimics "unknown
|
||||||
|
// token".
|
||||||
|
//
|
||||||
|
// kb#180 note: since the listener itself now requires auth, an unauthenticated
|
||||||
|
// or unknown-token client is rejected at the HTTP layer (401) during
|
||||||
|
// connect() — before any tool handler runs. That is a STRICTER outcome than
|
||||||
|
// the tool-level "vault access denied" this file originally asserted for
|
||||||
|
// those two cases, so those checks now assert the 401 instead. The
|
||||||
|
// authenticated-but-sandboxed (torgash) case is what still proves the kb#147
|
||||||
|
// vault gate itself, and the trusted (adolf) case still proves the gate lets
|
||||||
|
// a trusted caller through.
|
||||||
|
const AUTH_REJECTED = Symbol('listener-auth-rejected');
|
||||||
|
|
||||||
|
async function callToolAsCaller(token, tool) {
|
||||||
|
const requestInit = token ? { headers: { Authorization: `Bearer ${token}` } } : {};
|
||||||
|
const transport = new StreamableHTTPClientTransport(new URL(baseUrl), { requestInit });
|
||||||
|
const client = new Client({ name: 'kb179-test-client', version: '1.0.0' });
|
||||||
|
try {
|
||||||
|
await client.connect(transport);
|
||||||
|
} catch (e) {
|
||||||
|
if (/401|unauthenticated/i.test(e.message)) return AUTH_REJECTED;
|
||||||
|
throw e;
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
return await client.callTool({ name: tool, arguments: ARGS[tool] });
|
||||||
|
} catch (e) {
|
||||||
|
if (/401|unauthenticated/i.test(e.message)) return AUTH_REJECTED;
|
||||||
|
throw e;
|
||||||
|
} finally {
|
||||||
|
await client.close().catch(() => {});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function toolErrorText(result) {
|
||||||
|
const block = result?.content?.find(c => c.type === 'text');
|
||||||
|
return block?.text || '';
|
||||||
|
}
|
||||||
|
|
||||||
|
let passed = 0;
|
||||||
|
async function check(label, fn) {
|
||||||
|
await fn();
|
||||||
|
passed++;
|
||||||
|
console.log(`ok - ${label}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
for (const tool of VW_TOOLS) {
|
||||||
|
await check(`${tool}: no Authorization header -> DENIED over real HTTP (kb#180: 401 at the listener)`, async () => {
|
||||||
|
const result = await callToolAsCaller(null, tool);
|
||||||
|
assert.equal(result, AUTH_REJECTED, 'expected the listener to reject an unauthenticated MCP client');
|
||||||
|
});
|
||||||
|
|
||||||
|
await check(`${tool}: unknown/never-issued token -> DENIED over real HTTP (kb#180: 401 at the listener)`, async () => {
|
||||||
|
const result = await callToolAsCaller('this-token-was-never-issued', tool);
|
||||||
|
assert.equal(result, AUTH_REJECTED, 'expected the listener to reject an unknown bearer token');
|
||||||
|
});
|
||||||
|
|
||||||
|
await check(`${tool}: sandboxed agent (torgash) token -> DENIED over real HTTP`, async () => {
|
||||||
|
const result = await callToolAsCaller('tok-torgash-e2e-test', tool);
|
||||||
|
assert.equal(result.isError, true);
|
||||||
|
assert.match(toolErrorText(result), DENIED_RE);
|
||||||
|
});
|
||||||
|
|
||||||
|
await check(`${tool}: trusted agent (adolf) token -> gate ALLOWS (not blocked by requireVaultAccess) over real HTTP`, async () => {
|
||||||
|
const result = await callToolAsCaller('tok-adolf-e2e-test', tool);
|
||||||
|
// The gate must not be what blocks this call. Downstream vaultwarden.js
|
||||||
|
// has no bw session here (init() deliberately never ran), so the call
|
||||||
|
// may still fail -- just not with the gate's denial message.
|
||||||
|
assert.doesNotMatch(toolErrorText(result), DENIED_RE);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- Guard test (kb#179 acceptance bar #2) ---------------------------
|
||||||
|
// Fails if any vw_* tool is ever registered on createServer() without
|
||||||
|
// calling requireVaultAccess(). Rather than re-deriving this from source
|
||||||
|
// text (fragile to refactors), it drives the real registered handler for
|
||||||
|
// every tool name starting with "vw_" as an untrusted/denied caller and
|
||||||
|
// asserts the gate's specific denial message comes back. A vw_* tool
|
||||||
|
// that forgot to call requireVaultAccess would either succeed or throw a
|
||||||
|
// different (non-gate) error here, and this test would catch it.
|
||||||
|
await check('guard: every registered vw_* tool enforces requireVaultAccess()', async () => {
|
||||||
|
// A caller id that resolves to no agent in the (synthetic) registry --
|
||||||
|
// trustRankOf() returns -1 for it, so vaultAllowed() must be false and
|
||||||
|
// requireVaultAccess() must throw for every vw_* tool.
|
||||||
|
const probeServer = createServer('this-agent-id-does-not-exist-in-registry');
|
||||||
|
// McpServer keeps its registrations on `._registeredTools` (name ->
|
||||||
|
// { handler, ... }); read the real registry createServer() just
|
||||||
|
// populated rather than re-deriving tool names by hand, so a future
|
||||||
|
// vw_* tool is covered automatically.
|
||||||
|
const registeredTools = probeServer._registeredTools;
|
||||||
|
assert.ok(registeredTools && Object.keys(registeredTools).length > 0,
|
||||||
|
'expected createServer() to have registered tools onto the McpServer instance');
|
||||||
|
|
||||||
|
const vwToolNames = Object.keys(registeredTools).filter(n => n.startsWith('vw_'));
|
||||||
|
assert.ok(vwToolNames.length >= VW_TOOLS.length, `expected at least ${VW_TOOLS.length} vw_* tools registered, found: ${vwToolNames.join(', ')}`);
|
||||||
|
|
||||||
|
for (const name of vwToolNames) {
|
||||||
|
const handler = registeredTools[name].handler;
|
||||||
|
assert.ok(typeof handler === 'function', `could not locate callable handler for ${name}`);
|
||||||
|
const result = await handler(ARGS[name] || {}, {});
|
||||||
|
assert.equal(result.isError, true, `${name}: expected denial for an unregistered agent id, got success -- is requireVaultAccess() missing?`);
|
||||||
|
assert.match(toolErrorText(result), DENIED_RE, `${name}: expected the gate's denial message, got: ${toolErrorText(result)} -- is requireVaultAccess() missing or not the first check?`);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
// --- kb#180: the LISTENER is authenticated, not just vault tools --------
|
||||||
|
const base = `http://127.0.0.1:${port}`;
|
||||||
|
const authed = { Authorization: 'Bearer tok-torgash-e2e-test' }; // sandboxed but *known*
|
||||||
|
|
||||||
|
await check('kb#180: unauthenticated POST /mcp -> 401 (raw curl-equivalent)', async () => {
|
||||||
|
const res = await fetch(`${base}/mcp`, {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json', Accept: 'application/json, text/event-stream' },
|
||||||
|
body: JSON.stringify({ jsonrpc: '2.0', id: 1, method: 'tools/list', params: {} }),
|
||||||
|
});
|
||||||
|
assert.equal(res.status, 401);
|
||||||
|
assert.equal(res.headers.get('www-authenticate'), 'Bearer realm="agap-mcp"');
|
||||||
|
const body = await res.json();
|
||||||
|
assert.equal(body.jsonrpc, '2.0', 'MCP routes must answer with a JSON-RPC error envelope');
|
||||||
|
assert.match(body.error.message, /unauthenticated/);
|
||||||
|
});
|
||||||
|
|
||||||
|
await check('kb#180: unauthenticated POST /capture-idea -> 401 (no Todoist write)', async () => {
|
||||||
|
const res = await fetch(`${base}/capture-idea`, {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({ text: 'unauthenticated probe — must never reach Todoist' }),
|
||||||
|
});
|
||||||
|
assert.equal(res.status, 401);
|
||||||
|
const body = await res.json();
|
||||||
|
assert.match(body.error, /unauthenticated/);
|
||||||
|
});
|
||||||
|
|
||||||
|
await check('kb#180: unknown token on /capture-idea -> 401', async () => {
|
||||||
|
const res = await fetch(`${base}/capture-idea`, {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json', Authorization: 'Bearer nope' },
|
||||||
|
body: JSON.stringify({ text: 'probe' }),
|
||||||
|
});
|
||||||
|
assert.equal(res.status, 401);
|
||||||
|
});
|
||||||
|
|
||||||
|
await check('kb#180: unauthenticated GET /sse -> 401', async () => {
|
||||||
|
const res = await fetch(`${base}/sse`, { headers: { Accept: 'text/event-stream' } });
|
||||||
|
assert.equal(res.status, 401);
|
||||||
|
await res.arrayBuffer();
|
||||||
|
});
|
||||||
|
|
||||||
|
await check('kb#180: unauthenticated POST /messages -> 401 (was: any sessionId accepted)', async () => {
|
||||||
|
const res = await fetch(`${base}/messages?sessionId=whatever`, {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({ jsonrpc: '2.0', id: 1, method: 'tools/list' }),
|
||||||
|
});
|
||||||
|
assert.equal(res.status, 401);
|
||||||
|
});
|
||||||
|
|
||||||
|
await check('kb#180: /health stays open and reports listenerAuthEnabled=true', async () => {
|
||||||
|
const res = await fetch(`${base}/health`);
|
||||||
|
assert.equal(res.status, 200);
|
||||||
|
const body = await res.json();
|
||||||
|
assert.equal(body.listenerAuthEnabled, true);
|
||||||
|
});
|
||||||
|
|
||||||
|
await check('kb#180: an authenticated (known-token) caller still reaches the tool surface', async () => {
|
||||||
|
const res = await fetch(`${base}/mcp`, {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json', Accept: 'application/json, text/event-stream', ...authed },
|
||||||
|
body: JSON.stringify({
|
||||||
|
jsonrpc: '2.0', id: 1, method: 'initialize',
|
||||||
|
params: { protocolVersion: '2024-11-05', capabilities: {}, clientInfo: { name: 't', version: '1' } },
|
||||||
|
}),
|
||||||
|
});
|
||||||
|
assert.notEqual(res.status, 401, 'a known token must not be rejected by the listener gate');
|
||||||
|
assert.ok(res.status < 500, `expected the request to be served, got ${res.status}`);
|
||||||
|
await res.arrayBuffer();
|
||||||
|
});
|
||||||
|
|
||||||
|
// --- kb#180: /messages must match its /sse handshake identity ----------
|
||||||
|
await check('kb#180: /messages rejects a session opened by a DIFFERENT agent (hijack)', () => {
|
||||||
|
const sessions = new Map();
|
||||||
|
bindSseSession(sessions, 'sess-1', { id: 'transport-a' },
|
||||||
|
{ callerAgentId: 'adolf', callerToken: 'tok-adolf-e2e-test' });
|
||||||
|
|
||||||
|
const sameCaller = authorizeSseSession(sessions, 'sess-1',
|
||||||
|
{ callerAgentId: 'adolf', callerToken: 'tok-adolf-e2e-test' });
|
||||||
|
assert.equal(sameCaller.ok, true, 'the agent that opened the session must keep using it');
|
||||||
|
|
||||||
|
const otherAgent = authorizeSseSession(sessions, 'sess-1',
|
||||||
|
{ callerAgentId: 'torgash', callerToken: 'tok-torgash-e2e-test' });
|
||||||
|
assert.equal(otherAgent.ok, false);
|
||||||
|
assert.equal(otherAgent.status, 403);
|
||||||
|
assert.equal(otherAgent.reason, 'session-caller-mismatch');
|
||||||
|
|
||||||
|
const noCreds = authorizeSseSession(sessions, 'sess-1', { callerAgentId: null, callerToken: null });
|
||||||
|
assert.equal(noCreds.ok, false);
|
||||||
|
assert.equal(noCreds.status, 401);
|
||||||
|
|
||||||
|
const unknownSession = authorizeSseSession(sessions, 'nope',
|
||||||
|
{ callerAgentId: 'adolf', callerToken: 'tok-adolf-e2e-test' });
|
||||||
|
assert.equal(unknownSession.ok, false);
|
||||||
|
assert.equal(unknownSession.status, 400);
|
||||||
|
});
|
||||||
|
|
||||||
|
await check('kb#180: the live server binds real /sse sessions to a caller identity', () => {
|
||||||
|
// sseTransports is the exact map the /sse and /messages routes use; its
|
||||||
|
// entries must be {transport, agentId, token}, not a bare transport.
|
||||||
|
assert.ok(sseTransports instanceof Map);
|
||||||
|
bindSseSession(sseTransports, 'probe-session', { probe: true },
|
||||||
|
{ callerAgentId: 'adolf', callerToken: 'tok-adolf-e2e-test' });
|
||||||
|
const entry = sseTransports.get('probe-session');
|
||||||
|
assert.equal(entry.agentId, 'adolf');
|
||||||
|
assert.equal(entry.token, 'tok-adolf-e2e-test');
|
||||||
|
sseTransports.delete('probe-session');
|
||||||
|
});
|
||||||
|
|
||||||
|
// --- kb#180: boot-time config guard ------------------------------------
|
||||||
|
await check('kb#180: auth on + empty token map = refuse to boot (not silent all-deny)', () => {
|
||||||
|
assert.throws(() => assertListenerAuthConfig({}, { }), ListenerAuthConfigError);
|
||||||
|
assert.throws(() => assertListenerAuthConfig(Object.create(null), { AGAP_MCP_REQUIRE_AUTH: '1' }), ListenerAuthConfigError);
|
||||||
|
// Explicit opt-out is allowed (warns, does not throw) — the rollback path.
|
||||||
|
assert.doesNotThrow(() => assertListenerAuthConfig({}, { AGAP_MCP_REQUIRE_AUTH: '0' }));
|
||||||
|
// Configured normally: fine.
|
||||||
|
assert.doesNotThrow(() => assertListenerAuthConfig({ t: 'adolf' }, {}));
|
||||||
|
});
|
||||||
|
|
||||||
|
await check('kb#180: AGAP_MCP_REQUIRE_AUTH=0 is the only way to get the old open listener', () => {
|
||||||
|
assert.equal(requireAuthEnabled({}), true, 'auth must default ON');
|
||||||
|
assert.equal(requireAuthEnabled({ AGAP_MCP_REQUIRE_AUTH: '1' }), true);
|
||||||
|
assert.equal(requireAuthEnabled({ AGAP_MCP_REQUIRE_AUTH: '0' }), false);
|
||||||
|
// ...and with it off, an unauthenticated request passes through the
|
||||||
|
// middleware while still resolving an agent id when a token IS present.
|
||||||
|
const mw = listenerAuth({ 'tok-adolf-e2e-test': 'adolf' }, { AGAP_MCP_REQUIRE_AUTH: '0' });
|
||||||
|
let nexted = false;
|
||||||
|
mw({ headers: {}, path: '/mcp' }, null, () => { nexted = true; });
|
||||||
|
assert.equal(nexted, true);
|
||||||
|
const req = { headers: { authorization: 'Bearer tok-adolf-e2e-test' }, path: '/mcp' };
|
||||||
|
mw(req, null, () => {});
|
||||||
|
assert.equal(req.callerAgentId, 'adolf', 'vault gate must still see the caller id when listener auth is off');
|
||||||
|
});
|
||||||
|
|
||||||
|
console.log(`\n${passed} passed`);
|
||||||
|
} finally {
|
||||||
|
server.close();
|
||||||
|
_resetRegistryCacheForTests(null);
|
||||||
|
}
|
||||||
103
agap-mcp/src/trust-gate.js
Normal file
103
agap-mcp/src/trust-gate.js
Normal file
@@ -0,0 +1,103 @@
|
|||||||
|
// trust-gate — kb#147 (A2A-15): vault access = trusted-only, enforced here.
|
||||||
|
//
|
||||||
|
// agap-mcp has no per-caller identity today (every MCP client hits the same
|
||||||
|
// unauthenticated /mcp endpoint on :3100) — that's the gap DESIGN-a2a-agents.md
|
||||||
|
// v2.1 §5 flags as the crux of "vault access = trusted only" (DECIDED, alvis).
|
||||||
|
// This module is the enforcement point: it maps a bearer token off the request
|
||||||
|
// to an agent id (via AGAP_MCP_AGENT_TOKENS, a secret never committed to git),
|
||||||
|
// then looks up that agent's trust class in agent-registry.yaml (the
|
||||||
|
// version-controlled source of truth for grants, kb#134) to decide whether
|
||||||
|
// vault tools (vw_*) may run.
|
||||||
|
//
|
||||||
|
// FAIL-CLOSED PRINCIPLE (once enforcement is turned on): no token, an
|
||||||
|
// unrecognized token, or an agent below `trusted` rank all resolve to "no
|
||||||
|
// vault access" — there is no default-allow path once AGAP_MCP_ENFORCE_VAULT_TRUST
|
||||||
|
// is on. See server.js for the off-by-default activation gate: merging this
|
||||||
|
// module changes zero live behavior until an operator deliberately flips that
|
||||||
|
// flag AND supplies real per-agent tokens (kb#147 handover step).
|
||||||
|
|
||||||
|
import { readFileSync } from 'fs';
|
||||||
|
import yaml from 'js-yaml';
|
||||||
|
|
||||||
|
const VAULT_TOOL_PREFIX = 'vw_';
|
||||||
|
const DEFAULT_REGISTRY_PATH = '/agent-registry.yaml';
|
||||||
|
const DEFAULT_TRUSTED_RANK = 2; // matches agent-registry.yaml trust_classes.trusted.rank; used only if the registry can't be read
|
||||||
|
|
||||||
|
let _registryCache = null;
|
||||||
|
|
||||||
|
export function loadRegistry(path = process.env.AGENT_REGISTRY_PATH || DEFAULT_REGISTRY_PATH) {
|
||||||
|
if (_registryCache) return _registryCache;
|
||||||
|
try {
|
||||||
|
_registryCache = yaml.load(readFileSync(path, 'utf8'));
|
||||||
|
} catch (e) {
|
||||||
|
// Fail closed, not fail crash: no registry readable means no agent can be
|
||||||
|
// proven trusted, so vaultAllowed() below returns false for everyone
|
||||||
|
// rather than the process refusing to start (agap-mcp serves gitea/ha/
|
||||||
|
// zabbix/radicale/todoist tools too, which don't depend on this file).
|
||||||
|
console.error(`trust-gate: could not load agent registry from ${path}: ${e.message}`);
|
||||||
|
_registryCache = { agents: [], trust_classes: {} };
|
||||||
|
}
|
||||||
|
return _registryCache;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Test-only: let unit tests inject a registry object instead of touching the
|
||||||
|
// filesystem, and let the CLI/tests reset the module-level cache between runs.
|
||||||
|
export function _resetRegistryCacheForTests(registry = null) {
|
||||||
|
_registryCache = registry;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function trustedRankThreshold(registry = loadRegistry()) {
|
||||||
|
return registry.trust_classes?.trusted?.rank ?? DEFAULT_TRUSTED_RANK;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function trustRankOf(agentId, registry = loadRegistry()) {
|
||||||
|
if (!agentId) return -1; // unauthenticated caller: rank below every real trust class
|
||||||
|
const agent = (registry.agents || []).find(a => a.id === agentId);
|
||||||
|
if (!agent) return -1; // unknown agent id: fail closed, not "assume trusted"
|
||||||
|
const cls = registry.trust_classes?.[agent.trust_class];
|
||||||
|
return cls ? cls.rank : -1;
|
||||||
|
}
|
||||||
|
|
||||||
|
// tokenMap: { "<bearer-token>": "<agent-id>" } — parsed once at startup from
|
||||||
|
// AGAP_MCP_AGENT_TOKENS (JSON), itself sourced from per-agent tokens stored in
|
||||||
|
// Vaultwarden and injected via this container's .env, never inlined in git.
|
||||||
|
export function loadTokenMap(raw = process.env.AGAP_MCP_AGENT_TOKENS) {
|
||||||
|
if (!raw) return Object.create(null);
|
||||||
|
try {
|
||||||
|
const parsed = JSON.parse(raw);
|
||||||
|
if (!parsed || typeof parsed !== 'object') return Object.create(null);
|
||||||
|
// Rebuild onto a null-proto object so a token literally named
|
||||||
|
// "__proto__"/"constructor"/"prototype" in AGAP_MCP_AGENT_TOKENS can
|
||||||
|
// never merge into Object.prototype instead of becoming an own key.
|
||||||
|
return Object.assign(Object.create(null), parsed);
|
||||||
|
} catch (e) {
|
||||||
|
console.error(`trust-gate: AGAP_MCP_AGENT_TOKENS is not valid JSON: ${e.message}`);
|
||||||
|
return Object.create(null);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export function resolveCallerAgent(bearerToken, tokenMap) {
|
||||||
|
if (!bearerToken || !tokenMap) return null;
|
||||||
|
// Guard against prototype-key inputs: a bearer token of "__proto__",
|
||||||
|
// "constructor", or "prototype" must never resolve via the object's
|
||||||
|
// prototype chain (e.g. tokenMap['__proto__'] returning Object.prototype,
|
||||||
|
// which is truthy and would silently "authenticate" as a non-existent
|
||||||
|
// agent). Object.hasOwn only ever matches an actual own property that was
|
||||||
|
// set from AGAP_MCP_AGENT_TOKENS, so a __proto__ probe resolves to null
|
||||||
|
// explicitly regardless of the token map's shape (kb#182).
|
||||||
|
if (!Object.hasOwn(tokenMap, bearerToken)) return null;
|
||||||
|
return tokenMap[bearerToken] || null;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function isVaultTool(toolName) {
|
||||||
|
return toolName.startsWith(VAULT_TOOL_PREFIX);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function vaultAllowed(agentId, registry = loadRegistry()) {
|
||||||
|
return trustRankOf(agentId, registry) >= trustedRankThreshold(registry);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function authHeaderToken(req) {
|
||||||
|
const header = req.headers?.['authorization'] || req.headers?.['Authorization'] || '';
|
||||||
|
return header.startsWith('Bearer ') ? header.slice(7).trim() : null;
|
||||||
|
}
|
||||||
101
agap-mcp/src/trust-gate.test.mjs
Normal file
101
agap-mcp/src/trust-gate.test.mjs
Normal file
@@ -0,0 +1,101 @@
|
|||||||
|
// Proof-of-enforcement for kb#147, run with: node src/trust-gate.test.mjs
|
||||||
|
//
|
||||||
|
// Deliberately does NOT touch the live agap-mcp container, LiteLLM, or
|
||||||
|
// Vaultwarden — it exercises the exact exported functions server.js calls
|
||||||
|
// (trustRankOf/vaultAllowed/resolveCallerAgent/isVaultTool) against a
|
||||||
|
// synthetic registry + token map, so this is a real test of the enforcement
|
||||||
|
// logic itself, not a mock of it.
|
||||||
|
|
||||||
|
import assert from 'node:assert/strict';
|
||||||
|
import {
|
||||||
|
trustRankOf,
|
||||||
|
vaultAllowed,
|
||||||
|
resolveCallerAgent,
|
||||||
|
isVaultTool,
|
||||||
|
trustedRankThreshold,
|
||||||
|
_resetRegistryCacheForTests,
|
||||||
|
} from './trust-gate.js';
|
||||||
|
|
||||||
|
const registry = {
|
||||||
|
trust_classes: {
|
||||||
|
human: { rank: 3 },
|
||||||
|
trusted: { rank: 2 },
|
||||||
|
sandboxed: { rank: 1 },
|
||||||
|
untrusted: { rank: 0 },
|
||||||
|
},
|
||||||
|
agents: [
|
||||||
|
{ id: 'adolf', trust_class: 'trusted' },
|
||||||
|
{ id: 'claude-coder', trust_class: 'trusted' },
|
||||||
|
{ id: 'torgash', trust_class: 'sandboxed' },
|
||||||
|
{ id: 'researcher', trust_class: 'sandboxed' },
|
||||||
|
{ id: 'kimi-endpoint', trust_class: 'untrusted' },
|
||||||
|
],
|
||||||
|
};
|
||||||
|
|
||||||
|
const tokenMap = {
|
||||||
|
'tok-adolf': 'adolf',
|
||||||
|
'tok-claude-coder': 'claude-coder',
|
||||||
|
'tok-torgash': 'torgash',
|
||||||
|
'tok-researcher': 'researcher',
|
||||||
|
};
|
||||||
|
|
||||||
|
let passed = 0;
|
||||||
|
function check(label, fn) {
|
||||||
|
fn();
|
||||||
|
passed++;
|
||||||
|
console.log(`ok - ${label}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
check('trusted rank threshold resolves from registry', () => {
|
||||||
|
assert.equal(trustedRankThreshold(registry), 2);
|
||||||
|
});
|
||||||
|
|
||||||
|
check('trusted agents (adolf, claude-coder) can reach vault', () => {
|
||||||
|
assert.equal(vaultAllowed('adolf', registry), true);
|
||||||
|
assert.equal(vaultAllowed('claude-coder', registry), true);
|
||||||
|
});
|
||||||
|
|
||||||
|
check('sandboxed agents (torgash, researcher) CANNOT reach vault', () => {
|
||||||
|
assert.equal(vaultAllowed('torgash', registry), false);
|
||||||
|
assert.equal(vaultAllowed('researcher', registry), false);
|
||||||
|
});
|
||||||
|
|
||||||
|
check('untrusted agent cannot reach vault', () => {
|
||||||
|
assert.equal(vaultAllowed('kimi-endpoint', registry), false);
|
||||||
|
});
|
||||||
|
|
||||||
|
check('unauthenticated caller (no token resolved) cannot reach vault', () => {
|
||||||
|
assert.equal(vaultAllowed(null, registry), false);
|
||||||
|
assert.equal(trustRankOf(null, registry), -1);
|
||||||
|
});
|
||||||
|
|
||||||
|
check('unknown/unregistered agent id fails closed, not open', () => {
|
||||||
|
assert.equal(vaultAllowed('some-new-agent-nobody-declared', registry), false);
|
||||||
|
});
|
||||||
|
|
||||||
|
check('resolveCallerAgent maps bearer token -> agent id, else null', () => {
|
||||||
|
assert.equal(resolveCallerAgent('tok-torgash', tokenMap), 'torgash');
|
||||||
|
assert.equal(resolveCallerAgent('tok-adolf', tokenMap), 'adolf');
|
||||||
|
assert.equal(resolveCallerAgent('not-a-real-token', tokenMap), null);
|
||||||
|
assert.equal(resolveCallerAgent(null, tokenMap), null);
|
||||||
|
});
|
||||||
|
|
||||||
|
check('end-to-end: a sandboxed agent\'s token provably cannot reach vw_* tools', () => {
|
||||||
|
const callerAgentId = resolveCallerAgent('tok-torgash', tokenMap);
|
||||||
|
assert.equal(callerAgentId, 'torgash');
|
||||||
|
assert.equal(isVaultTool('vw_get_password'), true);
|
||||||
|
assert.equal(vaultAllowed(callerAgentId, registry), false); // <- the acceptance bar
|
||||||
|
});
|
||||||
|
|
||||||
|
check('end-to-end: a trusted agent\'s token can reach vw_* tools', () => {
|
||||||
|
const callerAgentId = resolveCallerAgent('tok-adolf', tokenMap);
|
||||||
|
assert.equal(vaultAllowed(callerAgentId, registry), true);
|
||||||
|
});
|
||||||
|
|
||||||
|
check('non-vault tool name is unaffected by the gate', () => {
|
||||||
|
assert.equal(isVaultTool('gitea_read_file'), false);
|
||||||
|
assert.equal(isVaultTool('zabbix_get_problems'), false);
|
||||||
|
});
|
||||||
|
|
||||||
|
_resetRegistryCacheForTests(null);
|
||||||
|
console.log(`\n${passed} passed`);
|
||||||
119
agap-mcp/src/vaultwarden.js
Normal file
119
agap-mcp/src/vaultwarden.js
Normal file
@@ -0,0 +1,119 @@
|
|||||||
|
import { execFileSync } from 'child_process';
|
||||||
|
|
||||||
|
const BW = 'bw';
|
||||||
|
const ORG_ID = '4bd75130-b4d3-48d4-a4cb-e52b70295a51';
|
||||||
|
const AI_COLLECTION = '5be27a82-8475-4c38-96b2-fa94ec8c957b';
|
||||||
|
|
||||||
|
let _session = null;
|
||||||
|
|
||||||
|
function bwEnv() {
|
||||||
|
const env = { ...process.env };
|
||||||
|
for (const k of ['HTTPS_PROXY','HTTP_PROXY','ALL_PROXY','https_proxy','http_proxy','all_proxy'])
|
||||||
|
delete env[k];
|
||||||
|
env.NODE_TLS_REJECT_UNAUTHORIZED = '0';
|
||||||
|
return env;
|
||||||
|
}
|
||||||
|
|
||||||
|
function run(args, input) {
|
||||||
|
try {
|
||||||
|
return execFileSync(BW, args, {
|
||||||
|
env: bwEnv(),
|
||||||
|
encoding: 'utf8',
|
||||||
|
input,
|
||||||
|
stdio: input ? ['pipe','pipe','pipe'] : ['ignore','pipe','pipe'],
|
||||||
|
}).trim();
|
||||||
|
} catch (e) {
|
||||||
|
// SECURITY: execFileSync puts the FULL argv in e.message
|
||||||
|
// ("Command failed: bw unlock <BW_PASSWORD> --raw"), and login/unlock pass
|
||||||
|
// the master password as argv. That message propagates to console.error /
|
||||||
|
// tool errors -> docker logs. Re-throw with the argv stripped: keep only the
|
||||||
|
// subcommand + exit code + stderr, and scrub the password out of stderr too
|
||||||
|
// (defensive; bw doesn't normally echo it). Never let argv reach a log.
|
||||||
|
const sub = Array.isArray(args) && args.length ? args[0] : '?';
|
||||||
|
let stderr = (e && e.stderr ? e.stderr.toString() : '').trim();
|
||||||
|
const secret = process.env.BW_PASSWORD;
|
||||||
|
if (secret && stderr.includes(secret)) stderr = stderr.split(secret).join('<redacted>');
|
||||||
|
const err = new Error(`bw ${sub} failed (exit ${e && e.status != null ? e.status : '?'})` +
|
||||||
|
(stderr ? `: ${stderr}` : ''));
|
||||||
|
err.status = e && e.status;
|
||||||
|
throw err;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function initVaultwarden() {
|
||||||
|
const email = process.env.BW_EMAIL || 'allogn@gmail.com';
|
||||||
|
const password = process.env.BW_PASSWORD;
|
||||||
|
|
||||||
|
// Data dir is mounted from host — server already configured, skip bw config server
|
||||||
|
|
||||||
|
let status = 'unauthenticated';
|
||||||
|
try {
|
||||||
|
status = JSON.parse(run(['status'])).status;
|
||||||
|
} catch {}
|
||||||
|
|
||||||
|
if (status === 'unauthenticated') {
|
||||||
|
run(['login', email, password, '--raw']);
|
||||||
|
}
|
||||||
|
|
||||||
|
_session = run(['unlock', password, '--raw']);
|
||||||
|
run(['sync', '--session', _session]);
|
||||||
|
console.log('Vaultwarden: ready');
|
||||||
|
}
|
||||||
|
|
||||||
|
function session() {
|
||||||
|
if (!_session) throw new Error('Vaultwarden not initialized');
|
||||||
|
return _session;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function vwGetPassword(name) {
|
||||||
|
return run(['get', 'password', name, '--session', session()]);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function vwGetItem(name) {
|
||||||
|
return JSON.parse(run(['get', 'item', name, '--session', session()]));
|
||||||
|
}
|
||||||
|
|
||||||
|
export function vwListItems(search) {
|
||||||
|
const args = ['list', 'items', '--session', session()];
|
||||||
|
if (search) args.push('--search', search);
|
||||||
|
return JSON.parse(run(args));
|
||||||
|
}
|
||||||
|
|
||||||
|
export function vwListOrgItems(search) {
|
||||||
|
const args = ['list', 'items', '--organizationid', ORG_ID, '--session', session()];
|
||||||
|
if (search) args.push('--search', search);
|
||||||
|
return JSON.parse(run(args));
|
||||||
|
}
|
||||||
|
|
||||||
|
export function vwCreateLogin({ name, username, password, url, notes }) {
|
||||||
|
const item = {
|
||||||
|
organizationId: ORG_ID,
|
||||||
|
collectionIds: [AI_COLLECTION],
|
||||||
|
folderId: null,
|
||||||
|
type: 1,
|
||||||
|
name,
|
||||||
|
notes: notes || null,
|
||||||
|
favorite: false,
|
||||||
|
login: {
|
||||||
|
username: username || null,
|
||||||
|
password,
|
||||||
|
uris: url ? [{ match: null, uri: url }] : [],
|
||||||
|
},
|
||||||
|
};
|
||||||
|
const encoded = run(['encode'], JSON.stringify(item));
|
||||||
|
return JSON.parse(run(['create', 'item', encoded, '--session', session()]));
|
||||||
|
}
|
||||||
|
|
||||||
|
export function vwUpdatePassword(nameOrId, newPassword) {
|
||||||
|
let item;
|
||||||
|
try {
|
||||||
|
item = JSON.parse(run(['get', 'item', nameOrId, '--session', session()]));
|
||||||
|
} catch {
|
||||||
|
const items = vwListOrgItems(nameOrId);
|
||||||
|
item = items.find(i => i.name === nameOrId);
|
||||||
|
if (!item) throw new Error(`Item not found: ${nameOrId}`);
|
||||||
|
}
|
||||||
|
item.login.password = newPassword;
|
||||||
|
const encoded = run(['encode'], JSON.stringify(item));
|
||||||
|
return JSON.parse(run(['edit', 'item', item.id, encoded, '--session', session()]));
|
||||||
|
}
|
||||||
75
agap-mcp/src/zabbix.js
Normal file
75
agap-mcp/src/zabbix.js
Normal file
@@ -0,0 +1,75 @@
|
|||||||
|
const BASE = () => `${process.env.ZABBIX_URL || 'http://localhost:81'}/api_jsonrpc.php`;
|
||||||
|
let _token = null;
|
||||||
|
let _reqId = 1;
|
||||||
|
|
||||||
|
export function initZabbix(token) {
|
||||||
|
_token = token;
|
||||||
|
console.log('Zabbix: ready');
|
||||||
|
}
|
||||||
|
|
||||||
|
async function api(method, params = {}) {
|
||||||
|
const res = await fetch(BASE(), {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${_token}` },
|
||||||
|
body: JSON.stringify({ jsonrpc: '2.0', method, params, id: _reqId++ }),
|
||||||
|
});
|
||||||
|
const data = await res.json();
|
||||||
|
if (data.error) throw new Error(`Zabbix ${method}: ${data.error.data}`);
|
||||||
|
return data.result;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function zabbixGetProblems() {
|
||||||
|
const problems = await api('problem.get', {
|
||||||
|
output: ['eventid', 'name', 'severity', 'clock', 'acknowledged'],
|
||||||
|
selectAcknowledges: 'count',
|
||||||
|
recent: true,
|
||||||
|
sortfield: 'eventid',
|
||||||
|
sortorder: 'DESC',
|
||||||
|
});
|
||||||
|
return problems.map(p => ({
|
||||||
|
id: p.eventid,
|
||||||
|
name: p.name,
|
||||||
|
severity: ['Not classified','Information','Warning','Average','High','Disaster'][+p.severity],
|
||||||
|
time: new Date(+p.clock * 1000).toISOString(),
|
||||||
|
acknowledged: +p.acknowledged > 0,
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function zabbixGetHosts() {
|
||||||
|
return api('host.get', {
|
||||||
|
output: ['hostid', 'host', 'name', 'available', 'status'],
|
||||||
|
selectInterfaces: ['ip'],
|
||||||
|
}).then(hosts => hosts.map(h => ({
|
||||||
|
id: h.hostid,
|
||||||
|
name: h.name || h.host,
|
||||||
|
available: ['Unknown','Available','Unavailable'][+h.available] || 'Unknown',
|
||||||
|
enabled: h.status === '0',
|
||||||
|
ip: h.interfaces?.[0]?.ip,
|
||||||
|
})));
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function zabbixGetItems(hostid) {
|
||||||
|
return api('item.get', {
|
||||||
|
output: ['itemid', 'name', 'key_', 'lastvalue', 'units', 'lastclock'],
|
||||||
|
hostids: hostid,
|
||||||
|
monitored: true,
|
||||||
|
sortfield: 'name',
|
||||||
|
}).then(items => items.map(i => ({
|
||||||
|
name: i.name,
|
||||||
|
key: i.key_,
|
||||||
|
value: i.lastvalue,
|
||||||
|
units: i.units,
|
||||||
|
updated: new Date(+i.lastclock * 1000).toISOString(),
|
||||||
|
})));
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function zabbixGetTriggers(hostid) {
|
||||||
|
const params = { output: ['triggerid','description','priority','value'], active: 1, monitored: 1 };
|
||||||
|
if (hostid) params.hostids = hostid;
|
||||||
|
return api('trigger.get', params).then(triggers => triggers.map(t => ({
|
||||||
|
id: t.triggerid,
|
||||||
|
name: t.description,
|
||||||
|
severity: ['Not classified','Information','Warning','Average','High','Disaster'][+t.priority],
|
||||||
|
problem: t.value === '1',
|
||||||
|
})));
|
||||||
|
}
|
||||||
2
agap-mcp/start.sh
Normal file
2
agap-mcp/start.sh
Normal file
@@ -0,0 +1,2 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
exec node src/server.js
|
||||||
11
anki/Dockerfile
Normal file
11
anki/Dockerfile
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
FROM python:3.11-slim
|
||||||
|
|
||||||
|
RUN pip install --no-cache-dir anki
|
||||||
|
|
||||||
|
EXPOSE 8080
|
||||||
|
|
||||||
|
VOLUME /anki_data
|
||||||
|
|
||||||
|
ENV SYNC_BASE=/anki_data
|
||||||
|
|
||||||
|
CMD ["python", "-m", "anki.syncserver"]
|
||||||
17
anki/docker-compose.yml
Normal file
17
anki/docker-compose.yml
Normal file
@@ -0,0 +1,17 @@
|
|||||||
|
name: anki
|
||||||
|
services:
|
||||||
|
app:
|
||||||
|
build: .
|
||||||
|
image: anki-sync-server:local
|
||||||
|
container_name: anki-sync-server
|
||||||
|
restart: always
|
||||||
|
ports:
|
||||||
|
- "127.0.0.1:8180:8080"
|
||||||
|
volumes:
|
||||||
|
- data:/anki_data
|
||||||
|
environment:
|
||||||
|
- SYNC_USER1=${ANKI_USER1:-admin:changeme}
|
||||||
|
- SYNC_BASE=/anki_data
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
data:
|
||||||
108
docker-maintenance/README.md
Normal file
108
docker-maintenance/README.md
Normal file
@@ -0,0 +1,108 @@
|
|||||||
|
# Docker Maintenance — Growth Guard (kb#184)
|
||||||
|
|
||||||
|
## Background
|
||||||
|
|
||||||
|
2026-07-24: root LV (`/`, holds Docker's data-root) filled to 1.5G free / 100% used —
|
||||||
|
risk of ENOSPC corruption across every service on Agap. Emergency reclaim (builder
|
||||||
|
prune + dangling images + stopped containers) took it to 25G free / 90% used.
|
||||||
|
|
||||||
|
2026-07-26 (this task): re-measured before any action — still 24G free / 90% used
|
||||||
|
(the emergency fix hadn't regrown, but hadn't been made recurring either). Ran
|
||||||
|
`prune.sh` live: root went to **40G free / 83% used**. Breakdown before/after:
|
||||||
|
|
||||||
|
| | Before | After |
|
||||||
|
|---|---|---|
|
||||||
|
| Images (total/reclaimable) | 88GB / 62.41GB (70%) | 87.2GB / 62.41GB (71%) |
|
||||||
|
| Build Cache (total/reclaimable) | 18.74GB / 14.49GB | 4.25GB / 0B |
|
||||||
|
| Root free | 24G (90% used) | 40G (83% used) |
|
||||||
|
|
||||||
|
The 62.41GB of image "reclaimable" space barely moved — that's almost entirely
|
||||||
|
**tagged-but-unused** images (`docker image prune -a` territory), not dangling ones.
|
||||||
|
See the decision below.
|
||||||
|
|
||||||
|
## What `prune.sh` does (safe scope only)
|
||||||
|
|
||||||
|
```
|
||||||
|
docker builder prune -f # build cache — always safe, fully rebuildable
|
||||||
|
docker container prune -f # stopped/exited containers only
|
||||||
|
docker image prune -f # DANGLING images only (untagged) — no -a
|
||||||
|
```
|
||||||
|
|
||||||
|
Logs before/after `docker system df` to `prune.log`, and pushes the post-prune
|
||||||
|
build-cache-reclaimable size (bytes) to Zabbix host `AgapHost` (hostid 10776),
|
||||||
|
trapper item `docker.buildcache.reclaimable.bytes` (itemid 70624, **type 2 —
|
||||||
|
confirmed to accept `history.push` on this Zabbix instance, unlike the type-2
|
||||||
|
"Calculated" items other tasks found dead; verified live 2026-07-26**).
|
||||||
|
|
||||||
|
Test without changing anything:
|
||||||
|
```bash
|
||||||
|
bash /home/alvis/agap_git/docker-maintenance/prune.sh --dry-run
|
||||||
|
```
|
||||||
|
|
||||||
|
## Explicitly NOT automated (human decision required)
|
||||||
|
|
||||||
|
- **`docker image prune -a`** — would reclaim ~62GB of tagged-but-currently-unused
|
||||||
|
images (e.g. `cognee/cognee-mcp:1.2.2` at 17.9GB, `cognee` variants, old
|
||||||
|
`ghcr.io/open-webui/open-webui`, `lscr.io/linuxserver/calibre`/`zotero` at
|
||||||
|
3.5GB each — many belong to services that are stopped/replaced but the image
|
||||||
|
may still be wanted for a quick restart). **Decision recorded, not made**:
|
||||||
|
someone with knowledge of which of these services are truly retired should
|
||||||
|
either (a) run `docker image prune -a` manually after reviewing the image
|
||||||
|
list (`docker system df -v`), or (b) curate a keep-list and prune around it.
|
||||||
|
Not scheduled — this script will never run `-a`.
|
||||||
|
- **`docker volume prune`** — never automated; can destroy live data for a
|
||||||
|
volume that's temporarily unmounted. Not touched by this script or its guard.
|
||||||
|
|
||||||
|
## Zabbix
|
||||||
|
|
||||||
|
- Item: `docker.buildcache.reclaimable.bytes` on host `AgapHost` (hostid 10776,
|
||||||
|
itemid 70624), type Trapper, units B. Pushed once per `prune.sh` run (post-prune
|
||||||
|
value — after a scheduled run this should almost always read near 0).
|
||||||
|
- Trigger (triggerid 32970): fires if `last()` of that item exceeds 20GB
|
||||||
|
(21474836480 bytes) — i.e. the build cache grew back past the safe threshold
|
||||||
|
between scheduled runs, meaning the timer isn't running or needs to run more
|
||||||
|
often.
|
||||||
|
- Zabbix runs on **lizacer** (`http://192.168.1.4:81`), not Agap.
|
||||||
|
|
||||||
|
## Scheduled run — NOT INSTALLED (handoff required)
|
||||||
|
|
||||||
|
This directory contains the tested script only. Installing the schedule is a
|
||||||
|
human step (per Agap automation policy: no unattended agent installs a cron/
|
||||||
|
timer that acts on a live target). To install:
|
||||||
|
|
||||||
|
**Cron** (matches the existing `agap_git/kanboard` pattern — user crontab, no sudo):
|
||||||
|
```bash
|
||||||
|
crontab -e
|
||||||
|
# add:
|
||||||
|
0 4 * * * /home/alvis/agap_git/docker-maintenance/prune.sh >> /home/alvis/agap_git/docker-maintenance/prune.log 2>&1
|
||||||
|
```
|
||||||
|
Runs daily at 04:00. Adjust frequency if build-cache growth between runs proves
|
||||||
|
faster than expected (watch the Zabbix trigger above).
|
||||||
|
|
||||||
|
Alternative — **systemd user timer**, unit files provided in this directory
|
||||||
|
(`docker-prune.service`, `docker-prune.timer`), not installed:
|
||||||
|
```bash
|
||||||
|
mkdir -p ~/.config/systemd/user
|
||||||
|
cp /home/alvis/agap_git/docker-maintenance/docker-prune.service ~/.config/systemd/user/
|
||||||
|
cp /home/alvis/agap_git/docker-maintenance/docker-prune.timer ~/.config/systemd/user/
|
||||||
|
systemctl --user daemon-reload
|
||||||
|
systemctl --user enable --now docker-prune.timer
|
||||||
|
```
|
||||||
|
|
||||||
|
Either mechanism is fine; cron matches existing Agap convention (kanboard
|
||||||
|
healthcheck/backup) so it's the recommended default.
|
||||||
|
|
||||||
|
## Acceptance status (kb#184)
|
||||||
|
|
||||||
|
- [x] Immediate reclaim (2026-07-24, prior task) — done.
|
||||||
|
- [x] Recurring prune script — written, tested (dry-run + live run confirmed
|
||||||
|
it reclaims builder cache + dangling images safely).
|
||||||
|
- [ ] **Schedule installed** — script + cron line + systemd unit files are ready;
|
||||||
|
installing them is a human step (see above). **Handoff: run the `crontab -e`
|
||||||
|
command above, or install the systemd timer.**
|
||||||
|
- [x] Build-cache Zabbix trigger — created (triggerid 32970), verified
|
||||||
|
`history.push` lands (item read back 14490000000 during test, then 0
|
||||||
|
after the live prune run).
|
||||||
|
- [x] `docker image prune -a` decision — recorded above as an open human
|
||||||
|
decision, not automated.
|
||||||
|
- [x] df / free space — 40G free (83% used) as of 2026-07-26, up from 24G/90%.
|
||||||
6
docker-maintenance/docker-prune.service
Normal file
6
docker-maintenance/docker-prune.service
Normal file
@@ -0,0 +1,6 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=Docker growth guard - safe prune (kb#184)
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=oneshot
|
||||||
|
ExecStart=/home/alvis/agap_git/docker-maintenance/prune.sh
|
||||||
9
docker-maintenance/docker-prune.timer
Normal file
9
docker-maintenance/docker-prune.timer
Normal file
@@ -0,0 +1,9 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=Run docker growth guard prune daily (kb#184)
|
||||||
|
|
||||||
|
[Timer]
|
||||||
|
OnCalendar=*-*-* 04:00:00
|
||||||
|
Persistent=true
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=timers.target
|
||||||
111
docker-maintenance/prune.sh
Executable file
111
docker-maintenance/prune.sh
Executable file
@@ -0,0 +1,111 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Docker growth guard — recurring safe prune (kb#184, stability audit 2026-07-24).
|
||||||
|
#
|
||||||
|
# BACKGROUND: root LV (/) holds Docker's data-root. On 2026-07-24 it filled to
|
||||||
|
# 1.5G free / 100% used, risking ENOSPC corruption across every service on Agap.
|
||||||
|
# A one-off safe reclaim (builder prune + dangling images + stopped containers)
|
||||||
|
# took it from 1.5G to 25G free. This script is the recurring guard so it can't
|
||||||
|
# silently refill between now and the next manual audit.
|
||||||
|
#
|
||||||
|
# SAFE SCOPE ONLY (learned from the 2026-07-24 incident — do not expand without
|
||||||
|
# a human decision, see README.md in this directory):
|
||||||
|
# - docker builder prune -f (build cache; always safe, fully rebuildable)
|
||||||
|
# - docker container prune -f (stopped/exited containers only)
|
||||||
|
# - docker image prune -f (DANGLING images only — untagged, no -a)
|
||||||
|
#
|
||||||
|
# EXPLICITLY NOT DONE HERE (human judgment call, see README.md):
|
||||||
|
# - docker image prune -a (removes TAGGED but currently-unused images —
|
||||||
|
# could remove images a human wants kept)
|
||||||
|
# - docker volume prune (destroys live data if a volume is unmounted
|
||||||
|
# but still wanted — never run unattended)
|
||||||
|
#
|
||||||
|
# Pushes freed-space and post-prune build-cache-reclaimable metrics to Zabbix
|
||||||
|
# (trapper items on host AgapHost, hostid 10776). Zabbix lives on lizacer
|
||||||
|
# (192.168.1.4:81), not Agap.
|
||||||
|
#
|
||||||
|
# Run manually to test: bash prune.sh --dry-run
|
||||||
|
# Scheduled via cron (see README.md for the exact line — NOT installed by this
|
||||||
|
# script; installation is a human/handoff step, not something this script does).
|
||||||
|
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
LOG_FILE="/home/alvis/agap_git/docker-maintenance/prune.log"
|
||||||
|
ZABBIX_TOKEN_FILE="/home/alvis/.zabbix_token"
|
||||||
|
ZABBIX_URL="http://192.168.1.4:81/api_jsonrpc.php"
|
||||||
|
ZABBIX_ITEM_ID="70624" # docker.buildcache.reclaimable.bytes on host AgapHost (10776)
|
||||||
|
|
||||||
|
DRY_RUN=0
|
||||||
|
if [[ "${1:-}" == "--dry-run" ]]; then
|
||||||
|
DRY_RUN=1
|
||||||
|
fi
|
||||||
|
|
||||||
|
log() {
|
||||||
|
echo "$(date '+%Y-%m-%d %H:%M:%S') $*"
|
||||||
|
}
|
||||||
|
|
||||||
|
{
|
||||||
|
log "=== docker-maintenance prune.sh start (dry_run=$DRY_RUN) ==="
|
||||||
|
|
||||||
|
log "--- before ---"
|
||||||
|
df -h / | tail -n +2
|
||||||
|
docker system df
|
||||||
|
|
||||||
|
if [[ "$DRY_RUN" -eq 1 ]]; then
|
||||||
|
log "DRY RUN: would run: docker builder prune -f"
|
||||||
|
log "DRY RUN: would run: docker container prune -f"
|
||||||
|
log "DRY RUN: would run: docker image prune -f (dangling only)"
|
||||||
|
else
|
||||||
|
log "Running: docker builder prune -f"
|
||||||
|
docker builder prune -f || true
|
||||||
|
|
||||||
|
log "Running: docker container prune -f"
|
||||||
|
docker container prune -f || true
|
||||||
|
|
||||||
|
log "Running: docker image prune -f (dangling only, no -a)"
|
||||||
|
docker image prune -f || true
|
||||||
|
fi
|
||||||
|
|
||||||
|
log "--- after ---"
|
||||||
|
df -h / | tail -n +2
|
||||||
|
DF_JSON=$(docker system df --format '{{json .}}')
|
||||||
|
echo "$DF_JSON"
|
||||||
|
|
||||||
|
# Extract build-cache reclaimable bytes for Zabbix. `docker system df` doesn't
|
||||||
|
# give bytes directly (only human-readable strings like "14.49GB"), so use
|
||||||
|
# `docker system df -v` reclaimable percentage isn't reliable either; instead
|
||||||
|
# compute it from `docker builder du`-equivalent: sum of build cache records
|
||||||
|
# not marked Shared/in-use is nontrivial to script robustly, so fall back to
|
||||||
|
# parsing the "Build Cache" line's RECLAIMABLE column via docker system df.
|
||||||
|
RECLAIM_STR=$(docker system df | awk '/^Build Cache/ {print $NF}')
|
||||||
|
# RECLAIM_STR looks like "14.49GB" or "0B" -- convert to bytes (approx, GB/MB/KB = *1000^n, matching docker's own decimal convention)
|
||||||
|
RECLAIM_BYTES=$(python3 -c "
|
||||||
|
import re, sys
|
||||||
|
s = '$RECLAIM_STR'
|
||||||
|
m = re.match(r'([0-9.]+)\s*([KMGTP]?B)', s)
|
||||||
|
if not m:
|
||||||
|
print(0)
|
||||||
|
else:
|
||||||
|
val, unit = float(m.group(1)), m.group(2)
|
||||||
|
mult = {'B':1,'KB':1000,'MB':1000**2,'GB':1000**3,'TB':1000**4,'PB':1000**5}[unit]
|
||||||
|
print(int(val*mult))
|
||||||
|
" 2>/dev/null || echo 0)
|
||||||
|
|
||||||
|
log "Build cache reclaimable: $RECLAIM_STR (~$RECLAIM_BYTES bytes)"
|
||||||
|
|
||||||
|
if [[ -f "$ZABBIX_TOKEN_FILE" && "$DRY_RUN" -eq 0 ]]; then
|
||||||
|
ZABBIX_TOKEN=$(cat "$ZABBIX_TOKEN_FILE")
|
||||||
|
env -u HTTPS_PROXY -u HTTP_PROXY -u ALL_PROXY -u https_proxy -u http_proxy -u all_proxy \
|
||||||
|
curl -s -X POST "$ZABBIX_URL" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-H "Authorization: Bearer $ZABBIX_TOKEN" \
|
||||||
|
-d "{\"jsonrpc\":\"2.0\",\"method\":\"history.push\",\"id\":1,\"params\":{\"itemid\":\"$ZABBIX_ITEM_ID\",\"value\":$RECLAIM_BYTES}}" > /dev/null \
|
||||||
|
&& log "Zabbix notified (docker.buildcache.reclaimable.bytes=$RECLAIM_BYTES)"
|
||||||
|
else
|
||||||
|
log "Skipped Zabbix push (dry-run or token file missing)"
|
||||||
|
fi
|
||||||
|
|
||||||
|
AVAIL=$(df / | tail -1 | awk '{print $4}')
|
||||||
|
log "Root free: ${AVAIL}KB"
|
||||||
|
|
||||||
|
log "=== docker-maintenance prune.sh end ==="
|
||||||
|
} | tee -a "$LOG_FILE"
|
||||||
6
family/Dockerfile
Normal file
6
family/Dockerfile
Normal file
@@ -0,0 +1,6 @@
|
|||||||
|
FROM mediawiki:latest
|
||||||
|
|
||||||
|
RUN apt-get update && apt-get install -y ffmpeg unzip && rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
RUN curl -sL "https://extdist.wmflabs.org/dist/extensions/TimedMediaHandler-REL1_44-ef5edcf.tar.gz" \
|
||||||
|
| tar -xz -C /var/www/html/extensions/
|
||||||
BIN
family/IMG_0448.JPG
Normal file
BIN
family/IMG_0448.JPG
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 124 KiB |
165
family/LocalSettings.php
Normal file
165
family/LocalSettings.php
Normal file
@@ -0,0 +1,165 @@
|
|||||||
|
<?php
|
||||||
|
# This file was automatically generated by the MediaWiki 1.45.3
|
||||||
|
# installer. If you make manual changes, please keep track in case you
|
||||||
|
# need to recreate them later.
|
||||||
|
#
|
||||||
|
# See includes/MainConfigSchema.php for all configurable settings
|
||||||
|
# and their default values, but don't forget to make changes in _this_
|
||||||
|
# file, not there.
|
||||||
|
#
|
||||||
|
# Further documentation for configuration settings may be found at:
|
||||||
|
# https://www.mediawiki.org/wiki/Manual:Configuration_settings
|
||||||
|
|
||||||
|
# Protect against web entry
|
||||||
|
if ( !defined( 'MEDIAWIKI' ) ) {
|
||||||
|
exit;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## Uncomment this to disable output compression
|
||||||
|
# $wgDisableOutputCompression = true;
|
||||||
|
|
||||||
|
$wgSitename = "РодоВики";
|
||||||
|
|
||||||
|
## The URL base path to the directory containing the wiki;
|
||||||
|
## defaults for all runtime URL paths are based off of this.
|
||||||
|
## For more information on customizing the URLs
|
||||||
|
## (like /w/index.php/Page_title to /wiki/Page_title) please see:
|
||||||
|
## https://www.mediawiki.org/wiki/Manual:Short_URL
|
||||||
|
$wgScriptPath = "";
|
||||||
|
|
||||||
|
## The protocol and server name to use in fully-qualified URLs
|
||||||
|
$wgServer = "https://family.alogins.net";
|
||||||
|
|
||||||
|
## The URL path to static resources (images, scripts, etc.)
|
||||||
|
$wgResourceBasePath = $wgScriptPath;
|
||||||
|
|
||||||
|
## The URL paths to the logo. Make sure you change this from the default,
|
||||||
|
## or else you'll overwrite your logo when you upgrade!
|
||||||
|
$wgLogos = [
|
||||||
|
'1x' => "$wgResourceBasePath/images/logo.jpg",
|
||||||
|
'icon' => "$wgResourceBasePath/images/logo.jpg",
|
||||||
|
];
|
||||||
|
|
||||||
|
## UPO means: this is also a user preference option
|
||||||
|
|
||||||
|
$wgEnableEmail = true;
|
||||||
|
$wgEnableUserEmail = true; # UPO
|
||||||
|
|
||||||
|
$wgEmergencyContact = "";
|
||||||
|
$wgPasswordSender = "";
|
||||||
|
|
||||||
|
$wgEnotifUserTalk = false; # UPO
|
||||||
|
$wgEnotifWatchlist = false; # UPO
|
||||||
|
$wgEmailAuthentication = true;
|
||||||
|
|
||||||
|
## Database settings
|
||||||
|
$wgDBtype = "mysql";
|
||||||
|
$wgDBserver = "db";
|
||||||
|
$wgDBname = "mediawiki";
|
||||||
|
$wgDBuser = "mw_k7px2q";
|
||||||
|
$wgDBpassword = "Vt9#mLqR4wXn8bZ2";
|
||||||
|
|
||||||
|
# MySQL specific settings
|
||||||
|
$wgDBprefix = "fw";
|
||||||
|
$wgDBssl = false;
|
||||||
|
|
||||||
|
# MySQL table options to use during installation or update
|
||||||
|
$wgDBTableOptions = "ENGINE=InnoDB, DEFAULT CHARSET=binary";
|
||||||
|
|
||||||
|
# Shared database table
|
||||||
|
# This has no effect unless $wgSharedDB is also set.
|
||||||
|
$wgSharedTables[] = "actor";
|
||||||
|
|
||||||
|
## Shared memory settings
|
||||||
|
$wgMainCacheType = CACHE_ACCEL;
|
||||||
|
$wgMemCachedServers = [];
|
||||||
|
|
||||||
|
## To enable image uploads, make sure the 'images' directory
|
||||||
|
## is writable, then set this to true:
|
||||||
|
$wgEnableUploads = true;
|
||||||
|
$wgMaxUploadSize = 20 * 1024 * 1024; // 20MB
|
||||||
|
$wgFileExtensions = array_merge( $wgFileExtensions, [
|
||||||
|
'png', 'gif', 'jpg', 'jpeg', 'webp',
|
||||||
|
'mp4', 'webm', 'ogv',
|
||||||
|
'mp3', 'ogg', 'oga', 'wav', 'flac',
|
||||||
|
] );
|
||||||
|
$wgUseImageMagick = true;
|
||||||
|
$wgImageMagickConvertCommand = "/usr/bin/convert";
|
||||||
|
|
||||||
|
# InstantCommons allows wiki to use images from https://commons.wikimedia.org
|
||||||
|
$wgUseInstantCommons = false;
|
||||||
|
|
||||||
|
# Periodically send a pingback to https://www.mediawiki.org/ with basic data
|
||||||
|
# about this MediaWiki instance. The Wikimedia Foundation shares this data
|
||||||
|
# with MediaWiki developers to help guide future development efforts.
|
||||||
|
$wgPingback = true;
|
||||||
|
|
||||||
|
# Site language code, should be one of the list in ./includes/languages/data/Names.php
|
||||||
|
$wgLanguageCode = "ru";
|
||||||
|
|
||||||
|
# Time zone
|
||||||
|
$wgLocaltimezone = "UTC";
|
||||||
|
|
||||||
|
## Set $wgCacheDirectory to a writable directory on the web server
|
||||||
|
## to make your wiki go slightly faster. The directory should not
|
||||||
|
## be publicly accessible from the web.
|
||||||
|
#$wgCacheDirectory = "$IP/cache";
|
||||||
|
|
||||||
|
$wgSecretKey = "5156b70f5486793efade503a2301caf53b9fe241505868a90e33509814fb7d1f";
|
||||||
|
|
||||||
|
# Changing this will log out all existing sessions.
|
||||||
|
$wgAuthenticationTokenVersion = "1";
|
||||||
|
|
||||||
|
# Bust ResourceLoader CSS cache
|
||||||
|
$wgCacheEpoch = '20260403000001';
|
||||||
|
|
||||||
|
# Site upgrade key. Must be set to a string (default provided) to turn on the
|
||||||
|
# web installer while LocalSettings.php is in place
|
||||||
|
$wgUpgradeKey = "da11c2f2774f498f";
|
||||||
|
|
||||||
|
## For attaching licensing metadata to pages, and displaying an
|
||||||
|
## appropriate copyright notice / icon. GNU Free Documentation
|
||||||
|
## License and Creative Commons licenses are supported so far.
|
||||||
|
$wgRightsPage = ""; # Set to the title of a wiki page that describes your license/copyright
|
||||||
|
$wgRightsUrl = "";
|
||||||
|
$wgRightsText = "";
|
||||||
|
$wgRightsIcon = "";
|
||||||
|
|
||||||
|
# Path to the GNU diff3 utility. Used for conflict resolution.
|
||||||
|
$wgDiff3 = "/usr/bin/diff3";
|
||||||
|
|
||||||
|
## Default skin: you can change the default skin. Use the internal symbolic
|
||||||
|
## names, e.g. 'vector' or 'monobook':
|
||||||
|
$wgDefaultSkin = "vector-2022";
|
||||||
|
|
||||||
|
# Enabled skins.
|
||||||
|
# The following skins were automatically enabled:
|
||||||
|
wfLoadSkin( 'MinervaNeue' );
|
||||||
|
wfLoadSkin( 'MonoBook' );
|
||||||
|
wfLoadSkin( 'Timeless' );
|
||||||
|
wfLoadSkin( 'Vector' );
|
||||||
|
|
||||||
|
|
||||||
|
wfLoadExtension( 'Cite' );
|
||||||
|
wfLoadExtension( 'MultimediaViewer' );
|
||||||
|
wfLoadExtension( 'ParserFunctions' );
|
||||||
|
wfLoadExtension( 'VisualEditor' );
|
||||||
|
wfLoadExtension( 'TimedMediaHandler' );
|
||||||
|
|
||||||
|
$wgDefaultUserOptions['visualeditor-enable'] = 1;
|
||||||
|
$wgDefaultUserOptions['visualeditor-editor'] = 'visualeditor';
|
||||||
|
$wgVisualEditorParsoidAutoConfig = true;
|
||||||
|
|
||||||
|
# End of automatically generated settings.
|
||||||
|
# Add more configuration options below.
|
||||||
|
|
||||||
|
# Restrict all access to logged-in users only
|
||||||
|
$wgGroupPermissions['*']['read'] = false;
|
||||||
|
$wgGroupPermissions['*']['edit'] = false;
|
||||||
|
$wgGroupPermissions['*']['createaccount'] = false;
|
||||||
|
|
||||||
|
# Only admins can create accounts
|
||||||
|
$wgGroupPermissions['sysop']['createaccount'] = true;
|
||||||
32
family/docker-compose.yml
Normal file
32
family/docker-compose.yml
Normal file
@@ -0,0 +1,32 @@
|
|||||||
|
services:
|
||||||
|
mediawiki:
|
||||||
|
build: .
|
||||||
|
image: mediawiki-tmh:latest
|
||||||
|
restart: unless-stopped
|
||||||
|
ports:
|
||||||
|
- "8099:80"
|
||||||
|
volumes:
|
||||||
|
- /mnt/ssd/dbs/wiki/mediawiki_images:/var/www/html/images
|
||||||
|
- ./LocalSettings.php:/var/www/html/LocalSettings.php # uncomment after initial setup
|
||||||
|
- ./IMG_0448.JPG:/var/www/html/images/logo.jpg
|
||||||
|
- ./uploads.ini:/usr/local/etc/php/conf.d/uploads.ini
|
||||||
|
environment:
|
||||||
|
MEDIAWIKI_DB_HOST: db
|
||||||
|
MEDIAWIKI_DB_NAME: mediawiki
|
||||||
|
MEDIAWIKI_DB_USER: mw_k7px2q
|
||||||
|
MEDIAWIKI_DB_PASSWORD: Vt9#mLqR4wXn8bZ2
|
||||||
|
depends_on:
|
||||||
|
- db
|
||||||
|
|
||||||
|
db:
|
||||||
|
image: mariadb:lts
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
MYSQL_DATABASE: mediawiki
|
||||||
|
MYSQL_USER: mw_k7px2q
|
||||||
|
MYSQL_PASSWORD: Vt9#mLqR4wXn8bZ2
|
||||||
|
MYSQL_RANDOM_ROOT_PASSWORD: "yes"
|
||||||
|
volumes:
|
||||||
|
- /mnt/ssd/dbs/wiki/mediawiki_db:/var/lib/mysql
|
||||||
|
|
||||||
|
|
||||||
458
family/migrate.py
Normal file
458
family/migrate.py
Normal file
@@ -0,0 +1,458 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""OtterWiki → MediaWiki migration script."""
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import requests
|
||||||
|
|
||||||
|
REPO = Path('/mnt/ssd/dbs/otter/app-data/repository')
|
||||||
|
API = 'http://localhost:8099/api.php'
|
||||||
|
|
||||||
|
_FN_DEF = re.compile(r'^\[(\^[^\]]+)\]:\s*(.*)')
|
||||||
|
|
||||||
|
# Cached pandoc availability (None = not yet checked)
|
||||||
|
_PANDOC_AVAILABLE: bool | None = None
|
||||||
|
|
||||||
|
|
||||||
|
def _cap(s: str) -> str:
|
||||||
|
"""Title-case: capitalize first letter of each word."""
|
||||||
|
return s.title() if s else s
|
||||||
|
|
||||||
|
|
||||||
|
def _pandoc_available() -> bool:
|
||||||
|
global _PANDOC_AVAILABLE
|
||||||
|
if _PANDOC_AVAILABLE is None:
|
||||||
|
try:
|
||||||
|
_PANDOC_AVAILABLE = subprocess.run(
|
||||||
|
['pandoc', '--version'], capture_output=True
|
||||||
|
).returncode == 0
|
||||||
|
except FileNotFoundError:
|
||||||
|
_PANDOC_AVAILABLE = False
|
||||||
|
return _PANDOC_AVAILABLE
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# MediaWiki session
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def mw_login(user: str, password: str):
|
||||||
|
s = requests.Session()
|
||||||
|
r = s.get(API, params={'action': 'query', 'meta': 'tokens', 'type': 'login', 'format': 'json'})
|
||||||
|
token = r.json()['query']['tokens']['logintoken']
|
||||||
|
s.post(API, data={'action': 'login', 'lgname': user, 'lgpassword': password,
|
||||||
|
'lgtoken': token, 'format': 'json'})
|
||||||
|
r = s.get(API, params={'action': 'query', 'meta': 'tokens', 'format': 'json'})
|
||||||
|
csrf = r.json()['query']['tokens']['csrftoken']
|
||||||
|
return s, csrf
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Title determination
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def page_title(md_path: Path) -> str:
|
||||||
|
parts = md_path.relative_to(REPO).parts
|
||||||
|
if md_path.name == 'home.md' and len(parts) == 1:
|
||||||
|
return 'Заглавная страница'
|
||||||
|
return _cap(md_path.stem)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Markdown → wikitext conversion
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def convert_pandoc(text: str) -> str:
|
||||||
|
return subprocess.run(
|
||||||
|
['pandoc', '-f', 'markdown', '-t', 'mediawiki'],
|
||||||
|
input=text, capture_output=True, text=True
|
||||||
|
).stdout
|
||||||
|
|
||||||
|
|
||||||
|
def convert_python(text: str, skip_info: bool = False) -> str:
|
||||||
|
lines = text.split('\n')
|
||||||
|
|
||||||
|
# Collect footnote definitions in one pass
|
||||||
|
footnotes: dict[str, str] = {}
|
||||||
|
for line in lines:
|
||||||
|
m = _FN_DEF.match(line)
|
||||||
|
if m:
|
||||||
|
footnotes[m.group(1)] = m.group(2)
|
||||||
|
|
||||||
|
def replace_fn(m):
|
||||||
|
key = m.group(0)[1:-1] # strip outer [ ] to match footnotes dict keys
|
||||||
|
content = footnotes.get(key, m.group(0))
|
||||||
|
content = re.sub(r'\[([^\]^][^\]]*)\]\(([^)]+)\)', r'[\2 \1]', content)
|
||||||
|
return f'<ref>{content}</ref>'
|
||||||
|
|
||||||
|
out = []
|
||||||
|
for line in lines:
|
||||||
|
if _FN_DEF.match(line):
|
||||||
|
continue
|
||||||
|
|
||||||
|
m = re.match(r'^(#{1,6})\s+(.*)', line)
|
||||||
|
if m:
|
||||||
|
eq = '=' * len(m.group(1))
|
||||||
|
out.append(f'{eq} {m.group(2)} {eq}')
|
||||||
|
continue
|
||||||
|
|
||||||
|
if re.match(r'^---+$', line.strip()):
|
||||||
|
out.append('----')
|
||||||
|
continue
|
||||||
|
|
||||||
|
line = re.sub(r'^(\s*)-(\s)', r'\1*\2', line)
|
||||||
|
line = re.sub(r'\*\*\*(.+?)\*\*\*', r"'''''\1'''''", line)
|
||||||
|
line = re.sub(r'\*\*(.+?)\*\*', r"'''\1'''", line)
|
||||||
|
line = re.sub(r'\*(.+?)\*', r"''\1''", line)
|
||||||
|
line = re.sub(r'(?<!!)\[([^\]^!][^\]]*)\]\(([^)]+)\)', r'[\2 \1]', line)
|
||||||
|
line = re.sub(r'\[\^\S+?\]', replace_fn, line)
|
||||||
|
out.append(line)
|
||||||
|
|
||||||
|
result = convert_tables('\n'.join(out), skip_info=skip_info)
|
||||||
|
if footnotes:
|
||||||
|
result += '\n<references />'
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def _split_cells(line: str) -> list[str]:
|
||||||
|
"""Split a markdown table row on | but not inside [[ ]]."""
|
||||||
|
cells = []
|
||||||
|
depth = 0
|
||||||
|
current = []
|
||||||
|
i = 0
|
||||||
|
# Strip leading/trailing |
|
||||||
|
line = line.strip()
|
||||||
|
if line.startswith('|'):
|
||||||
|
line = line[1:]
|
||||||
|
if line.endswith('|'):
|
||||||
|
line = line[:-1]
|
||||||
|
while i < len(line):
|
||||||
|
if line[i:i+2] == '[[':
|
||||||
|
depth += 1
|
||||||
|
current.append('[[')
|
||||||
|
i += 2
|
||||||
|
elif line[i:i+2] == ']]':
|
||||||
|
depth -= 1
|
||||||
|
current.append(']]')
|
||||||
|
i += 2
|
||||||
|
elif line[i] == '|' and depth == 0:
|
||||||
|
cells.append(''.join(current).strip())
|
||||||
|
current = []
|
||||||
|
i += 1
|
||||||
|
else:
|
||||||
|
current.append(line[i])
|
||||||
|
i += 1
|
||||||
|
cells.append(''.join(current).strip())
|
||||||
|
return cells
|
||||||
|
|
||||||
|
|
||||||
|
def convert_tables(text: str, skip_info: bool = False) -> str:
|
||||||
|
lines = text.split('\n')
|
||||||
|
out = []
|
||||||
|
in_table = False
|
||||||
|
skip_table = False
|
||||||
|
|
||||||
|
for line in lines:
|
||||||
|
if re.match(r'^\|', line):
|
||||||
|
cells = _split_cells(line)
|
||||||
|
if all(re.match(r'^:?-+:?$', c) for c in cells if c):
|
||||||
|
# Separator row: start or continue table
|
||||||
|
if not in_table:
|
||||||
|
header_line = out.pop() if out else ''
|
||||||
|
hcells = _split_cells(header_line)
|
||||||
|
# Blank-header table (all-empty header cells) = OtterWiki info table
|
||||||
|
if skip_info and all(c == '' for c in hcells):
|
||||||
|
skip_table = True
|
||||||
|
in_table = True
|
||||||
|
else:
|
||||||
|
out += ['{| class="wikitable"', '|-', '! ' + ' !! '.join(hcells)]
|
||||||
|
in_table = True
|
||||||
|
skip_table = False
|
||||||
|
if not skip_table:
|
||||||
|
out.append('|-')
|
||||||
|
else:
|
||||||
|
if in_table:
|
||||||
|
if not skip_table:
|
||||||
|
out.append('|-')
|
||||||
|
out.append('| ' + ' || '.join(cells))
|
||||||
|
# else: skip info table row
|
||||||
|
else:
|
||||||
|
out.append(line)
|
||||||
|
else:
|
||||||
|
if in_table:
|
||||||
|
if not skip_table:
|
||||||
|
out.append('|}')
|
||||||
|
in_table = False
|
||||||
|
skip_table = False
|
||||||
|
out.append(line)
|
||||||
|
|
||||||
|
if in_table and not skip_table:
|
||||||
|
out.append('|}')
|
||||||
|
|
||||||
|
return '\n'.join(out)
|
||||||
|
|
||||||
|
|
||||||
|
_PERSON_FIELD_MAP = {
|
||||||
|
'родился': 'родился', 'родилась': 'родился',
|
||||||
|
'умер': 'умер', 'умерла': 'умер',
|
||||||
|
'отец': 'отец', 'мать': 'мать',
|
||||||
|
'супруг': 'супруг', 'супруга': 'супруг', 'муж': 'супруг', 'жена': 'супруг',
|
||||||
|
'дети': 'дети', 'ребёнок': 'дети',
|
||||||
|
'братья': 'братья', 'брат': 'братья', 'сестра': 'братья',
|
||||||
|
'сёстры': 'братья', 'сестры': 'братья',
|
||||||
|
'место рождения': 'место_рождения', 'место_рождения': 'место_рождения',
|
||||||
|
'прочее': 'прочее',
|
||||||
|
}
|
||||||
|
_PERSONA_PARAM_ORDER = ['родился', 'место_рождения', 'умер', 'отец', 'мать', 'супруг', 'дети', 'братья', 'прочее']
|
||||||
|
|
||||||
|
_PLACE_FIELD_MAP = {
|
||||||
|
'тип': 'тип',
|
||||||
|
'статус': 'статус',
|
||||||
|
'страна': 'страна',
|
||||||
|
'регион': 'регион', 'область': 'регион',
|
||||||
|
'район': 'район', 'расположение': 'район', 'самоуправление': 'район',
|
||||||
|
'река': 'река',
|
||||||
|
'основана': 'основана', 'основан': 'основана',
|
||||||
|
'население': 'население',
|
||||||
|
'адрес': 'адрес',
|
||||||
|
'период': 'период', 'годы': 'период',
|
||||||
|
'жильцы': 'жильцы', 'жильцы/семья': 'жильцы', 'семейное имя': 'прочее',
|
||||||
|
'латв. название': 'назв_латыш',
|
||||||
|
'белор. название': 'назв_белор',
|
||||||
|
'координаты': 'координаты',
|
||||||
|
'сайт': 'сайт',
|
||||||
|
'телефон': 'телефон', 'email': 'телефон',
|
||||||
|
'полное название': 'прочее', 'классы': 'прочее',
|
||||||
|
'штаб-квартира': 'прочее', 'сотрудников': 'прочее',
|
||||||
|
}
|
||||||
|
_PLACE_PARAM_ORDER = ['тип', 'статус', 'страна', 'регион', 'район', 'река', 'основана',
|
||||||
|
'население', 'адрес', 'период', 'жильцы', 'назв_латыш', 'назв_белор',
|
||||||
|
'координаты', 'сайт', 'телефон', 'прочее']
|
||||||
|
|
||||||
|
|
||||||
|
def _extract_infobox(text: str, title: str, name_param: str, template: str,
|
||||||
|
field_map: dict, param_order: list) -> tuple[str, str]:
|
||||||
|
"""Extract photo + blank-header info table; return ({{Template|...}}, cleaned_text)."""
|
||||||
|
lines = text.split('\n')
|
||||||
|
photo = None
|
||||||
|
fields: dict[str, str] = {}
|
||||||
|
remove: set[int] = set()
|
||||||
|
|
||||||
|
for i, line in enumerate(lines):
|
||||||
|
if re.match(r'^\|', line):
|
||||||
|
break # reached info table — stop looking for photo
|
||||||
|
m = re.match(r'^\s*\[!\[[^\]]*\]\(\./(?:[^/)]+/)?([^)?]+?)(?:\?[^)]*)?\)\]', line)
|
||||||
|
if m:
|
||||||
|
photo = m.group(1)
|
||||||
|
remove.add(i)
|
||||||
|
break
|
||||||
|
|
||||||
|
in_table = False
|
||||||
|
for i, line in enumerate(lines):
|
||||||
|
if re.match(r'^\|', line):
|
||||||
|
cells = _split_cells(line)
|
||||||
|
if all(re.match(r'^:?-+:?$', c) for c in cells if c):
|
||||||
|
if not in_table:
|
||||||
|
prev = i - 1
|
||||||
|
if prev >= 0 and re.match(r'^\|', lines[prev]):
|
||||||
|
hcells = _split_cells(lines[prev])
|
||||||
|
if all(c == '' for c in hcells):
|
||||||
|
in_table = True
|
||||||
|
remove.add(prev)
|
||||||
|
if in_table:
|
||||||
|
remove.add(i)
|
||||||
|
elif in_table:
|
||||||
|
remove.add(i)
|
||||||
|
if len(cells) >= 2:
|
||||||
|
key = re.sub(r'\*\*(.+?)\*\*', r'\1', cells[0]).strip().lower()
|
||||||
|
val = re.sub(r'\*\*(.+?)\*\*', r"'''\1'''", cells[1].strip())
|
||||||
|
param = field_map.get(key)
|
||||||
|
if param and param not in fields:
|
||||||
|
fields[param] = val
|
||||||
|
elif in_table:
|
||||||
|
in_table = False
|
||||||
|
|
||||||
|
if not photo and not fields:
|
||||||
|
return '', text
|
||||||
|
|
||||||
|
parts = ['{{' + template, f'| {name_param:<16} = {title}']
|
||||||
|
if photo:
|
||||||
|
parts.append(f'| фото = {photo}')
|
||||||
|
for param in param_order:
|
||||||
|
if param in fields:
|
||||||
|
parts.append(f'| {param:<16} = {fields[param]}')
|
||||||
|
infobox = '\n'.join(parts) + '\n}}'
|
||||||
|
cleaned = '\n'.join(line for i, line in enumerate(lines) if i not in remove)
|
||||||
|
return infobox, cleaned
|
||||||
|
|
||||||
|
|
||||||
|
def extract_person_infobox(text: str, title: str) -> tuple[str, str]:
|
||||||
|
return _extract_infobox(text, title, 'имя', 'Персона', _PERSON_FIELD_MAP, _PERSONA_PARAM_ORDER)
|
||||||
|
|
||||||
|
|
||||||
|
def extract_place_infobox(text: str, title: str) -> tuple[str, str]:
|
||||||
|
return _extract_infobox(text, title, 'название', 'Место', _PLACE_FIELD_MAP, _PLACE_PARAM_ORDER)
|
||||||
|
|
||||||
|
|
||||||
|
def strip_first_heading(text: str) -> str:
|
||||||
|
"""Remove the first H1 line — MW displays the page title itself."""
|
||||||
|
return re.sub(r'^#[^#][^\n]*\n?', '', text, count=1)
|
||||||
|
|
||||||
|
|
||||||
|
def convert(text: str, skip_info: bool = False, is_place: bool = False, title: str = '') -> str:
|
||||||
|
text = strip_first_heading(text)
|
||||||
|
infobox = ''
|
||||||
|
if skip_info:
|
||||||
|
infobox, text = extract_person_infobox(text, title)
|
||||||
|
elif is_place:
|
||||||
|
infobox, text = extract_place_infobox(text, title)
|
||||||
|
result = convert_pandoc(text) if _pandoc_available() else convert_python(text, skip_info=skip_info)
|
||||||
|
if infobox:
|
||||||
|
result = infobox + '\n' + result.lstrip('\n')
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Post-processing
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def fix_links(text: str) -> str:
|
||||||
|
pattern = (r'\[\[([^\]|]+)\|'
|
||||||
|
r'(?:Люди|Места|Воспоминания|люди|места|воспоминания|Место)'
|
||||||
|
r'/([^\]]+)\]\]')
|
||||||
|
|
||||||
|
def replace_link(m):
|
||||||
|
display = m.group(1).strip()
|
||||||
|
page = _cap(m.group(2).strip().lower())
|
||||||
|
if display.lower() == page.lower():
|
||||||
|
return f'[[{page}]]'
|
||||||
|
return f'[[{page}|{display}]]'
|
||||||
|
|
||||||
|
text = re.sub(pattern, replace_link, text)
|
||||||
|
# Also handle bare section paths: [[Section/PageName]] → [[PageName]]
|
||||||
|
text = re.sub(
|
||||||
|
r'\[\[(?:Люди|Места|Воспоминания|люди|места|воспоминания|Место)/([^\]|]+)\]\]',
|
||||||
|
lambda m: f'[[{m.group(1).strip().lower().title()}]]',
|
||||||
|
text
|
||||||
|
)
|
||||||
|
return text
|
||||||
|
|
||||||
|
|
||||||
|
def fix_images(text: str) -> str:
|
||||||
|
# Handle linked images: [](./file.jpg)
|
||||||
|
# and plain images: 
|
||||||
|
pattern = r'(?:\[)?!\[[^\]]*\]\(\./(?:[^/)]+/)?([^)?]+?)(\?[^)]*)?\)(?:\]\([^)]*\))?'
|
||||||
|
|
||||||
|
def replace_img(m):
|
||||||
|
filename = m.group(1)
|
||||||
|
size_m = re.search(r'thumbnail=(\d+)', m.group(2) or '')
|
||||||
|
return f'[[File:{filename}|{size_m.group(1)}px]]' if size_m else f'[[File:{filename}]]'
|
||||||
|
|
||||||
|
return re.sub(pattern, replace_img, text)
|
||||||
|
|
||||||
|
|
||||||
|
_CATEGORY_MAP = {'люди': 'Люди', 'места': 'Места', 'воспоминания': 'Воспоминания'}
|
||||||
|
|
||||||
|
|
||||||
|
def category_suffix(md_path: Path) -> str:
|
||||||
|
parts = md_path.relative_to(REPO).parts
|
||||||
|
if len(parts) == 1:
|
||||||
|
return '' if md_path.name == 'home.md' else '\n\n[[Category:Статьи]]'
|
||||||
|
cat = _CATEGORY_MAP.get(parts[0].lower())
|
||||||
|
return f'\n\n[[Category:{cat}]]' if cat else ''
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# MW operations
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def post_page(session, csrf: str, title: str, text: str, dry_run: bool) -> bool:
|
||||||
|
if dry_run:
|
||||||
|
print(f'[DRY] {title}')
|
||||||
|
return True
|
||||||
|
data = session.post(API, data={
|
||||||
|
'action': 'edit', 'title': title, 'text': text,
|
||||||
|
'token': csrf, 'format': 'json'
|
||||||
|
}).json()
|
||||||
|
if 'error' in data:
|
||||||
|
print(f'[ERR] {title}: {data["error"].get("info", data["error"])}')
|
||||||
|
return False
|
||||||
|
print(f'[OK] {title}')
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def upload_image(session, csrf: str, image_path: Path, dry_run: bool) -> bool:
|
||||||
|
basename = image_path.name
|
||||||
|
if dry_run:
|
||||||
|
print(f'[DRY] File:{basename}')
|
||||||
|
return True
|
||||||
|
with open(image_path, 'rb') as f:
|
||||||
|
data = session.post(API, data={
|
||||||
|
'action': 'upload', 'filename': basename,
|
||||||
|
'token': csrf, 'format': 'json', 'ignorewarnings': '1'
|
||||||
|
}, files={'file': f}).json()
|
||||||
|
if 'error' in data:
|
||||||
|
print(f'[ERR] File:{basename}: {data["error"].get("info", data["error"])}')
|
||||||
|
return False
|
||||||
|
if data.get('upload', {}).get('result') == 'Success':
|
||||||
|
print(f'[OK] File:{basename}')
|
||||||
|
else:
|
||||||
|
print(f'[SKIP] File:{basename} (already exists or no change)')
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Main
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def collect_images() -> list[Path]:
|
||||||
|
images = []
|
||||||
|
for folder in ('люди', 'места'):
|
||||||
|
p = REPO / folder
|
||||||
|
if p.exists():
|
||||||
|
for ext in ('*.jpg', '*.jpeg', '*.JPG', '*.JPEG', '*.png', '*.PNG'):
|
||||||
|
images.extend(p.rglob(ext))
|
||||||
|
return images
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
parser = argparse.ArgumentParser(description='Migrate OtterWiki to MediaWiki')
|
||||||
|
parser.add_argument('--user', required=True)
|
||||||
|
parser.add_argument('--password', required=True)
|
||||||
|
parser.add_argument('--dry-run', action='store_true')
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
session = csrf = None
|
||||||
|
if not args.dry_run:
|
||||||
|
session, csrf = mw_login(args.user, args.password)
|
||||||
|
|
||||||
|
pages_ok = pages_err = images_ok = images_err = 0
|
||||||
|
|
||||||
|
for md_path in sorted(REPO.rglob('*.md')):
|
||||||
|
title = page_title(md_path)
|
||||||
|
raw = md_path.read_text(encoding='utf-8')
|
||||||
|
parts = md_path.relative_to(REPO).parts
|
||||||
|
is_people = len(parts) > 0 and parts[0].lower() == 'люди'
|
||||||
|
is_place = len(parts) > 0 and parts[0].lower() == 'места'
|
||||||
|
wikitext = convert(raw, skip_info=is_people, is_place=is_place, title=title)
|
||||||
|
wikitext = fix_links(wikitext)
|
||||||
|
wikitext = fix_images(wikitext)
|
||||||
|
wikitext += category_suffix(md_path)
|
||||||
|
if post_page(session, csrf, title, wikitext, args.dry_run):
|
||||||
|
pages_ok += 1
|
||||||
|
else:
|
||||||
|
pages_err += 1
|
||||||
|
|
||||||
|
for img in collect_images():
|
||||||
|
if upload_image(session, csrf, img, args.dry_run):
|
||||||
|
images_ok += 1
|
||||||
|
else:
|
||||||
|
images_err += 1
|
||||||
|
|
||||||
|
print(f'\nDone: {pages_ok} pages, {images_ok} images, {pages_err + images_err} errors')
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
main()
|
||||||
2
family/uploads.ini
Normal file
2
family/uploads.ini
Normal file
@@ -0,0 +1,2 @@
|
|||||||
|
upload_max_filesize = 20M
|
||||||
|
post_max_size = 25M
|
||||||
11
freshrss/.env
Normal file
11
freshrss/.env
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
BASE_URL=https://news.alogins.net
|
||||||
|
ADMIN_EMAIL=allogn@gmail.com
|
||||||
|
ADMIN_PASSWORD=ff221f4d1!
|
||||||
|
ADMIN_API_PASSWORD=sfs32003r2
|
||||||
|
# Published port if running locally
|
||||||
|
PUBLISHED_PORT=8091
|
||||||
|
# Database credentials (not relevant if using default SQLite database)
|
||||||
|
DB_HOST=freshrss-db
|
||||||
|
DB_BASE=freshrss
|
||||||
|
DB_PASSWORD=freshrss1945133
|
||||||
|
DB_USER=freshrss12
|
||||||
36
freshrss/docker-compose.yml
Normal file
36
freshrss/docker-compose.yml
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
services:
|
||||||
|
freshrss:
|
||||||
|
image: freshrss/freshrss:latest
|
||||||
|
# # Optional build section if you want to build the image locally:
|
||||||
|
# build:
|
||||||
|
# # Pick #latest (slow releases) or #edge (rolling release) or a specific release like #1.27.1
|
||||||
|
# context: https://github.com/FreshRSS/FreshRSS.git#latest
|
||||||
|
# dockerfile: Docker/Dockerfile-Alpine
|
||||||
|
container_name: freshrss
|
||||||
|
hostname: freshrss
|
||||||
|
restart: unless-stopped
|
||||||
|
ports:
|
||||||
|
# If you want to open a port 8080 on the local machine:
|
||||||
|
- "8091:80"
|
||||||
|
logging:
|
||||||
|
options:
|
||||||
|
max-size: 10m
|
||||||
|
volumes:
|
||||||
|
- /mnt/dbs/freshrss/data:/var/www/FreshRSS/data
|
||||||
|
- /mnt/dbs/freshrss/extensions:/var/www/FreshRSS/extensions
|
||||||
|
- /mnt/dbs/freshrss/db:/var/lib/postgresql
|
||||||
|
environment:
|
||||||
|
POSTGRES_DB: ${DB_BASE:-freshrss}
|
||||||
|
POSTGRES_USER: ${DB_USER:-freshrss}
|
||||||
|
POSTGRES_PASSWORD: ${DB_PASSWORD:-freshrss}
|
||||||
|
TZ: Europe/Moscow
|
||||||
|
CRON_MIN: '3,33'
|
||||||
|
LISTEN: 0.0.0.0:80
|
||||||
|
# Optional healthcheck section:
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "cli/health.php"]
|
||||||
|
timeout: 10s
|
||||||
|
start_period: 60s
|
||||||
|
start_interval: 11s
|
||||||
|
interval: 75s
|
||||||
|
retries: 3
|
||||||
191
haos/CLAUDE.md
191
haos/CLAUDE.md
@@ -1,191 +0,0 @@
|
|||||||
# Home Assistant REST API
|
|
||||||
|
|
||||||
## Connection
|
|
||||||
|
|
||||||
- **Base URL**: `http://<HA_IP>:8123/api/`
|
|
||||||
- **Auth header**: `Authorization: Bearer <TOKEN>`
|
|
||||||
- **Token**: Generate at `http://<HA_IP>:8123/profile` → Long-Lived Access Tokens
|
|
||||||
- **Response format**: JSON (except `/api/error_log` which is plaintext)
|
|
||||||
|
|
||||||
Store token in env var, never hardcode:
|
|
||||||
```bash
|
|
||||||
export HA_TOKEN="your_token_here"
|
|
||||||
export HA_URL="http://<HA_IP>:8123"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Status Codes
|
|
||||||
|
|
||||||
| Code | Meaning |
|
|
||||||
|------|---------|
|
|
||||||
| 200 | Success (existing resource) |
|
|
||||||
| 201 | Created (new resource) |
|
|
||||||
| 400 | Bad request |
|
|
||||||
| 401 | Unauthorized |
|
|
||||||
| 404 | Not found |
|
|
||||||
| 405 | Method not allowed |
|
|
||||||
|
|
||||||
## GET Endpoints
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Health check
|
|
||||||
GET /api/
|
|
||||||
|
|
||||||
# Current HA configuration
|
|
||||||
GET /api/config
|
|
||||||
|
|
||||||
# Loaded components
|
|
||||||
GET /api/components
|
|
||||||
|
|
||||||
# All entity states
|
|
||||||
GET /api/states
|
|
||||||
|
|
||||||
# Specific entity state
|
|
||||||
GET /api/states/<entity_id>
|
|
||||||
|
|
||||||
# Available services
|
|
||||||
GET /api/services
|
|
||||||
|
|
||||||
# Available events
|
|
||||||
GET /api/events
|
|
||||||
|
|
||||||
# Error log (plaintext)
|
|
||||||
GET /api/error_log
|
|
||||||
|
|
||||||
# Camera image
|
|
||||||
GET /api/camera_proxy/<camera_entity_id>
|
|
||||||
|
|
||||||
# All calendar entities
|
|
||||||
GET /api/calendars
|
|
||||||
|
|
||||||
# Calendar events (start and end are required ISO timestamps)
|
|
||||||
GET /api/calendars/<calendar_entity_id>?start=<ISO>&end=<ISO>
|
|
||||||
|
|
||||||
# Historical state changes
|
|
||||||
GET /api/history/period/<ISO_timestamp>?filter_entity_id=<entity_id>
|
|
||||||
# Optional params: end_time, minimal_response, no_attributes, significant_changes_only
|
|
||||||
|
|
||||||
# Logbook entries
|
|
||||||
GET /api/logbook/<ISO_timestamp>
|
|
||||||
# Optional params: entity=<entity_id>, end_time=<ISO>
|
|
||||||
```
|
|
||||||
|
|
||||||
## POST Endpoints
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Create or update entity state (virtual, not device)
|
|
||||||
POST /api/states/<entity_id>
|
|
||||||
{"state": "on", "attributes": {"brightness": 255}}
|
|
||||||
|
|
||||||
# Fire an event
|
|
||||||
POST /api/events/<event_type>
|
|
||||||
{"optional": "event_data"}
|
|
||||||
|
|
||||||
# Call a service
|
|
||||||
POST /api/services/<domain>/<service>
|
|
||||||
{"entity_id": "light.living_room"}
|
|
||||||
|
|
||||||
# Call service and get its response
|
|
||||||
POST /api/services/<domain>/<service>?return_response
|
|
||||||
{"entity_id": "..."}
|
|
||||||
|
|
||||||
# Render a Jinja2 template
|
|
||||||
POST /api/template
|
|
||||||
{"template": "{{ states('sensor.temperature') }}"}
|
|
||||||
|
|
||||||
# Validate configuration
|
|
||||||
POST /api/config/core/check_config
|
|
||||||
|
|
||||||
# Handle an intent
|
|
||||||
POST /api/intent/handle
|
|
||||||
{"name": "HassTurnOn", "data": {"name": "lights"}}
|
|
||||||
```
|
|
||||||
|
|
||||||
## DELETE Endpoints
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Remove an entity
|
|
||||||
DELETE /api/states/<entity_id>
|
|
||||||
```
|
|
||||||
|
|
||||||
## Example curl Usage
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Health check
|
|
||||||
curl -s -H "Authorization: Bearer $HA_TOKEN" $HA_URL/api/
|
|
||||||
|
|
||||||
# Get all states
|
|
||||||
curl -s -H "Authorization: Bearer $HA_TOKEN" $HA_URL/api/states | jq .
|
|
||||||
|
|
||||||
# Get specific entity
|
|
||||||
curl -s -H "Authorization: Bearer $HA_TOKEN" $HA_URL/api/states/light.living_room
|
|
||||||
|
|
||||||
# Turn on a light
|
|
||||||
curl -s -X POST \
|
|
||||||
-H "Authorization: Bearer $HA_TOKEN" \
|
|
||||||
-H "Content-Type: application/json" \
|
|
||||||
-d '{"entity_id": "light.living_room"}' \
|
|
||||||
$HA_URL/api/services/light/turn_on
|
|
||||||
|
|
||||||
# Render template
|
|
||||||
curl -s -X POST \
|
|
||||||
-H "Authorization: Bearer $HA_TOKEN" \
|
|
||||||
-H "Content-Type: application/json" \
|
|
||||||
-d '{"template": "{{ states(\"sensor.temperature\") }}"}' \
|
|
||||||
$HA_URL/api/template
|
|
||||||
```
|
|
||||||
|
|
||||||
## Devices
|
|
||||||
|
|
||||||
### Lights
|
|
||||||
4x Zigbee Tuya lights (TZ3210 TS0505B):
|
|
||||||
- `light.tz3210_r5afgmkl_ts0505b` (G2)
|
|
||||||
- `light.tz3210_r5afgmkl_ts0505b_g2` (G22)
|
|
||||||
- `light.tz3210_r5afgmkl_ts0505b_2`
|
|
||||||
- `light.tz3210_r5afgmkl_ts0505b_3`
|
|
||||||
|
|
||||||
Support: color_temp (2000-6535K), xy color mode, brightness (0-254)
|
|
||||||
|
|
||||||
### Vacuum Cleaner
|
|
||||||
**Entity**: `vacuum.xiaomi_ru_1173505785_ov71gl` (Петя Петя)
|
|
||||||
**Status**: Docked
|
|
||||||
**Type**: Xiaomi robot vacuum with mop
|
|
||||||
|
|
||||||
**Rooms** (from `sensor.xiaomi_ru_1173505785_ov71gl_room_information_p_2_16`):
|
|
||||||
- ID 4: Спальня (Bedroom)
|
|
||||||
- ID 3: Гостиная (Living Room)
|
|
||||||
- ID 5: Кухня (Kitchen)
|
|
||||||
- ID 6: Прихожая (Hallway)
|
|
||||||
- ID 7: Ванная комната (Bathroom)
|
|
||||||
|
|
||||||
**Services**:
|
|
||||||
- `vacuum.start` — Start cleaning
|
|
||||||
- `vacuum.pause` — Pause
|
|
||||||
- `vacuum.stop` — Stop
|
|
||||||
- `vacuum.return_to_base` — Dock
|
|
||||||
- `vacuum.clean_spot` — Clean spot
|
|
||||||
- `vacuum.set_fan_speed` — Set fan (param: `fan_speed`)
|
|
||||||
- `vacuum.send_command` — Raw command (params: `command`, `params`)
|
|
||||||
- Room-aware: `start_vacuum_room_sweep`, `start_zone_sweep`, `get_room_configs`, `set_room_clean_configs`
|
|
||||||
|
|
||||||
**Key attributes**:
|
|
||||||
- `sensor.xiaomi_ru_1173505785_ov71gl_room_information_p_2_16` — Room data (JSON)
|
|
||||||
- `sensor.xiaomi_ru_1173505785_ov71gl_zone_ids_p_2_12` — Zone IDs
|
|
||||||
- `button.xiaomi_ru_1173505785_ov71gl_auto_room_partition_a_10_5` — Auto-detect room boundaries
|
|
||||||
|
|
||||||
### Water Leak Sensors
|
|
||||||
3x HOBEIAN ZG-222Z Zigbee moisture sensors:
|
|
||||||
- `binary_sensor.hobeian_zg_222z` — Kitchen
|
|
||||||
- `binary_sensor.hobeian_zg_222z_2` — Bathroom
|
|
||||||
- `binary_sensor.hobeian_zg_222z_3` — Laundry
|
|
||||||
|
|
||||||
Battery sensors: `sensor.hobeian_zg_222z_battery`, `_2`, `_3`
|
|
||||||
|
|
||||||
**Automations** (push to Zabbix via `rest_command`):
|
|
||||||
- "Water Leak Alert" (`water_leak_alert`) — any sensor ON → `rest_command.zabbix_water_leak` with room name
|
|
||||||
- "Water Leak Clear" (`water_leak_clear`) — all sensors OFF → `rest_command.zabbix_water_leak_clear`
|
|
||||||
|
|
||||||
## Notes
|
|
||||||
|
|
||||||
- `POST /api/states/<entity_id>` creates a virtual state representation only — it does NOT control physical devices. Use `POST /api/services/...` for actual device control.
|
|
||||||
- Timestamp format: `YYYY-MM-DDThh:mm:ssTZD` (ISO 8601)
|
|
||||||
- Using `?return_response` on a service that doesn't support it returns a 400 error
|
|
||||||
@@ -5,9 +5,9 @@
|
|||||||
# You can find documentation for all the supported env variables at https://docs.immich.app/install/environment-variables
|
# You can find documentation for all the supported env variables at https://docs.immich.app/install/environment-variables
|
||||||
|
|
||||||
# The location where your uploaded files are stored
|
# The location where your uploaded files are stored
|
||||||
UPLOAD_LOCATION=/mnt/media/upload
|
UPLOAD_LOCATION=/mnt/smsg/media/upload
|
||||||
THUMB_LOCATION=/mnt/ssd/media/thumbs
|
THUMB_LOCATION=/mnt/smsg/media/thumbs
|
||||||
ENCODED_VIDEO_LOCATION=/mnt/ssd/media/encoded-video
|
ENCODED_VIDEO_LOCATION=/mnt/smsg/media/encoded-video
|
||||||
|
|
||||||
# The location where your database files are stored. Network shares are not supported for the database
|
# The location where your database files are stored. Network shares are not supported for the database
|
||||||
DB_DATA_LOCATION=/mnt/ssd/media/postgres
|
DB_DATA_LOCATION=/mnt/ssd/media/postgres
|
||||||
|
|||||||
@@ -1,30 +1,42 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
BACKUP_DIR=/mnt/backups/media
|
BACKUP_DIR=/mnt/toshiba/backups/media
|
||||||
DB_BACKUP_DIR="$BACKUP_DIR/backups"
|
|
||||||
LOG="$BACKUP_DIR/backup.log"
|
LOG="$BACKUP_DIR/backup.log"
|
||||||
RETAIN_DAYS=14
|
VERBOSE=0
|
||||||
|
|
||||||
mkdir -p "$DB_BACKUP_DIR"
|
for arg in "$@"; do
|
||||||
|
case $arg in
|
||||||
echo "[$(date)] Starting Immich backup" >> "$LOG"
|
-v|--verbose) VERBOSE=1 ;;
|
||||||
|
esac
|
||||||
# 1. Database dump (must come before file sync)
|
|
||||||
DUMP_FILE="$DB_BACKUP_DIR/immich-db-$(date +%Y%m%dT%H%M%S).sql.gz"
|
|
||||||
docker exec immich_postgres pg_dump --clean --if-exists \
|
|
||||||
--dbname=immich --username=postgres | gzip > "$DUMP_FILE"
|
|
||||||
echo "[$(date)] DB dump: $DUMP_FILE" >> "$LOG"
|
|
||||||
|
|
||||||
# 2. Rsync critical asset folders (skip thumbs and encoded-video — regeneratable)
|
|
||||||
for DIR in library upload profile; do
|
|
||||||
rsync -a --delete /mnt/media/upload/$DIR/ "$BACKUP_DIR/$DIR/" >> "$LOG" 2>&1
|
|
||||||
echo "[$(date)] Synced $DIR" >> "$LOG"
|
|
||||||
done
|
done
|
||||||
|
|
||||||
# 3. Remove old DB dumps
|
log() { echo "[$(date)] $*" >> "$LOG"; }
|
||||||
find "$DB_BACKUP_DIR" -name "immich-db-*.sql.gz" -mtime +$RETAIN_DAYS -delete
|
say() { [[ $VERBOSE -eq 1 ]] && echo "$*" || true; }
|
||||||
echo "[$(date)] Cleaned dumps older than ${RETAIN_DAYS}d" >> "$LOG"
|
|
||||||
|
|
||||||
|
mkdir -p "$BACKUP_DIR"
|
||||||
|
|
||||||
|
say ""
|
||||||
|
say "┌─────────────────────────────────────┐"
|
||||||
|
say "│ Immich Backup Starting │"
|
||||||
|
say "└─────────────────────────────────────┘"
|
||||||
|
say ""
|
||||||
|
log "Starting Immich backup"
|
||||||
|
|
||||||
|
# Rsync critical asset folders (skip thumbs and encoded-video — regeneratable)
|
||||||
|
RSYNC_OPTS="-a --ignore-existing"
|
||||||
|
[[ $VERBOSE -eq 1 ]] && RSYNC_OPTS="$RSYNC_OPTS --info=progress2"
|
||||||
|
|
||||||
|
for DIR in library upload profile; do
|
||||||
|
say " Syncing $DIR/ ..."
|
||||||
|
rsync $RSYNC_OPTS /mnt/smsg/media/upload/$DIR/ "$BACKUP_DIR/$DIR/" 2>&1 | \
|
||||||
|
tee -a "$LOG" | { [[ $VERBOSE -eq 0 ]] && cat > /dev/null || cat; }
|
||||||
|
log "Synced $DIR"
|
||||||
|
say ""
|
||||||
|
done
|
||||||
|
|
||||||
touch "$BACKUP_DIR/.last_sync"
|
touch "$BACKUP_DIR/.last_sync"
|
||||||
echo "[$(date)] Immich backup complete" >> "$LOG"
|
log "Immich backup complete"
|
||||||
|
say "✓ Done"
|
||||||
|
say ""
|
||||||
|
|||||||
@@ -30,6 +30,7 @@ services:
|
|||||||
- redis
|
- redis
|
||||||
- database
|
- database
|
||||||
restart: always
|
restart: always
|
||||||
|
mem_limit: 1500m
|
||||||
healthcheck:
|
healthcheck:
|
||||||
disable: false
|
disable: false
|
||||||
|
|
||||||
@@ -37,15 +38,16 @@ services:
|
|||||||
container_name: immich_machine_learning
|
container_name: immich_machine_learning
|
||||||
# For hardware acceleration, add one of -[armnn, cuda, rocm, openvino, rknn] to the image tag.
|
# For hardware acceleration, add one of -[armnn, cuda, rocm, openvino, rknn] to the image tag.
|
||||||
# Example tag: ${IMMICH_VERSION:-release}-cuda
|
# Example tag: ${IMMICH_VERSION:-release}-cuda
|
||||||
image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
|
image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-cuda
|
||||||
# extends: # uncomment this section for hardware acceleration - see https://docs.immich.app/features/ml-hardware-acceleration
|
extends: # uncomment this section for hardware acceleration - see https://docs.immich.app/features/ml-hardware-acceleration
|
||||||
# file: hwaccel.ml.yml
|
file: hwaccel.ml.yml
|
||||||
# service: cpu # set to one of [armnn, cuda, rocm, openvino, openvino-wsl, rknn] for accelerated inference - use the `-wsl` version for WSL2 where applicable
|
service: cuda # set to one of [armnn, cuda, rocm, openvino, openvino-wsl, rknn] for accelerated inference - use the `-wsl` version for WSL2 where applicable
|
||||||
volumes:
|
volumes:
|
||||||
- model-cache:/cache
|
- model-cache:/cache
|
||||||
env_file:
|
env_file:
|
||||||
- .env
|
- .env
|
||||||
restart: always
|
restart: always
|
||||||
|
mem_limit: 750m
|
||||||
healthcheck:
|
healthcheck:
|
||||||
disable: false
|
disable: false
|
||||||
|
|
||||||
@@ -55,6 +57,7 @@ services:
|
|||||||
healthcheck:
|
healthcheck:
|
||||||
test: redis-cli ping || exit 1
|
test: redis-cli ping || exit 1
|
||||||
restart: always
|
restart: always
|
||||||
|
mem_limit: 256m
|
||||||
|
|
||||||
database:
|
database:
|
||||||
container_name: immich_postgres
|
container_name: immich_postgres
|
||||||
@@ -71,6 +74,7 @@ services:
|
|||||||
- ${DB_DATA_LOCATION}:/var/lib/postgresql/data
|
- ${DB_DATA_LOCATION}:/var/lib/postgresql/data
|
||||||
shm_size: 128mb
|
shm_size: 128mb
|
||||||
restart: always
|
restart: always
|
||||||
|
mem_limit: 1500m
|
||||||
|
|
||||||
volumes:
|
volumes:
|
||||||
model-cache:
|
model-cache:
|
||||||
|
|||||||
57
immich-app/hwaccel.ml.yml
Normal file
57
immich-app/hwaccel.ml.yml
Normal file
@@ -0,0 +1,57 @@
|
|||||||
|
# Configurations for hardware-accelerated machine learning
|
||||||
|
|
||||||
|
# If using Unraid or another platform that doesn't allow multiple Compose files,
|
||||||
|
# you can inline the config for a backend by copying its contents
|
||||||
|
# into the immich-machine-learning service in the docker-compose.yml file.
|
||||||
|
|
||||||
|
# See https://docs.immich.app/features/ml-hardware-acceleration for info on usage.
|
||||||
|
|
||||||
|
services:
|
||||||
|
armnn:
|
||||||
|
devices:
|
||||||
|
- /dev/mali0:/dev/mali0
|
||||||
|
volumes:
|
||||||
|
- /lib/firmware/mali_csffw.bin:/lib/firmware/mali_csffw.bin:ro # Mali firmware for your chipset (not always required depending on the driver)
|
||||||
|
- /usr/lib/libmali.so:/usr/lib/libmali.so:ro # Mali driver for your chipset (always required)
|
||||||
|
|
||||||
|
rknn:
|
||||||
|
security_opt:
|
||||||
|
- systempaths=unconfined
|
||||||
|
- apparmor=unconfined
|
||||||
|
devices:
|
||||||
|
- /dev/dri:/dev/dri
|
||||||
|
|
||||||
|
cpu: {}
|
||||||
|
|
||||||
|
cuda:
|
||||||
|
deploy:
|
||||||
|
resources:
|
||||||
|
reservations:
|
||||||
|
devices:
|
||||||
|
- driver: nvidia
|
||||||
|
count: 1
|
||||||
|
capabilities:
|
||||||
|
- gpu
|
||||||
|
|
||||||
|
rocm:
|
||||||
|
group_add:
|
||||||
|
- video
|
||||||
|
devices:
|
||||||
|
- /dev/dri:/dev/dri
|
||||||
|
- /dev/kfd:/dev/kfd
|
||||||
|
|
||||||
|
openvino:
|
||||||
|
device_cgroup_rules:
|
||||||
|
- 'c 189:* rmw'
|
||||||
|
devices:
|
||||||
|
- /dev/dri:/dev/dri
|
||||||
|
volumes:
|
||||||
|
- /dev/bus/usb:/dev/bus/usb
|
||||||
|
|
||||||
|
openvino-wsl:
|
||||||
|
devices:
|
||||||
|
- /dev/dri:/dev/dri
|
||||||
|
- /dev/dxg:/dev/dxg
|
||||||
|
volumes:
|
||||||
|
- /dev/bus/usb:/dev/bus/usb
|
||||||
|
- /usr/lib/wsl:/usr/lib/wsl
|
||||||
BIN
iperf3/CellularLab-v2.2.apk
Normal file
BIN
iperf3/CellularLab-v2.2.apk
Normal file
Binary file not shown.
BIN
iperf3/apk/CellularLab-v2.2.apk
Normal file
BIN
iperf3/apk/CellularLab-v2.2.apk
Normal file
Binary file not shown.
9
iperf3/apk/index.html
Normal file
9
iperf3/apk/index.html
Normal file
@@ -0,0 +1,9 @@
|
|||||||
|
<!DOCTYPE html>
|
||||||
|
<html>
|
||||||
|
<head><title>iperf3 Android</title></head>
|
||||||
|
<body>
|
||||||
|
<h2>iperf3 for Android</h2>
|
||||||
|
<p><a href="CellularLab-v2.2.apk">Download CellularLab v2.2 (iperf3 client)</a></p>
|
||||||
|
<p>Server: <code>iperf.alogins.net</code> — port 5201</p>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
18
iperf3/docker-compose.yml
Normal file
18
iperf3/docker-compose.yml
Normal file
@@ -0,0 +1,18 @@
|
|||||||
|
services:
|
||||||
|
iperf3:
|
||||||
|
image: networkstatic/iperf3
|
||||||
|
container_name: iperf3
|
||||||
|
command: -s
|
||||||
|
ports:
|
||||||
|
- "5201:5201"
|
||||||
|
- "5201:5201/udp"
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
|
iperf3-files:
|
||||||
|
image: nginx:alpine
|
||||||
|
container_name: iperf3-files
|
||||||
|
ports:
|
||||||
|
- "8095:80"
|
||||||
|
volumes:
|
||||||
|
- ./apk:/usr/share/nginx/html:ro
|
||||||
|
restart: unless-stopped
|
||||||
25
kanboard/CLAUDE.md
Normal file
25
kanboard/CLAUDE.md
Normal file
@@ -0,0 +1,25 @@
|
|||||||
|
# CLAUDE.md — Kanboard
|
||||||
|
|
||||||
|
Guidance for Claude Code when working with the Kanboard service on the Agap server.
|
||||||
|
|
||||||
|
This directory holds only the **live** Docker Compose config for the `kanboard`
|
||||||
|
container (board data lives in a Docker volume). Everything else Kanboard-related —
|
||||||
|
service docs, the task-orchestration ruleset, the `claude-usage` quota script, and a
|
||||||
|
reference copy of the `kanboard_*` MCP tool implementation — has been consolidated into
|
||||||
|
its own repo: **`/home/alvis/kanboard`** (Gitea: `alvis/kanboard`). Read
|
||||||
|
`/home/alvis/kanboard/CLAUDE.md` before working with Kanboard or the orchestration
|
||||||
|
pipeline.
|
||||||
|
|
||||||
|
- Container `kanboard`, image `kanboard/kanboard:latest`, port `127.0.0.1:4800:80`
|
||||||
|
- The MCP server implementing `mcp__agap__kanboard_*` (`src/kanboard.js`) lives in
|
||||||
|
`agap-mcp` (`/home/alvis/agap_git/agap-mcp`) — shared with other tool sets (`vw_*`,
|
||||||
|
`gitea_*`, `ha_*`, `zabbix_*`, `radicale_*`), so it stays here rather than moving to
|
||||||
|
the kanboard repo.
|
||||||
|
|
||||||
|
## Common Commands
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose up -d # start
|
||||||
|
docker compose restart # restart
|
||||||
|
docker compose logs -f # logs
|
||||||
|
```
|
||||||
69
kanboard/backup.sh
Executable file
69
kanboard/backup.sh
Executable file
@@ -0,0 +1,69 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Kanboard backup — tier-0 hardening (kb#158, A2A-26, DESIGN-a2a-agents.md v2.1 §6c).
|
||||||
|
# Mirrors the vaultwarden backup.sh pattern (same repo, ~/agap_git/vaultwarden/backup.sh):
|
||||||
|
# scheduled dump -> /mnt/backups, retention of last 5, Zabbix freshness trapper.
|
||||||
|
#
|
||||||
|
# Runs every 3 days via alvis's user crontab (NOT root crontab like vaultwarden's --
|
||||||
|
# /mnt/backups/kanboard was bootstrapped chown'd to alvis specifically so this backup,
|
||||||
|
# like the rest of the kanboard tooling, needs no root/sudo at all. alvis is in the
|
||||||
|
# `docker` group so `docker exec`/`docker cp` need no privilege escalation either).
|
||||||
|
#
|
||||||
|
# DB dump method: kanboard's container has no sqlite3 CLI and no PHP `sqlite3`
|
||||||
|
# extension (only pdo_sqlite) -- checked directly (kb#158). Instead we run SQLite's
|
||||||
|
# own `VACUUM INTO` via PDO, which is SQLite's supported way to take an atomic,
|
||||||
|
# consistent online snapshot of a live database (safe against concurrent writers,
|
||||||
|
# same safety property `vaultwarden backup` gives us for that service).
|
||||||
|
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
BACKUP_DIR="/mnt/backups/kanboard"
|
||||||
|
ZABBIX_TOKEN_FILE="/home/alvis/.zabbix_token"
|
||||||
|
ZABBIX_URL="http://192.168.1.4:81/api_jsonrpc.php"
|
||||||
|
ZABBIX_ITEM_ID="70605" # kanboard.backup.ts on host AgapHost (10776)
|
||||||
|
|
||||||
|
DATE=$(date '+%Y%m%d-%H%M')
|
||||||
|
DEST="$BACKUP_DIR/$DATE"
|
||||||
|
TMP_NAME="backup_${DATE}.sqlite"
|
||||||
|
|
||||||
|
mkdir -p "$DEST"
|
||||||
|
|
||||||
|
# Online, consistent snapshot via SQLite's VACUUM INTO (PDO sqlite driver is present
|
||||||
|
# in the image; the sqlite3 CLI/extension is not, so this replaces the vaultwarden
|
||||||
|
# `docker exec vaultwarden /vaultwarden backup` equivalent for this service).
|
||||||
|
docker exec kanboard php -r '
|
||||||
|
$db = new PDO("sqlite:/var/www/app/data/db.sqlite");
|
||||||
|
$db->exec("VACUUM INTO \"/var/www/app/data/'"$TMP_NAME"'\"");
|
||||||
|
'
|
||||||
|
|
||||||
|
# Pull the snapshot out to the host, then remove the temp copy from the live data dir
|
||||||
|
# (mirrors vaultwarden's "move the file out of DATA_DIR" step).
|
||||||
|
docker cp "kanboard:/var/www/app/data/$TMP_NAME" "$DEST/db.sqlite"
|
||||||
|
docker exec kanboard rm -f "/var/www/app/data/$TMP_NAME"
|
||||||
|
|
||||||
|
# Plugins volume (PLUGIN_INSTALLER=true means plugins can be installed at runtime,
|
||||||
|
# not just baked into the image) -- back it up too so a restore doesn't silently
|
||||||
|
# drop installed plugins.
|
||||||
|
docker run --rm --user 1000:1000 -v kanboard_plugins:/plugins:ro -v "$DEST":/dest alpine \
|
||||||
|
sh -c 'cd /plugins && tar -czf /dest/plugins.tar.gz . 2>/dev/null || true'
|
||||||
|
|
||||||
|
echo "$(date): Backup complete: $DEST"
|
||||||
|
ls -la "$DEST/"
|
||||||
|
|
||||||
|
# Notify Zabbix (trapper item kanboard.backup.ts, unixtime) -- pushes a real epoch
|
||||||
|
# timestamp, unlike vaultwarden.backup.ts which (kb#158 finding) pushes a formatted
|
||||||
|
# date STRING into a numeric item and has therefore never recorded a valid value.
|
||||||
|
if [[ -f "$ZABBIX_TOKEN_FILE" ]]; then
|
||||||
|
ZABBIX_TOKEN=$(cat "$ZABBIX_TOKEN_FILE")
|
||||||
|
NOW_EPOCH=$(date '+%s')
|
||||||
|
env -u HTTPS_PROXY -u HTTP_PROXY -u ALL_PROXY -u https_proxy -u http_proxy -u all_proxy \
|
||||||
|
curl -s -X POST "$ZABBIX_URL" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-H "Authorization: Bearer $ZABBIX_TOKEN" \
|
||||||
|
-d "{\"jsonrpc\":\"2.0\",\"method\":\"history.push\",\"id\":1,\"params\":{\"itemid\":\"$ZABBIX_ITEM_ID\",\"value\":$NOW_EPOCH}}" > /dev/null \
|
||||||
|
&& echo "Zabbix notified (kanboard.backup.ts=$NOW_EPOCH)."
|
||||||
|
else
|
||||||
|
echo "WARNING: $ZABBIX_TOKEN_FILE not found -- skipped Zabbix freshness push." >&2
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Rotate: keep last 5 backups
|
||||||
|
ls -1dt "$BACKUP_DIR"/[0-9]*-[0-9]* 2>/dev/null | tail -n +6 | xargs -r rm -rf
|
||||||
17
kanboard/docker-compose.yml
Normal file
17
kanboard/docker-compose.yml
Normal file
@@ -0,0 +1,17 @@
|
|||||||
|
name: kanboard
|
||||||
|
services:
|
||||||
|
app:
|
||||||
|
image: kanboard/kanboard:latest
|
||||||
|
container_name: kanboard
|
||||||
|
restart: always
|
||||||
|
ports:
|
||||||
|
- "127.0.0.1:4800:80"
|
||||||
|
volumes:
|
||||||
|
- data:/var/www/app/data
|
||||||
|
- plugins:/var/www/app/plugins
|
||||||
|
environment:
|
||||||
|
- PLUGIN_INSTALLER=true
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
data:
|
||||||
|
plugins:
|
||||||
92
kanboard/healthcheck.sh
Executable file
92
kanboard/healthcheck.sh
Executable file
@@ -0,0 +1,92 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Kanboard service + JSON-RPC API health -- tier-0 hardening (kb#158, A2A-26,
|
||||||
|
# DESIGN-a2a-agents.md v2.1 §6c: "Zabbix monitoring of the service and API").
|
||||||
|
#
|
||||||
|
# Runs every 2 minutes via alvis's user crontab (no root needed -- alvis is in the
|
||||||
|
# `docker` group). Pushes two Zabbix trapper items on host AgapHost:
|
||||||
|
# kanboard.service.up -- 1 if the `kanboard` container is running AND its own
|
||||||
|
# Docker healthcheck reports "healthy", else 0
|
||||||
|
# kanboard.jsonrpc.up -- 1 if an authenticated JSON-RPC call round-trips
|
||||||
|
# correctly, else 0
|
||||||
|
# Both are pushed every run (unlike backup.sh, which only pushes on success) so a
|
||||||
|
# transition to "down" is reported immediately rather than waiting for nodata() --
|
||||||
|
# the triggers additionally use nodata(...,10m) as a backstop in case this script
|
||||||
|
# itself stops running.
|
||||||
|
|
||||||
|
set -uo pipefail # no -e: we want to push a "0" and continue, not abort, on failure
|
||||||
|
|
||||||
|
ZABBIX_TOKEN_FILE="/home/alvis/.zabbix_token"
|
||||||
|
KANBOARD_TOKEN_FILE="/home/alvis/.kanboard_token"
|
||||||
|
ZABBIX_URL="http://192.168.1.4:81/api_jsonrpc.php"
|
||||||
|
SERVICE_ITEM_ID="70606" # kanboard.service.up
|
||||||
|
JSONRPC_ITEM_ID="70607" # kanboard.jsonrpc.up
|
||||||
|
|
||||||
|
# kb#188: tier-0 fabric container up/down. Piggybacks on this same 2-minute cron
|
||||||
|
# slot (no new cron entry) -- container:itemid map, plain `docker inspect
|
||||||
|
# .State.Running` since most of these have no HEALTHCHECK defined (only `adolf`
|
||||||
|
# does; checking bare Running is what's available uniformly here). Pushed
|
||||||
|
# unconditionally like the two items above, with nodata(...,10m) as trigger backstop.
|
||||||
|
declare -A FABRIC_ITEMS=(
|
||||||
|
["litellm"]="70625"
|
||||||
|
["litellm-db"]="70626"
|
||||||
|
["adolf"]="70627"
|
||||||
|
["adolf-llm"]="70628"
|
||||||
|
["hindsight"]="70629"
|
||||||
|
["hindsight-llm"]="70630"
|
||||||
|
["tei-reranker"]="70631"
|
||||||
|
["ollama"]="70632"
|
||||||
|
["kanboard-mcp-kanboard-mcp-1"]="70633"
|
||||||
|
["kanboard-mcp-adolf"]="70634"
|
||||||
|
["agap-mcp-agap-mcp-1"]="70635"
|
||||||
|
["fabric-keeper"]="70636"
|
||||||
|
)
|
||||||
|
|
||||||
|
# --- 1. Container health ---
|
||||||
|
HEALTH=$(docker inspect --format '{{.State.Health.Status}}' kanboard 2>/dev/null)
|
||||||
|
if [[ "$HEALTH" == "healthy" ]]; then
|
||||||
|
SERVICE_UP=1
|
||||||
|
else
|
||||||
|
SERVICE_UP=0
|
||||||
|
fi
|
||||||
|
|
||||||
|
# --- 2. JSON-RPC API health (authenticated round-trip, not just a TCP/HTTP check) ---
|
||||||
|
JSONRPC_UP=0
|
||||||
|
if [[ -f "$KANBOARD_TOKEN_FILE" ]]; then
|
||||||
|
KANBOARD_TOKEN=$(cat "$KANBOARD_TOKEN_FILE")
|
||||||
|
RESP=$(env -u HTTPS_PROXY -u HTTP_PROXY -u ALL_PROXY -u https_proxy -u http_proxy -u all_proxy \
|
||||||
|
curl -s --max-time 10 -u "jsonrpc:$KANBOARD_TOKEN" -X POST http://127.0.0.1:4800/jsonrpc.php \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{"jsonrpc":"2.0","method":"getVersion","id":1}' 2>/dev/null)
|
||||||
|
if echo "$RESP" | python3 -c "import sys,json; d=json.load(sys.stdin); sys.exit(0 if 'result' in d else 1)" 2>/dev/null; then
|
||||||
|
JSONRPC_UP=1
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
# --- 3. Fabric tier-0 container up/down (kb#188) ---
|
||||||
|
FABRIC_PUSH_PARAMS="{\"itemid\":\"$SERVICE_ITEM_ID\",\"value\":$SERVICE_UP},{\"itemid\":\"$JSONRPC_ITEM_ID\",\"value\":$JSONRPC_UP}"
|
||||||
|
FABRIC_SUMMARY=""
|
||||||
|
for container in "${!FABRIC_ITEMS[@]}"; do
|
||||||
|
itemid="${FABRIC_ITEMS[$container]}"
|
||||||
|
running=$(docker inspect --format '{{.State.Running}}' "$container" 2>/dev/null)
|
||||||
|
if [[ "$running" == "true" ]]; then
|
||||||
|
up=1
|
||||||
|
else
|
||||||
|
up=0
|
||||||
|
fi
|
||||||
|
FABRIC_PUSH_PARAMS="$FABRIC_PUSH_PARAMS,{\"itemid\":\"$itemid\",\"value\":$up}"
|
||||||
|
FABRIC_SUMMARY="$FABRIC_SUMMARY $container=$up"
|
||||||
|
done
|
||||||
|
|
||||||
|
# --- 4. Push everything to Zabbix in one batch, unconditionally (down is a real value, not a gap) ---
|
||||||
|
if [[ -f "$ZABBIX_TOKEN_FILE" ]]; then
|
||||||
|
ZABBIX_TOKEN=$(cat "$ZABBIX_TOKEN_FILE")
|
||||||
|
env -u HTTPS_PROXY -u HTTP_PROXY -u ALL_PROXY -u https_proxy -u http_proxy -u all_proxy \
|
||||||
|
curl -s --max-time 10 -X POST "$ZABBIX_URL" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-H "Authorization: Bearer $ZABBIX_TOKEN" \
|
||||||
|
-d "{\"jsonrpc\":\"2.0\",\"method\":\"history.push\",\"id\":1,\"params\":[$FABRIC_PUSH_PARAMS]}" > /dev/null
|
||||||
|
else
|
||||||
|
echo "WARNING: $ZABBIX_TOKEN_FILE not found -- skipped Zabbix push." >&2
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "$(date '+%Y-%m-%d %H:%M:%S') service_up=$SERVICE_UP jsonrpc_up=$JSONRPC_UP$FABRIC_SUMMARY"
|
||||||
473
linkwarden/.env
Normal file
473
linkwarden/.env
Normal file
@@ -0,0 +1,473 @@
|
|||||||
|
NEXTAUTH_URL=https://lw.alogins.net/api/v1/auth
|
||||||
|
NEXTAUTH_SECRET=sdf2323frghjkj211
|
||||||
|
|
||||||
|
# Manual installation database settings
|
||||||
|
# Example: DATABASE_URL=postgresql://user:password@localhost:5432/linkwarden
|
||||||
|
DATABASE_URL=
|
||||||
|
|
||||||
|
# Docker installation database settings
|
||||||
|
POSTGRES_PASSWORD=KAf122!!fsdf2w
|
||||||
|
|
||||||
|
# Additional Optional Settings
|
||||||
|
PAGINATION_TAKE_COUNT=
|
||||||
|
STORAGE_FOLDER=
|
||||||
|
AUTOSCROLL_TIMEOUT=
|
||||||
|
NEXT_PUBLIC_DISABLE_REGISTRATION=true
|
||||||
|
NEXT_PUBLIC_CREDENTIALS_ENABLED=
|
||||||
|
DISABLE_NEW_SSO_USERS=
|
||||||
|
MAX_LINKS_PER_USER=
|
||||||
|
ARCHIVE_TAKE_COUNT=
|
||||||
|
BROWSER_TIMEOUT=
|
||||||
|
IGNORE_URL_SIZE_LIMIT=
|
||||||
|
NEXT_PUBLIC_DEMO=
|
||||||
|
NEXT_PUBLIC_DEMO_USERNAME=
|
||||||
|
NEXT_PUBLIC_DEMO_PASSWORD=
|
||||||
|
NEXT_PUBLIC_ADMIN=
|
||||||
|
NEXT_PUBLIC_MAX_FILE_BUFFER=
|
||||||
|
PDF_MAX_BUFFER=
|
||||||
|
SCREENSHOT_MAX_BUFFER=
|
||||||
|
READABILITY_MAX_BUFFER=
|
||||||
|
PREVIEW_MAX_BUFFER=
|
||||||
|
MONOLITH_MAX_BUFFER=
|
||||||
|
MONOLITH_CUSTOM_OPTIONS=
|
||||||
|
IMPORT_LIMIT=
|
||||||
|
PLAYWRIGHT_LAUNCH_OPTIONS_EXECUTABLE_PATH=
|
||||||
|
PLAYWRIGHT_WS_URL=
|
||||||
|
MAX_WORKERS=
|
||||||
|
DISABLE_PRESERVATION=
|
||||||
|
NEXT_PUBLIC_RSS_POLLING_INTERVAL_MINUTES=
|
||||||
|
RSS_SUBSCRIPTION_LIMIT_PER_USER=
|
||||||
|
TEXT_CONTENT_LIMIT=
|
||||||
|
SEARCH_FILTER_LIMIT=
|
||||||
|
INDEX_TAKE_COUNT=
|
||||||
|
MEILI_TIMEOUT=
|
||||||
|
ALLOW_PRIVATE_NETWORK_ACCESS=
|
||||||
|
ALLOW_INSECURE_TLS=
|
||||||
|
|
||||||
|
# AI Settings
|
||||||
|
NEXT_PUBLIC_OLLAMA_ENDPOINT_URL=
|
||||||
|
OLLAMA_MODEL=
|
||||||
|
|
||||||
|
# https://ai-sdk.dev/providers/openai-compatible-providers
|
||||||
|
OPENAI_API_KEY=
|
||||||
|
OPENAI_MODEL=
|
||||||
|
# Optional: Set a custom OpenAI base URL and name (for third-party providers)
|
||||||
|
CUSTOM_OPENAI_BASE_URL=
|
||||||
|
CUSTOM_OPENAI_NAME=
|
||||||
|
|
||||||
|
# https://sdk.vercel.ai/providers/ai-sdk-providers/azure
|
||||||
|
AZURE_API_KEY=
|
||||||
|
AZURE_RESOURCE_NAME=
|
||||||
|
AZURE_MODEL=
|
||||||
|
|
||||||
|
# https://sdk.vercel.ai/providers/ai-sdk-providers/anthropic
|
||||||
|
ANTHROPIC_API_KEY=
|
||||||
|
ANTHROPIC_MODEL=
|
||||||
|
|
||||||
|
# https://github.com/OpenRouterTeam/ai-sdk-provider
|
||||||
|
OPENROUTER_API_KEY=
|
||||||
|
OPENROUTER_MODEL=
|
||||||
|
|
||||||
|
# https://ai-sdk.dev/providers/ai-sdk-providers/perplexity
|
||||||
|
PERPLEXITY_API_KEY=
|
||||||
|
PERPLEXITY_MODEL=
|
||||||
|
|
||||||
|
# MeiliSearch Settings
|
||||||
|
MEILI_HOST=
|
||||||
|
MEILI_MASTER_KEY=FAfg24!@bbqq
|
||||||
|
|
||||||
|
# AWS S3 Settings
|
||||||
|
SPACES_KEY=
|
||||||
|
SPACES_SECRET=
|
||||||
|
SPACES_ENDPOINT=
|
||||||
|
SPACES_BUCKET_NAME=
|
||||||
|
SPACES_REGION=
|
||||||
|
SPACES_FORCE_PATH_STYLE=
|
||||||
|
|
||||||
|
# SMTP Settings
|
||||||
|
NEXT_PUBLIC_EMAIL_PROVIDER=
|
||||||
|
EMAIL_FROM=
|
||||||
|
EMAIL_SERVER=
|
||||||
|
BASE_URL=
|
||||||
|
|
||||||
|
# Proxy settings
|
||||||
|
PROXY=
|
||||||
|
PROXY_USERNAME=
|
||||||
|
PROXY_PASSWORD=
|
||||||
|
PROXY_BYPASS=
|
||||||
|
|
||||||
|
# PDF archive settings
|
||||||
|
PDF_MARGIN_TOP=
|
||||||
|
PDF_MARGIN_BOTTOM=
|
||||||
|
|
||||||
|
#################
|
||||||
|
# SSO Providers #
|
||||||
|
#################
|
||||||
|
|
||||||
|
# 42 School
|
||||||
|
NEXT_PUBLIC_FORTYTWO_ENABLED=
|
||||||
|
FORTYTWO_CUSTOM_NAME=
|
||||||
|
FORTYTWO_CLIENT_ID=
|
||||||
|
FORTYTWO_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Apple
|
||||||
|
NEXT_PUBLIC_APPLE_ENABLED=
|
||||||
|
APPLE_CUSTOM_NAME=
|
||||||
|
APPLE_ID=
|
||||||
|
APPLE_SECRET=
|
||||||
|
|
||||||
|
# Atlassian
|
||||||
|
NEXT_PUBLIC_ATLASSIAN_ENABLED=
|
||||||
|
ATLASSIAN_CUSTOM_NAME=
|
||||||
|
ATLASSIAN_CLIENT_ID=
|
||||||
|
ATLASSIAN_CLIENT_SECRET=
|
||||||
|
ATLASSIAN_SCOPE=
|
||||||
|
|
||||||
|
# Auth0
|
||||||
|
NEXT_PUBLIC_AUTH0_ENABLED=
|
||||||
|
AUTH0_CUSTOM_NAME=
|
||||||
|
AUTH0_ISSUER=
|
||||||
|
AUTH0_CLIENT_SECRET=
|
||||||
|
AUTH0_CLIENT_ID=
|
||||||
|
|
||||||
|
# Authelia
|
||||||
|
NEXT_PUBLIC_AUTHELIA_ENABLED=
|
||||||
|
AUTHELIA_CLIENT_ID=
|
||||||
|
AUTHELIA_CLIENT_SECRET=
|
||||||
|
AUTHELIA_WELLKNOWN_URL=
|
||||||
|
|
||||||
|
# Authentik
|
||||||
|
NEXT_PUBLIC_AUTHENTIK_ENABLED=
|
||||||
|
AUTHENTIK_CUSTOM_NAME=
|
||||||
|
AUTHENTIK_ISSUER=
|
||||||
|
AUTHENTIK_CLIENT_ID=
|
||||||
|
AUTHENTIK_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Azure AD B2C
|
||||||
|
NEXT_PUBLIC_AZURE_AD_B2C_ENABLED=
|
||||||
|
AZURE_AD_B2C_TENANT_NAME=
|
||||||
|
AZURE_AD_B2C_CLIENT_ID=
|
||||||
|
AZURE_AD_B2C_CLIENT_SECRET=
|
||||||
|
AZURE_AD_B2C_PRIMARY_USER_FLOW=
|
||||||
|
|
||||||
|
# Azure AD
|
||||||
|
NEXT_PUBLIC_AZURE_AD_ENABLED=
|
||||||
|
AZURE_AD_CLIENT_ID=
|
||||||
|
AZURE_AD_CLIENT_SECRET=
|
||||||
|
AZURE_AD_TENANT_ID=
|
||||||
|
|
||||||
|
# Battle.net
|
||||||
|
NEXT_PUBLIC_BATTLENET_ENABLED=
|
||||||
|
BATTLENET_CUSTOM_NAME=
|
||||||
|
BATTLENET_CLIENT_ID=
|
||||||
|
BATTLENET_CLIENT_SECRET=
|
||||||
|
BATTLENET_ISSUER=
|
||||||
|
|
||||||
|
# Box
|
||||||
|
NEXT_PUBLIC_BOX_ENABLED=
|
||||||
|
BOX_CUSTOM_NAME=
|
||||||
|
BOX_CLIENT_ID=
|
||||||
|
BOX_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Bungie
|
||||||
|
NEXT_PUBLIC_BUNGIE_ENABLED=
|
||||||
|
BUNGIE_CUSTOM_NAME=
|
||||||
|
BUNGIE_CLIENT_ID=
|
||||||
|
BUNGIE_CLIENT_SECRET=
|
||||||
|
BUNGIE_API_KEY=
|
||||||
|
|
||||||
|
# Cognito
|
||||||
|
NEXT_PUBLIC_COGNITO_ENABLED=
|
||||||
|
COGNITO_CUSTOM_NAME=
|
||||||
|
COGNITO_CLIENT_ID=
|
||||||
|
COGNITO_CLIENT_SECRET=
|
||||||
|
COGNITO_ISSUER=
|
||||||
|
|
||||||
|
# Coinbase
|
||||||
|
NEXT_PUBLIC_COINBASE_ENABLED=
|
||||||
|
COINBASE_CUSTOM_NAME=
|
||||||
|
COINBASE_CLIENT_ID=
|
||||||
|
COINBASE_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Discord
|
||||||
|
NEXT_PUBLIC_DISCORD_ENABLED=
|
||||||
|
DISCORD_CUSTOM_NAME=
|
||||||
|
DISCORD_CLIENT_ID=
|
||||||
|
DISCORD_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Dropbox
|
||||||
|
NEXT_PUBLIC_DROPBOX_ENABLED=
|
||||||
|
DROPBOX_CUSTOM_NAME=
|
||||||
|
DROPBOX_CLIENT_ID=
|
||||||
|
DROPBOX_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# DuendeIndentityServer6
|
||||||
|
NEXT_PUBLIC_DUENDE_IDS6_ENABLED=
|
||||||
|
DUENDE_IDS6_CUSTOM_NAME=
|
||||||
|
DUENDE_IDS6_CLIENT_ID=
|
||||||
|
DUENDE_IDS6_CLIENT_SECRET=
|
||||||
|
DUENDE_IDS6_ISSUER=
|
||||||
|
|
||||||
|
# EVE Online
|
||||||
|
NEXT_PUBLIC_EVEONLINE_ENABLED=
|
||||||
|
EVEONLINE_CUSTOM_NAME=
|
||||||
|
EVEONLINE_CLIENT_ID=
|
||||||
|
EVEONLINE_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Facebook
|
||||||
|
NEXT_PUBLIC_FACEBOOK_ENABLED=
|
||||||
|
FACEBOOK_CUSTOM_NAME=
|
||||||
|
FACEBOOK_CLIENT_ID=
|
||||||
|
FACEBOOK_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# FACEIT
|
||||||
|
NEXT_PUBLIC_FACEIT_ENABLED=
|
||||||
|
FACEIT_CUSTOM_NAME=
|
||||||
|
FACEIT_CLIENT_ID=
|
||||||
|
FACEIT_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Foursquare
|
||||||
|
NEXT_PUBLIC_FOURSQUARE_ENABLED=
|
||||||
|
FOURSQUARE_CUSTOM_NAME=
|
||||||
|
FOURSQUARE_CLIENT_ID=
|
||||||
|
FOURSQUARE_CLIENT_SECRET=
|
||||||
|
FOURSQUARE_APIVERSION=
|
||||||
|
|
||||||
|
# Freshbooks
|
||||||
|
NEXT_PUBLIC_FRESHBOOKS_ENABLED=
|
||||||
|
FRESHBOOKS_CUSTOM_NAME=
|
||||||
|
FRESHBOOKS_CLIENT_ID=
|
||||||
|
FRESHBOOKS_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# FusionAuth
|
||||||
|
NEXT_PUBLIC_FUSIONAUTH_ENABLED=
|
||||||
|
FUSIONAUTH_CUSTOM_NAME=
|
||||||
|
FUSIONAUTH_CLIENT_ID=
|
||||||
|
FUSIONAUTH_CLIENT_SECRET=
|
||||||
|
FUSIONAUTH_ISSUER=
|
||||||
|
FUSIONAUTH_TENANT_ID=
|
||||||
|
|
||||||
|
# GitHub
|
||||||
|
NEXT_PUBLIC_GITHUB_ENABLED=
|
||||||
|
GITHUB_CUSTOM_NAME=
|
||||||
|
GITHUB_ID=
|
||||||
|
GITHUB_SECRET=
|
||||||
|
|
||||||
|
# GitLab
|
||||||
|
NEXT_PUBLIC_GITLAB_ENABLED=
|
||||||
|
GITLAB_CUSTOM_NAME=
|
||||||
|
GITLAB_CLIENT_ID=
|
||||||
|
GITLAB_CLIENT_SECRET=
|
||||||
|
GITLAB_AUTH_URL=
|
||||||
|
|
||||||
|
# Google
|
||||||
|
NEXT_PUBLIC_GOOGLE_ENABLED=
|
||||||
|
GOOGLE_CUSTOM_NAME=
|
||||||
|
GOOGLE_CLIENT_ID=
|
||||||
|
GOOGLE_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# HubSpot
|
||||||
|
NEXT_PUBLIC_HUBSPOT_ENABLED=
|
||||||
|
HUBSPOT_CUSTOM_NAME=
|
||||||
|
HUBSPOT_CLIENT_ID=
|
||||||
|
HUBSPOT_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# IdentityServer4
|
||||||
|
NEXT_PUBLIC_IDS4_ENABLED=
|
||||||
|
IDS4_CUSTOM_NAME=
|
||||||
|
IDS4_CLIENT_ID=
|
||||||
|
IDS4_CLIENT_SECRET=
|
||||||
|
IDS4_ISSUER=
|
||||||
|
|
||||||
|
# Kakao
|
||||||
|
NEXT_PUBLIC_KAKAO_ENABLED=
|
||||||
|
KAKAO_CUSTOM_NAME=
|
||||||
|
KAKAO_CLIENT_ID=
|
||||||
|
KAKAO_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Keycloak
|
||||||
|
NEXT_PUBLIC_KEYCLOAK_ENABLED=
|
||||||
|
KEYCLOAK_CUSTOM_NAME=
|
||||||
|
KEYCLOAK_ISSUER=
|
||||||
|
KEYCLOAK_CLIENT_ID=
|
||||||
|
KEYCLOAK_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# LINE
|
||||||
|
NEXT_PUBLIC_LINE_ENABLED=
|
||||||
|
LINE_CUSTOM_NAME=
|
||||||
|
LINE_CLIENT_ID=
|
||||||
|
LINE_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# LinkedIn
|
||||||
|
NEXT_PUBLIC_LINKEDIN_ENABLED=
|
||||||
|
LINKEDIN_CUSTOM_NAME=
|
||||||
|
LINKEDIN_CLIENT_ID=
|
||||||
|
LINKEDIN_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Mailchimp
|
||||||
|
NEXT_PUBLIC_MAILCHIMP_ENABLED=
|
||||||
|
MAILCHIMP_CUSTOM_NAME=
|
||||||
|
MAILCHIMP_CLIENT_ID=
|
||||||
|
MAILCHIMP_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Mail.ru
|
||||||
|
NEXT_PUBLIC_MAILRU_ENABLED=
|
||||||
|
MAILRU_CUSTOM_NAME=
|
||||||
|
MAILRU_CLIENT_ID=
|
||||||
|
MAILRU_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Naver
|
||||||
|
NEXT_PUBLIC_NAVER_ENABLED=
|
||||||
|
NAVER_CUSTOM_NAME=
|
||||||
|
NAVER_CLIENT_ID=
|
||||||
|
NAVER_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Netlify
|
||||||
|
NEXT_PUBLIC_NETLIFY_ENABLED=
|
||||||
|
NETLIFY_CUSTOM_NAME=
|
||||||
|
NETLIFY_CLIENT_ID=
|
||||||
|
NETLIFY_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Okta
|
||||||
|
NEXT_PUBLIC_OKTA_ENABLED=
|
||||||
|
OKTA_CUSTOM_NAME=
|
||||||
|
OKTA_CLIENT_ID=
|
||||||
|
OKTA_CLIENT_SECRET=
|
||||||
|
OKTA_ISSUER=
|
||||||
|
|
||||||
|
# OneLogin
|
||||||
|
NEXT_PUBLIC_ONELOGIN_ENABLED=
|
||||||
|
ONELOGIN_CUSTOM_NAME=
|
||||||
|
ONELOGIN_CLIENT_ID=
|
||||||
|
ONELOGIN_CLIENT_SECRET=
|
||||||
|
ONELOGIN_ISSUER=
|
||||||
|
|
||||||
|
# Osso
|
||||||
|
NEXT_PUBLIC_OSSO_ENABLED=
|
||||||
|
OSSO_CUSTOM_NAME=
|
||||||
|
OSSO_CLIENT_ID=
|
||||||
|
OSSO_CLIENT_SECRET=
|
||||||
|
OSSO_ISSUER=
|
||||||
|
|
||||||
|
# osu!
|
||||||
|
NEXT_PUBLIC_OSU_ENABLED=
|
||||||
|
OSU_CUSTOM_NAME=
|
||||||
|
OSU_CLIENT_ID=
|
||||||
|
OSU_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Patreon
|
||||||
|
NEXT_PUBLIC_PATREON_ENABLED=
|
||||||
|
PATREON_CUSTOM_NAME=
|
||||||
|
PATREON_CLIENT_ID=
|
||||||
|
PATREON_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Pinterest
|
||||||
|
NEXT_PUBLIC_PINTEREST_ENABLED=
|
||||||
|
PINTEREST_CUSTOM_NAME=
|
||||||
|
PINTEREST_CLIENT_ID=
|
||||||
|
PINTEREST_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Pipedrive
|
||||||
|
NEXT_PUBLIC_PIPEDRIVE_ENABLED=
|
||||||
|
PIPEDRIVE_CUSTOM_NAME=
|
||||||
|
PIPEDRIVE_CLIENT_ID=
|
||||||
|
PIPEDRIVE_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Reddit
|
||||||
|
NEXT_PUBLIC_REDDIT_ENABLED=
|
||||||
|
REDDIT_CUSTOM_NAME=
|
||||||
|
REDDIT_CLIENT_ID=
|
||||||
|
REDDIT_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Salesforce
|
||||||
|
NEXT_PUBLIC_SALESFORCE_ENABLED=
|
||||||
|
SALESFORCE_CUSTOM_NAME=
|
||||||
|
SALESFORCE_CLIENT_ID=
|
||||||
|
SALESFORCE_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Slack
|
||||||
|
NEXT_PUBLIC_SLACK_ENABLED=
|
||||||
|
SLACK_CUSTOM_NAME=
|
||||||
|
SLACK_CLIENT_ID=
|
||||||
|
SLACK_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Spotify
|
||||||
|
NEXT_PUBLIC_SPOTIFY_ENABLED=
|
||||||
|
SPOTIFY_CUSTOM_NAME=
|
||||||
|
SPOTIFY_CLIENT_ID=
|
||||||
|
SPOTIFY_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Strava
|
||||||
|
NEXT_PUBLIC_STRAVA_ENABLED=
|
||||||
|
STRAVA_CUSTOM_NAME=
|
||||||
|
STRAVA_CLIENT_ID=
|
||||||
|
STRAVA_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Synology
|
||||||
|
NEXT_PUBLIC_SYNOLOGY_ENABLED=
|
||||||
|
SYNOLOGY_CUSTOM_NAME=
|
||||||
|
SYNOLOGY_CLIENT_ID=
|
||||||
|
SYNOLOGY_CLIENT_SECRET=
|
||||||
|
SYNOLOGY_WELLKNOWN_URL=
|
||||||
|
|
||||||
|
# Todoist
|
||||||
|
NEXT_PUBLIC_TODOIST_ENABLED=
|
||||||
|
TODOIST_CUSTOM_NAME=
|
||||||
|
TODOIST_CLIENT_ID=
|
||||||
|
TODOIST_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Twitch
|
||||||
|
NEXT_PUBLIC_TWITCH_ENABLED=
|
||||||
|
TWITCH_CUSTOM_NAME=
|
||||||
|
TWITCH_CLIENT_ID=
|
||||||
|
TWITCH_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# United Effects
|
||||||
|
NEXT_PUBLIC_UNITED_EFFECTS_ENABLED=
|
||||||
|
UNITED_EFFECTS_CUSTOM_NAME=
|
||||||
|
UNITED_EFFECTS_CLIENT_ID=
|
||||||
|
UNITED_EFFECTS_CLIENT_SECRET=
|
||||||
|
UNITED_EFFECTS_ISSUER=
|
||||||
|
|
||||||
|
# VK
|
||||||
|
NEXT_PUBLIC_VK_ENABLED=
|
||||||
|
VK_CUSTOM_NAME=
|
||||||
|
VK_CLIENT_ID=
|
||||||
|
VK_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Wikimedia
|
||||||
|
NEXT_PUBLIC_WIKIMEDIA_ENABLED=
|
||||||
|
WIKIMEDIA_CUSTOM_NAME=
|
||||||
|
WIKIMEDIA_CLIENT_ID=
|
||||||
|
WIKIMEDIA_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Wordpress.com
|
||||||
|
NEXT_PUBLIC_WORDPRESS_ENABLED=
|
||||||
|
WORDPRESS_CUSTOM_NAME=
|
||||||
|
WORDPRESS_CLIENT_ID=
|
||||||
|
WORDPRESS_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Yandex
|
||||||
|
NEXT_PUBLIC_YANDEX_ENABLED=
|
||||||
|
YANDEX_CUSTOM_NAME=
|
||||||
|
YANDEX_CLIENT_ID=
|
||||||
|
YANDEX_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Zitadel
|
||||||
|
NEXT_PUBLIC_ZITADEL_ENABLED=
|
||||||
|
ZITADEL_CUSTOM_NAME=
|
||||||
|
ZITADEL_CLIENT_ID=
|
||||||
|
ZITADEL_CLIENT_SECRET=
|
||||||
|
ZITADEL_ISSUER=
|
||||||
|
|
||||||
|
# Zoho
|
||||||
|
NEXT_PUBLIC_ZOHO_ENABLED=
|
||||||
|
ZOHO_CUSTOM_NAME=
|
||||||
|
ZOHO_CLIENT_ID=
|
||||||
|
ZOHO_CLIENT_SECRET=
|
||||||
|
|
||||||
|
# Zoom
|
||||||
|
NEXT_PUBLIC_ZOOM_ENABLED=
|
||||||
|
ZOOM_CUSTOM_NAME=
|
||||||
|
ZOOM_CLIENT_ID=
|
||||||
|
ZOOM_CLIENT_SECRET=
|
||||||
3
linkwarden/.gitignore
vendored
Normal file
3
linkwarden/.gitignore
vendored
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
data/
|
||||||
|
meili_data/
|
||||||
|
pgdata/
|
||||||
28
linkwarden/docker-compose.yml
Normal file
28
linkwarden/docker-compose.yml
Normal file
@@ -0,0 +1,28 @@
|
|||||||
|
services:
|
||||||
|
postgres:
|
||||||
|
image: postgres:16-alpine
|
||||||
|
env_file: .env
|
||||||
|
restart: always
|
||||||
|
volumes:
|
||||||
|
- /mnt/ssd/dbs/linkwarden/pgdata:/var/lib/postgresql/data
|
||||||
|
linkwarden:
|
||||||
|
env_file: .env
|
||||||
|
environment:
|
||||||
|
- DATABASE_URL=postgresql://postgres:${POSTGRES_PASSWORD}@postgres:5432/postgres
|
||||||
|
restart: always
|
||||||
|
# build: . # uncomment to build from source
|
||||||
|
image: ghcr.io/linkwarden/linkwarden:latest # comment to build from source
|
||||||
|
ports:
|
||||||
|
- 3012:3000
|
||||||
|
volumes:
|
||||||
|
- /mnt/ssd/dbs/linkwarden/data:/data/data
|
||||||
|
depends_on:
|
||||||
|
- postgres
|
||||||
|
- meilisearch
|
||||||
|
meilisearch:
|
||||||
|
image: getmeili/meilisearch:v1.12.8
|
||||||
|
restart: always
|
||||||
|
env_file:
|
||||||
|
- .env
|
||||||
|
volumes:
|
||||||
|
- /mnt/ssd/dbs/linkwarden/meili_data:/meili_data
|
||||||
@@ -26,6 +26,7 @@ Connect clients to: `https://mtx.alogins.net`
|
|||||||
| admin | yes |
|
| admin | yes |
|
||||||
| elizaveta | no |
|
| elizaveta | no |
|
||||||
| aleksandra | no |
|
| aleksandra | no |
|
||||||
|
| juris | no |
|
||||||
|
|
||||||
## Managing Users
|
## Managing Users
|
||||||
|
|
||||||
|
|||||||
6
mood/.env.example
Normal file
6
mood/.env.example
Normal file
@@ -0,0 +1,6 @@
|
|||||||
|
# mood service config — optional tuning only, no credentials needed.
|
||||||
|
# (Source is a directly-readable local SQLite file, not an HTTP API.)
|
||||||
|
# Copy to .env if you want to override the defaults baked into docker-compose.yml.
|
||||||
|
|
||||||
|
MOOD_SYNC_INTERVAL_SECONDS=3600
|
||||||
|
MOOD_OVERLAP_ROWS=3
|
||||||
6
mood/.gitignore
vendored
Normal file
6
mood/.gitignore
vendored
Normal file
@@ -0,0 +1,6 @@
|
|||||||
|
__pycache__/
|
||||||
|
*.pyc
|
||||||
|
*.sqlite
|
||||||
|
*.sqlite-wal
|
||||||
|
*.sqlite-shm
|
||||||
|
.env
|
||||||
13
mood/Dockerfile
Normal file
13
mood/Dockerfile
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
# Stdlib only (sqlite3 + argparse + csv) — no pip install needed.
|
||||||
|
FROM python:3.12-slim
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
COPY schema.sql ./schema.sql
|
||||||
|
COPY src ./src
|
||||||
|
|
||||||
|
ENV MOOD_DB_PATH=/data/mood_archive.sqlite \
|
||||||
|
MOOD_SOURCE_DB_PATH=/source/moodtracker/mood.db \
|
||||||
|
PYTHONUNBUFFERED=1
|
||||||
|
|
||||||
|
# Long-lived service: cli.py `sync` loops on MOOD_SYNC_INTERVAL_SECONDS.
|
||||||
|
CMD ["python", "-m", "src.cli", "sync"]
|
||||||
269
mood/README.md
Normal file
269
mood/README.md
Normal file
@@ -0,0 +1,269 @@
|
|||||||
|
# mood — local archive of mood.alogins.net
|
||||||
|
|
||||||
|
Local SQLite copy of mood entries logged at **mood.alogins.net**, plus simple
|
||||||
|
reports and a cross-source correlation hook. Kanboard **Adolf #107** (data-pipeline
|
||||||
|
half only — see "Scope" below).
|
||||||
|
|
||||||
|
Status: **built and proven against the real live data** (28 real entries pulled
|
||||||
|
and queried successfully). Not yet deployed as a running service — `docker
|
||||||
|
compose up -d` is a one-line handover, see "Deploy".
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Scope of this build
|
||||||
|
|
||||||
|
Task #107 has four parts. This directory implements **1, 3, and 4 only**:
|
||||||
|
|
||||||
|
1. ✅ Investigate whether mood.alogins.net has an API/export — done, see below.
|
||||||
|
3. ✅ Regular ingestion into local SQLite storage on Agap — done, this service.
|
||||||
|
4. ✅ Simple reports/correlations over the stored data — done, `query` CLI.
|
||||||
|
|
||||||
|
**2 is explicitly NOT built here**: "proactive Matrix/Telegram reminder" is an
|
||||||
|
*outward-facing, scheduled* message to the user. That capability belongs to
|
||||||
|
the proactive-cadence framework (Kanboard **Adolf #124**), which is currently
|
||||||
|
**parked, tagged `blocked`, awaiting a human decision** on whether to enable
|
||||||
|
its cron at all (a sibling run already had to back out an unauthorized live
|
||||||
|
crontab install — see #124 comments). Wiring a second scheduled outward
|
||||||
|
message before that decision lands would repeat the same mistake.
|
||||||
|
|
||||||
|
**Follow-up task to create**: "Wire the mood.alogins.net proactive reminder"
|
||||||
|
— *depends on #124's cron being authorized*. Once #124 is resolved, adding
|
||||||
|
this reminder is small: a text-only Backlog-card-generation step reusing
|
||||||
|
whatever executor #124 lands on, with the message
|
||||||
|
`Как день? Запиши в mood.alogins.net`. See "Ready-to-hand-over reminder"
|
||||||
|
below for a schedule a human can enable manually right now if they don't want
|
||||||
|
to wait for #124.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Investigation: does mood.alogins.net have an API?
|
||||||
|
|
||||||
|
**mood.alogins.net is not a third-party tracker** — it's a small self-built
|
||||||
|
Flask app already running on Agap:
|
||||||
|
|
||||||
|
- Source: `/home/alvis/moodtracker/app.py` (+ `templates/`)
|
||||||
|
- Container: `moodtracker` (compose at `/home/alvis/moodtracker/docker-compose.yml`)
|
||||||
|
- Caddy: `mood.alogins.net { reverse_proxy localhost:5177 }` (`/etc/caddy/Caddyfile`)
|
||||||
|
- DB: SQLite at `/home/alvis/moodtracker/data/mood.db`, one table `entries`
|
||||||
|
(`id, ts, mood, tags, note, affirmation`), mood on a 1–5 scale, tags a JSON
|
||||||
|
array of free-text strings, `ts` ISO8601 UTC.
|
||||||
|
|
||||||
|
**It does have a JSON API**, but it's session-cookie gated, not token-based:
|
||||||
|
- `POST /login` (form `username`/`password` = `AUTH_USER`/`AUTH_PASS` env vars,
|
||||||
|
currently plaintext in its own `docker-compose.yml` — pre-existing, not
|
||||||
|
something this task introduced) → sets a Flask session cookie.
|
||||||
|
- `GET /api/history?limit=N`, `POST /api/log`, `DELETE /api/entry/<id>` — all
|
||||||
|
require that session cookie (`@require_auth`); no HTTP token/API-key.
|
||||||
|
|
||||||
|
**Chosen ingestion path: read the SQLite file directly, not the HTTP API.**
|
||||||
|
`data/mood.db` is host-readable (`644`, owned by `root:root`, world-read bit
|
||||||
|
set) — no credential needed. This is strictly more robust than replicating a
|
||||||
|
cookie-login flow: it survives moodtracker adding/changing auth, needs no
|
||||||
|
secret in this service at all, and is read-only by construction (bind-mounted
|
||||||
|
`:ro`), so it can never corrupt or lock the live app's database.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Ingestion architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
moodtracker's own SQLite file ──(read-only bind mount)──► src/mood_source.py
|
||||||
|
/data/mood.db │
|
||||||
|
▼
|
||||||
|
src/sync.py
|
||||||
|
(cursor + idempotent upsert)
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
local archive: mood_entries
|
||||||
|
/mnt/dbs/mood/mood_archive.sqlite
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
src/cli.py query (reports)
|
||||||
|
```
|
||||||
|
|
||||||
|
Mirrors the sibling `googlefit` service's shape (same author idiom, see
|
||||||
|
`agap_git/googlefit/`):
|
||||||
|
|
||||||
|
- `schema.sql` — `mood_entries` (PK `source, source_id`), `sync_state`
|
||||||
|
(per-stream cursor), `ingest_runs` (audit log).
|
||||||
|
- `src/config.py` — paths + tuning, no credentials (none needed).
|
||||||
|
- `src/mood_source.py` — read-only reader for moodtracker's SQLite file.
|
||||||
|
- `src/store.py` — schema init, idempotent UPSERTs, read/report queries.
|
||||||
|
- `src/sync.py` — cursor-based incremental sync, isolated failure handling.
|
||||||
|
- `src/cli.py` — `init-db | sync | query`.
|
||||||
|
|
||||||
|
### Why SQLite, not InfluxDB
|
||||||
|
Same reasoning as `googlefit`: single-user, a handful of manually-logged
|
||||||
|
entries a week — trivial volume. Agap storage doctrine is SQLite-first. No
|
||||||
|
extra always-on TSDB service for ~30 rows/month of data.
|
||||||
|
|
||||||
|
### Idempotency / cursor
|
||||||
|
`sync_state.last_synced_id` is the high-water mark on moodtracker's own
|
||||||
|
`entries.id`. Each run re-checks the last `MOOD_OVERLAP_ROWS` (default 3)
|
||||||
|
already-synced ids too, as a cheap safety net — moodtracker currently has no
|
||||||
|
edit endpoint (only insert + delete), so this is mostly redundant today but
|
||||||
|
costs nothing.
|
||||||
|
|
||||||
|
**Deleted-upstream entries are kept.** If an entry is deleted via moodtracker's
|
||||||
|
`DELETE /api/entry/<id>`, this archive does not remove its copy — it's an
|
||||||
|
append-only journal by design, so history survives accidental or intentional
|
||||||
|
deletes in the live app.
|
||||||
|
|
||||||
|
### Proven against real data
|
||||||
|
```
|
||||||
|
$ python -m src.cli sync --once
|
||||||
|
{"synced": {"entries": 28, "errors": []}}
|
||||||
|
$ python -m src.cli query summary
|
||||||
|
{"entries": 28, "coverage": {"earliest": "2026-05-18T04:44:34...", "latest": "2026-07-07T05:49:34..."}, "avg_mood_all_time": 3.64, ...}
|
||||||
|
```
|
||||||
|
Re-running `sync --once` twice more produced **zero row growth** (idempotent).
|
||||||
|
The source file's mtime was unchanged after every sync run (proves read-only).
|
||||||
|
Also verified end-to-end through the built Docker image (`docker build` +
|
||||||
|
`docker run --rm ... sync --once` against the real, live-mounted
|
||||||
|
`/home/alvis/moodtracker/data`), then removed the test image/container —
|
||||||
|
nothing was left running.
|
||||||
|
|
||||||
|
### Test suite
|
||||||
|
```
|
||||||
|
$ python -m pytest tests/ -q
|
||||||
|
............ [100%]
|
||||||
|
12 passed in 0.11s
|
||||||
|
```
|
||||||
|
Covers: upsert idempotency, conflict updates, sync-cursor high-water-mark
|
||||||
|
behavior, a mock-moodtracker-schema DB driven through `run_sync` (first run,
|
||||||
|
idempotent re-run, incremental pickup of a newly-inserted row, error handling
|
||||||
|
when the source is missing, and read-only-ness), plus the report/correlation
|
||||||
|
math below.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Reports / correlations
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m src.cli query summary # counts, coverage, freshness
|
||||||
|
python -m src.cli query entries --days 30 # raw recent entries
|
||||||
|
python -m src.cli query daily --days 30 # avg mood + entry count per day
|
||||||
|
python -m src.cli query tags --days 90 # avg mood per tag (min 2 occurrences)
|
||||||
|
python -m src.cli query correlate <csv> --days 90
|
||||||
|
```
|
||||||
|
|
||||||
|
`daily` and `tags` are the "mood over time" and "which tags coincide with
|
||||||
|
low/high mood" reports from point 4. Real output against the live data
|
||||||
|
(tags, 90-day window): lowest avg mood tags were `exhausted` (1.0, n=2),
|
||||||
|
`depressed` (2.0, n=4); highest were `happy` (5.0, n=4), `energetic` (4.6, n=5).
|
||||||
|
|
||||||
|
### Correlation hook (cross-source, not wired)
|
||||||
|
`query correlate <csv_path>` computes a Pearson `r` between the daily average
|
||||||
|
mood and an arbitrary external daily series supplied as a plain
|
||||||
|
`day,value` CSV (`YYYY-MM-DD,float`). This is a deliberate seam: it takes
|
||||||
|
**data**, not a live connection to another service's DB or container, so
|
||||||
|
wiring a real second source later (e.g. `googlefit query metric
|
||||||
|
heart_rate_avg` or sleep minutes, once that service is deployed) is a one-line
|
||||||
|
change — build the CSV/dict from that service's own read-only query CLI. This
|
||||||
|
build does **not** reach into `googlefit` or any other service's DB, per the
|
||||||
|
task's "leave hooks, don't wire other services" instruction.
|
||||||
|
|
||||||
|
Tested with a synthetic series (`tests/test_store.py::test_correlate_with_series`,
|
||||||
|
near-perfect correlation asserted `r > 0.99`) and manually against the real
|
||||||
|
mood data with a hand-built "hours slept" CSV (`r = 0.933`, n=5, small sample —
|
||||||
|
illustrative only, not a real finding).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Deploy
|
||||||
|
|
||||||
|
```bash
|
||||||
|
mkdir -p /mnt/dbs/mood # (needs sudo — /mnt/dbs is root-owned; see googlefit precedent)
|
||||||
|
cd /home/alvis/agap_git/mood
|
||||||
|
docker compose up -d --build
|
||||||
|
```
|
||||||
|
The container loops `mood sync` every `MOOD_SYNC_INTERVAL_SECONDS` (default
|
||||||
|
3600s / hourly — mood entries are logged manually, hourly polling of a local
|
||||||
|
file is effectively free and gives fresh reports without any real cost).
|
||||||
|
|
||||||
|
One-off / cron alternative (no long-lived container):
|
||||||
|
```bash
|
||||||
|
docker compose run --rm mood-archive python -m src.cli sync --once
|
||||||
|
```
|
||||||
|
|
||||||
|
**This was not started as a live service in this build** — only proven via
|
||||||
|
`docker build` + `docker run --rm ... --once` against a scratch data
|
||||||
|
directory, then torn down. Bringing up the persistent `restart: unless-stopped`
|
||||||
|
container is a one-line `docker compose up -d --build` for a human/operator to run.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Read tool for Adolf
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose run --rm mood-archive python -m src.cli query summary
|
||||||
|
docker compose run --rm mood-archive python -m src.cli query daily --days 14
|
||||||
|
docker compose run --rm mood-archive python -m src.cli query tags --days 90
|
||||||
|
```
|
||||||
|
JSON on stdout, read-only, no credentials.
|
||||||
|
|
||||||
|
Follow-up (adjacent, not in this task, same idiom as `googlefit`'s README):
|
||||||
|
promote `query` to a native `agap-mcp` tool once that server's active edit
|
||||||
|
window (sibling task touching `shared-mcp.json`/`openclaw.json`) is clear.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Ready-to-hand-over reminder (point 2, NOT installed)
|
||||||
|
|
||||||
|
Per the scope note above, the proactive reminder is intentionally not built
|
||||||
|
or scheduled here. If a human wants it live **without** waiting for #124's
|
||||||
|
cron decision, here is a self-contained one-liner using the existing Telegram
|
||||||
|
bot credentials already in Vaultwarden (`TELEGRAM_BOT_TOKEN`,
|
||||||
|
`TELEGRAM_CHAT_ID`) — nothing new to build, nothing in this repo depends on
|
||||||
|
it:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
BW=/home/alvis/bin/bw
|
||||||
|
SESSION=$(env -u HTTPS_PROXY -u HTTP_PROXY -u ALL_PROXY -u https_proxy -u http_proxy -u all_proxy \
|
||||||
|
NODE_TLS_REJECT_UNAUTHORIZED=0 $BW unlock "$BW_PASSWORD" --raw 2>/dev/null)
|
||||||
|
BOT=$(env -u HTTPS_PROXY -u HTTP_PROXY -u ALL_PROXY -u https_proxy -u http_proxy -u all_proxy \
|
||||||
|
NODE_TLS_REJECT_UNAUTHORIZED=0 $BW get password "TELEGRAM_BOT_TOKEN" --session "$SESSION" 2>/dev/null)
|
||||||
|
CHAT=$(env -u HTTPS_PROXY -u HTTP_PROXY -u ALL_PROXY -u https_proxy -u http_proxy -u all_proxy \
|
||||||
|
NODE_TLS_REJECT_UNAUTHORIZED=0 $BW get password "TELEGRAM_CHAT_ID" --session "$SESSION" 2>/dev/null)
|
||||||
|
env -u HTTPS_PROXY -u HTTP_PROXY -u ALL_PROXY -u https_proxy -u http_proxy -u all_proxy \
|
||||||
|
curl -s -X POST "https://api.telegram.org/bot${BOT}/sendMessage" \
|
||||||
|
-d "chat_id=${CHAT}" -d "text=Как день? Запиши в mood.alogins.net"
|
||||||
|
```
|
||||||
|
|
||||||
|
Proposed cron (a human adds this — **not installed by this task**, per the
|
||||||
|
"never install an unattended cron on a live target" rule):
|
||||||
|
```
|
||||||
|
0 21 * * * /home/alvis/agap_git/mood/scripts/remind.sh # hypothetical path if built
|
||||||
|
```
|
||||||
|
No `scripts/remind.sh` exists yet — the command above is the full logic; if
|
||||||
|
approved, wrapping it in a script + crontab line is a ~2-minute follow-up, but
|
||||||
|
it is outward-facing (sends a message unattended) so it needs the same
|
||||||
|
explicit human go-ahead #124 is waiting on, not a unilateral install by an
|
||||||
|
agent.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Files
|
||||||
|
|
||||||
|
```
|
||||||
|
mood/
|
||||||
|
├── README.md
|
||||||
|
├── schema.sql
|
||||||
|
├── Dockerfile
|
||||||
|
├── docker-compose.yml
|
||||||
|
├── requirements.txt
|
||||||
|
├── .env.example
|
||||||
|
├── .gitignore
|
||||||
|
├── src/
|
||||||
|
│ ├── config.py
|
||||||
|
│ ├── mood_source.py # read-only reader for moodtracker's SQLite file
|
||||||
|
│ ├── store.py # schema, upserts, reports, correlation hook
|
||||||
|
│ ├── sync.py # cursor-based incremental sync
|
||||||
|
│ └── cli.py # init-db | sync | query
|
||||||
|
└── tests/
|
||||||
|
├── test_store.py
|
||||||
|
└── test_sync.py # drives run_sync against a mock moodtracker DB
|
||||||
|
```
|
||||||
|
|
||||||
|
Nothing in this directory is committed to git — `agap_git` is a git repo but
|
||||||
|
no `git add`/`git commit` was run for this task.
|
||||||
21
mood/docker-compose.yml
Normal file
21
mood/docker-compose.yml
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
services:
|
||||||
|
mood-archive:
|
||||||
|
build: .
|
||||||
|
container_name: mood-archive
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
TZ: Europe/Moscow
|
||||||
|
MOOD_DB_PATH: /data/mood_archive.sqlite
|
||||||
|
MOOD_SOURCE_DB_PATH: /source/moodtracker/mood.db
|
||||||
|
# Manual-entry data; hourly polling is more than enough (local file read).
|
||||||
|
MOOD_SYNC_INTERVAL_SECONDS: ${MOOD_SYNC_INTERVAL_SECONDS:-3600}
|
||||||
|
MOOD_OVERLAP_ROWS: ${MOOD_OVERLAP_ROWS:-3}
|
||||||
|
volumes:
|
||||||
|
# Local mood archive lives alongside the other Agap databases.
|
||||||
|
- /mnt/dbs/mood:/data
|
||||||
|
# moodtracker's own SQLite file, READ-ONLY — no credentials, no HTTP,
|
||||||
|
# no risk of this service ever writing into the live app's DB.
|
||||||
|
- /home/alvis/moodtracker/data:/source/moodtracker:ro
|
||||||
|
logging:
|
||||||
|
options:
|
||||||
|
max-size: 10m
|
||||||
4
mood/requirements.txt
Normal file
4
mood/requirements.txt
Normal file
@@ -0,0 +1,4 @@
|
|||||||
|
# Runtime (sync + store + query CLI): stdlib only — nothing required here.
|
||||||
|
#
|
||||||
|
# Dev only:
|
||||||
|
pytest>=8.0 # tests/
|
||||||
57
mood/schema.sql
Normal file
57
mood/schema.sql
Normal file
@@ -0,0 +1,57 @@
|
|||||||
|
-- Mood archive — SQLite schema.
|
||||||
|
--
|
||||||
|
-- mood.alogins.net (container `moodtracker`, /home/alvis/moodtracker) is a small
|
||||||
|
-- self-built Flask app with its own SQLite DB. It has no token-based API — its
|
||||||
|
-- only auth is a session-cookie login (/login with AUTH_USER/AUTH_PASS) guarding
|
||||||
|
-- /api/log, /api/history, /api/entry/<id>. Its DB file, however, is directly
|
||||||
|
-- readable on the host (world-readable, 644) at
|
||||||
|
-- /home/alvis/moodtracker/data/mood.db. This archiver reads that file straight
|
||||||
|
-- (via a read-only bind mount) instead of scraping the HTTP session API — no
|
||||||
|
-- credential handling needed, and it is immune to any future change in the
|
||||||
|
-- moodtracker app's auth scheme.
|
||||||
|
--
|
||||||
|
-- Why SQLite (not InfluxDB): single-user, few-entries-per-day mood logging is
|
||||||
|
-- tiny volume; Agap storage doctrine is SQLite-first (see googlefit/schema.sql
|
||||||
|
-- for the same reasoning). Mirrors that service's shape: idempotent upserts,
|
||||||
|
-- a sync cursor, an ingest-run audit log.
|
||||||
|
|
||||||
|
PRAGMA journal_mode = WAL;
|
||||||
|
|
||||||
|
-- One row per moodtracker entry. PK is the *source* row id (moodtracker's own
|
||||||
|
-- autoincrement id) + source name, so re-syncing never duplicates and a future
|
||||||
|
-- second mood source (were one ever added) can't collide ids with this one.
|
||||||
|
CREATE TABLE IF NOT EXISTS mood_entries (
|
||||||
|
source TEXT NOT NULL DEFAULT 'moodtracker',
|
||||||
|
source_id INTEGER NOT NULL, -- moodtracker entries.id
|
||||||
|
ts TEXT NOT NULL, -- ISO8601 UTC, as recorded by moodtracker
|
||||||
|
mood INTEGER NOT NULL, -- 1-5 scale used by moodtracker
|
||||||
|
tags TEXT NOT NULL DEFAULT '[]', -- JSON array of tag strings
|
||||||
|
note TEXT,
|
||||||
|
affirmation TEXT,
|
||||||
|
ingested_at TEXT NOT NULL,
|
||||||
|
PRIMARY KEY (source, source_id)
|
||||||
|
);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_mood_ts ON mood_entries (ts);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_mood_mood ON mood_entries (mood);
|
||||||
|
|
||||||
|
-- Incremental-sync cursor per stream (one stream today: 'moodtracker_entries').
|
||||||
|
-- last_synced_id is the high-water mark on source_id; each run re-checks a
|
||||||
|
-- small overlap of already-synced ids too (cheap, guards against any future
|
||||||
|
-- edit capability moodtracker doesn't have today).
|
||||||
|
CREATE TABLE IF NOT EXISTS sync_state (
|
||||||
|
stream_key TEXT PRIMARY KEY,
|
||||||
|
last_synced_id INTEGER NOT NULL DEFAULT 0,
|
||||||
|
last_run_at TEXT,
|
||||||
|
last_status TEXT, -- ok | error
|
||||||
|
last_error TEXT
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Audit log of ingestion runs (observability; Zabbix can read staleness later).
|
||||||
|
CREATE TABLE IF NOT EXISTS ingest_runs (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
started_at TEXT NOT NULL,
|
||||||
|
finished_at TEXT,
|
||||||
|
status TEXT, -- ok | error
|
||||||
|
entries_upserted INTEGER DEFAULT 0,
|
||||||
|
error TEXT
|
||||||
|
);
|
||||||
0
mood/src/__init__.py
Normal file
0
mood/src/__init__.py
Normal file
109
mood/src/cli.py
Normal file
109
mood/src/cli.py
Normal file
@@ -0,0 +1,109 @@
|
|||||||
|
"""mood CLI — single entrypoint for ingestion and reads.
|
||||||
|
|
||||||
|
Commands:
|
||||||
|
init-db create the SQLite schema
|
||||||
|
sync [--once] pull from moodtracker's SQLite; loop unless --once
|
||||||
|
query summary counts, coverage, freshness, last run
|
||||||
|
query entries [--days N] recent raw entries
|
||||||
|
query daily [--days N] average mood + entry count per day
|
||||||
|
query tags [--days N] average mood per tag (simple correlation)
|
||||||
|
query correlate <csv> [--days N] Pearson r between daily mood and an external
|
||||||
|
day,value CSV (generic cross-source hook —
|
||||||
|
see store.correlate_with_series docstring)
|
||||||
|
|
||||||
|
The `query` commands are the read tool for Adolf: JSON on stdout, no creds needed.
|
||||||
|
"""
|
||||||
|
import argparse
|
||||||
|
import csv
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
|
||||||
|
from . import config, store
|
||||||
|
from .sync import run_sync, sync_loop
|
||||||
|
|
||||||
|
|
||||||
|
def _conn():
|
||||||
|
return store.connect(config.DB_PATH)
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_init_db(_):
|
||||||
|
conn = _conn()
|
||||||
|
store.init_db(conn)
|
||||||
|
print(f"Initialized schema at {config.DB_PATH}")
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_sync(args):
|
||||||
|
conn = _conn()
|
||||||
|
store.init_db(conn)
|
||||||
|
exit_code = 0
|
||||||
|
for totals in sync_loop(conn, once=args.once):
|
||||||
|
print(json.dumps({"synced": totals}))
|
||||||
|
if totals["errors"]:
|
||||||
|
print("WARN: " + " | ".join(totals["errors"]), file=sys.stderr)
|
||||||
|
exit_code = 1
|
||||||
|
return exit_code
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_query(args):
|
||||||
|
conn = _conn()
|
||||||
|
store.init_db(conn)
|
||||||
|
if args.what == "summary":
|
||||||
|
out = store.summary(conn)
|
||||||
|
elif args.what == "entries":
|
||||||
|
out = store.recent_entries(conn, days=args.days)
|
||||||
|
elif args.what == "daily":
|
||||||
|
out = store.daily_mood(conn, days=args.days)
|
||||||
|
elif args.what == "tags":
|
||||||
|
out = store.tag_breakdown(conn, days=args.days)
|
||||||
|
elif args.what == "correlate":
|
||||||
|
external = {}
|
||||||
|
with open(args.csv_path, newline="") as f:
|
||||||
|
for row in csv.reader(f):
|
||||||
|
if len(row) < 2 or row[0].lower() == "day":
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
external[row[0].strip()] = float(row[1])
|
||||||
|
except ValueError:
|
||||||
|
continue
|
||||||
|
out = store.correlate_with_series(conn, external, days=args.days)
|
||||||
|
else:
|
||||||
|
print(f"unknown query: {args.what}", file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
print(json.dumps(out, indent=2, ensure_ascii=False))
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def build_parser():
|
||||||
|
p = argparse.ArgumentParser(prog="mood")
|
||||||
|
sub = p.add_subparsers(dest="cmd", required=True)
|
||||||
|
|
||||||
|
sub.add_parser("init-db").set_defaults(func=cmd_init_db)
|
||||||
|
|
||||||
|
ps = sub.add_parser("sync")
|
||||||
|
ps.add_argument("--once", action="store_true", help="run one pass and exit")
|
||||||
|
ps.set_defaults(func=cmd_sync)
|
||||||
|
|
||||||
|
pq = sub.add_parser("query")
|
||||||
|
pqs = pq.add_subparsers(dest="what", required=True)
|
||||||
|
pqs.add_parser("summary")
|
||||||
|
pe = pqs.add_parser("entries")
|
||||||
|
pe.add_argument("--days", type=int, default=30)
|
||||||
|
pd = pqs.add_parser("daily")
|
||||||
|
pd.add_argument("--days", type=int, default=30)
|
||||||
|
pt = pqs.add_parser("tags")
|
||||||
|
pt.add_argument("--days", type=int, default=90)
|
||||||
|
pc = pqs.add_parser("correlate")
|
||||||
|
pc.add_argument("csv_path", help="CSV with 'day,value' rows (day=YYYY-MM-DD)")
|
||||||
|
pc.add_argument("--days", type=int, default=90)
|
||||||
|
pq.set_defaults(func=cmd_query)
|
||||||
|
|
||||||
|
return p
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv=None):
|
||||||
|
args = build_parser().parse_args(argv)
|
||||||
|
return args.func(args) or 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
23
mood/src/config.py
Normal file
23
mood/src/config.py
Normal file
@@ -0,0 +1,23 @@
|
|||||||
|
"""Configuration for the mood archiver.
|
||||||
|
|
||||||
|
No credentials required: the source is a directly-readable SQLite file
|
||||||
|
(mood.alogins.net / container `moodtracker`), bind-mounted read-only into this
|
||||||
|
container. There is nothing to fetch from Vaultwarden for this service.
|
||||||
|
"""
|
||||||
|
import os
|
||||||
|
|
||||||
|
# Our own local archive.
|
||||||
|
DB_PATH = os.environ.get("MOOD_DB_PATH", "/data/mood_archive.sqlite")
|
||||||
|
|
||||||
|
# moodtracker's SQLite file, read-only bind mount (see docker-compose.yml).
|
||||||
|
SOURCE_DB_PATH = os.environ.get("MOOD_SOURCE_DB_PATH", "/source/moodtracker/mood.db")
|
||||||
|
|
||||||
|
# Re-check this many already-synced ids on every run (cheap safety net in case
|
||||||
|
# moodtracker ever grows an edit capability; it currently only supports
|
||||||
|
# insert + delete, so this is mostly a no-op today).
|
||||||
|
OVERLAP_ROWS = int(os.environ.get("MOOD_OVERLAP_ROWS", "3"))
|
||||||
|
|
||||||
|
# Seconds between automatic sync cycles when run as a long-lived service.
|
||||||
|
# Mood entries are logged manually a few times a week at most; hourly is far
|
||||||
|
# more than enough and the read is essentially free (local SQLite file).
|
||||||
|
SYNC_INTERVAL_SECONDS = int(os.environ.get("MOOD_SYNC_INTERVAL_SECONDS", "3600"))
|
||||||
34
mood/src/mood_source.py
Normal file
34
mood/src/mood_source.py
Normal file
@@ -0,0 +1,34 @@
|
|||||||
|
"""Reader for moodtracker's own SQLite file (the mood.alogins.net source DB).
|
||||||
|
|
||||||
|
moodtracker's schema (see /home/alvis/moodtracker/app.py):
|
||||||
|
|
||||||
|
CREATE TABLE entries (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
ts TEXT NOT NULL,
|
||||||
|
mood INTEGER NOT NULL,
|
||||||
|
tags TEXT NOT NULL, -- JSON array, e.g. '["sad","tired"]'
|
||||||
|
note TEXT,
|
||||||
|
affirmation TEXT
|
||||||
|
)
|
||||||
|
|
||||||
|
We open it read-only (URI mode=ro) so this archiver can never corrupt or lock
|
||||||
|
the live app's database.
|
||||||
|
"""
|
||||||
|
import sqlite3
|
||||||
|
|
||||||
|
|
||||||
|
def connect_source(path):
|
||||||
|
"""Read-only connection to the moodtracker SQLite file."""
|
||||||
|
conn = sqlite3.connect(f"file:{path}?mode=ro", uri=True)
|
||||||
|
conn.row_factory = sqlite3.Row
|
||||||
|
return conn
|
||||||
|
|
||||||
|
|
||||||
|
def fetch_entries_since(source_conn, since_id=0, limit=100000):
|
||||||
|
"""Entries with id > since_id, oldest first."""
|
||||||
|
rows = source_conn.execute(
|
||||||
|
"""SELECT id, ts, mood, tags, note, affirmation
|
||||||
|
FROM entries WHERE id > ? ORDER BY id ASC LIMIT ?""",
|
||||||
|
(since_id, limit),
|
||||||
|
).fetchall()
|
||||||
|
return [dict(r) for r in rows]
|
||||||
202
mood/src/store.py
Normal file
202
mood/src/store.py
Normal file
@@ -0,0 +1,202 @@
|
|||||||
|
"""SQLite storage layer: schema init, idempotent upserts, and read queries.
|
||||||
|
|
||||||
|
All writes are UPSERTs keyed on (source, source_id), so re-running a sync over
|
||||||
|
an overlapping id range is a no-op rather than a duplicate. Reads back the
|
||||||
|
Adolf query CLI (`cli.py query ...`).
|
||||||
|
"""
|
||||||
|
import os
|
||||||
|
import sqlite3
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
|
||||||
|
SCHEMA_PATH = os.path.join(os.path.dirname(os.path.dirname(__file__)), "schema.sql")
|
||||||
|
|
||||||
|
|
||||||
|
def _now_iso():
|
||||||
|
return datetime.now(tz=timezone.utc).isoformat()
|
||||||
|
|
||||||
|
|
||||||
|
def connect(db_path):
|
||||||
|
os.makedirs(os.path.dirname(os.path.abspath(db_path)), exist_ok=True)
|
||||||
|
conn = sqlite3.connect(db_path)
|
||||||
|
conn.row_factory = sqlite3.Row
|
||||||
|
return conn
|
||||||
|
|
||||||
|
|
||||||
|
def init_db(conn):
|
||||||
|
with open(SCHEMA_PATH) as f:
|
||||||
|
conn.executescript(f.read())
|
||||||
|
conn.commit()
|
||||||
|
|
||||||
|
|
||||||
|
# --- writes ---------------------------------------------------------------
|
||||||
|
|
||||||
|
def upsert_entries(conn, rows, source="moodtracker"):
|
||||||
|
now = _now_iso()
|
||||||
|
n = 0
|
||||||
|
for r in rows:
|
||||||
|
conn.execute(
|
||||||
|
"""
|
||||||
|
INSERT INTO mood_entries
|
||||||
|
(source, source_id, ts, mood, tags, note, affirmation, ingested_at)
|
||||||
|
VALUES (?,?,?,?,?,?,?,?)
|
||||||
|
ON CONFLICT(source, source_id) DO UPDATE SET
|
||||||
|
ts=excluded.ts,
|
||||||
|
mood=excluded.mood,
|
||||||
|
tags=excluded.tags,
|
||||||
|
note=excluded.note,
|
||||||
|
affirmation=excluded.affirmation,
|
||||||
|
ingested_at=excluded.ingested_at
|
||||||
|
""",
|
||||||
|
(
|
||||||
|
source, r["id"], r["ts"], r["mood"], r.get("tags", "[]"),
|
||||||
|
r.get("note"), r.get("affirmation"), now,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
n += 1
|
||||||
|
conn.commit()
|
||||||
|
return n
|
||||||
|
|
||||||
|
|
||||||
|
# --- sync cursor & run audit ---------------------------------------------
|
||||||
|
|
||||||
|
def get_last_synced_id(conn, stream_key):
|
||||||
|
row = conn.execute(
|
||||||
|
"SELECT last_synced_id FROM sync_state WHERE stream_key=?", (stream_key,)
|
||||||
|
).fetchone()
|
||||||
|
return row["last_synced_id"] if row else 0
|
||||||
|
|
||||||
|
|
||||||
|
def set_sync_state(conn, stream_key, last_synced_id, status="ok", error=None):
|
||||||
|
conn.execute(
|
||||||
|
"""
|
||||||
|
INSERT INTO sync_state (stream_key, last_synced_id, last_run_at, last_status, last_error)
|
||||||
|
VALUES (?,?,?,?,?)
|
||||||
|
ON CONFLICT(stream_key) DO UPDATE SET
|
||||||
|
last_synced_id=MAX(sync_state.last_synced_id, excluded.last_synced_id),
|
||||||
|
last_run_at=excluded.last_run_at,
|
||||||
|
last_status=excluded.last_status,
|
||||||
|
last_error=excluded.last_error
|
||||||
|
""",
|
||||||
|
(stream_key, last_synced_id, _now_iso(), status, error),
|
||||||
|
)
|
||||||
|
conn.commit()
|
||||||
|
|
||||||
|
|
||||||
|
def start_run(conn):
|
||||||
|
cur = conn.execute(
|
||||||
|
"INSERT INTO ingest_runs (started_at, status) VALUES (?, 'running')", (_now_iso(),)
|
||||||
|
)
|
||||||
|
conn.commit()
|
||||||
|
return cur.lastrowid
|
||||||
|
|
||||||
|
|
||||||
|
def finish_run(conn, run_id, status, entries=0, error=None):
|
||||||
|
conn.execute(
|
||||||
|
"""UPDATE ingest_runs SET finished_at=?, status=?, entries_upserted=?, error=?
|
||||||
|
WHERE id=?""",
|
||||||
|
(_now_iso(), status, entries, error, run_id),
|
||||||
|
)
|
||||||
|
conn.commit()
|
||||||
|
|
||||||
|
|
||||||
|
# --- reads (Adolf query tool + reports) -----------------------------------
|
||||||
|
|
||||||
|
def summary(conn):
|
||||||
|
"""Compact snapshot: count, coverage, freshness, last run."""
|
||||||
|
out = {}
|
||||||
|
out["entries"] = conn.execute("SELECT COUNT(*) c FROM mood_entries").fetchone()["c"]
|
||||||
|
span = conn.execute("SELECT MIN(ts) a, MAX(ts) b FROM mood_entries").fetchone()
|
||||||
|
out["coverage"] = {"earliest": span["a"], "latest": span["b"]}
|
||||||
|
out["avg_mood_all_time"] = conn.execute(
|
||||||
|
"SELECT ROUND(AVG(mood), 2) a FROM mood_entries"
|
||||||
|
).fetchone()["a"]
|
||||||
|
out["sync_state"] = [dict(r) for r in conn.execute(
|
||||||
|
"SELECT stream_key, last_synced_id, last_run_at, last_status FROM sync_state"
|
||||||
|
).fetchall()]
|
||||||
|
last = conn.execute(
|
||||||
|
"SELECT started_at, finished_at, status, entries_upserted FROM ingest_runs "
|
||||||
|
"ORDER BY id DESC LIMIT 1"
|
||||||
|
).fetchone()
|
||||||
|
out["last_run"] = dict(last) if last else None
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def recent_entries(conn, days=30, limit=200):
|
||||||
|
return [dict(r) for r in conn.execute(
|
||||||
|
"""SELECT source_id, ts, mood, tags, note, affirmation
|
||||||
|
FROM mood_entries WHERE ts >= datetime('now', ?)
|
||||||
|
ORDER BY ts DESC LIMIT ?""",
|
||||||
|
(f"-{int(days)} days", int(limit)),
|
||||||
|
).fetchall()]
|
||||||
|
|
||||||
|
|
||||||
|
def daily_mood(conn, days=30):
|
||||||
|
"""Average mood and entry count per calendar day (report #1: mood over time)."""
|
||||||
|
return [dict(r) for r in conn.execute(
|
||||||
|
"""SELECT date(ts) AS day, ROUND(AVG(mood), 2) AS avg_mood, COUNT(*) AS entries
|
||||||
|
FROM mood_entries WHERE ts >= datetime('now', ?)
|
||||||
|
GROUP BY day ORDER BY day DESC""",
|
||||||
|
(f"-{int(days)} days",),
|
||||||
|
).fetchall()]
|
||||||
|
|
||||||
|
|
||||||
|
def tag_breakdown(conn, days=90, min_count=2):
|
||||||
|
"""Average mood per tag (simple correlation: which tags co-occur with
|
||||||
|
higher/lower mood). Tags are stored as a JSON array per entry; this
|
||||||
|
unpacks them in Python since SQLite has no native JSON array explode
|
||||||
|
without the (not always compiled-in) json1 table-valued functions."""
|
||||||
|
import json
|
||||||
|
rows = conn.execute(
|
||||||
|
"SELECT mood, tags FROM mood_entries WHERE ts >= datetime('now', ?)",
|
||||||
|
(f"-{int(days)} days",),
|
||||||
|
).fetchall()
|
||||||
|
by_tag = {}
|
||||||
|
for r in rows:
|
||||||
|
try:
|
||||||
|
tags = json.loads(r["tags"]) or []
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
tags = []
|
||||||
|
for t in tags:
|
||||||
|
by_tag.setdefault(t, []).append(r["mood"])
|
||||||
|
out = [
|
||||||
|
{"tag": t, "avg_mood": round(sum(v) / len(v), 2), "count": len(v)}
|
||||||
|
for t, v in by_tag.items()
|
||||||
|
if len(v) >= min_count
|
||||||
|
]
|
||||||
|
out.sort(key=lambda x: x["avg_mood"])
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
# --- correlation hook (generic, no other Agap service wired) --------------
|
||||||
|
|
||||||
|
def correlate_with_series(conn, external_daily, days=90):
|
||||||
|
"""Pearson correlation between daily average mood and an arbitrary
|
||||||
|
externally-supplied daily series.
|
||||||
|
|
||||||
|
`external_daily` is a dict {'YYYY-MM-DD': float}. This is a deliberate
|
||||||
|
seam for future cross-source correlation (e.g. googlefit sleep/steps) —
|
||||||
|
it takes plain data, not a live connection to another service's DB, so
|
||||||
|
wiring a second source later is a one-line change at the call site
|
||||||
|
(build the dict from that source's own query CLI) and never requires
|
||||||
|
this service to know about the other service's schema or container.
|
||||||
|
|
||||||
|
Returns {'n': overlap_days, 'r': pearson_r_or_None, 'points': [...]}."""
|
||||||
|
mood_by_day = {
|
||||||
|
r["day"]: r["avg_mood"] for r in daily_mood(conn, days=days)
|
||||||
|
}
|
||||||
|
common_days = sorted(set(mood_by_day) & set(external_daily))
|
||||||
|
xs = [mood_by_day[d] for d in common_days]
|
||||||
|
ys = [external_daily[d] for d in common_days]
|
||||||
|
n = len(xs)
|
||||||
|
if n < 2:
|
||||||
|
return {"n": n, "r": None, "points": list(zip(common_days, xs, ys))}
|
||||||
|
mx, my = sum(xs) / n, sum(ys) / n
|
||||||
|
cov = sum((x - mx) * (y - my) for x, y in zip(xs, ys))
|
||||||
|
varx = sum((x - mx) ** 2 for x in xs)
|
||||||
|
vary = sum((y - my) ** 2 for y in ys)
|
||||||
|
r = cov / (varx ** 0.5 * vary ** 0.5) if varx > 0 and vary > 0 else None
|
||||||
|
return {
|
||||||
|
"n": n,
|
||||||
|
"r": round(r, 3) if r is not None else None,
|
||||||
|
"points": [{"day": d, "mood": x, "value": y} for d, x, y in zip(common_days, xs, ys)],
|
||||||
|
}
|
||||||
50
mood/src/sync.py
Normal file
50
mood/src/sync.py
Normal file
@@ -0,0 +1,50 @@
|
|||||||
|
"""Sync orchestration: read moodtracker's SQLite file -> idempotent upsert.
|
||||||
|
|
||||||
|
Single stream ('moodtracker_entries'), cursor = highest source_id ingested so
|
||||||
|
far. Each run re-checks a small overlap of already-synced ids (config.OVERLAP_ROWS)
|
||||||
|
as a cheap safety net, then upserts anything with id > cursor - overlap.
|
||||||
|
"""
|
||||||
|
import time
|
||||||
|
|
||||||
|
from . import config, store
|
||||||
|
from .mood_source import connect_source, fetch_entries_since
|
||||||
|
|
||||||
|
STREAM_KEY = "moodtracker_entries"
|
||||||
|
|
||||||
|
|
||||||
|
def run_sync(conn, source_db_path=None):
|
||||||
|
"""One sync pass. Returns a counts dict. Never raises — errors are
|
||||||
|
recorded in ingest_runs/sync_state and returned in totals['errors']."""
|
||||||
|
source_db_path = source_db_path or config.SOURCE_DB_PATH
|
||||||
|
run_id = store.start_run(conn)
|
||||||
|
totals = {"entries": 0, "errors": []}
|
||||||
|
try:
|
||||||
|
last = store.get_last_synced_id(conn, STREAM_KEY)
|
||||||
|
since = max(0, last - config.OVERLAP_ROWS)
|
||||||
|
source_conn = connect_source(source_db_path)
|
||||||
|
try:
|
||||||
|
rows = fetch_entries_since(source_conn, since)
|
||||||
|
finally:
|
||||||
|
source_conn.close()
|
||||||
|
n = store.upsert_entries(conn, rows)
|
||||||
|
totals["entries"] = n
|
||||||
|
max_id = max((r["id"] for r in rows), default=last)
|
||||||
|
store.set_sync_state(conn, STREAM_KEY, max(max_id, last))
|
||||||
|
store.finish_run(conn, run_id, "ok", entries=n)
|
||||||
|
except Exception as e: # noqa: BLE001 - isolate failures, keep the loop alive
|
||||||
|
totals["errors"].append(str(e))
|
||||||
|
store.set_sync_state(
|
||||||
|
conn, STREAM_KEY, store.get_last_synced_id(conn, STREAM_KEY),
|
||||||
|
status="error", error=str(e),
|
||||||
|
)
|
||||||
|
store.finish_run(conn, run_id, "error", entries=0, error=str(e))
|
||||||
|
return totals
|
||||||
|
|
||||||
|
|
||||||
|
def sync_loop(conn, source_db_path=None, interval=None, once=False):
|
||||||
|
interval = interval or config.SYNC_INTERVAL_SECONDS
|
||||||
|
while True:
|
||||||
|
yield run_sync(conn, source_db_path)
|
||||||
|
if once:
|
||||||
|
return
|
||||||
|
time.sleep(interval)
|
||||||
0
mood/tests/__init__.py
Normal file
0
mood/tests/__init__.py
Normal file
112
mood/tests/test_store.py
Normal file
112
mood/tests/test_store.py
Normal file
@@ -0,0 +1,112 @@
|
|||||||
|
import json
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
|
||||||
|
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||||
|
|
||||||
|
from src import store
|
||||||
|
|
||||||
|
|
||||||
|
def _fresh_db():
|
||||||
|
path = os.path.join(tempfile.mkdtemp(), "t.sqlite")
|
||||||
|
conn = store.connect(path)
|
||||||
|
store.init_db(conn)
|
||||||
|
return conn
|
||||||
|
|
||||||
|
|
||||||
|
def _row(id_, ts, mood, tags, note="", affirmation=""):
|
||||||
|
return {"id": id_, "ts": ts, "mood": mood, "tags": json.dumps(tags),
|
||||||
|
"note": note, "affirmation": affirmation}
|
||||||
|
|
||||||
|
|
||||||
|
def test_upsert_idempotent():
|
||||||
|
conn = _fresh_db()
|
||||||
|
rows = [
|
||||||
|
_row(1, "2026-07-01T08:00:00+00:00", 4, ["calm"]),
|
||||||
|
_row(2, "2026-07-02T08:00:00+00:00", 2, ["sad", "tired"]),
|
||||||
|
]
|
||||||
|
store.upsert_entries(conn, rows)
|
||||||
|
store.upsert_entries(conn, rows) # re-run same rows
|
||||||
|
count = conn.execute("SELECT COUNT(*) c FROM mood_entries").fetchone()["c"]
|
||||||
|
assert count == 2 # no duplication despite double ingest
|
||||||
|
|
||||||
|
|
||||||
|
def test_upsert_updates_on_conflict():
|
||||||
|
conn = _fresh_db()
|
||||||
|
rows = [_row(1, "2026-07-01T08:00:00+00:00", 4, ["calm"], note="first")]
|
||||||
|
store.upsert_entries(conn, rows)
|
||||||
|
rows[0]["note"] = "corrected"
|
||||||
|
store.upsert_entries(conn, rows)
|
||||||
|
got = conn.execute("SELECT note FROM mood_entries WHERE source_id=1").fetchone()["note"]
|
||||||
|
assert got == "corrected"
|
||||||
|
|
||||||
|
|
||||||
|
def test_sync_state_high_water_mark():
|
||||||
|
conn = _fresh_db()
|
||||||
|
store.set_sync_state(conn, "moodtracker_entries", 5)
|
||||||
|
store.set_sync_state(conn, "moodtracker_entries", 3) # older cursor must not regress
|
||||||
|
assert store.get_last_synced_id(conn, "moodtracker_entries") == 5
|
||||||
|
store.set_sync_state(conn, "moodtracker_entries", 9)
|
||||||
|
assert store.get_last_synced_id(conn, "moodtracker_entries") == 9
|
||||||
|
|
||||||
|
|
||||||
|
def test_summary_and_daily_mood():
|
||||||
|
conn = _fresh_db()
|
||||||
|
rows = [
|
||||||
|
_row(1, "2026-07-01T08:00:00+00:00", 4, ["calm"]),
|
||||||
|
_row(2, "2026-07-01T20:00:00+00:00", 2, ["tired"]),
|
||||||
|
_row(3, "2026-07-02T08:00:00+00:00", 5, ["happy"]),
|
||||||
|
]
|
||||||
|
store.upsert_entries(conn, rows)
|
||||||
|
s = store.summary(conn)
|
||||||
|
assert s["entries"] == 3
|
||||||
|
assert s["coverage"]["earliest"] is not None
|
||||||
|
|
||||||
|
daily = store.daily_mood(conn, days=30)
|
||||||
|
by_day = {d["day"]: d for d in daily}
|
||||||
|
assert by_day["2026-07-01"]["entries"] == 2
|
||||||
|
assert by_day["2026-07-01"]["avg_mood"] == 3.0 # (4+2)/2
|
||||||
|
assert by_day["2026-07-02"]["avg_mood"] == 5.0
|
||||||
|
|
||||||
|
|
||||||
|
def test_tag_breakdown():
|
||||||
|
conn = _fresh_db()
|
||||||
|
rows = [
|
||||||
|
_row(1, "2026-07-01T08:00:00+00:00", 5, ["happy", "energetic"]),
|
||||||
|
_row(2, "2026-07-02T08:00:00+00:00", 1, ["sad", "tired"]),
|
||||||
|
_row(3, "2026-07-03T08:00:00+00:00", 2, ["tired"]),
|
||||||
|
]
|
||||||
|
store.upsert_entries(conn, rows)
|
||||||
|
tags = store.tag_breakdown(conn, days=90, min_count=2)
|
||||||
|
by_tag = {t["tag"]: t for t in tags}
|
||||||
|
assert by_tag["tired"]["count"] == 2
|
||||||
|
assert by_tag["tired"]["avg_mood"] == 1.5
|
||||||
|
assert "happy" not in by_tag # min_count=2 filters singletons
|
||||||
|
|
||||||
|
|
||||||
|
def test_correlate_with_series():
|
||||||
|
conn = _fresh_db()
|
||||||
|
rows = [
|
||||||
|
_row(1, "2026-07-01T08:00:00+00:00", 5, []),
|
||||||
|
_row(2, "2026-07-02T08:00:00+00:00", 4, []),
|
||||||
|
_row(3, "2026-07-03T08:00:00+00:00", 2, []),
|
||||||
|
_row(4, "2026-07-04T08:00:00+00:00", 1, []),
|
||||||
|
]
|
||||||
|
store.upsert_entries(conn, rows)
|
||||||
|
# perfectly correlated external series (e.g. "hours slept")
|
||||||
|
external = {
|
||||||
|
"2026-07-01": 8.0, "2026-07-02": 7.0,
|
||||||
|
"2026-07-03": 5.0, "2026-07-04": 4.0,
|
||||||
|
}
|
||||||
|
result = store.correlate_with_series(conn, external, days=30)
|
||||||
|
assert result["n"] == 4
|
||||||
|
assert result["r"] > 0.99 # near-perfect positive correlation
|
||||||
|
|
||||||
|
|
||||||
|
def test_correlate_too_few_points():
|
||||||
|
conn = _fresh_db()
|
||||||
|
store.upsert_entries(conn, [_row(1, "2026-07-01T08:00:00+00:00", 3, [])])
|
||||||
|
result = store.correlate_with_series(conn, {"2026-07-01": 5.0}, days=30)
|
||||||
|
assert result["n"] == 1
|
||||||
|
assert result["r"] is None
|
||||||
108
mood/tests/test_sync.py
Normal file
108
mood/tests/test_sync.py
Normal file
@@ -0,0 +1,108 @@
|
|||||||
|
import json
|
||||||
|
import os
|
||||||
|
import sqlite3
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
|
||||||
|
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||||
|
|
||||||
|
from src import store
|
||||||
|
from src.sync import run_sync
|
||||||
|
|
||||||
|
|
||||||
|
def _mock_moodtracker_db(entries):
|
||||||
|
"""Build a SQLite file with moodtracker's exact `entries` schema
|
||||||
|
(see /home/alvis/moodtracker/app.py init_db) and seed rows."""
|
||||||
|
path = os.path.join(tempfile.mkdtemp(), "mood.db")
|
||||||
|
conn = sqlite3.connect(path)
|
||||||
|
conn.execute("""
|
||||||
|
CREATE TABLE entries (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
ts TEXT NOT NULL,
|
||||||
|
mood INTEGER NOT NULL,
|
||||||
|
tags TEXT NOT NULL,
|
||||||
|
note TEXT,
|
||||||
|
affirmation TEXT
|
||||||
|
)
|
||||||
|
""")
|
||||||
|
for ts, mood, tags, note, aff in entries:
|
||||||
|
conn.execute(
|
||||||
|
"INSERT INTO entries (ts, mood, tags, note, affirmation) VALUES (?,?,?,?,?)",
|
||||||
|
(ts, mood, json.dumps(tags), note, aff),
|
||||||
|
)
|
||||||
|
conn.commit()
|
||||||
|
conn.close()
|
||||||
|
return path
|
||||||
|
|
||||||
|
|
||||||
|
def _archive_conn():
|
||||||
|
path = os.path.join(tempfile.mkdtemp(), "archive.sqlite")
|
||||||
|
conn = store.connect(path)
|
||||||
|
store.init_db(conn)
|
||||||
|
return conn
|
||||||
|
|
||||||
|
|
||||||
|
def test_sync_pulls_all_rows_first_run():
|
||||||
|
source = _mock_moodtracker_db([
|
||||||
|
("2026-07-01T08:00:00+00:00", 4, ["calm"], "n1", "a1"),
|
||||||
|
("2026-07-02T08:00:00+00:00", 2, ["sad", "tired"], "n2", ""),
|
||||||
|
])
|
||||||
|
conn = _archive_conn()
|
||||||
|
totals = run_sync(conn, source_db_path=source)
|
||||||
|
assert totals["errors"] == []
|
||||||
|
assert totals["entries"] == 2
|
||||||
|
assert conn.execute("SELECT COUNT(*) c FROM mood_entries").fetchone()["c"] == 2
|
||||||
|
|
||||||
|
|
||||||
|
def test_sync_is_idempotent_across_runs():
|
||||||
|
source = _mock_moodtracker_db([
|
||||||
|
("2026-07-01T08:00:00+00:00", 4, ["calm"], "", ""),
|
||||||
|
])
|
||||||
|
conn = _archive_conn()
|
||||||
|
run_sync(conn, source_db_path=source)
|
||||||
|
run_sync(conn, source_db_path=source) # nothing new, cursor unchanged
|
||||||
|
run_sync(conn, source_db_path=source)
|
||||||
|
assert conn.execute("SELECT COUNT(*) c FROM mood_entries").fetchone()["c"] == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_sync_picks_up_new_rows_incrementally():
|
||||||
|
path = _mock_moodtracker_db([
|
||||||
|
("2026-07-01T08:00:00+00:00", 4, ["calm"], "", ""),
|
||||||
|
])
|
||||||
|
conn = _archive_conn()
|
||||||
|
run_sync(conn, source_db_path=path)
|
||||||
|
assert conn.execute("SELECT COUNT(*) c FROM mood_entries").fetchone()["c"] == 1
|
||||||
|
|
||||||
|
# a new entry gets logged upstream between syncs
|
||||||
|
src_conn = sqlite3.connect(path)
|
||||||
|
src_conn.execute(
|
||||||
|
"INSERT INTO entries (ts, mood, tags, note, affirmation) VALUES (?,?,?,?,?)",
|
||||||
|
("2026-07-03T09:00:00+00:00", 5, json.dumps(["happy"]), "", ""),
|
||||||
|
)
|
||||||
|
src_conn.commit()
|
||||||
|
src_conn.close()
|
||||||
|
|
||||||
|
run_sync(conn, source_db_path=path)
|
||||||
|
assert conn.execute("SELECT COUNT(*) c FROM mood_entries").fetchone()["c"] == 2
|
||||||
|
|
||||||
|
|
||||||
|
def test_sync_records_error_when_source_missing():
|
||||||
|
conn = _archive_conn()
|
||||||
|
totals = run_sync(conn, source_db_path="/nonexistent/path/mood.db")
|
||||||
|
assert totals["errors"]
|
||||||
|
row = conn.execute(
|
||||||
|
"SELECT last_status, last_error FROM sync_state WHERE stream_key='moodtracker_entries'"
|
||||||
|
).fetchone()
|
||||||
|
assert row["last_status"] == "error"
|
||||||
|
assert row["last_error"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_sync_never_writes_to_source_db():
|
||||||
|
"""The source connection is opened read-only; a failed write attempt
|
||||||
|
would raise, and run_sync should never attempt one in the first place."""
|
||||||
|
source = _mock_moodtracker_db([("2026-07-01T08:00:00+00:00", 3, [], "", "")])
|
||||||
|
before = os.path.getmtime(source)
|
||||||
|
conn = _archive_conn()
|
||||||
|
run_sync(conn, source_db_path=source)
|
||||||
|
after = os.path.getmtime(source)
|
||||||
|
assert before == after
|
||||||
4
moodtracker/.env.example
Normal file
4
moodtracker/.env.example
Normal file
@@ -0,0 +1,4 @@
|
|||||||
|
# moodtracker host config — copy to .env and fill in the real value.
|
||||||
|
# Credential is stored in Vaultwarden (AI collection) as MOODTRACKER_AUTH_PASS.
|
||||||
|
|
||||||
|
MOODTRACKER_AUTH_PASS=changeme
|
||||||
1
moodtracker/.gitignore
vendored
Normal file
1
moodtracker/.gitignore
vendored
Normal file
@@ -0,0 +1 @@
|
|||||||
|
.env
|
||||||
12
moodtracker/docker-compose.yml
Normal file
12
moodtracker/docker-compose.yml
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
services:
|
||||||
|
moodtracker:
|
||||||
|
build: /home/alvis/moodtracker
|
||||||
|
container_name: moodtracker
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
AUTH_USER: admin
|
||||||
|
AUTH_PASS: ${MOODTRACKER_AUTH_PASS}
|
||||||
|
volumes:
|
||||||
|
- /home/alvis/moodtracker/data:/data
|
||||||
|
ports:
|
||||||
|
- "127.0.0.1:5177:5000"
|
||||||
29
ollama/docker-compose.yml
Normal file
29
ollama/docker-compose.yml
Normal file
@@ -0,0 +1,29 @@
|
|||||||
|
services:
|
||||||
|
ollama:
|
||||||
|
image: ollama/ollama
|
||||||
|
container_name: ollama
|
||||||
|
ports:
|
||||||
|
- "11436:11434"
|
||||||
|
volumes:
|
||||||
|
- /mnt/ssd/ai/ollama:/root/.ollama
|
||||||
|
restart: always
|
||||||
|
environment:
|
||||||
|
# Allow qwen3:8b + qwen2.5:1.5b to coexist in VRAM (~6.7-7.7 GB on 8 GB GPU)
|
||||||
|
- OLLAMA_MAX_LOADED_MODELS=2
|
||||||
|
# One GPU inference at a time — prevents compute contention between models
|
||||||
|
- OLLAMA_NUM_PARALLEL=1
|
||||||
|
# Force all layers to GPU — fail instead of falling back to CPU
|
||||||
|
- OLLAMA_NUM_GPU=999
|
||||||
|
runtime: nvidia
|
||||||
|
mem_limit: 4g
|
||||||
|
# kb#190: `ollama list` just queries the local server's model registry --
|
||||||
|
# no model load/inference, cheap. This is a SEPARATE compose project from
|
||||||
|
# openai/docker-compose.yml (reached from there via
|
||||||
|
# host.docker.internal:11436), so it cannot be wired into that file's
|
||||||
|
# depends_on/condition chain -- this only gives it its own status.
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "ollama list || exit 1"]
|
||||||
|
interval: 15s
|
||||||
|
timeout: 10s
|
||||||
|
retries: 5
|
||||||
|
start_period: 20s
|
||||||
12
omo/Dockerfile
Normal file
12
omo/Dockerfile
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
FROM node:22-slim
|
||||||
|
|
||||||
|
# Install oh-my-opencode (ships its own opencode binary with omo extensions)
|
||||||
|
RUN npm install -g oh-my-opencode
|
||||||
|
|
||||||
|
# Run non-interactive setup — all cloud providers disabled, bifrost configured via opencode.json
|
||||||
|
RUN oh-my-opencode install --no-tui --claude=no --openai=no --gemini=no --copilot=no
|
||||||
|
|
||||||
|
# Config is mounted at runtime via volume
|
||||||
|
WORKDIR /workspace
|
||||||
|
|
||||||
|
ENTRYPOINT ["oh-my-opencode"]
|
||||||
17
omo/docker-compose.yml
Normal file
17
omo/docker-compose.yml
Normal file
@@ -0,0 +1,17 @@
|
|||||||
|
services:
|
||||||
|
omo:
|
||||||
|
build: .
|
||||||
|
container_name: omo
|
||||||
|
volumes:
|
||||||
|
- /home/alvis:/workspace
|
||||||
|
- ./opencode.json:/root/.config/opencode/opencode.json:ro
|
||||||
|
entrypoint: ["sleep", "infinity"]
|
||||||
|
stdin_open: true
|
||||||
|
tty: true
|
||||||
|
networks:
|
||||||
|
- adolf_default
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
|
networks:
|
||||||
|
adolf_default:
|
||||||
|
external: true
|
||||||
28
omo/opencode.json
Normal file
28
omo/opencode.json
Normal file
@@ -0,0 +1,28 @@
|
|||||||
|
{
|
||||||
|
"$schema": "https://opencode.ai/config.json",
|
||||||
|
"provider": {
|
||||||
|
"bifrost": {
|
||||||
|
"npm": "@ai-sdk/openai-compatible",
|
||||||
|
"name": "Bifrost",
|
||||||
|
"options": {
|
||||||
|
"baseURL": "http://bifrost:8080/v1",
|
||||||
|
"apiKey": "dummy"
|
||||||
|
},
|
||||||
|
"models": {
|
||||||
|
"ollama/qwen3:8b": {
|
||||||
|
"name": "qwen3:8b"
|
||||||
|
},
|
||||||
|
"ollama/qwen3:4b": {
|
||||||
|
"name": "qwen3:4b"
|
||||||
|
},
|
||||||
|
"ollama/qwen2.5:1.5b": {
|
||||||
|
"name": "qwen2.5:1.5b"
|
||||||
|
},
|
||||||
|
"ollama/gemma3:4b": {
|
||||||
|
"name": "gemma3:4b"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"model": "bifrost/ollama/qwen3:8b"
|
||||||
|
}
|
||||||
11
openai/adolf-llm/Dockerfile
Normal file
11
openai/adolf-llm/Dockerfile
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
FROM node:22-slim
|
||||||
|
|
||||||
|
RUN npm install -g @moonshot-ai/kimi-code
|
||||||
|
|
||||||
|
WORKDIR /workspace
|
||||||
|
|
||||||
|
COPY server.js /app/server.js
|
||||||
|
|
||||||
|
EXPOSE 8010
|
||||||
|
|
||||||
|
ENTRYPOINT ["node", "/app/server.js"]
|
||||||
777
openai/adolf-llm/server.js
Normal file
777
openai/adolf-llm/server.js
Normal file
@@ -0,0 +1,777 @@
|
|||||||
|
const http = require('http');
|
||||||
|
const fs = require('fs');
|
||||||
|
const path = require('path');
|
||||||
|
const crypto = require('crypto');
|
||||||
|
const { spawn } = require('child_process');
|
||||||
|
|
||||||
|
const PORT = 8010;
|
||||||
|
const MODEL_ID = 'adolf';
|
||||||
|
const TIMEOUT_MS = 15 * 60 * 1000;
|
||||||
|
|
||||||
|
const WORKSPACE = '/workspace';
|
||||||
|
const CONV_ROOT = path.join(WORKSPACE, 'conversations');
|
||||||
|
const STATE_DIR = path.join(WORKSPACE, '.adolf-llm');
|
||||||
|
const MAP_FILE = path.join(STATE_DIR, 'sessions.json');
|
||||||
|
const MAX_ENTRIES = 1000; // prune oldest beyond this
|
||||||
|
|
||||||
|
fs.mkdirSync(CONV_ROOT, { recursive: true });
|
||||||
|
fs.mkdirSync(STATE_DIR, { recursive: true });
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Shared MCP layer (Gate 1). Kimi Code CLI has NO `--mcp-config-file` flag and
|
||||||
|
// no `kimi mcp` subcommand; it auto-discovers a project-root `.mcp.json` by
|
||||||
|
// walking up from its cwd to the nearest `.git` (falling back to cwd itself
|
||||||
|
// when none is found). So we drop a `.mcp.json` into each session's working
|
||||||
|
// directory before spawning kimi.
|
||||||
|
//
|
||||||
|
// Single source of truth: `/shared-mcp.json` (mounted read-only from the repo
|
||||||
|
// root's `shared-mcp.json`, the same file P6 wires into OpenClaw's own
|
||||||
|
// `mcp.servers` registry). Adding a server is then a one-file change — no
|
||||||
|
// server list is hardcoded here anymore.
|
||||||
|
//
|
||||||
|
// Gate-1 transport finding (P5, verified by decompiling the installed
|
||||||
|
// @moonshot-ai/kimi-code package, packages/agent-core/src/config/schema.ts's
|
||||||
|
// McpServerConfigSchema): Kimi's own field name for remote MCP servers is
|
||||||
|
// `transport` (literal "stdio" | "http" | "sse"), not `type`. When `transport`
|
||||||
|
// is omitted, Kimi's config preprocessor infers it from shape: `command` ->
|
||||||
|
// "stdio", `url` -> "http" (never "sse" — sse requires an explicit
|
||||||
|
// `transport: "sse"`). It does NOT recognize a `type` key at all; unknown keys
|
||||||
|
// are silently stripped by the (non-strict) zod schema.
|
||||||
|
// OpenClaw's own canonical `mcp.servers` schema (docs/gateway/
|
||||||
|
// configuration-reference.md) uses different literals for the same
|
||||||
|
// transport: `transport: "streamable-http"` or `"sse"`, with `type: "http"`
|
||||||
|
// documented as a *CLI-native alias* that `openclaw mcp set` / `openclaw
|
||||||
|
// doctor --fix` normalize into canonical `transport: "streamable-http"`.
|
||||||
|
// So the two consumers disagree on the literal value for HTTP streaming
|
||||||
|
// ("http" vs "streamable-http") under the same field name `transport` --
|
||||||
|
// writing `transport` explicitly in shared-mcp.json would satisfy at most one
|
||||||
|
// side. `type: "http"` is the one shape both sides tolerate today: Kimi
|
||||||
|
// ignores the unrecognized `type` key and correctly infers transport "http"
|
||||||
|
// from the `url` field alone; OpenClaw recognizes `type` as its documented
|
||||||
|
// alias and normalizes it on its own terms (P6 concern, not touched here).
|
||||||
|
// Hence shared-mcp.json intentionally keeps `"type": "http"` for both cognee
|
||||||
|
// and openclaw-tools rather than switching to `transport`.
|
||||||
|
let SHARED_MCP_SERVERS = {};
|
||||||
|
try {
|
||||||
|
const raw = fs.readFileSync('/shared-mcp.json', 'utf8');
|
||||||
|
SHARED_MCP_SERVERS = JSON.parse(raw).mcpServers || {};
|
||||||
|
} catch (err) {
|
||||||
|
console.error(`shared-mcp.json not loaded (${err.message}); sessions will get no shared MCP servers`);
|
||||||
|
}
|
||||||
|
|
||||||
|
function writeMcpConfig(dir) {
|
||||||
|
const cfg = { mcpServers: SHARED_MCP_SERVERS };
|
||||||
|
fs.writeFileSync(path.join(dir, '.mcp.json'), JSON.stringify(cfg, null, 2));
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Memory lives at the OpenClaw layer, not here (P8). The Adolf gateway loads
|
||||||
|
// the `cognee-memory` OpenClaw plugin, which owns all memory touchpoints:
|
||||||
|
// - before_prompt_build => LLM-free cognee graph recall, injected into the
|
||||||
|
// prompt this wrapper then receives from OpenClaw.
|
||||||
|
// - agent_end => raw `add` of the turn to cognee.
|
||||||
|
// - background sweep => async `cognify` on cognee-llm (Kimi).
|
||||||
|
// - cognee_recall tool + cognee-mcp `recall` for on-demand deep queries.
|
||||||
|
// This wrapper is therefore a dumb model endpoint again: it must never call
|
||||||
|
// cognee itself. The former cogneeSearch/cogneeAdd stubs (and their call sites)
|
||||||
|
// were deleted when the plugin took over (P8).
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Persistent conversation -> Kimi session map.
|
||||||
|
// Primary key: `chat:<chat_id>` parsed from OpenClaw's "Conversation info" block
|
||||||
|
// (Gate 2). Fallback key: `hist:<sha256(prior history)>` when no chat_id is
|
||||||
|
// present (e.g. webchat surface / future OpenClaw layout change).
|
||||||
|
// value = { convId, sessionId, dir, ts }
|
||||||
|
let sessionMap = {};
|
||||||
|
try {
|
||||||
|
sessionMap = JSON.parse(fs.readFileSync(MAP_FILE, 'utf8'));
|
||||||
|
} catch {
|
||||||
|
sessionMap = {};
|
||||||
|
}
|
||||||
|
|
||||||
|
let writeQueue = Promise.resolve();
|
||||||
|
function persistMap() {
|
||||||
|
// prune to the MAX_ENTRIES most-recently-used before writing
|
||||||
|
const keys = Object.keys(sessionMap);
|
||||||
|
if (keys.length > MAX_ENTRIES) {
|
||||||
|
keys.sort((a, b) => (sessionMap[a].ts || 0) - (sessionMap[b].ts || 0));
|
||||||
|
for (const k of keys.slice(0, keys.length - MAX_ENTRIES)) delete sessionMap[k];
|
||||||
|
}
|
||||||
|
const snapshot = JSON.stringify(sessionMap);
|
||||||
|
writeQueue = writeQueue.then(
|
||||||
|
() => fs.promises.writeFile(MAP_FILE, snapshot),
|
||||||
|
() => fs.promises.writeFile(MAP_FILE, snapshot),
|
||||||
|
);
|
||||||
|
return writeQueue;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Message helpers.
|
||||||
|
function textOf(msg) {
|
||||||
|
const c = msg.content;
|
||||||
|
if (Array.isArray(c)) return c.map(p => (typeof p === 'string' ? p : p.text || '')).join('\n');
|
||||||
|
return c == null ? '' : String(c);
|
||||||
|
}
|
||||||
|
|
||||||
|
// only user/assistant turns define conversation identity (system is constant)
|
||||||
|
function convTurns(messages) {
|
||||||
|
return messages.filter(m => m.role === 'user' || m.role === 'assistant');
|
||||||
|
}
|
||||||
|
|
||||||
|
function historyKey(turns) {
|
||||||
|
const norm = turns.map(m => ({ role: m.role, text: textOf(m).trim() }));
|
||||||
|
return crypto.createHash('sha256').update(JSON.stringify(norm)).digest('hex');
|
||||||
|
}
|
||||||
|
|
||||||
|
function renderTranscript(turns) {
|
||||||
|
return turns
|
||||||
|
.map(m => `${m.role === 'assistant' ? 'Assistant' : 'User'}: ${textOf(m)}`)
|
||||||
|
.join('\n\n');
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Gate 2 — parse the stable chat_id out of OpenClaw's untrusted-metadata block.
|
||||||
|
// OpenClaw injects, into the *user-role* content, a block that looks like:
|
||||||
|
// Conversation info (untrusted metadata):
|
||||||
|
// ```json
|
||||||
|
// { "chat_id": "matrix:!room:server", "message_id": "...", ... }
|
||||||
|
// ```
|
||||||
|
// We grep on the label string (never a fixed line offset) then pull the first
|
||||||
|
// balanced JSON object after it and read chat_id. Robust to edited/truncated
|
||||||
|
// history, which is exactly why it beats a history hash for the common case.
|
||||||
|
const CONV_INFO_LABEL = 'Conversation info (untrusted metadata):';
|
||||||
|
|
||||||
|
function extractBalancedJson(str, from) {
|
||||||
|
const start = str.indexOf('{', from);
|
||||||
|
if (start === -1) return null;
|
||||||
|
let depth = 0;
|
||||||
|
let inStr = false;
|
||||||
|
let esc = false;
|
||||||
|
for (let i = start; i < str.length; i++) {
|
||||||
|
const ch = str[i];
|
||||||
|
if (inStr) {
|
||||||
|
if (esc) esc = false;
|
||||||
|
else if (ch === '\\') esc = true;
|
||||||
|
else if (ch === '"') inStr = false;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (ch === '"') inStr = true;
|
||||||
|
else if (ch === '{') depth++;
|
||||||
|
else if (ch === '}') {
|
||||||
|
depth--;
|
||||||
|
if (depth === 0) return str.slice(start, i + 1);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
function extractChatId(userMsg) {
|
||||||
|
const text = textOf(userMsg);
|
||||||
|
const at = text.indexOf(CONV_INFO_LABEL);
|
||||||
|
if (at === -1) return null;
|
||||||
|
const jsonStr = extractBalancedJson(text, at + CONV_INFO_LABEL.length);
|
||||||
|
if (!jsonStr) return null;
|
||||||
|
try {
|
||||||
|
const obj = JSON.parse(jsonStr);
|
||||||
|
const id = obj.chat_id;
|
||||||
|
return typeof id === 'string' && id ? id : null;
|
||||||
|
} catch {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Gate 3 — media. Persist inbound image parts into the session dir and return
|
||||||
|
// relative path references; the CLI autonomously calls its built-in
|
||||||
|
// ReadMediaFile tool on referenced paths (no flag/placeholder syntax needed).
|
||||||
|
const MIME_EXT = {
|
||||||
|
'image/png': 'png',
|
||||||
|
'image/jpeg': 'jpg',
|
||||||
|
'image/jpg': 'jpg',
|
||||||
|
'image/webp': 'webp',
|
||||||
|
'image/gif': 'gif',
|
||||||
|
'image/bmp': 'bmp',
|
||||||
|
'image/heic': 'heic',
|
||||||
|
'image/heif': 'heif',
|
||||||
|
};
|
||||||
|
|
||||||
|
function extFromMime(mime) {
|
||||||
|
return MIME_EXT[(mime || '').toLowerCase()] || 'img';
|
||||||
|
}
|
||||||
|
|
||||||
|
// Persist one image_url part; returns "./img_N.ext" or null if it couldn't.
|
||||||
|
async function persistImage(url, dir, n) {
|
||||||
|
if (typeof url !== 'string' || !url) return null;
|
||||||
|
if (url.startsWith('data:')) {
|
||||||
|
const m = /^data:([^;,]+)?(;base64)?,(.*)$/s.exec(url);
|
||||||
|
if (!m) return null;
|
||||||
|
const mime = m[1] || 'application/octet-stream';
|
||||||
|
const isB64 = !!m[2];
|
||||||
|
const ext = extFromMime(mime);
|
||||||
|
const name = `img_${n}.${ext}`;
|
||||||
|
const buf = isB64
|
||||||
|
? Buffer.from(m[3], 'base64')
|
||||||
|
: Buffer.from(decodeURIComponent(m[3]), 'utf8');
|
||||||
|
fs.writeFileSync(path.join(dir, name), buf);
|
||||||
|
return `./${name}`;
|
||||||
|
}
|
||||||
|
// Remote URL: best-effort fetch so the CLI gets a local path to ReadMediaFile.
|
||||||
|
// On any failure fall back to handing the raw URL to the model as text.
|
||||||
|
if (/^https?:\/\//i.test(url)) {
|
||||||
|
try {
|
||||||
|
const resp = await fetch(url);
|
||||||
|
if (!resp.ok) return url;
|
||||||
|
const mime = resp.headers.get('content-type') || '';
|
||||||
|
const ext = extFromMime(mime.split(';')[0].trim());
|
||||||
|
const name = `img_${n}.${ext}`;
|
||||||
|
const buf = Buffer.from(await resp.arrayBuffer());
|
||||||
|
fs.writeFileSync(path.join(dir, name), buf);
|
||||||
|
return `./${name}`;
|
||||||
|
} catch {
|
||||||
|
return url;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Build the prompt for the current user turn: join text parts, persist any
|
||||||
|
// image parts, append path references.
|
||||||
|
async function buildPrompt(userMsg, dir) {
|
||||||
|
const content = userMsg.content;
|
||||||
|
const textParts = [];
|
||||||
|
const imageRefs = [];
|
||||||
|
|
||||||
|
if (Array.isArray(content)) {
|
||||||
|
let n = 0;
|
||||||
|
for (const part of content) {
|
||||||
|
if (typeof part === 'string') {
|
||||||
|
textParts.push(part);
|
||||||
|
} else if (part && (part.type === 'text' || typeof part.text === 'string')) {
|
||||||
|
textParts.push(part.text || '');
|
||||||
|
} else if (part && part.type === 'image_url' && part.image_url && part.image_url.url) {
|
||||||
|
n++;
|
||||||
|
const ref = await persistImage(part.image_url.url, dir, n);
|
||||||
|
if (ref) imageRefs.push(ref);
|
||||||
|
} else if (part && part.type === 'input_image' && (part.image_url || part.url)) {
|
||||||
|
n++;
|
||||||
|
const u = typeof part.image_url === 'string' ? part.image_url : part.url;
|
||||||
|
const ref = await persistImage(u, dir, n);
|
||||||
|
if (ref) imageRefs.push(ref);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
textParts.push(content == null ? '' : String(content));
|
||||||
|
}
|
||||||
|
|
||||||
|
let prompt = textParts.join('\n');
|
||||||
|
if (imageRefs.length) {
|
||||||
|
prompt += '\n\n' + imageRefs.map(r => `See attached image: ${r}`).join('\n');
|
||||||
|
}
|
||||||
|
return prompt;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Kimi invocation with REAL streaming. Parses `--output-format stream-json`
|
||||||
|
// incrementally: each complete stdout line is one JSON object.
|
||||||
|
// {"role":"assistant","content":"..."} -> emit as a delta
|
||||||
|
// {"type":"session.resume_hint","session_id":"..."} -> capture session id
|
||||||
|
// onDelta(chunk) is called per assistant content fragment as it arrives.
|
||||||
|
// Resolves { text, sessionId } once the process closes.
|
||||||
|
function runKimi({ prompt, cwd, resumeId, onDelta, signal }) {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
if (signal?.aborted) { reject(new Error('aborted before start')); return; }
|
||||||
|
const args = [];
|
||||||
|
if (resumeId) args.push('-r', resumeId);
|
||||||
|
args.push('-p', prompt, '--output-format', 'stream-json');
|
||||||
|
|
||||||
|
const child = spawn('kimi', args, { cwd, timeout: TIMEOUT_MS });
|
||||||
|
|
||||||
|
let buf = '';
|
||||||
|
let stderr = '';
|
||||||
|
const parts = [];
|
||||||
|
let sessionId = null;
|
||||||
|
let settled = false;
|
||||||
|
let aborted = false;
|
||||||
|
|
||||||
|
// If the caller aborts (the gateway/client disconnected — e.g. its idle
|
||||||
|
// watchdog gave up), kill the child so it doesn't keep grinding an
|
||||||
|
// orphaned agent turn to completion, wasting Kimi quota and streaming into
|
||||||
|
// a dead socket. SIGTERM first, hard SIGKILL if it lingers.
|
||||||
|
const onAbort = () => {
|
||||||
|
aborted = true;
|
||||||
|
try { child.kill('SIGTERM'); } catch {}
|
||||||
|
setTimeout(() => { try { child.kill('SIGKILL'); } catch {} }, 3000).unref();
|
||||||
|
};
|
||||||
|
if (signal) signal.addEventListener('abort', onAbort, { once: true });
|
||||||
|
|
||||||
|
function handleLine(line) {
|
||||||
|
const t = line.trim();
|
||||||
|
if (!t) return;
|
||||||
|
let obj;
|
||||||
|
try { obj = JSON.parse(t); } catch { return; }
|
||||||
|
if (obj.role === 'assistant' && typeof obj.content === 'string' && obj.content) {
|
||||||
|
parts.push(obj.content);
|
||||||
|
if (onDelta) onDelta(obj.content);
|
||||||
|
}
|
||||||
|
if (obj.type === 'session.resume_hint' && obj.session_id) sessionId = obj.session_id;
|
||||||
|
}
|
||||||
|
|
||||||
|
child.stdout.on('data', d => {
|
||||||
|
buf += d;
|
||||||
|
let nl;
|
||||||
|
while ((nl = buf.indexOf('\n')) !== -1) {
|
||||||
|
const line = buf.slice(0, nl);
|
||||||
|
buf = buf.slice(nl + 1);
|
||||||
|
handleLine(line);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
child.stderr.on('data', d => { stderr += d; });
|
||||||
|
|
||||||
|
child.on('error', err => {
|
||||||
|
if (settled) return;
|
||||||
|
settled = true;
|
||||||
|
if (signal) signal.removeEventListener('abort', onAbort);
|
||||||
|
reject(err);
|
||||||
|
});
|
||||||
|
child.on('close', code => {
|
||||||
|
if (settled) return;
|
||||||
|
settled = true;
|
||||||
|
if (signal) signal.removeEventListener('abort', onAbort);
|
||||||
|
if (buf) handleLine(buf); // flush any trailing partial line
|
||||||
|
const text = parts.join('').trim();
|
||||||
|
if (aborted) {
|
||||||
|
reject(new Error('aborted: client disconnected'));
|
||||||
|
} else if (!text && code !== 0) {
|
||||||
|
reject(new Error(`kimi exited ${code}: ${stderr.slice(0, 2000)}`));
|
||||||
|
} else {
|
||||||
|
resolve({ text, sessionId });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// One turn: resolve session (chat_id primary, history-hash fallback), persist
|
||||||
|
// media + .mcp.json, run kimi (streaming through onDelta), record the mapping,
|
||||||
|
// and fire the async cognee ingest. Returns { text }.
|
||||||
|
async function handleTurn(messages, onDelta, signal) {
|
||||||
|
const turns = convTurns(messages);
|
||||||
|
let lastUserIdx = -1;
|
||||||
|
for (let i = turns.length - 1; i >= 0; i--) {
|
||||||
|
if (turns[i].role === 'user') { lastUserIdx = i; break; }
|
||||||
|
}
|
||||||
|
if (lastUserIdx === -1) throw new Error('no user message found');
|
||||||
|
|
||||||
|
const userMsg = turns[lastUserIdx];
|
||||||
|
const prior = turns.slice(0, lastUserIdx);
|
||||||
|
const chatId = extractChatId(userMsg);
|
||||||
|
|
||||||
|
// Resolve the session key + working dir + resume id.
|
||||||
|
let key;
|
||||||
|
let convId;
|
||||||
|
let dir;
|
||||||
|
let resumeId = null;
|
||||||
|
let reseed = false; // when true, prompt with the full transcript to rebuild continuity
|
||||||
|
|
||||||
|
if (chatId) {
|
||||||
|
key = `chat:${chatId}`;
|
||||||
|
const entry = sessionMap[key];
|
||||||
|
if (entry) {
|
||||||
|
convId = entry.convId;
|
||||||
|
dir = entry.dir;
|
||||||
|
resumeId = entry.sessionId;
|
||||||
|
} else {
|
||||||
|
convId = crypto.randomUUID();
|
||||||
|
dir = path.join(CONV_ROOT, convId);
|
||||||
|
// First time we see this chat_id but history exists (server restart / lost
|
||||||
|
// map): reseed the fresh session with the transcript so context survives.
|
||||||
|
reseed = prior.length > 0;
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
// Fallback: no chat_id -> forward history-hash mapping (kimi-agent style).
|
||||||
|
if (prior.length === 0) {
|
||||||
|
convId = crypto.randomUUID();
|
||||||
|
dir = path.join(CONV_ROOT, convId);
|
||||||
|
} else {
|
||||||
|
const entry = sessionMap[`hist:${historyKey(prior)}`];
|
||||||
|
if (entry) {
|
||||||
|
convId = entry.convId;
|
||||||
|
dir = entry.dir;
|
||||||
|
resumeId = entry.sessionId;
|
||||||
|
} else {
|
||||||
|
convId = crypto.randomUUID();
|
||||||
|
dir = path.join(CONV_ROOT, convId);
|
||||||
|
reseed = true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fs.mkdirSync(dir, { recursive: true });
|
||||||
|
writeMcpConfig(dir); // Gate 1: shared MCP via project-root .mcp.json
|
||||||
|
|
||||||
|
let prompt;
|
||||||
|
if (reseed) {
|
||||||
|
// Rebuild the whole conversation for a fresh session, plus current media.
|
||||||
|
const base = renderTranscript(turns.slice(0, lastUserIdx + 1));
|
||||||
|
const media = await buildPrompt(userMsg, dir);
|
||||||
|
// buildPrompt already includes the current user text; for reseed we want the
|
||||||
|
// transcript to carry it, so only append image refs.
|
||||||
|
prompt = base;
|
||||||
|
const extra = media.replace(textOf(userMsg), '').trim();
|
||||||
|
if (extra) prompt += `\n\n${extra}`;
|
||||||
|
} else {
|
||||||
|
prompt = await buildPrompt(userMsg, dir);
|
||||||
|
}
|
||||||
|
|
||||||
|
const { text, sessionId } = await runKimi({ prompt, cwd: dir, resumeId, onDelta, signal });
|
||||||
|
|
||||||
|
// Record the forward mapping.
|
||||||
|
const entry = { convId, sessionId: sessionId || resumeId, dir, ts: Date.now() };
|
||||||
|
if (chatId) {
|
||||||
|
sessionMap[`chat:${chatId}`] = entry;
|
||||||
|
} else {
|
||||||
|
const forward = turns.slice(0, lastUserIdx + 1).concat([{ role: 'assistant', content: text }]);
|
||||||
|
sessionMap[`hist:${historyKey(forward)}`] = entry;
|
||||||
|
}
|
||||||
|
persistMap();
|
||||||
|
|
||||||
|
return { text };
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Kimi quota readout (kb #62). GET /usage — the claude-usage analog for
|
||||||
|
// Adolf. LLM-free: hits Kimi's own managed-usage endpoint directly, never
|
||||||
|
// spawns `kimi`. Mirrors the parsing logic of the installed
|
||||||
|
// @moonshot-ai/kimi-code CLI itself (decompiled from dist/main.mjs's
|
||||||
|
// parseManagedUsagePayload/toUsageRow/limitLabel/resetHintFrom — same
|
||||||
|
// endpoint, same response shape) so bucket labels/derivations stay in sync
|
||||||
|
// with what `kimi` would show via its own /usage-equivalent.
|
||||||
|
//
|
||||||
|
// Token source: the CLI's own OAuth creds file, kept fresh by the running
|
||||||
|
// `kimi` process (adolf-llm-home volume). We ONLY read the file's live
|
||||||
|
// access_token and never refresh here. Kimi's OAuth rotates the refresh_token
|
||||||
|
// on every refresh (single-use), so an independent refresh from this route
|
||||||
|
// invalidates the refresh_token the CLI's file still holds -> the CLI's next
|
||||||
|
// refresh fails `invalid_grant` and wipes the whole login (kb#87: this was the
|
||||||
|
// recurring Adolf logout, incl. the 2026-07-17 06:15 wipe / task #86). Making
|
||||||
|
// the CLI the sole refresher removes that race.
|
||||||
|
//
|
||||||
|
// Cost of that trade, measured 2026-07-22: the access token's `expires_in` is
|
||||||
|
// 900s, so it is only valid for 15 minutes after the CLI last refreshed it —
|
||||||
|
// i.e. only within 15 minutes of an actual Adolf turn. Adolf is idle most of
|
||||||
|
// the day, so a bare read failed far more often than it succeeded, which made
|
||||||
|
// quota gating effectively blind. Rather than refresh here (see above: that
|
||||||
|
// wipes the login), /usage now falls back to the LAST GOOD reading, clearly
|
||||||
|
// labelled `stale` with `as_of` + `age_s` so callers can decide whether it is
|
||||||
|
// fresh enough. The cache is written on every success and persisted to the
|
||||||
|
// workspace volume so it survives a container restart. Auth is untouched:
|
||||||
|
// this route still only ever READS the creds file.
|
||||||
|
const KIMI_CREDS_PATH = '/root/.kimi-code/credentials/kimi-code.json';
|
||||||
|
const KIMI_USAGES_URL = 'https://api.kimi.com/coding/v1/usages';
|
||||||
|
const KIMI_USAGE_CACHE_PATH = '/workspace/.adolf-llm/usage-cache.json';
|
||||||
|
|
||||||
|
// Last successful /usage payload, kept in memory and mirrored to disk.
|
||||||
|
let kimiUsageCache = null;
|
||||||
|
|
||||||
|
function readKimiUsageCache() {
|
||||||
|
if (kimiUsageCache) return kimiUsageCache;
|
||||||
|
try {
|
||||||
|
const parsed = JSON.parse(fs.readFileSync(KIMI_USAGE_CACHE_PATH, 'utf8'));
|
||||||
|
if (parsed && parsed.payload && parsed.cached_at) kimiUsageCache = parsed;
|
||||||
|
} catch { /* no cache yet, or unreadable — treated as "no cache" */ }
|
||||||
|
return kimiUsageCache;
|
||||||
|
}
|
||||||
|
|
||||||
|
function writeKimiUsageCache(payload) {
|
||||||
|
kimiUsageCache = { payload, cached_at: new Date().toISOString() };
|
||||||
|
try {
|
||||||
|
fs.mkdirSync(path.dirname(KIMI_USAGE_CACHE_PATH), { recursive: true });
|
||||||
|
fs.writeFileSync(KIMI_USAGE_CACHE_PATH, JSON.stringify(kimiUsageCache));
|
||||||
|
} catch { /* cache is best-effort; an unwritable volume must not break /usage */ }
|
||||||
|
}
|
||||||
|
|
||||||
|
async function loadKimiCreds() {
|
||||||
|
const raw = await fs.promises.readFile(KIMI_CREDS_PATH, 'utf8');
|
||||||
|
return JSON.parse(raw);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Read the live access_token from the CLI's creds file. We deliberately do NOT
|
||||||
|
// refresh here (see the note above): the Kimi CLI is the sole refresher, so
|
||||||
|
// this route can never rotate the single-use refresh_token out from under it.
|
||||||
|
// A stale file token surfaces as an error -> /usage 502 -> "quota unavailable".
|
||||||
|
async function getKimiAccessToken() {
|
||||||
|
const creds = await loadKimiCreds();
|
||||||
|
const now = Math.floor(Date.now() / 1000);
|
||||||
|
if (creds.access_token && creds.expires_at && now < creds.expires_at - 30) {
|
||||||
|
return creds.access_token;
|
||||||
|
}
|
||||||
|
throw new Error('kimi access token stale (CLI refreshes on next use); quota temporarily unavailable');
|
||||||
|
}
|
||||||
|
|
||||||
|
async function fetchKimiUsagesRaw() {
|
||||||
|
const token = await getKimiAccessToken();
|
||||||
|
const res = await fetch(KIMI_USAGES_URL, { headers: { Authorization: `Bearer ${token}`, Accept: 'application/json' } });
|
||||||
|
if (!res.ok) {
|
||||||
|
const text = await res.text().catch(() => '');
|
||||||
|
throw new Error(`kimi /usages HTTP ${res.status}: ${text.slice(0, 500)}`);
|
||||||
|
}
|
||||||
|
return res.json();
|
||||||
|
}
|
||||||
|
|
||||||
|
function isRecord(v) { return v !== null && typeof v === 'object' && !Array.isArray(v); }
|
||||||
|
|
||||||
|
function toInt(v) {
|
||||||
|
if (typeof v === 'number') return Number.isFinite(v) ? Math.trunc(v) : null;
|
||||||
|
if (typeof v === 'string') { const n = Number(v); return Number.isFinite(n) ? Math.trunc(n) : null; }
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Port of the CLI's limitLabel(): prefer an explicit name/title/scope field,
|
||||||
|
// else derive "<N>h limit" / "<N>m limit" / "<N>d limit" from the window's
|
||||||
|
// duration+timeUnit.
|
||||||
|
function kimiLimitLabel(item, detail, window, idx) {
|
||||||
|
for (const key of ['name', 'title', 'scope']) {
|
||||||
|
const v = item[key] ?? detail[key];
|
||||||
|
if (typeof v === 'string' && v) return v;
|
||||||
|
}
|
||||||
|
const duration = toInt(window.duration ?? item.duration ?? detail.duration);
|
||||||
|
const rawUnit = window.timeUnit ?? item.timeUnit ?? detail.timeUnit;
|
||||||
|
const timeUnit = typeof rawUnit === 'string' ? rawUnit : '';
|
||||||
|
if (duration !== null) {
|
||||||
|
if (timeUnit.includes('MINUTE')) {
|
||||||
|
if (duration >= 60 && duration % 60 === 0) return `${duration / 60}h limit`;
|
||||||
|
return `${duration}m limit`;
|
||||||
|
}
|
||||||
|
if (timeUnit.includes('HOUR')) return `${duration}h limit`;
|
||||||
|
if (timeUnit.includes('DAY')) return `${duration}d limit`;
|
||||||
|
return `${duration}s limit`;
|
||||||
|
}
|
||||||
|
return `Limit #${idx + 1}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
function kimiResetIso(raw) {
|
||||||
|
for (const key of ['reset_at', 'resetAt', 'reset_time', 'resetTime']) {
|
||||||
|
const v = raw[key];
|
||||||
|
if (typeof v === 'string' && v) return v;
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Port of the CLI's toUsageRow(): used = raw.used, or limit-remaining when
|
||||||
|
// used is absent.
|
||||||
|
function kimiUsageRow(raw, defaultLabel) {
|
||||||
|
if (!isRecord(raw)) return null;
|
||||||
|
const limit = toInt(raw.limit);
|
||||||
|
let used = toInt(raw.used);
|
||||||
|
const remaining = toInt(raw.remaining);
|
||||||
|
if (used === null && remaining !== null && limit !== null) used = limit - remaining;
|
||||||
|
if (used === null && limit === null) return null;
|
||||||
|
const name = typeof raw.name === 'string' ? raw.name : (typeof raw.title === 'string' ? raw.title : defaultLabel);
|
||||||
|
return {
|
||||||
|
label: name,
|
||||||
|
used: used ?? 0,
|
||||||
|
limit: limit ?? 0,
|
||||||
|
remaining: remaining !== null ? remaining : (limit !== null && used !== null ? limit - used : null),
|
||||||
|
resets: kimiResetIso(raw),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function kimiRowOut(row) {
|
||||||
|
if (!row) return null;
|
||||||
|
const pct = row.limit > 0 ? Math.round((row.used / row.limit) * 100) : null;
|
||||||
|
return { pct, used: row.used, limit: row.limit, remaining: row.remaining, resets: row.resets };
|
||||||
|
}
|
||||||
|
|
||||||
|
// Normalize Kimi's /usages payload ({ usage, limits: [...] }) into the
|
||||||
|
// claude-usage-analog shape: weekly / window_5h / window_7d, each
|
||||||
|
// pct/used/limit/remaining/resets, plus a raw `limits` passthrough so no
|
||||||
|
// bucket is lost if label text ever drifts from what we match on below.
|
||||||
|
function normalizeKimiUsage(payload) {
|
||||||
|
const rec = isRecord(payload) ? payload : {};
|
||||||
|
const summaryRow = kimiUsageRow(rec.usage, 'Weekly limit');
|
||||||
|
const limitRows = [];
|
||||||
|
const rawLimits = Array.isArray(rec.limits) ? rec.limits : [];
|
||||||
|
rawLimits.forEach((item, idx) => {
|
||||||
|
if (!isRecord(item)) return;
|
||||||
|
const detail = isRecord(item.detail) ? item.detail : item;
|
||||||
|
const window = isRecord(item.window) ? item.window : {};
|
||||||
|
const label = kimiLimitLabel(item, detail, window, idx);
|
||||||
|
const row = kimiUsageRow(detail, label);
|
||||||
|
if (row) limitRows.push(row);
|
||||||
|
});
|
||||||
|
|
||||||
|
const findByLabel = re => limitRows.find(r => re.test(r.label));
|
||||||
|
const weekly = summaryRow || findByLabel(/week/i) || null;
|
||||||
|
const window5h = findByLabel(/^5\s*h(our)?\b|5h limit/i) || null;
|
||||||
|
const window7d = findByLabel(/^7\s*d(ay)?\b|7d limit/i) || null;
|
||||||
|
|
||||||
|
return {
|
||||||
|
timestamp: new Date().toISOString(),
|
||||||
|
weekly: kimiRowOut(weekly),
|
||||||
|
window_5h: kimiRowOut(window5h),
|
||||||
|
window_7d: kimiRowOut(window7d),
|
||||||
|
limits: limitRows.map(r => ({ label: r.label, ...kimiRowOut(r) })),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// OpenAI-compatible HTTP surface.
|
||||||
|
function completionBody(text) {
|
||||||
|
return {
|
||||||
|
id: `chatcmpl-${Date.now()}`,
|
||||||
|
object: 'chat.completion',
|
||||||
|
created: Math.floor(Date.now() / 1000),
|
||||||
|
model: MODEL_ID,
|
||||||
|
choices: [{
|
||||||
|
index: 0,
|
||||||
|
message: { role: 'assistant', content: text },
|
||||||
|
finish_reason: 'stop',
|
||||||
|
}],
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function sseChunk(id, created, delta, finishReason) {
|
||||||
|
return `data: ${JSON.stringify({
|
||||||
|
id, object: 'chat.completion.chunk', created, model: MODEL_ID,
|
||||||
|
choices: [{ index: 0, delta, finish_reason: finishReason }],
|
||||||
|
})}\n\n`;
|
||||||
|
}
|
||||||
|
|
||||||
|
const server = http.createServer((req, res) => {
|
||||||
|
if (req.method === 'GET' && req.url === '/v1/models') {
|
||||||
|
res.writeHead(200, { 'Content-Type': 'application/json' });
|
||||||
|
res.end(JSON.stringify({
|
||||||
|
object: 'list',
|
||||||
|
data: [{ id: MODEL_ID, object: 'model', owned_by: 'moonshot' }],
|
||||||
|
}));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (req.method === 'GET' && req.url === '/usage') {
|
||||||
|
(async () => {
|
||||||
|
try {
|
||||||
|
const raw = await fetchKimiUsagesRaw();
|
||||||
|
const out = normalizeKimiUsage(raw);
|
||||||
|
writeKimiUsageCache(out);
|
||||||
|
res.writeHead(200, { 'Content-Type': 'application/json' });
|
||||||
|
res.end(JSON.stringify({ ...out, stale: false }));
|
||||||
|
} catch (err) {
|
||||||
|
// Token stale (the common case when Adolf has been idle >15min) or Kimi
|
||||||
|
// unreachable. Serve the last good reading rather than nothing, labelled
|
||||||
|
// so a caller can reject it if it is too old to gate on.
|
||||||
|
const cached = readKimiUsageCache();
|
||||||
|
if (cached) {
|
||||||
|
const ageS = Math.max(0, Math.round((Date.now() - Date.parse(cached.cached_at)) / 1000));
|
||||||
|
res.writeHead(200, { 'Content-Type': 'application/json' });
|
||||||
|
res.end(JSON.stringify({
|
||||||
|
...cached.payload,
|
||||||
|
stale: true,
|
||||||
|
as_of: cached.cached_at,
|
||||||
|
age_s: ageS,
|
||||||
|
stale_reason: String(err.message || err),
|
||||||
|
}));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
res.writeHead(502, { 'Content-Type': 'application/json' });
|
||||||
|
res.end(JSON.stringify({ error: String(err.message || err) }));
|
||||||
|
}
|
||||||
|
})();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (req.method === 'POST' && req.url === '/v1/chat/completions') {
|
||||||
|
let body = '';
|
||||||
|
req.on('data', d => { body += d; });
|
||||||
|
req.on('end', async () => {
|
||||||
|
let parsed;
|
||||||
|
try {
|
||||||
|
parsed = JSON.parse(body);
|
||||||
|
} catch {
|
||||||
|
res.writeHead(400, { 'Content-Type': 'application/json' });
|
||||||
|
res.end(JSON.stringify({ error: 'invalid JSON body' }));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const messages = parsed.messages || [];
|
||||||
|
|
||||||
|
if (parsed.stream) {
|
||||||
|
// Real streaming: open SSE, emit role chunk, then forward kimi deltas.
|
||||||
|
res.writeHead(200, {
|
||||||
|
'Content-Type': 'text/event-stream',
|
||||||
|
'Cache-Control': 'no-cache',
|
||||||
|
Connection: 'keep-alive',
|
||||||
|
});
|
||||||
|
const id = `chatcmpl-${Date.now()}`;
|
||||||
|
const created = Math.floor(Date.now() / 1000);
|
||||||
|
|
||||||
|
// Heartbeat keepalive: OpenClaw's LLM idle watchdog aborts a turn on
|
||||||
|
// any >120s gap between SSE stream events (default timeoutSeconds),
|
||||||
|
// not on total run length. During long thinking/tool/MCP phases Kimi
|
||||||
|
// emits stream-json events we don't forward, so the SSE stream can go
|
||||||
|
// silent well past that window. Every write resets `lastWrite`; a 5s
|
||||||
|
// ticker emits an empty-content delta once 25s of silence elapses —
|
||||||
|
// still a stream event (resets the watchdog) but appends nothing
|
||||||
|
// visible to the rendered reply or cognee-persisted text.
|
||||||
|
let lastWrite = Date.now();
|
||||||
|
const write = (delta, finish) => {
|
||||||
|
if (res.writableEnded || res.destroyed) return;
|
||||||
|
res.write(sseChunk(id, created, delta, finish));
|
||||||
|
lastWrite = Date.now();
|
||||||
|
};
|
||||||
|
write({ role: 'assistant' }, null);
|
||||||
|
const hb = setInterval(() => {
|
||||||
|
if (Date.now() - lastWrite >= 25_000) write({ content: '' }, null);
|
||||||
|
}, 5_000);
|
||||||
|
|
||||||
|
// Propagate a client/gateway disconnect down to the kimi child so an
|
||||||
|
// abandoned turn (e.g. OpenClaw's idle watchdog gave up) is killed
|
||||||
|
// instead of finishing invisibly and burning quota. `done` guards
|
||||||
|
// against the normal res.end() 'close' also aborting.
|
||||||
|
const ac = new AbortController();
|
||||||
|
let done = false;
|
||||||
|
res.on('close', () => { if (!done) ac.abort(); });
|
||||||
|
|
||||||
|
try {
|
||||||
|
await handleTurn(messages, delta => write({ content: delta }, null), ac.signal);
|
||||||
|
done = true;
|
||||||
|
write({}, 'stop');
|
||||||
|
res.write('data: [DONE]\n\n');
|
||||||
|
} catch (err) {
|
||||||
|
done = true;
|
||||||
|
// Headers already sent — surface the error inside the stream (unless
|
||||||
|
// the socket is already gone, in which case there is nowhere to write).
|
||||||
|
if (!res.writableEnded && !res.destroyed) {
|
||||||
|
write({ content: `\n[error: ${String(err.message || err)}]` }, 'stop');
|
||||||
|
res.write('data: [DONE]\n\n');
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
clearInterval(hb);
|
||||||
|
if (!res.writableEnded) res.end();
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const ac = new AbortController();
|
||||||
|
let done = false;
|
||||||
|
res.on('close', () => { if (!done) ac.abort(); });
|
||||||
|
try {
|
||||||
|
const { text } = await handleTurn(messages, null, ac.signal);
|
||||||
|
done = true;
|
||||||
|
res.writeHead(200, { 'Content-Type': 'application/json' });
|
||||||
|
res.end(JSON.stringify(completionBody(text)));
|
||||||
|
} catch (err) {
|
||||||
|
done = true;
|
||||||
|
if (!res.writableEnded && !res.destroyed) {
|
||||||
|
res.writeHead(500, { 'Content-Type': 'application/json' });
|
||||||
|
res.end(JSON.stringify({ error: String(err.message || err) }));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
res.writeHead(404, { 'Content-Type': 'application/json' });
|
||||||
|
res.end(JSON.stringify({ error: 'not found' }));
|
||||||
|
});
|
||||||
|
|
||||||
|
server.listen(PORT, () => console.log(`adolf-llm wrapper listening on :${PORT}`));
|
||||||
20
openai/adolf-llm/service-block.yml
Normal file
20
openai/adolf-llm/service-block.yml
Normal file
@@ -0,0 +1,20 @@
|
|||||||
|
# adolf-llm — conversational Kimi-CLI wrapper (P2, :8010). OpenAI-compatible,
|
||||||
|
# model id "adolf". Real streaming, chat_id session mapping (1:1 kimi -r resume),
|
||||||
|
# media persistence, shared .mcp.json per session. cognee/MCP wiring is stubbed
|
||||||
|
# until P4/P5. Needs `kimi login` credentials seeded into its own home volume.
|
||||||
|
# Orchestrator: merge this `adolf-llm` service + the two named volumes into
|
||||||
|
# openai/docker-compose.yml (do NOT edit that file here).
|
||||||
|
services:
|
||||||
|
adolf-llm:
|
||||||
|
build: ./adolf-llm
|
||||||
|
container_name: adolf-llm
|
||||||
|
ports:
|
||||||
|
- "8010:8010"
|
||||||
|
volumes:
|
||||||
|
- adolf-llm-workspace:/workspace
|
||||||
|
- adolf-llm-home:/root/.kimi-code
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
adolf-llm-workspace:
|
||||||
|
adolf-llm-home:
|
||||||
492
openai/agent-registry.yaml
Normal file
492
openai/agent-registry.yaml
Normal file
@@ -0,0 +1,492 @@
|
|||||||
|
# Agent registry — agents are personas, not queues.
|
||||||
|
#
|
||||||
|
# Per DESIGN-a2a-agents.md v2.1 §2, §5, §5b (commit 714a9ca7), kanboard task
|
||||||
|
# #134 (A2A-2), sibling of #133's model-registry.yaml. An agent is
|
||||||
|
# (identity, Card, Policy, State): {persona/system prompt, memory bank(s),
|
||||||
|
# tool scope, trust class, preferred tier, current backbone}.
|
||||||
|
#
|
||||||
|
# THE ONE-FIELD-BACKBONE-SWAP MECHANISM: an agent's `backbone:` field is the
|
||||||
|
# ONLY thing here that names a concrete model/runtime. Everything a Card
|
||||||
|
# would otherwise duplicate from the model plane (tier the backbone actually
|
||||||
|
# delivers, cost_class, availability a(t), context window) is NOT stored
|
||||||
|
# statically on the agent — it is resolved at read time by dereferencing
|
||||||
|
# `backbone` into model-registry.yaml (via model_registry.py) or, for
|
||||||
|
# non-metered flat-subscription runtimes model-registry.yaml deliberately
|
||||||
|
# excludes (kb#133: "Claude Code ... belongs in the agent registry"), into
|
||||||
|
# the `runtimes:` section below. See agent_registry.py:effective_card().
|
||||||
|
# This is what makes "switching a backbone is one field" true rather than
|
||||||
|
# aspirational: edit `backbone: kimi` -> `backbone: local-small` and the
|
||||||
|
# derived tier/cost/a(t) change with it, with nothing else to keep in sync.
|
||||||
|
#
|
||||||
|
# `preferred_tier` is a POLICY field (what this persona asks for), distinct
|
||||||
|
# from the backbone's actual delivered tier (a FACT, resolved above) — they
|
||||||
|
# usually agree but don't have to (e.g. a large-preferring agent temporarily
|
||||||
|
# pinned to a small backbone during a quota outage).
|
||||||
|
#
|
||||||
|
# v2.1: personas + Cards live in git, deployed to runtimes — never
|
||||||
|
# live-edited in volumes (kb#156). Several agents below still point at a
|
||||||
|
# live-volume persona (adolf's SOUL.md) because #156 (git-deploy pipeline)
|
||||||
|
# hasn't landed yet; this registry records that fact, it doesn't fix it.
|
||||||
|
#
|
||||||
|
# Trust class is on every Card (§5) and is what kb#140 (capability routing)
|
||||||
|
# and kb#147 (grant enforcement / vault access) consume — see
|
||||||
|
# routing_consumption: at the bottom. Several Cards below (torgash,
|
||||||
|
# researcher, elizaveta's KB identity) describe TARGET state for agents/
|
||||||
|
# accounts that don't exist yet; each is flagged `status: target` /
|
||||||
|
# `note:` rather than silently implying they're live. Building them is
|
||||||
|
# explicitly out of kb#134's scope (registry only).
|
||||||
|
#
|
||||||
|
# Read with agent_registry.py (same directory): load_registry(),
|
||||||
|
# get_agent(), effective_card(), trust_rank().
|
||||||
|
|
||||||
|
schema_version: 1
|
||||||
|
|
||||||
|
# ── trust classes (§5) ──────────────────────────────────────────────────
|
||||||
|
# "human > trusted > sandboxed > untrusted". Numeric rank lets routing/grant
|
||||||
|
# code do `>=` comparisons instead of string-matching an ordered list.
|
||||||
|
#
|
||||||
|
# default_budget_usd/budget_duration (kb#147): defense-in-depth defaults fed
|
||||||
|
# into each agent's LiteLLM virtual-key spec (agent_registry.py:
|
||||||
|
# litellm_key_spec()) when the agent doesn't override them. Moot for money
|
||||||
|
# TODAY (§3a: no metered API by default, only free local-small is reachable
|
||||||
|
# without opt-in) but real once anything metered is opted into a tier pool —
|
||||||
|
# a budget cap should already exist rather than being bolted on later.
|
||||||
|
trust_classes:
|
||||||
|
human: { rank: 3, note: "vault access yes; not MCP-tool-scoped, IS the reasoning" }
|
||||||
|
trusted: { rank: 2, note: "vault access yes (DECIDED, kb#147); outward actions per ask-first rules", default_budget_usd: 50.0, budget_duration: "30d" }
|
||||||
|
sandboxed: { rank: 1, note: "no vault, no outward sends; scoped MCP allowlist; KB access project-scoped", default_budget_usd: 5.0, budget_duration: "30d" }
|
||||||
|
untrusted: { rank: 0, note: "anything ingesting the open web; its OUTPUTS are tainted, not just its access restricted", default_budget_usd: 0.0, budget_duration: "30d" }
|
||||||
|
|
||||||
|
# ── runtimes ─────────────────────────────────────────────────────────────
|
||||||
|
# Backbones deliberately OUTSIDE model-registry.yaml's `models:` list.
|
||||||
|
# kb#133 excluded Claude Code explicitly: "a flat-subscription runtime, not
|
||||||
|
# a metered API deployment — it belongs in the agent registry (#134), not
|
||||||
|
# here." This is that home. Same lookup contract as a model-registry entry
|
||||||
|
# (id, tier, cost_class, lifecycle, context_tokens) so agent_registry.py's
|
||||||
|
# backbone resolver can treat the two sources uniformly.
|
||||||
|
runtimes:
|
||||||
|
- id: claude-code-cli
|
||||||
|
role: "Claude Code CLI — flat Anthropic subscription runtime (this session's own kind)"
|
||||||
|
tier: large
|
||||||
|
context_tokens: 200000 # current Sonnet/Opus family context window; re-verify on model upgrades
|
||||||
|
cost_class: subscription
|
||||||
|
lifecycle: always-on # gated by Claude Code's own usage windows, not GPU/quota probes
|
||||||
|
quota_probe: "claude-usage (kanboard/bin/claude-usage); orchestration/quota-gating rules documented in kanboard/CLAUDE.md"
|
||||||
|
metered: false
|
||||||
|
opt_in_required: false
|
||||||
|
|
||||||
|
# ── agents ───────────────────────────────────────────────────────────────
|
||||||
|
agents:
|
||||||
|
|
||||||
|
# ── Adolf — proactive auditor / personal assistant ──────────────────────
|
||||||
|
- id: adolf
|
||||||
|
capabilities: [matrix-chat, task-triage, proactive-monitoring, memory-recall, cron-scheduling, browser]
|
||||||
|
persona:
|
||||||
|
role: "proactive auditor / personal assistant, talks to alvis and elizaveta over Matrix"
|
||||||
|
trivial: false
|
||||||
|
prompt_source:
|
||||||
|
current: "container adolf:/home/node/.openclaw/workspace/SOUL.md — lives in the adolf-state Docker VOLUME, live-editable, zero git history. This is the kb#156 migration-debt state; recorded here per kb#134's brief, not migrated."
|
||||||
|
companion_files: ["AGENTS.md", "IDENTITY.md", "TOOLS.md", "USER.md", "HEARTBEAT.md"] # same workspace, same volume, same debt
|
||||||
|
target: "git-controlled path once kb#156 lands (e.g. openai/personas/adolf/SOUL.md), deployed read-only into the volume"
|
||||||
|
trust_class: trusted
|
||||||
|
preferred_tier: large
|
||||||
|
backbone: kimi # resolves via model-registry.yaml (adolf-llm Kimi-CLI wrapper endpoint). Swap this ONE field to rebackbone Adolf.
|
||||||
|
tool_allowlist:
|
||||||
|
mcp_servers: [hindsight, openclaw-tools, kanboard, marketplace, agap]
|
||||||
|
gateway_tools: [cron, nodes, browser] # openclaw.json gateway.tools.allow, live 2026-07-21
|
||||||
|
vault_access: true # trusted-only per §5 DECIDED; reaches vw_* via the `agap` MCP server
|
||||||
|
# kb#144 (A2A-12): PER-TOOL scoping, TWO layers — 2026-07-22, second
|
||||||
|
# pass after a live-verified miss on the first. Each server below
|
||||||
|
# keeps its own justification comment in its config; this is the
|
||||||
|
# registry's copy of the same lists (source of truth this side —
|
||||||
|
# validate_capability_grants.py cross-checks against BOTH).
|
||||||
|
#
|
||||||
|
# Layer 1 — OpenClaw's own mcp.servers.*.toolFilter.include in
|
||||||
|
# adolf/openclaw.json, applied when OpenClaw (the `adolf` container)
|
||||||
|
# builds ITS OWN tool bundle. Landed first pass, verified schema-valid
|
||||||
|
# via `openclaw config validate` / `openclaw mcp probe`. Real, correct
|
||||||
|
# for OpenClaw's own client — but NOT what determines the model's
|
||||||
|
# actual per-turn context on Adolf's kimi backbone.
|
||||||
|
#
|
||||||
|
# Layer 2 — shared-mcp.json's per-server `enabledTools`, which
|
||||||
|
# adolf-llm/server.js's writeMcpConfig() seeds into each Kimi CLI
|
||||||
|
# session's project-root .mcp.json (Gate 1). THIS is the layer that
|
||||||
|
# actually reaches the model: Kimi CLI auto-discovers that file, not
|
||||||
|
# OpenClaw's config, and applies its own McpServerCommonFields.
|
||||||
|
# enabledTools/disabledTools via computeEnabledNames (an allowlist
|
||||||
|
# when only enabledTools is set — confirmed by decompiling the
|
||||||
|
# installed @moonshot-ai/kimi-code package's dist/main.mjs, both
|
||||||
|
# copies of the function, packages/agent-core{,-v2}/src/agent/mcp/
|
||||||
|
# connection-manager.ts). The first kb#144 pass got this backwards —
|
||||||
|
# see the release comment on kb#144 for the exact wire.jsonl proof
|
||||||
|
# (tool counts unchanged post-restart) that caught it: Layer 1 alone
|
||||||
|
# is invisible to Kimi.
|
||||||
|
#
|
||||||
|
# Counts (same tool lists both layers, confirmed identical by
|
||||||
|
# validate_capability_grants.py, exit 0):
|
||||||
|
# agap 32->28 (includes kb#95 wiki_* and kb#170 todoist_capture_idea),
|
||||||
|
# hindsight 29->9, kanboard 23->14, marketplace 13->7 (now in shared-
|
||||||
|
# mcp.json, reaches Kimi), openclaw-tools 5->5 (already minimal).
|
||||||
|
# Reachable-by-Kimi total (2026-07-26): 9+14+7+5+28=63 tools.
|
||||||
|
# Previous total was 52 (excluding marketplace, pre-shared-mcp.json);
|
||||||
|
# byte-measure against each server's real tools/list JSON schemas:
|
||||||
|
# est. ~9K tokens/turn. Estimate pending the real wire.jsonl number,
|
||||||
|
# which needs the adolf-llm container restart alvis owns
|
||||||
|
# (shared-mcp.json is bind-mounted read-only but adolf-llm's
|
||||||
|
# server.js caches its content at process start, so editing the file
|
||||||
|
# alone does not take effect — see capability_grant_status below for
|
||||||
|
# the confirm-post-restart command).
|
||||||
|
#
|
||||||
|
# kb#95 (2026-07-23): added wiki_search/wiki_read/wiki_edit (family
|
||||||
|
# MediaWiki / РодоВики, family.alogins.net) to agap-mcp and to both
|
||||||
|
# layers' agap allowlist below — Adolf's persona domain (relatives,
|
||||||
|
# dates, events), same reasoning as HA/Zabbix/Todoist above. This ages
|
||||||
|
# the counts comment above (agap 32->24, total 52) by +3/+3; not
|
||||||
|
# recomputed here since it needs the same live wire.jsonl proof kb#144
|
||||||
|
# used and this task does not touch the running containers (see
|
||||||
|
# shared_mcp_kimi_allowlist below for the exact confirm command).
|
||||||
|
#
|
||||||
|
# kb#170: added todoist_capture_idea (agap-mcp/src/capture.js —
|
||||||
|
# classify with local bge-m3 nearest-centroid, no LLM call, then
|
||||||
|
# create the labelled Todoist task in one round trip) to agap-mcp
|
||||||
|
# and to both layers' agap allowlist below. Ages the counts comment
|
||||||
|
# above by +1/+1 for the same reason as kb#95's note.
|
||||||
|
mcp_tool_filter:
|
||||||
|
hindsight: [recall, retain, reflect, list_memories, get_memory, update_memory, list_directives, create_directive, delete_directive]
|
||||||
|
kanboard: [kanboard_list_projects, kanboard_get_project, kanboard_list_tasks, kanboard_my_tasks, kanboard_get_task, kanboard_search_tasks, kanboard_list_users, kanboard_project_activity, kanboard_create_task, kanboard_update_task, kanboard_move_task, kanboard_change_task_status, kanboard_assign_task, kanboard_add_comment]
|
||||||
|
marketplace: [marketplace_find_best, marketplace_search, marketplace_get_product, marketplace_get_recommendations, marketplace_get_reviews, marketplace_compare_prices, marketplace_status]
|
||||||
|
agap: [vw_get_password, vw_get_item, vw_list_items, vw_create_login, vw_update_password, ha_get_state, ha_list_entities, ha_call_service, ha_get_history, zabbix_get_problems, zabbix_get_hosts, zabbix_get_items, zabbix_get_triggers, radicale_list_calendars, radicale_list_events, radicale_get_event, radicale_put_event, radicale_delete_event, radicale_move_event, todoist_list_tasks, todoist_list_projects, todoist_create_task, todoist_update_task, todoist_complete_task, todoist_capture_idea, wiki_search, wiki_read, wiki_edit]
|
||||||
|
openclaw-tools: null # no filter in openclaw.json — already minimal (5/5 kept)
|
||||||
|
note: >
|
||||||
|
"scoped core tools" per kb#134's brief, now REAL at both levels: this
|
||||||
|
IS openai/openclaw.json's live mcp.servers block (server selection)
|
||||||
|
plus its per-server toolFilter.include (tool selection, kb#144). Not
|
||||||
|
a narrower aspirational allowlist — validate_capability_grants.py
|
||||||
|
cross-checks both against the live config. LiteLLM virtual-key
|
||||||
|
budgets remain kb#147's separate job; this field is the input #147
|
||||||
|
consumes for MCP scope (litellm_key_spec() handles the model side).
|
||||||
|
capability_grant: # kb#147 — the enforcement input for LiteLLM + agap-mcp
|
||||||
|
litellm_key_alias: adolf
|
||||||
|
mcp_auth_token_env: AGAP_MCP_TOKEN_ADOLF # secret lives in Vaultwarden + this container's env, never in git; see agap-mcp/docker-compose.yml AGAP_MCP_AGENT_TOKENS
|
||||||
|
note: >
|
||||||
|
Reachable model tiers derived at read time from preferred_tier via
|
||||||
|
agent_registry.py:litellm_key_spec() — not duplicated here. Actual
|
||||||
|
virtual-key provisioning happens via openai/provision_litellm_keys.py
|
||||||
|
against the live LiteLLM proxy; NOT run automatically by this
|
||||||
|
registry (privileged action, requires the LiteLLM master key —
|
||||||
|
kb#147 handover step, see task comment).
|
||||||
|
memory:
|
||||||
|
banks:
|
||||||
|
- { id: adolf-alvis, role: private, interlocutor: alvis }
|
||||||
|
- { id: adolf-elizaveta, role: private, interlocutor: elizaveta }
|
||||||
|
- { id: adolf-shared, role: shared, interlocutor: household }
|
||||||
|
current_state: >
|
||||||
|
NOT split yet for the plugin's recall/retain hooks: a single live
|
||||||
|
bank "adolf" (525+ facts) serves every interlocutor today with no
|
||||||
|
per-human isolation — the exact defect kb#153 exists to fix
|
||||||
|
(depends on this registry existing first). The three banks above
|
||||||
|
are kb#153's target, not current fact for the hooks.
|
||||||
|
kb#169 (2026-07-26): the SEPARATE raw hindsight MCP tool surface
|
||||||
|
(mcp.servers.hindsight in adolf/openclaw.json + openai/shared-
|
||||||
|
mcp.json — recall/retain/reflect/etc. callable directly by the
|
||||||
|
model, bypassing #153's interlocutor-scoping entirely) has been
|
||||||
|
repointed from http://hindsight:8888/mcp/adolf/ (the unpartitioned
|
||||||
|
bank, still what the hooks use) to http://hindsight:8888/mcp/
|
||||||
|
adolf-shared/ (pre-existing, 0 facts). That surface can now only
|
||||||
|
ever touch the shared bank — never a private one, never the mixed
|
||||||
|
"adolf" bank — regardless of who's talking to Adolf.
|
||||||
|
kb_identity: { username: adolf, user_id: 3 }
|
||||||
|
availability_note: "a(t) inherited from backbone at read time (kimi: quota-gated, ~60msg/5h ~300/wk — see model-registry.yaml)"
|
||||||
|
|
||||||
|
# ── claude-coder — the Claude Code loop as an ordinary consumer ─────────
|
||||||
|
- id: claude-coder
|
||||||
|
capabilities: [coding, kanboard-dispatch, git, infra-ops, code-review]
|
||||||
|
persona:
|
||||||
|
role: "implementer — pulls complex coding tasks from the fabric (design §2 theorem 5, 'never special')"
|
||||||
|
trivial: false
|
||||||
|
prompt_source:
|
||||||
|
current: "git-native, layered: ~/.claude/CLAUDE.md (global user memory) + kanboard/CLAUDE.md (canonical kb orchestration ruleset) + per-repo CLAUDE.md files (e.g. agap_git/CLAUDE.md)"
|
||||||
|
note: "No single SOUL.md — persona is these CLAUDE.md conventions, already git-controlled. No kb#156 debt for this agent."
|
||||||
|
trust_class: trusted
|
||||||
|
preferred_tier: large
|
||||||
|
backbone: claude-code-cli # resolves via runtimes: above (kb#133 exclusion), NOT model-registry.yaml
|
||||||
|
tool_allowlist:
|
||||||
|
mcp_servers: [kanboard, agap]
|
||||||
|
native_tools: [Bash, Read, Edit, Write, Agent, WebFetch, WebSearch, git]
|
||||||
|
vault_access: true # trusted
|
||||||
|
note: >
|
||||||
|
Widest-scoped agent by design (this session's own tool surface) —
|
||||||
|
gated by ask-first rules on outward/destructive actions rather than
|
||||||
|
MCP allowlisting. Per kb#147, a real virtual-key budget still applies.
|
||||||
|
capability_grant:
|
||||||
|
litellm_key_alias: claude-coder
|
||||||
|
mcp_auth_token_env: AGAP_MCP_TOKEN_CLAUDE_CODER
|
||||||
|
note: "same mechanism as adolf's capability_grant above; see that note."
|
||||||
|
memory:
|
||||||
|
banks: []
|
||||||
|
model: "session (ephemeral, per invocation) + repo state (git history, CLAUDE.md files, kanboard task/comment history) — no persistent Hindsight bank"
|
||||||
|
kb_identity: { username: claude, user_id: 2 }
|
||||||
|
availability_note: "a(t) inherited from backbone at read time (claude-code-cli: always-on, gated by claude-usage windows)"
|
||||||
|
completion_convention: >
|
||||||
|
Verified-completion flow (DESIGN v2.1 §2, kb#159): when completing a task,
|
||||||
|
the worker/agent NEVER closes it — only moves it to Done (unverified
|
||||||
|
completion) and leaves it open. Closing is verification, done by someone
|
||||||
|
OTHER than the producer (the submitter, a human, or a reviewer-agent after
|
||||||
|
checking acceptance criteria). The fabric-keeper audits this: closed tasks
|
||||||
|
where the producer also closed them are flagged as kb#159 violations in
|
||||||
|
the daily digest.
|
||||||
|
|
||||||
|
# ── torgash — marketplace analyst (sandboxed) ───────────────────────────
|
||||||
|
- id: torgash
|
||||||
|
capabilities: [price-comparison, marketplace-search, cart-ops, product-recommendations]
|
||||||
|
persona:
|
||||||
|
role: "marketplace analyst — price comparison / shopping across Ozon, Yandex Market, etc."
|
||||||
|
trivial: false
|
||||||
|
prompt_source:
|
||||||
|
current: null
|
||||||
|
note: "NOT yet stood up as a running persona — this Card is target state per design §2's agent table and the A2A design-review plan. Building the runtime is out of kb#134's scope (registry only); flagging as follow-up work."
|
||||||
|
trust_class: sandboxed
|
||||||
|
preferred_tier: small
|
||||||
|
backbone: local-small
|
||||||
|
tool_allowlist:
|
||||||
|
mcp_servers: [marketplace]
|
||||||
|
vault_access: false # sandboxed — hard rule §5, no exceptions
|
||||||
|
outward_sends: false
|
||||||
|
note: "scoped to mcp__marketplace__* tools only; no gitea/ha/zabbix/radicale, no kanboard project outside its own."
|
||||||
|
capability_grant:
|
||||||
|
litellm_key_alias: torgash
|
||||||
|
mcp_auth_token_env: AGAP_MCP_TOKEN_TORGASH
|
||||||
|
note: >
|
||||||
|
Not provisioned yet (agent not built — see persona.prompt_source
|
||||||
|
above). litellm_key_spec('torgash') already resolves correctly
|
||||||
|
against the registry today (preferred_tier: small -> models=[local-
|
||||||
|
small's litellm_model_name] only, no large/paid-fallback) — verified
|
||||||
|
by kb#147's provision_litellm_keys.py --dry-run.
|
||||||
|
memory:
|
||||||
|
banks: [{ id: torgash, role: private }]
|
||||||
|
current_state: "bank not yet created — target state, same as the persona itself"
|
||||||
|
kb_identity:
|
||||||
|
username: null
|
||||||
|
note: "no Kanboard account provisioned yet; §5 calls for KB access project-scoped to a dedicated project once created — gap, not in kb#134's scope."
|
||||||
|
availability_note: "a(t) inherited from backbone at read time (local-small: always-on, VRAM-bound)"
|
||||||
|
|
||||||
|
# ── researcher — autonomous, sandboxed/untrusted-input loop ─────────────
|
||||||
|
- id: researcher
|
||||||
|
capabilities: [web-research, synthesis, low-priority-background-loop]
|
||||||
|
persona:
|
||||||
|
role: "autonomous researcher — low-priority self-submitting loop (design §6, §8)"
|
||||||
|
trivial: false
|
||||||
|
prompt_source: { current: null, note: "not yet built; target-state Card, same caveat as torgash." }
|
||||||
|
trust_class: sandboxed
|
||||||
|
trust_note: >
|
||||||
|
Ingests the open web, so its OUTPUTS are tainted regardless of its own
|
||||||
|
sandboxed access level (the untrusted-INPUT rule, §5). Promotion of
|
||||||
|
tainted output into a trusted agent's memory or into any action
|
||||||
|
requires a gate — initially a task to alvis's inbox.
|
||||||
|
preferred_tier: small
|
||||||
|
backbone: local-small # escalates to large via always-ask policy (§5) for synthesis — never silent retry-on-bigger-model
|
||||||
|
tool_allowlist:
|
||||||
|
mcp_servers: [] # target: a scoped web-search/fetch surface once built
|
||||||
|
native_tools: [WebSearch, WebFetch]
|
||||||
|
vault_access: false
|
||||||
|
outward_sends: false
|
||||||
|
capability_grant:
|
||||||
|
litellm_key_alias: researcher
|
||||||
|
mcp_auth_token_env: AGAP_MCP_TOKEN_RESEARCHER
|
||||||
|
note: "not provisioned yet (agent not built) — same verification status as torgash's capability_grant above."
|
||||||
|
memory:
|
||||||
|
banks: [{ id: researcher, role: private }]
|
||||||
|
current_state: "bank not yet created — target state"
|
||||||
|
kb_identity:
|
||||||
|
username: null
|
||||||
|
note: "needs its own KB project(s) per §5 ('researcher gets its own KB project(s)') — not yet created, gap outside kb#134's scope."
|
||||||
|
availability_note: "a(t) inherited from backbone at read time (local-small: always-on, VRAM-bound)"
|
||||||
|
|
||||||
|
# ── model-agents — trivial personas (from kb#133) ───────────────────────
|
||||||
|
# Design §2 table row 1: "LLM endpoint (kimi, gemma3:4b, ...) | persona:
|
||||||
|
# trivial (identity) | memory: none". These are direct-address targets
|
||||||
|
# (router mode target=agent-id) for a caller that wants THIS backbone
|
||||||
|
# specifically, bypassing any persona/tool-scope layer — distinct from
|
||||||
|
# e.g. `adolf`, which happens to use the `kimi` backbone today but adds a
|
||||||
|
# persona, memory, and tool scope on top. Only the two chat-capable
|
||||||
|
# models get an agent entry; bge-m3/tei-reranker are non-chat sidecars in
|
||||||
|
# model-registry.yaml, not addressable A2A targets.
|
||||||
|
- id: kimi-endpoint
|
||||||
|
capabilities: [raw-completion]
|
||||||
|
persona: { role: "trivial identity — the kimi backbone exposed directly, no persona layer", trivial: true, prompt_source: { current: null } }
|
||||||
|
trust_class: untrusted # a bare model endpoint makes no trust decisions itself; the caller's trust class governs what reaches it
|
||||||
|
preferred_tier: large
|
||||||
|
backbone: kimi
|
||||||
|
tool_allowlist: { mcp_servers: [], native_tools: [], vault_access: false }
|
||||||
|
memory: { banks: [], model: "none (trivial persona)" }
|
||||||
|
kb_identity: { username: null }
|
||||||
|
|
||||||
|
- id: local-small-endpoint
|
||||||
|
capabilities: [raw-completion]
|
||||||
|
persona: { role: "trivial identity — the local-small backbone exposed directly, no persona layer", trivial: true, prompt_source: { current: null } }
|
||||||
|
trust_class: untrusted
|
||||||
|
preferred_tier: small
|
||||||
|
backbone: local-small
|
||||||
|
tool_allowlist: { mcp_servers: [], native_tools: [], vault_access: false }
|
||||||
|
memory: { banks: [], model: "none (trivial persona)" }
|
||||||
|
kb_identity: { username: null }
|
||||||
|
|
||||||
|
# ── humans — first-class agents (§2, §5b) ───────────────────────────────
|
||||||
|
# "The human being an agent is not a metaphor": approval gates and
|
||||||
|
# decisions are ordinary tasks submitted to a human's inbox, which IS the
|
||||||
|
# Kanboard column/assignment he already processes. No `backbone` — humans
|
||||||
|
# ARE the reasoning, not a resolvable model.
|
||||||
|
- id: alvis
|
||||||
|
capabilities: [approval, decision, escalation-target]
|
||||||
|
persona: { role: "human — primary user/owner of Agap" }
|
||||||
|
trust_class: human
|
||||||
|
preferred_tier: null
|
||||||
|
backbone: null
|
||||||
|
tool_allowlist: null # n/a — outranks `trusted`, not MCP-scoped
|
||||||
|
memory:
|
||||||
|
banks:
|
||||||
|
- { id: adolf-alvis, role: private }
|
||||||
|
- { id: adolf-shared, role: shared }
|
||||||
|
current_state: "today's single unsplit 'adolf' bank mixes alvis + elizaveta content; split pending kb#153"
|
||||||
|
matrix_id: "@admin:mtx.alogins.net"
|
||||||
|
kb_identity:
|
||||||
|
username: admin
|
||||||
|
user_id: 1
|
||||||
|
note: "assumed == alvis (sole non-bot app-admin account); the Kanboard user record's email field is unpopulated so this can't be confirmed via API — verify by hand if it's ever ambiguous."
|
||||||
|
inbox: "tasks assigned to Kanboard user 'admin' (id 1) across projects — his approval/escalation inbox (design §2)"
|
||||||
|
availability: "a(t) = waking hours (informal, no fixed function yet); vacation mode sets a(t)=0 and parks his inbox per §5b"
|
||||||
|
|
||||||
|
- id: elizaveta
|
||||||
|
capabilities: [approval, decision, escalation-target]
|
||||||
|
persona: { role: "human — household member" }
|
||||||
|
trust_class: human
|
||||||
|
preferred_tier: null
|
||||||
|
backbone: null
|
||||||
|
tool_allowlist: null
|
||||||
|
memory:
|
||||||
|
banks:
|
||||||
|
- { id: adolf-elizaveta, role: private }
|
||||||
|
- { id: adolf-shared, role: shared }
|
||||||
|
current_state: "not yet split out of the single 'adolf' bank — kb#153, flagged urgent-ish there since she is on Adolf's Matrix allowlist TODAY with a shared, unpartitioned bank."
|
||||||
|
matrix_id: "@elizaveta:mtx.alogins.net"
|
||||||
|
kb_identity:
|
||||||
|
username: null
|
||||||
|
note: "no Kanboard account provisioned for her yet — her only inbox today is the Matrix DM channel (Adolf's dm.allowFrom), not a KB column. Gap / candidate follow-up task, out of kb#134's scope."
|
||||||
|
inbox: "none in Kanboard yet (see kb_identity note above); Matrix DM is the only channel today"
|
||||||
|
|
||||||
|
# ── memory bank policy (§5b) ─────────────────────────────────────────────
|
||||||
|
# Hard rules the hindsight-memory plugin's recall/retain hooks must enforce
|
||||||
|
# once kb#153 implements bank selection by interlocutor. Registry mirror of
|
||||||
|
# model-registry.yaml's gpu_residency_policy: the parameters a consumer
|
||||||
|
# reads, not the enforcement logic itself.
|
||||||
|
memory_bank_policy:
|
||||||
|
hard_rules:
|
||||||
|
- "content from one human's conversations must never surface to another human (correctness property, not preference)"
|
||||||
|
- "promotion private -> shared happens only by the owning human's explicit action or an approval task — never automatically"
|
||||||
|
- "recall is interlocutor-scoped: the hindsight-memory plugin selects the bank by interlocutor identity (Matrix sender)"
|
||||||
|
- "sandboxed agents (torgash, researcher) read at most the shared bank, never any private bank"
|
||||||
|
banks:
|
||||||
|
- { id: adolf-alvis, owner: alvis, role: private, status: target, note: "kb#153 not yet done" }
|
||||||
|
- { id: adolf-elizaveta, owner: elizaveta, role: private, status: target, note: "kb#153 not yet done" }
|
||||||
|
- { id: adolf-shared, owner: household, role: shared, status: target, note: "kb#153 not yet done" }
|
||||||
|
- { id: adolf, owner: null, role: legacy, status: live-today, note: "single unsplit bank every interlocutor currently reads/writes; superseded by the three rows above once kb#153 lands" }
|
||||||
|
- { id: torgash, owner: torgash, role: private, status: target, note: "agent not yet built" }
|
||||||
|
- { id: researcher, owner: researcher, role: private, status: target, note: "agent not yet built" }
|
||||||
|
|
||||||
|
# ── how downstream tasks consume this file ────────────────────────────────
|
||||||
|
# Documents the contract kb#140 (routing) and kb#147 (grant enforcement)
|
||||||
|
# build against, mirroring model-registry.yaml's `routing:` section.
|
||||||
|
routing_consumption:
|
||||||
|
trust_gate: >
|
||||||
|
kb#147: a task requiring vault access may target only an agent whose
|
||||||
|
trust_classes[...].rank >= trust_classes.trusted.rank (i.e. trusted or
|
||||||
|
human). Sandboxed/untrusted agents' tool_allowlist.vault_access is
|
||||||
|
always false by construction above — kb#147's job is to make that
|
||||||
|
provably true at the MCP/LiteLLM enforcement points, not just on paper.
|
||||||
|
capability_routing: >
|
||||||
|
kb#140: given a task's required capabilities + trust constraint, the
|
||||||
|
router filters agents by capabilities ⊆ agent.capabilities, trust rank
|
||||||
|
>= required, and current a(t) == 1 (resolved via agent_registry.py's
|
||||||
|
effective_card(), which dereferences `backbone` into model-registry.yaml
|
||||||
|
or `runtimes:` above) — no submitter-side hardcoded agent id needed.
|
||||||
|
|
||||||
|
# ── kb#147 (A2A-15) implementation status ────────────────────────────────
|
||||||
|
# What "enforced" means as of this task, per enforcement point (§5 lists
|
||||||
|
# three: agap-mcp vault tools, OpenClaw per-agent MCP scoping, LiteLLM
|
||||||
|
# virtual keys). Recorded here — not just in the KB task comment — because
|
||||||
|
# this file is the grants source of truth the design demands.
|
||||||
|
capability_grant_status:
|
||||||
|
agap_mcp_vault_gate: >
|
||||||
|
IMPLEMENTED, tested, NOT ACTIVATED. agap-mcp/src/trust-gate.js resolves
|
||||||
|
a bearer token -> agent id -> trust rank (reading THIS file, mounted
|
||||||
|
read-only) and agap-mcp/src/server.js gates every vw_* tool behind it.
|
||||||
|
Proven with two real, run-now test suites (no live container touched):
|
||||||
|
src/trust-gate.test.mjs (pure logic, 10/10) and
|
||||||
|
src/trust-gate-http.test.mjs (real HTTP request/response, 4/4) — the
|
||||||
|
latter shows a sandboxed-agent token, no token, and an unknown token all
|
||||||
|
get "vault access denied" while a trusted-agent token passes. OFF by
|
||||||
|
default (AGAP_MCP_ENFORCE_VAULT_TRUST=0 in docker-compose.yml) so this
|
||||||
|
change is zero-impact until an operator: (1) generates real per-agent
|
||||||
|
bearer tokens, stores them in Vaultwarden, wires them into
|
||||||
|
AGAP_MCP_AGENT_TOKENS, (2) sets AGAP_MCP_ENFORCE_VAULT_TRUST=1, (3)
|
||||||
|
restarts the agap-mcp container — deliberately not done by this task
|
||||||
|
(never restart the live agap-mcp service unattended).
|
||||||
|
litellm_virtual_keys: >
|
||||||
|
SPEC'D, NOT PROVISIONED. agent_registry.py:litellm_key_spec() computes
|
||||||
|
each agent's model allow-list (derived from preferred_tier x
|
||||||
|
model-registry.yaml routing.tiers, non-metered only unless opted in)
|
||||||
|
and a default budget from trust_classes[...].default_budget_usd.
|
||||||
|
openai/provision_litellm_keys.py turns that spec into LiteLLM
|
||||||
|
/key/generate calls. Verified with --dry-run (prints the exact payload
|
||||||
|
per agent, no network call) — actually creating keys needs
|
||||||
|
LITELLM_MASTER_KEY against the live proxy, a privileged write this task
|
||||||
|
does not perform unattended; see the kb#147 task comment for the exact
|
||||||
|
command to run once approved.
|
||||||
|
openclaw_mcp_allowlist: >
|
||||||
|
STRUCTURAL at both server AND tool level, cross-checked (kb#144 extended
|
||||||
|
this from server-only). adolf/openclaw.json's mcp.servers block is
|
||||||
|
adolf's real, live MCP surface (git-controlled): its server set matches
|
||||||
|
tool_allowlist.mcp_servers, and each server's toolFilter.include (added
|
||||||
|
kb#144 first pass) matches tool_allowlist.mcp_tool_filter — exactly.
|
||||||
|
openai/validate_capability_grants.py checks this automatically, read-only,
|
||||||
|
no live changes. Real and correct for OpenClaw's OWN MCP client surface —
|
||||||
|
but per the kb#144 first-pass release comment's wire.jsonl proof, this
|
||||||
|
layer alone does NOT reach the model on Adolf's kimi backbone (see
|
||||||
|
shared_mcp_kimi_allowlist below, the layer that does). NOT restarted by
|
||||||
|
this task for this file's change either way (openclaw.json is
|
||||||
|
bind-mounted read-only as the live config; `docker compose restart adolf`
|
||||||
|
is the activation step, alvis's call).
|
||||||
|
shared_mcp_kimi_allowlist: >
|
||||||
|
STRUCTURAL, cross-checked, NOT YET ACTIVATED — kb#144 SECOND pass
|
||||||
|
(2026-07-22), added after live verification (restart + one real turn,
|
||||||
|
wire.jsonl inspection) proved the first pass's openclaw.json-only fix
|
||||||
|
left Kimi's actual tool bundle unchanged (23/32/29/5, not 14/24/9/5).
|
||||||
|
Root cause: Kimi CLI auto-discovers a project-root `.mcp.json` that
|
||||||
|
adolf-llm/server.js's writeMcpConfig() seeds from openai/shared-mcp.json
|
||||||
|
— a completely separate config from openclaw.json, read by a separate
|
||||||
|
MCP client (Kimi CLI inside the adolf-llm container, not OpenClaw inside
|
||||||
|
the adolf container). shared-mcp.json now carries the same per-server
|
||||||
|
tool lists as `enabledTools` (Kimi's own allowlist field —
|
||||||
|
McpServerCommonFields.enabledTools, applied via computeEnabledNames;
|
||||||
|
confirmed by decompiling the installed @moonshot-ai/kimi-code package's
|
||||||
|
dist/main.mjs, both copies of the function/schema, no live container
|
||||||
|
touched). openai/validate_capability_grants.py now cross-checks THIS
|
||||||
|
file too (load_shared_mcp_enabled_tools), same exit-0-or-fail contract
|
||||||
|
as the openclaw.json check. marketplace is absent from shared-mcp.json
|
||||||
|
entirely (pre-existing: Kimi's session never had it) — not asserted by
|
||||||
|
the validator for that server, by design, not a gap this task opened.
|
||||||
|
NOT YET ACTIVATED: shared-mcp.json IS bind-mounted read-only into
|
||||||
|
adolf-llm (`./shared-mcp.json:/shared-mcp.json:ro` in
|
||||||
|
openai/docker-compose.yml) so the file on disk is already what the
|
||||||
|
container would read — but adolf-llm/server.js loads it ONCE into a
|
||||||
|
module-level variable at process start (not per-request), so editing the
|
||||||
|
file alone does not take effect; `docker compose restart adolf-llm` is
|
||||||
|
the only remaining step, deliberately not run by this task (never
|
||||||
|
restart a live service unattended). Confirm the real post-restart
|
||||||
|
per-turn token delta via the adolf-llm container's Kimi session wire
|
||||||
|
log: `docker exec adolf-llm sh -c "tail -1
|
||||||
|
/root/.kimi-code/sessions/*/agents/main/wire.jsonl"` (after one real
|
||||||
|
turn against a NEW session, since existing sessions' .mcp.json is
|
||||||
|
rewritten on their next turn too) and compare per-server tool counts
|
||||||
|
against 9/14/5/24 (hindsight/kanboard/openclaw-tools/agap) — the exact
|
||||||
|
same command the first-pass verification used to catch the miss.
|
||||||
282
openai/agent_registry.py
Executable file
282
openai/agent_registry.py
Executable file
@@ -0,0 +1,282 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""agent_registry — reads agent-registry.yaml (kb#134, A2A-2).
|
||||||
|
|
||||||
|
Sibling of model_registry.py (kb#133): same load/get/CLI shape, same
|
||||||
|
philosophy ("registry is data, not logic"). This module's one piece of
|
||||||
|
real logic is effective_card() — the mechanism that makes "switching a
|
||||||
|
backbone is one field" true rather than aspirational (see the header
|
||||||
|
comment in agent-registry.yaml): an agent's static fields (persona, trust
|
||||||
|
class, tool scope, memory banks, preferred tier) never duplicate what the
|
||||||
|
CURRENT backbone provides (tier, cost_class, availability a(t), context
|
||||||
|
window). Those are resolved at read time by dereferencing `backbone` into
|
||||||
|
model-registry.yaml (via model_registry.get_model) or, for runtimes
|
||||||
|
model-registry.yaml deliberately excludes (kb#133: Claude Code), into this
|
||||||
|
file's own `runtimes:` section.
|
||||||
|
|
||||||
|
Usage (library):
|
||||||
|
from agent_registry import load_registry, get_agent, effective_card, trust_rank
|
||||||
|
reg = load_registry()
|
||||||
|
adolf = get_agent(reg, "adolf")
|
||||||
|
card = effective_card(reg, "adolf") # -> persona/trust/tools/memory + resolved tier/cost/a(t)
|
||||||
|
trust_rank(reg, "torgash") < trust_rank(reg, "adolf") # sandboxed < trusted
|
||||||
|
|
||||||
|
Usage (CLI, for manual verification):
|
||||||
|
./agent_registry.py list
|
||||||
|
./agent_registry.py get --id adolf
|
||||||
|
./agent_registry.py effective-card --id adolf
|
||||||
|
./agent_registry.py trust-rank --id torgash
|
||||||
|
./agent_registry.py can-reach-vault --id torgash # kb#147 sanity check
|
||||||
|
"""
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
import model_registry as mr
|
||||||
|
|
||||||
|
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||||
|
DEFAULT_REGISTRY_PATH = os.path.join(HERE, "agent-registry.yaml")
|
||||||
|
|
||||||
|
|
||||||
|
class AgentRegistryError(Exception):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def load_registry(path=None):
|
||||||
|
"""Load and lightly validate agent-registry.yaml."""
|
||||||
|
path = path or DEFAULT_REGISTRY_PATH
|
||||||
|
with open(path) as f:
|
||||||
|
reg = yaml.safe_load(f)
|
||||||
|
if not reg or "agents" not in reg:
|
||||||
|
raise AgentRegistryError(f"{path}: missing top-level 'agents' list")
|
||||||
|
ids = [a["id"] for a in reg["agents"]]
|
||||||
|
if len(ids) != len(set(ids)):
|
||||||
|
raise AgentRegistryError(f"{path}: duplicate agent ids in {ids}")
|
||||||
|
runtime_ids = [r["id"] for r in reg.get("runtimes", [])]
|
||||||
|
if len(runtime_ids) != len(set(runtime_ids)):
|
||||||
|
raise AgentRegistryError(f"{path}: duplicate runtime ids in {runtime_ids}")
|
||||||
|
return reg
|
||||||
|
|
||||||
|
|
||||||
|
def get_agent(registry, agent_id):
|
||||||
|
for a in registry["agents"]:
|
||||||
|
if a["id"] == agent_id:
|
||||||
|
return a
|
||||||
|
raise AgentRegistryError(f"unknown agent id: {agent_id!r}")
|
||||||
|
|
||||||
|
|
||||||
|
def get_runtime(registry, runtime_id):
|
||||||
|
for r in registry.get("runtimes", []):
|
||||||
|
if r["id"] == runtime_id:
|
||||||
|
return r
|
||||||
|
raise AgentRegistryError(f"unknown runtime id: {runtime_id!r}")
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# trust_rank — §5 "human > trusted > sandboxed > untrusted" as a comparable
|
||||||
|
# integer, for kb#140 (routing) / kb#147 (grant enforcement) to use directly
|
||||||
|
# instead of re-deriving an ordering from the string.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def trust_rank(registry, agent_id):
|
||||||
|
a = get_agent(registry, agent_id)
|
||||||
|
classes = registry.get("trust_classes", {})
|
||||||
|
cls = a["trust_class"]
|
||||||
|
if cls not in classes:
|
||||||
|
raise AgentRegistryError(f"{agent_id}: unknown trust_class {cls!r} (have: {sorted(classes)})")
|
||||||
|
return classes[cls]["rank"]
|
||||||
|
|
||||||
|
|
||||||
|
def can_reach_vault(registry, agent_id):
|
||||||
|
"""kb#147 sanity check: vault access = trusted-or-human only (DECIDED).
|
||||||
|
Cross-checks the agent's declared tool_allowlist.vault_access flag
|
||||||
|
against its trust rank, so a registry typo (vault_access: true on a
|
||||||
|
sandboxed agent) is a raised error, not a silent enforcement gap."""
|
||||||
|
a = get_agent(registry, agent_id)
|
||||||
|
classes = registry.get("trust_classes", {})
|
||||||
|
trusted_rank = classes.get("trusted", {}).get("rank")
|
||||||
|
declared = ((a.get("tool_allowlist") or {}).get("vault_access")) if a.get("tool_allowlist") else False
|
||||||
|
rank = trust_rank(registry, agent_id)
|
||||||
|
if declared and rank < trusted_rank:
|
||||||
|
raise AgentRegistryError(
|
||||||
|
f"{agent_id}: tool_allowlist.vault_access=true but trust_class="
|
||||||
|
f"{a['trust_class']!r} (rank {rank}) < trusted (rank {trusted_rank}) — "
|
||||||
|
"registry inconsistency, fix before this is load-bearing for kb#147"
|
||||||
|
)
|
||||||
|
return bool(declared) and rank >= trusted_rank
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# effective_card — the full A2A Agent Card: static agent fields merged with
|
||||||
|
# the CURRENT backbone's resolved tier/cost_class/a(t)/context_tokens.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _resolve_backbone(registry, backbone_id, model_registry=None):
|
||||||
|
"""backbone can point into either this file's runtimes: (flat-subscription
|
||||||
|
runtimes model-registry.yaml deliberately excludes, kb#133) or into
|
||||||
|
model-registry.yaml's models: (everything else). Try runtimes first —
|
||||||
|
it's the smaller, local list."""
|
||||||
|
try:
|
||||||
|
return "runtime", get_runtime(registry, backbone_id)
|
||||||
|
except AgentRegistryError:
|
||||||
|
pass
|
||||||
|
model_registry = model_registry if model_registry is not None else mr.load_registry()
|
||||||
|
return "model", mr.get_model(model_registry, backbone_id)
|
||||||
|
|
||||||
|
|
||||||
|
def effective_card(registry, agent_id, model_registry=None):
|
||||||
|
"""Return the full A2A Agent Card for `agent_id`: its own persona/trust/
|
||||||
|
tools/memory fields plus tier/cost_class/context_tokens/lifecycle
|
||||||
|
resolved from whatever `backbone` currently names. Humans and trivial
|
||||||
|
endpoints with backbone=None get resolved_backbone=None — there's
|
||||||
|
nothing to dereference."""
|
||||||
|
a = get_agent(registry, agent_id)
|
||||||
|
card = dict(a) # shallow copy; don't mutate the loaded registry
|
||||||
|
backbone_id = a.get("backbone")
|
||||||
|
if backbone_id is None:
|
||||||
|
card["resolved_backbone"] = None
|
||||||
|
return card
|
||||||
|
|
||||||
|
source, backbone = _resolve_backbone(registry, backbone_id, model_registry)
|
||||||
|
card["resolved_backbone"] = {
|
||||||
|
"source": source, # "runtime" | "model" — which file it came from
|
||||||
|
"id": backbone_id,
|
||||||
|
"tier": backbone["tier"],
|
||||||
|
"cost_class": backbone["cost_class"],
|
||||||
|
"lifecycle": backbone["lifecycle"],
|
||||||
|
"context_tokens": backbone.get("context_tokens"),
|
||||||
|
"metered": backbone.get("metered", False),
|
||||||
|
}
|
||||||
|
return card
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# litellm_key_spec — kb#147 (A2A-15): turn an agent's static registry fields
|
||||||
|
# into the LiteLLM virtual-key grant provision_litellm_keys.py provisions.
|
||||||
|
# "Grants live in the agent registry, not scattered configs" (kb#147 accept-
|
||||||
|
# ance bar) means the model allow-list and budget are COMPUTED here from
|
||||||
|
# preferred_tier + trust_class, never hand-typed per agent.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
# Ascending order matching model-registry.yaml's routing.tiers keys. An
|
||||||
|
# agent may use its preferred tier and anything below it (a "large"-
|
||||||
|
# preferring agent degrades to "small" gracefully; a "small"-only agent
|
||||||
|
# never gets "large" — that asymmetry IS the sandboxed/trusted split this
|
||||||
|
# key spec exists to enforce).
|
||||||
|
_TIER_ORDER = ["small", "large"]
|
||||||
|
|
||||||
|
|
||||||
|
def _reachable_tiers(preferred_tier):
|
||||||
|
if preferred_tier not in _TIER_ORDER:
|
||||||
|
return []
|
||||||
|
return _TIER_ORDER[: _TIER_ORDER.index(preferred_tier) + 1]
|
||||||
|
|
||||||
|
|
||||||
|
def litellm_key_spec(registry, agent_id, model_registry=None):
|
||||||
|
"""Return the LiteLLM virtual-key grant for `agent_id`: which
|
||||||
|
litellm_model_name values it may use and its default budget, derived
|
||||||
|
from THIS registry's data (preferred_tier, trust_class) plus
|
||||||
|
model-registry.yaml's routing.tiers/metered_opt_in — never hand-entered
|
||||||
|
per agent. Models with no litellm_model_name (e.g. `kimi`, called
|
||||||
|
directly via the adolf-llm wrapper, never through LiteLLM) are outside
|
||||||
|
LiteLLM's enforcement surface by construction and are excluded, not
|
||||||
|
silently allowed.
|
||||||
|
|
||||||
|
provision_litellm_keys.py consumes this dict's `models`/`max_budget`/
|
||||||
|
`budget_duration`/`key_alias` as the body of a LiteLLM /key/generate (or
|
||||||
|
/key/update) call. This function makes no network call itself.
|
||||||
|
"""
|
||||||
|
a = get_agent(registry, agent_id)
|
||||||
|
model_registry = model_registry if model_registry is not None else mr.load_registry()
|
||||||
|
grant = a.get("capability_grant") or {}
|
||||||
|
key_alias = grant.get("litellm_key_alias", agent_id)
|
||||||
|
|
||||||
|
opted_in = set(model_registry.get("routing", {}).get("metered_opt_in", []) or [])
|
||||||
|
opted_in_key = f"agent:{agent_id}"
|
||||||
|
pools = model_registry.get("routing", {}).get("tiers", {})
|
||||||
|
|
||||||
|
models = []
|
||||||
|
for tier in _reachable_tiers(a.get("preferred_tier")):
|
||||||
|
for model_id in pools.get(tier, []):
|
||||||
|
m = mr.get_model(model_registry, model_id)
|
||||||
|
name = m.get("litellm_model_name")
|
||||||
|
if not name:
|
||||||
|
continue # not LiteLLM-routed (e.g. kimi's adolf-llm wrapper) -- nothing to grant/deny here
|
||||||
|
if m.get("metered") and opted_in_key not in opted_in:
|
||||||
|
continue # §3a: no metered API by default, per-key opt-in only
|
||||||
|
if name not in models:
|
||||||
|
models.append(name)
|
||||||
|
|
||||||
|
classes = registry.get("trust_classes", {})
|
||||||
|
cls = classes.get(a["trust_class"], {})
|
||||||
|
return {
|
||||||
|
"agent_id": agent_id,
|
||||||
|
"key_alias": key_alias,
|
||||||
|
"trust_class": a["trust_class"],
|
||||||
|
"models": models,
|
||||||
|
"max_budget": cls.get("default_budget_usd"),
|
||||||
|
"budget_duration": cls.get("budget_duration"),
|
||||||
|
"mcp_auth_token_env": grant.get("mcp_auth_token_env"),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# CLI — manual verification only, not part of the library contract.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def main():
|
||||||
|
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||||
|
ap.add_argument("--registry", default=None, help="path to agent-registry.yaml (default: sibling file)")
|
||||||
|
ap.add_argument("--model-registry", default=None, help="path to model-registry.yaml (default: sibling file)")
|
||||||
|
sub = ap.add_subparsers(dest="cmd", required=True)
|
||||||
|
|
||||||
|
p = sub.add_parser("get")
|
||||||
|
p.add_argument("--id", required=True)
|
||||||
|
|
||||||
|
p = sub.add_parser("effective-card")
|
||||||
|
p.add_argument("--id", required=True)
|
||||||
|
|
||||||
|
p = sub.add_parser("trust-rank")
|
||||||
|
p.add_argument("--id", required=True)
|
||||||
|
|
||||||
|
p = sub.add_parser("can-reach-vault")
|
||||||
|
p.add_argument("--id", required=True)
|
||||||
|
|
||||||
|
p = sub.add_parser("litellm-key-spec")
|
||||||
|
p.add_argument("--id", required=True)
|
||||||
|
|
||||||
|
sub.add_parser("list")
|
||||||
|
|
||||||
|
args = ap.parse_args()
|
||||||
|
reg = load_registry(args.registry)
|
||||||
|
model_reg = mr.load_registry(args.model_registry) if args.model_registry else None
|
||||||
|
|
||||||
|
try:
|
||||||
|
if args.cmd == "get":
|
||||||
|
print(json.dumps(get_agent(reg, args.id), indent=2))
|
||||||
|
elif args.cmd == "effective-card":
|
||||||
|
print(json.dumps(effective_card(reg, args.id, model_reg), indent=2))
|
||||||
|
elif args.cmd == "trust-rank":
|
||||||
|
print(trust_rank(reg, args.id))
|
||||||
|
elif args.cmd == "can-reach-vault":
|
||||||
|
ok = can_reach_vault(reg, args.id)
|
||||||
|
print(json.dumps({"id": args.id, "can_reach_vault": ok}))
|
||||||
|
sys.exit(0 if ok else 1)
|
||||||
|
elif args.cmd == "litellm-key-spec":
|
||||||
|
print(json.dumps(litellm_key_spec(reg, args.id, model_reg), indent=2))
|
||||||
|
elif args.cmd == "list":
|
||||||
|
for a in reg["agents"]:
|
||||||
|
backbone = a.get("backbone") or "-"
|
||||||
|
print(f"{a['id']:22} trust={a['trust_class']:9} "
|
||||||
|
f"tier={str(a.get('preferred_tier')):6} backbone={backbone:16} "
|
||||||
|
f"role={a['persona']['role']}")
|
||||||
|
except AgentRegistryError as exc:
|
||||||
|
print(f"error: {exc}", file=sys.stderr)
|
||||||
|
sys.exit(2)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
39
openai/auto-router-routes.json
Normal file
39
openai/auto-router-routes.json
Normal file
@@ -0,0 +1,39 @@
|
|||||||
|
{
|
||||||
|
"_note": "kb#128 (A2A-16): human-readable source of truth for the auto_router route set. NOT loaded from this path at runtime -- litellm-config.yaml's `auto_router` deployment inlines this same `routes` array as a literal JSON string via litellm_params.auto_router_config. Reason (verified hands-on 2026-07-26 against litellm:main-latest): the auto_router_config_path loader (AutoRouter._load_semantic_routing_routes -> SemanticRouter.from_json) unconditionally builds a raw semantic_router encoder from encoder_type/encoder_name and requires a real provider API key even for a local model name like bge-m3 -- this IS the open Auto Router v2 embedding bug the task brief warned about. The auto_router_config (inline-string) loader (_load_auto_router_routes_from_config_json) only reads the `routes` key and builds Route objects directly, with zero encoder bootstrap -- confirmed working end-to-end: real litellm.embedding(model=ollama/bge-m3) calls, zero metered API spend, 'hi there' -> ollama/gemma3:4b, a refactor/dependency-injection prompt -> kimi-agent. Keep the two `routes` arrays in sync by hand when editing either.",
|
||||||
|
"encoder_type": "litellm",
|
||||||
|
"encoder_name": "bge-m3",
|
||||||
|
"routes": [
|
||||||
|
{
|
||||||
|
"name": "ollama/gemma3:4b",
|
||||||
|
"description": "Simple, short, low-stakes requests — greetings, quick factual lookups, formatting, one-line questions.",
|
||||||
|
"utterances": [
|
||||||
|
"hi",
|
||||||
|
"hello",
|
||||||
|
"what time is it",
|
||||||
|
"what's the weather",
|
||||||
|
"thanks",
|
||||||
|
"what does this word mean",
|
||||||
|
"summarize this in one sentence",
|
||||||
|
"give me a quick yes or no",
|
||||||
|
"format this as a list",
|
||||||
|
"what is 2 plus 2"
|
||||||
|
],
|
||||||
|
"score_threshold": 0.5
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "kimi-agent",
|
||||||
|
"description": "Complex reasoning, multi-step planning, coding, or anything needing tool use and deep context.",
|
||||||
|
"utterances": [
|
||||||
|
"write a function that parses this log file and extracts errors",
|
||||||
|
"refactor this class to use dependency injection",
|
||||||
|
"think through the tradeoffs of these two architectures step by step",
|
||||||
|
"debug why this docker container keeps crashing",
|
||||||
|
"plan out the migration from cognee to hindsight across five tasks",
|
||||||
|
"analyze this design document and find inconsistencies",
|
||||||
|
"write a SQL query that joins these three tables and aggregates by month",
|
||||||
|
"review this pull request for security issues"
|
||||||
|
],
|
||||||
|
"score_threshold": 0.5
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user