Backups

Daily backups run from root's crontab (sudo crontab -e), except Kanboard's (see below), which runs from alvis's own crontab (crontab -e, no sudo).

Gitea

See Gitea#Backup for details.

0 3 * * * /home/alvis/agap_git/gitea/backup.sh >> /mnt/backups/gitea/cron.log 2>&1

Output: /mnt/backups/gitea/gitea-dump-*.zip

Immich (photos)

Script: immich-app/backup.sh — dumps DB first, then rsyncs originals to backup volume. Runs at 02:30 (after Immich's built-in dump at 02:00).

30 2 * * * /home/alvis/agap_git/immich-app/backup.sh >> /mnt/toshiba/backups/media/cron.log 2>&1

What	Source	Destination
DB dump	`immich_postgres` container	`/mnt/toshiba/backups/media/backups/immich-db-*.sql.gz`
Photos	`/mnt/smsg/media/upload/library/`	`/mnt/toshiba/backups/media/library/`
Uploads	`/mnt/smsg/media/upload/upload/`	`/mnt/toshiba/backups/media/upload/`
Profiles	`/mnt/smsg/media/upload/profile/`	`/mnt/toshiba/backups/media/profile/`

Thumbnails and encoded video are skipped — they can be regenerated by Immich. DB dumps are retained for 14 days.

DBS (databases)

Rsync of all databases from SSD to backup volume.

30 3 * * * rsync -a --delete /mnt/ssd/dbs/ /mnt/backups/dbs/ >> /mnt/backups/dbs/cron.log 2>&1 && touch /mnt/backups/dbs/.last_sync

Source: /mnt/ssd/dbs/ (gitea, otter, qdrant, zabbix) Destination: /mnt/backups/dbs/

Vaultwarden

Script: vaultwarden/backup.sh — uses built-in container backup, copies db/config/keys. Runs every 3 days.

0 2 */3 * * /home/alvis/agap_git/vaultwarden/backup.sh >> /var/log/vaultwarden-backup.log 2>&1

Destination: /mnt/backups/vaultwarden/<timestamp>/ Retention: last 5 backups Zabbix: trapper item vaultwarden.backup.ts (id 70368) — trigger if no backup in 4 days

Known bug (found 2026-07-21, kb#158): the script pushes a formatted date string (date '+%Y-%m-%d %H:%M') into vaultwarden.backup.ts, which is a numeric (unsigned) item — every push is silently rejected, so the item has never recorded a real value (lastclock: 0) and its "overdue" trigger has been permanently in the problem state. Not fixed as part of kb#158 (out of scope — flagged here for a follow-up task). Kanboard's own backup below pushes a proper unix timestamp and was verified to actually populate the item.

Seafile

Script: seafile/backup.sh — mysqldump of 3 DBs + rsync of seafile data. Runs every 3 days.

0 2 */3 * * /home/alvis/agap_git/seafile/backup.sh >> /var/log/seafile-backup.log 2>&1

Destination: /mnt/backups/seafile/<timestamp>/ Retention: last 5 backups Zabbix: trapper item seafile.backup.ts (id 70369) — trigger if no backup in 4 days

Users (/mnt/misc)

Script: users-backup.sh — rsyncs alvis and liza home dirs from /mnt/misc to backup volume. Runs every 3 days.

0 4 */3 * * /home/alvis/agap_git/users-backup.sh >> /var/log/users-backup.log 2>&1

Source	Destination
`/mnt/misc/alvis/`	`/mnt/backups/users/alvis/`
`/mnt/misc/liza/`	`/mnt/backups/users/liza/`

Zabbix: trapper item users.backup.ts (id 70379) — trigger (id 32869) if no backup in 4 days

Kanboard (kb#158, A2A-26 — tier-0 hardening)

Kanboard was promoted to tier-0 (fabric backbone, DESIGN-a2a-agents.md v2.1 §6c) on 2026-07-21: backups on par with the vault, plus live service/API monitoring (see Zabbix Monitoring below).

Script: /home/alvis/agap_git/kanboard/backup.sh — the kanboard container has no sqlite3 CLI and no PHP sqlite3 extension (only pdo_sqlite), so instead of a CLI dump it runs SQLite's own VACUUM INTO via PDO inside the container — an atomic, consistent online snapshot safe against concurrent writers (the same safety property vaultwarden backup gives that service). Also tars up the kanboard_plugins volume (PLUGIN_INSTALLER=true means plugins can be installed at runtime, not just baked into the image). Runs every 3 days.

0 3 */3 * * /home/alvis/agap_git/kanboard/backup.sh >> /home/alvis/agap_git/kanboard/backup.log 2>&1

Runs from alvis's own crontab, not root's — unlike the other services above. /mnt/backups/kanboard/ is chown'd to alvis (one-time bootstrap: alvis is in the docker group, which is root-equivalent on this host, so a throwaway alpine container did the mkdir+chown without needing an interactive sudo password) so the rest of the pipeline — the dump itself, the Zabbix push, the crontab — needs no root at all.

Destination: /mnt/backups/kanboard/<timestamp>/ (db.sqlite + plugins.tar.gz) Retention: last 5 backups Zabbix: trapper item kanboard.backup.ts (id 70605, real unix timestamp) — trigger Kanboard backup overdue (>4 days) (id 32958)

Kanboard service + JSON-RPC health

Script: /home/alvis/agap_git/kanboard/healthcheck.sh, every 2 minutes from alvis's crontab:

*/2 * * * * /home/alvis/agap_git/kanboard/healthcheck.sh >> /home/alvis/agap_git/kanboard/healthcheck.log 2>&1

Pushes two Zabbix trapper items every run (a "down" value is real signal, not a gap to wait out via nodata()):

Item key	id	Meaning	Trigger
`kanboard.service.up`	70606	1 if the `kanboard` container's own Docker healthcheck reports `healthy`	`Kanboard service is down` (id 32959): `last()=0` or `nodata(10m)`
`kanboard.jsonrpc.up`	70607	1 if an authenticated `getVersion` JSON-RPC call round-trips	`Kanboard JSON-RPC API is down` (id 32960): `last()=0` or `nodata(10m)`

Verified live (kb#158): pushed a manual service.up=0, watched the trigger fire (problem.get returned the event within seconds), then restored the real value and watched it clear.

Degraded mode (fabric ops fail soft when Kanboard is down)

Verified/hardened as part of kb#158, tested by pointing clients at an unreachable KANBOARD_URL rather than stopping the live container:

kanboard-mcp (Adolf's and Claude's MCP tool server) already failed soft — every kanboard_* tool handler wraps its call in try/catch and returns an MCP isError: true result on failure; the server process itself never crashes. Verified by invoking kanboard.js's functions directly against an unreachable URL — both calls returned clean tool errors, process stayed alive. So when Kanboard is down, an Adolf turn that touches a kanboard_* tool just sees that one tool call fail; the rest of the turn (and Matrix chat in general) is unaffected.
kb_worker.py's daemon loop had a real bug, fixed in kb#158: kb-claim's rpc() calls sys.exit() on an unreachable/erroring Kanboard, and the daemon's main loop didn't catch that — an unreachable Kanboard killed the whole worker process instead of parking it (reproduced live: pointed a worker at an unreachable KANBOARD_URL, it died on its first loop iteration). Fixed by wrapping the loop body in except (SystemExit, Exception), logging parked: kanboard unreachable... and sleeping park_interval before retrying — same treatment as an a(t)=0 probe. Re-verified live after the fix: the same unreachable-URL run now parks and keeps looping instead of dying.
A task claimed but not yet reported when Kanboard drops keeps its lease; the lease expires and the fabric-keeper (#154) returns it to Ready — nothing is silently lost, just delayed.

Zabbix Monitoring

Backups monitored via Zabbix on host AgapHost.

Item key	Description	Trigger
`gitea.backup.status`	`1` if last dump succeeded	—
`gitea.backup.age`	Seconds since last dump zip	High if > 25h
`dbs.backup.age`	Seconds since last rsync (`.last_sync` mtime)	High if > 25h
`immich.backup.age`	Seconds since last immich backup (`.last_sync` mtime)	High if > 25h
`vaultwarden.backup.ts`	Timestamp of last backup (trapper)	Warning if > 4 days (⚠ never actually populated — see bug note above)
`seafile.backup.ts`	Timestamp of last backup (trapper)	Warning if > 4 days
`users.backup.ts`	Timestamp of last backup (trapper)	Warning if > 4 days
`kanboard.backup.ts`	Timestamp of last backup (trapper, real unixtime)	High if no backup in 4 days
`kanboard.service.up`	Container health, pushed every 2 min	High if down or stale >10 min
`kanboard.jsonrpc.up`	JSON-RPC API health, pushed every 2 min	High if down or stale >10 min

Agent config: /etc/zabbix/zabbix_agent2.d/gitea_backup.conf