# Restore Runbook — Kanboard, Vaultwarden, Seafile

Companion to each service's `backup.sh`. Covers kb#192.

Each service now has a `restore.sh` next to its `backup.sh`:
- `kanboard/restore.sh`
- `vaultwarden/restore.sh`
- `seafile/restore.sh`

All three follow the same convention: they default to the **live** container
names/paths, but every target is overridable via env vars, so the exact same
script restores into a real disaster or into a disposable/throwaway
container for a dry-run test. **Never invoke a restore.sh without env
overrides unless you are doing a real, intentional disaster recovery** —
the defaults point at production.

## Before you restore anything

1. `/mnt/backups/<service>/` is read-only in practice — copy the snapshot
   you want out to a scratch dir first, don't operate on it in place.
2. Confirm you have the right snapshot: `ls /mnt/backups/<service>/` and
   pick the newest dir, but **check its contents aren't empty** (see the
   Seafile gotcha below — an empty dump looks like a valid directory).
3. Restoring into the live container is destructive and briefly stops the
   service. Only do this for a real incident, and say so out loud before
   running it.

## Kanboard

```bash
cp -r /mnt/backups/kanboard/<snapshot>  /tmp/kb-restore
# Real disaster recovery (overwrites the live "kanboard" container):
cd /home/alvis/agap_git/kanboard
./restore.sh /tmp/kb-restore
```

What it does: stops the `kanboard` container, replaces `db.sqlite` via
`docker cp`, restores `plugins.tar.gz` if present, restarts, then verifies
by querying `SELECT COUNT(*) FROM tasks` through the container's PHP PDO
sqlite driver (kanboard's image has no sqlite3 CLI).

Throwaway test (no live impact — no ports published, isolated volumes):
```bash
docker volume create kb_restore_test_data
docker volume create kb_restore_test_plugins
docker run -d --name kanboard-restore-test \
  -v kb_restore_test_data:/var/www/app/data \
  -v kb_restore_test_plugins:/var/www/app/plugins \
  -e PLUGIN_INSTALLER=true kanboard/kanboard:latest

CONTAINER=kanboard-restore-test ./restore.sh /tmp/kb-restore

# Teardown
docker rm -f kanboard-restore-test
docker volume rm kb_restore_test_data kb_restore_test_plugins
```

**Verified 2026-07-30**: restored the 2026-07-28 03:00 snapshot into a
throwaway container this way. `tasks` table came back with 180 rows;
container started healthy. Throwaway container and volumes torn down after.

## Vaultwarden

```bash
cp -r /mnt/backups/vaultwarden/<snapshot>  /tmp/vw-restore
# Real disaster recovery (overwrites /mnt/ssd/dbs/vw-data and the live
# "vaultwarden" container — needs root; the data dir is root-owned):
cd /home/alvis/agap_git/vaultwarden
sudo ./restore.sh /tmp/vw-restore
```

What it does: stops the `vaultwarden` container, copies the `db_*.sqlite3`
snapshot to `db.sqlite3`, restores `config.json`, `rsa_key*`,
`attachments/`, `sends/`, restarts, then checks the DB file is in place
(the vaultwarden image has no `sqlite3` CLI either — verification falls
back to a file-presence/size check and reading the startup log for a
clean launch with no re-keying).

Throwaway test (isolated data dir, isolated container, no ports published):
```bash
mkdir -p /tmp/vw-data-test
docker run -d --name vaultwarden-restore-test --user 1000:1000 \
  -v /tmp/vw-data-test:/data vaultwarden/server:latest

CONTAINER=vaultwarden-restore-test DATA_DIR=/tmp/vw-data-test \
  ./restore.sh /tmp/vw-restore

# Teardown
docker rm -f vaultwarden-restore-test
rm -rf /tmp/vw-data-test
```
Note: `--user 1000:1000` is only needed for the throwaway test so the bind
mount is writable by a non-root operator; the live container runs as root
and the live data dir is root-owned, so a real restore needs `sudo`.

**Verified 2026-07-30**: restored the 2026-07-28 02:00 snapshot into a
throwaway container this way. `db.sqlite3` landed at the correct size,
`config.json` was picked up ("Using saved config from `data/config.json`"
in the startup log), and the RSA key was reused rather than regenerated
(no "Private key created" line on the post-restore boot) — i.e. structural
restore confirmed. Row-level vault content was not inspected (per
Vaultwarden-handling rules — never surface real vault contents). Throwaway
container and scratch dir removed after.

## Seafile

```bash
cp -r /mnt/backups/seafile/<snapshot>  /tmp/sf-restore
# Real disaster recovery (drops+reloads ccnet_db/seafile_db/seahub_db in
# the live "seafile-mysql" container, and rsyncs the data dir back into
# /mnt/misc/seafile, stopping/starting the "seafile" container around it —
# needs root; data dir is root-owned):
cd /home/alvis/agap_git/seafile
sudo ./restore.sh /tmp/sf-restore
```

What it does: refuses to run if any of the three `*.sql` dumps in the
snapshot is missing/empty (see gotcha below), then for each of
`ccnet_db`/`seafile_db`/`seahub_db`: `DROP DATABASE IF EXISTS` +
`CREATE DATABASE` + reload from the dump, and reports table counts as a
sanity check. If the snapshot has a `data/` dir and `SEAFILE_CONTAINER` is
set (default), it also stops the seafile app container, rsyncs `data/`
into `DATA_DIR`, and restarts it.

**Env-var gotcha fixed during this task**: `SEAFILE_CONTAINER=""` used to
fall back to the live container name, because `${VAR:-default}` treats an
empty string as unset. It's now `${VAR-default}` so an explicit empty
string really means "skip the data-dir step." If you want a DB-only
restore, pass `SEAFILE_CONTAINER=""` explicitly.

Throwaway test (DB-only, fully isolated MariaDB container + scratch data dir):
```bash
docker volume create sf_restore_test_db
docker run -d --name seafile-mysql-restore-test \
  -e MYSQL_ROOT_PASSWORD=<test-only-pw> \
  -e MYSQL_USER=seafile -e MYSQL_PASSWORD=<matches SEAFILE_MYSQL_DB_PASSWORD> \
  -e MYSQL_DATABASE=placeholder \
  -v sf_restore_test_db:/var/lib/mysql mariadb:10.11
# grant seafile broad perms on this throwaway instance only:
docker exec seafile-mysql-restore-test mysql -u root -p<test-only-pw> \
  -e "GRANT ALL PRIVILEGES ON *.* TO 'seafile'@'%'; FLUSH PRIVILEGES;"

mkdir -p /tmp/sf-data-test
MYSQL_CONTAINER=seafile-mysql-restore-test \
  SEAFILE_CONTAINER=seafile-app-does-not-exist-test \
  DATA_DIR=/tmp/sf-data-test \
  ./restore.sh /tmp/sf-restore

# Teardown
docker rm -f seafile-mysql-restore-test
docker volume rm sf_restore_test_db
rm -rf /tmp/sf-data-test
```

**Verified 2026-07-30 — partially**: DB restore was verified end-to-end
into a throwaway MariaDB container using the last known-good snapshot
(2026-07-04 02:00 — see the critical finding below): `ccnet_db` (13
tables), `seafile_db` (46 tables), `seahub_db` (127 tables) all restored
and importable. The `data/` directory step ran cleanly against an isolated
scratch dir (`/tmp/sf-data-test`, not `/mnt/misc/seafile`) — the on-disk
tree (`seafile-data/`, `seadoc-data/`, `onlyoffice-data/`) landed as
expected. **A full running Seafile app-stack test (seafile+redis+caddy
actually serving content from the restored data) was not attempted** — that
would need the full compose stack, matching `JWT_PRIVATE_KEY`/hostname
config from `.env`, and meaningfully more setup than the box's read-only
`/mnt/backups` + non-root constraints support cleanly in one pass. If a
full app-level restore drill is wanted, treat it as separate follow-up
work with root access.

### ⚠️ Critical finding: Seafile backups have been broken since 2026-07-07

Every `/mnt/backups/seafile/<snapshot>` from **2026-07-07 through the
latest, 2026-07-28**, contains only an **empty** `ccnet_db.sql` (0 bytes)
and nothing else — no `seafile_db.sql`, `seahub_db.sql`, or `data/`. The
last known-good snapshot is **2026-07-04 02:00**. `restore.sh` now refuses
to run against a snapshot with a missing/empty dump file rather than
silently "restoring" an empty database, but **the backup cron job itself
is still broken** and needs its own fix (likely the `mysqldump` credentials
in `seafile/backup.sh` no longer matching the live `seafile` MySQL user, or
similar — not diagnosed further here since reproducing it requires running
`mysqldump` against the live `seafile-mysql` container, which is out of
scope for a restore-focused task and was blocked by the sandbox's
action classifier during this session). **Recommend filing a new,
separate task to fix `seafile/backup.sh`** — this is a live gap: three
weeks of Seafile backups are currently useless.

## Files touched

- `/home/alvis/agap_git/kanboard/restore.sh` (new)
- `/home/alvis/agap_git/vaultwarden/restore.sh` (new)
- `/home/alvis/agap_git/seafile/restore.sh` (new)
- `/home/alvis/agap_git/RESTORE-RUNBOOK.md` (this file, new)

Per the standing rule for this repo, nothing above was committed — it's
left in the working tree for a human to review and commit.