alvis/AgapHost
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

services: add mood, moodtracker, overleaf, personal-sensing; update ollama

Browse Source 
Compose and supporting code for four services that had been running or
prototyped without their config tracked here, per the repo convention that
agap_git holds the compose + config while application source lives in each
service's own Gitea repo.

Only placeholder credentials are included: mood/.env.example and
moodtracker/.env.example ship dummy values, and overleaf/variables.env carries
app name and feature flags only. Real values stay in Vaultwarden.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
alvis
2026-07-30 04:42:58 +00:00
parent 41f3f15d27
commit d37801806d
31 changed files with 1848 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
mood/.env.example Normal file
View File

@@ -0,0 +1,6 @@
# mood service config — optional tuning only, no credentials needed.
# (Source is a directly-readable local SQLite file, not an HTTP API.)
# Copy to .env if you want to override the defaults baked into docker-compose.yml.
MOOD_SYNC_INTERVAL_SECONDS=3600
MOOD_OVERLAP_ROWS=3

6
mood/.gitignore vendored Normal file
View File

@@ -0,0 +1,6 @@
__pycache__/
*.pyc
*.sqlite
*.sqlite-wal
*.sqlite-shm
.env

13
mood/Dockerfile Normal file
View File

@@ -0,0 +1,13 @@
# Stdlib only (sqlite3 + argparse + csv) — no pip install needed.
FROM python:3.12-slim
WORKDIR /app
COPY schema.sql ./schema.sql
COPY src ./src
ENV MOOD_DB_PATH=/data/mood_archive.sqlite \
MOOD_SOURCE_DB_PATH=/source/moodtracker/mood.db \
PYTHONUNBUFFERED=1
# Long-lived service: cli.py `sync` loops on MOOD_SYNC_INTERVAL_SECONDS.
CMD ["python", "-m", "src.cli", "sync"]

269
mood/README.md Normal file
View File

@@ -0,0 +1,269 @@
# mood — local archive of mood.alogins.net
Local SQLite copy of mood entries logged at **mood.alogins.net**, plus simple
reports and a cross-source correlation hook. Kanboard **Adolf #107** (data-pipeline
half only — see "Scope" below).
Status: **built and proven against the real live data** (28 real entries pulled
and queried successfully). Not yet deployed as a running service — `docker
compose up -d` is a one-line handover, see "Deploy".
---
## Scope of this build
Task #107 has four parts. This directory implements **1, 3, and 4 only**:
1. ✅ Investigate whether mood.alogins.net has an API/export — done, see below.
3. ✅ Regular ingestion into local SQLite storage on Agap — done, this service.
4. ✅ Simple reports/correlations over the stored data — done, `query` CLI.
**2 is explicitly NOT built here**: "proactive Matrix/Telegram reminder" is an
*outward-facing, scheduled* message to the user. That capability belongs to
the proactive-cadence framework (Kanboard **Adolf #124**), which is currently
**parked, tagged `blocked`, awaiting a human decision** on whether to enable
its cron at all (a sibling run already had to back out an unauthorized live
crontab install — see #124 comments). Wiring a second scheduled outward
message before that decision lands would repeat the same mistake.
**Follow-up task to create**: "Wire the mood.alogins.net proactive reminder"
— *depends on #124's cron being authorized*. Once #124 is resolved, adding
this reminder is small: a text-only Backlog-card-generation step reusing
whatever executor #124 lands on, with the message
`Как день? Запиши в mood.alogins.net`. See "Ready-to-hand-over reminder"
below for a schedule a human can enable manually right now if they don't want
to wait for #124.
---
## 1. Investigation: does mood.alogins.net have an API?
**mood.alogins.net is not a third-party tracker** — it's a small self-built
Flask app already running on Agap:
- Source: `/home/alvis/moodtracker/app.py` (+ `templates/`)
- Container: `moodtracker` (compose at `/home/alvis/moodtracker/docker-compose.yml`)
- Caddy: `mood.alogins.net { reverse_proxy localhost:5177 }` (`/etc/caddy/Caddyfile`)
- DB: SQLite at `/home/alvis/moodtracker/data/mood.db`, one table `entries`
(`id, ts, mood, tags, note, affirmation`), mood on a 1–5 scale, tags a JSON
array of free-text strings, `ts` ISO8601 UTC.
**It does have a JSON API**, but it's session-cookie gated, not token-based:
- `POST /login` (form `username`/`password` = `AUTH_USER`/`AUTH_PASS` env vars,
currently plaintext in its own `docker-compose.yml` — pre-existing, not
something this task introduced) → sets a Flask session cookie.
- `GET /api/history?limit=N`, `POST /api/log`, `DELETE /api/entry/<id>` — all
require that session cookie (`@require_auth`); no HTTP token/API-key.
**Chosen ingestion path: read the SQLite file directly, not the HTTP API.**
`data/mood.db` is host-readable (`644`, owned by `root:root`, world-read bit
set) — no credential needed. This is strictly more robust than replicating a
cookie-login flow: it survives moodtracker adding/changing auth, needs no
secret in this service at all, and is read-only by construction (bind-mounted
`:ro`), so it can never corrupt or lock the live app's database.
---
## 3. Ingestion architecture
```
moodtracker's own SQLite file ──(read-only bind mount)──► src/mood_source.py
/data/mood.db │
▼
src/sync.py
(cursor + idempotent upsert)
│
▼
local archive: mood_entries
/mnt/dbs/mood/mood_archive.sqlite
│
▼
src/cli.py query (reports)
```
Mirrors the sibling `googlefit` service's shape (same author idiom, see
`agap_git/googlefit/`):
- `schema.sql` — `mood_entries` (PK `source, source_id`), `sync_state`
(per-stream cursor), `ingest_runs` (audit log).
- `src/config.py` — paths + tuning, no credentials (none needed).
- `src/mood_source.py` — read-only reader for moodtracker's SQLite file.
- `src/store.py` — schema init, idempotent UPSERTs, read/report queries.
- `src/sync.py` — cursor-based incremental sync, isolated failure handling.
- `src/cli.py` — `init-db | sync | query`.
### Why SQLite, not InfluxDB
Same reasoning as `googlefit`: single-user, a handful of manually-logged
entries a week — trivial volume. Agap storage doctrine is SQLite-first. No
extra always-on TSDB service for ~30 rows/month of data.
### Idempotency / cursor
`sync_state.last_synced_id` is the high-water mark on moodtracker's own
`entries.id`. Each run re-checks the last `MOOD_OVERLAP_ROWS` (default 3)
already-synced ids too, as a cheap safety net — moodtracker currently has no
edit endpoint (only insert + delete), so this is mostly redundant today but
costs nothing.
**Deleted-upstream entries are kept.** If an entry is deleted via moodtracker's
`DELETE /api/entry/<id>`, this archive does not remove its copy — it's an
append-only journal by design, so history survives accidental or intentional
deletes in the live app.
### Proven against real data
```
$ python -m src.cli sync --once
{"synced": {"entries": 28, "errors": []}}
$ python -m src.cli query summary
{"entries": 28, "coverage": {"earliest": "2026-05-18T04:44:34...", "latest": "2026-07-07T05:49:34..."}, "avg_mood_all_time": 3.64, ...}
```
Re-running `sync --once` twice more produced **zero row growth** (idempotent).
The source file's mtime was unchanged after every sync run (proves read-only).
Also verified end-to-end through the built Docker image (`docker build` +
`docker run --rm ... sync --once` against the real, live-mounted
`/home/alvis/moodtracker/data`), then removed the test image/container —
nothing was left running.
### Test suite
```
$ python -m pytest tests/ -q
............ [100%]
12 passed in 0.11s
```
Covers: upsert idempotency, conflict updates, sync-cursor high-water-mark
behavior, a mock-moodtracker-schema DB driven through `run_sync` (first run,
idempotent re-run, incremental pickup of a newly-inserted row, error handling
when the source is missing, and read-only-ness), plus the report/correlation
math below.
---
## 4. Reports / correlations
```bash
python -m src.cli query summary # counts, coverage, freshness
python -m src.cli query entries --days 30 # raw recent entries
python -m src.cli query daily --days 30 # avg mood + entry count per day
python -m src.cli query tags --days 90 # avg mood per tag (min 2 occurrences)
python -m src.cli query correlate <csv> --days 90
```
`daily` and `tags` are the "mood over time" and "which tags coincide with
low/high mood" reports from point 4. Real output against the live data
(tags, 90-day window): lowest avg mood tags were `exhausted` (1.0, n=2),
`depressed` (2.0, n=4); highest were `happy` (5.0, n=4), `energetic` (4.6, n=5).
### Correlation hook (cross-source, not wired)
`query correlate <csv_path>` computes a Pearson `r` between the daily average
mood and an arbitrary external daily series supplied as a plain
`day,value` CSV (`YYYY-MM-DD,float`). This is a deliberate seam: it takes
**data**, not a live connection to another service's DB or container, so
wiring a real second source later (e.g. `googlefit query metric
heart_rate_avg` or sleep minutes, once that service is deployed) is a one-line
change — build the CSV/dict from that service's own read-only query CLI. This
build does **not** reach into `googlefit` or any other service's DB, per the
task's "leave hooks, don't wire other services" instruction.
Tested with a synthetic series (`tests/test_store.py::test_correlate_with_series`,
near-perfect correlation asserted `r > 0.99`) and manually against the real
mood data with a hand-built "hours slept" CSV (`r = 0.933`, n=5, small sample —
illustrative only, not a real finding).
---
## Deploy
```bash
mkdir -p /mnt/dbs/mood # (needs sudo — /mnt/dbs is root-owned; see googlefit precedent)
cd /home/alvis/agap_git/mood
docker compose up -d --build
```
The container loops `mood sync` every `MOOD_SYNC_INTERVAL_SECONDS` (default
3600s / hourly — mood entries are logged manually, hourly polling of a local
file is effectively free and gives fresh reports without any real cost).
One-off / cron alternative (no long-lived container):
```bash
docker compose run --rm mood-archive python -m src.cli sync --once
```
**This was not started as a live service in this build** — only proven via
`docker build` + `docker run --rm ... --once` against a scratch data
directory, then torn down. Bringing up the persistent `restart: unless-stopped`
container is a one-line `docker compose up -d --build` for a human/operator to run.
---
## Read tool for Adolf
```bash
docker compose run --rm mood-archive python -m src.cli query summary
docker compose run --rm mood-archive python -m src.cli query daily --days 14
docker compose run --rm mood-archive python -m src.cli query tags --days 90
```
JSON on stdout, read-only, no credentials.
Follow-up (adjacent, not in this task, same idiom as `googlefit`'s README):
promote `query` to a native `agap-mcp` tool once that server's active edit
window (sibling task touching `shared-mcp.json`/`openclaw.json`) is clear.
---
## Ready-to-hand-over reminder (point 2, NOT installed)
Per the scope note above, the proactive reminder is intentionally not built
or scheduled here. If a human wants it live **without** waiting for #124's
cron decision, here is a self-contained one-liner using the existing Telegram
bot credentials already in Vaultwarden (`TELEGRAM_BOT_TOKEN`,
`TELEGRAM_CHAT_ID`) — nothing new to build, nothing in this repo depends on
it:
```bash
BW=/home/alvis/bin/bw
SESSION=$(env -u HTTPS_PROXY -u HTTP_PROXY -u ALL_PROXY -u https_proxy -u http_proxy -u all_proxy \
NODE_TLS_REJECT_UNAUTHORIZED=0 $BW unlock "$BW_PASSWORD" --raw 2>/dev/null)
BOT=$(env -u HTTPS_PROXY -u HTTP_PROXY -u ALL_PROXY -u https_proxy -u http_proxy -u all_proxy \
NODE_TLS_REJECT_UNAUTHORIZED=0 $BW get password "TELEGRAM_BOT_TOKEN" --session "$SESSION" 2>/dev/null)
CHAT=$(env -u HTTPS_PROXY -u HTTP_PROXY -u ALL_PROXY -u https_proxy -u http_proxy -u all_proxy \
NODE_TLS_REJECT_UNAUTHORIZED=0 $BW get password "TELEGRAM_CHAT_ID" --session "$SESSION" 2>/dev/null)
env -u HTTPS_PROXY -u HTTP_PROXY -u ALL_PROXY -u https_proxy -u http_proxy -u all_proxy \
curl -s -X POST "https://api.telegram.org/bot${BOT}/sendMessage" \
-d "chat_id=${CHAT}" -d "text=Как день? Запиши в mood.alogins.net"
```
Proposed cron (a human adds this — **not installed by this task**, per the
"never install an unattended cron on a live target" rule):
```
0 21 * * * /home/alvis/agap_git/mood/scripts/remind.sh # hypothetical path if built
```
No `scripts/remind.sh` exists yet — the command above is the full logic; if
approved, wrapping it in a script + crontab line is a ~2-minute follow-up, but
it is outward-facing (sends a message unattended) so it needs the same
explicit human go-ahead #124 is waiting on, not a unilateral install by an
agent.
---
## Files
```
mood/
├── README.md
├── schema.sql
├── Dockerfile
├── docker-compose.yml
├── requirements.txt
├── .env.example
├── .gitignore
├── src/
│ ├── config.py
│ ├── mood_source.py # read-only reader for moodtracker's SQLite file
│ ├── store.py # schema, upserts, reports, correlation hook
│ ├── sync.py # cursor-based incremental sync
│ └── cli.py # init-db | sync | query
└── tests/
├── test_store.py
└── test_sync.py # drives run_sync against a mock moodtracker DB
```
Nothing in this directory is committed to git — `agap_git` is a git repo but
no `git add`/`git commit` was run for this task.

21
mood/docker-compose.yml Normal file
View File

@@ -0,0 +1,21 @@
services:
mood-archive:
build: .
container_name: mood-archive
restart: unless-stopped
environment:
TZ: Europe/Moscow
MOOD_DB_PATH: /data/mood_archive.sqlite
MOOD_SOURCE_DB_PATH: /source/moodtracker/mood.db
# Manual-entry data; hourly polling is more than enough (local file read).
MOOD_SYNC_INTERVAL_SECONDS: ${MOOD_SYNC_INTERVAL_SECONDS:-3600}
MOOD_OVERLAP_ROWS: ${MOOD_OVERLAP_ROWS:-3}
volumes:
# Local mood archive lives alongside the other Agap databases.
- /mnt/dbs/mood:/data
# moodtracker's own SQLite file, READ-ONLY — no credentials, no HTTP,
# no risk of this service ever writing into the live app's DB.
- /home/alvis/moodtracker/data:/source/moodtracker:ro
logging:
options:
max-size: 10m

4
mood/requirements.txt Normal file
View File

@@ -0,0 +1,4 @@
# Runtime (sync + store + query CLI): stdlib only — nothing required here.
#
# Dev only:
pytest>=8.0 # tests/

57
mood/schema.sql Normal file
View File

@@ -0,0 +1,57 @@
-- Mood archive — SQLite schema.
--
-- mood.alogins.net (container `moodtracker`, /home/alvis/moodtracker) is a small
-- self-built Flask app with its own SQLite DB. It has no token-based API — its
-- only auth is a session-cookie login (/login with AUTH_USER/AUTH_PASS) guarding
-- /api/log, /api/history, /api/entry/<id>. Its DB file, however, is directly
-- readable on the host (world-readable, 644) at
-- /home/alvis/moodtracker/data/mood.db. This archiver reads that file straight
-- (via a read-only bind mount) instead of scraping the HTTP session API — no
-- credential handling needed, and it is immune to any future change in the
-- moodtracker app's auth scheme.
--
-- Why SQLite (not InfluxDB): single-user, few-entries-per-day mood logging is
-- tiny volume; Agap storage doctrine is SQLite-first (see googlefit/schema.sql
-- for the same reasoning). Mirrors that service's shape: idempotent upserts,
-- a sync cursor, an ingest-run audit log.
PRAGMA journal_mode = WAL;
-- One row per moodtracker entry. PK is the *source* row id (moodtracker's own
-- autoincrement id) + source name, so re-syncing never duplicates and a future
-- second mood source (were one ever added) can't collide ids with this one.
CREATE TABLE IF NOT EXISTS mood_entries (
source TEXT NOT NULL DEFAULT 'moodtracker',
source_id INTEGER NOT NULL, -- moodtracker entries.id
ts TEXT NOT NULL, -- ISO8601 UTC, as recorded by moodtracker
mood INTEGER NOT NULL, -- 1-5 scale used by moodtracker
tags TEXT NOT NULL DEFAULT '[]', -- JSON array of tag strings
note TEXT,
affirmation TEXT,
ingested_at TEXT NOT NULL,
PRIMARY KEY (source, source_id)
);
CREATE INDEX IF NOT EXISTS idx_mood_ts ON mood_entries (ts);
CREATE INDEX IF NOT EXISTS idx_mood_mood ON mood_entries (mood);
-- Incremental-sync cursor per stream (one stream today: 'moodtracker_entries').
-- last_synced_id is the high-water mark on source_id; each run re-checks a
-- small overlap of already-synced ids too (cheap, guards against any future
-- edit capability moodtracker doesn't have today).
CREATE TABLE IF NOT EXISTS sync_state (
stream_key TEXT PRIMARY KEY,
last_synced_id INTEGER NOT NULL DEFAULT 0,
last_run_at TEXT,
last_status TEXT, -- ok | error
last_error TEXT
);
-- Audit log of ingestion runs (observability; Zabbix can read staleness later).
CREATE TABLE IF NOT EXISTS ingest_runs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
started_at TEXT NOT NULL,
finished_at TEXT,
status TEXT, -- ok | error
entries_upserted INTEGER DEFAULT 0,
error TEXT
);

0
mood/src/__init__.py Normal file
View File

109
mood/src/cli.py Normal file
View File

@@ -0,0 +1,109 @@
"""mood CLI — single entrypoint for ingestion and reads.
Commands:
init-db create the SQLite schema
sync [--once] pull from moodtracker's SQLite; loop unless --once
query summary counts, coverage, freshness, last run
query entries [--days N] recent raw entries
query daily [--days N] average mood + entry count per day
query tags [--days N] average mood per tag (simple correlation)
query correlate <csv> [--days N] Pearson r between daily mood and an external
day,value CSV (generic cross-source hook —
see store.correlate_with_series docstring)
The `query` commands are the read tool for Adolf: JSON on stdout, no creds needed.
"""
import argparse
import csv
import json
import sys
from . import config, store
from .sync import run_sync, sync_loop
def _conn():
return store.connect(config.DB_PATH)
def cmd_init_db(_):
conn = _conn()
store.init_db(conn)
print(f"Initialized schema at {config.DB_PATH}")
def cmd_sync(args):
conn = _conn()
store.init_db(conn)
exit_code = 0
for totals in sync_loop(conn, once=args.once):
print(json.dumps({"synced": totals}))
if totals["errors"]:
print("WARN: " + " | ".join(totals["errors"]), file=sys.stderr)
exit_code = 1
return exit_code
def cmd_query(args):
conn = _conn()
store.init_db(conn)
if args.what == "summary":
out = store.summary(conn)
elif args.what == "entries":
out = store.recent_entries(conn, days=args.days)
elif args.what == "daily":
out = store.daily_mood(conn, days=args.days)
elif args.what == "tags":
out = store.tag_breakdown(conn, days=args.days)
elif args.what == "correlate":
external = {}
with open(args.csv_path, newline="") as f:
for row in csv.reader(f):
if len(row) < 2 or row[0].lower() == "day":
continue
try:
external[row[0].strip()] = float(row[1])
except ValueError:
continue
out = store.correlate_with_series(conn, external, days=args.days)
else:
print(f"unknown query: {args.what}", file=sys.stderr)
return 2
print(json.dumps(out, indent=2, ensure_ascii=False))
return 0
def build_parser():
p = argparse.ArgumentParser(prog="mood")
sub = p.add_subparsers(dest="cmd", required=True)
sub.add_parser("init-db").set_defaults(func=cmd_init_db)
ps = sub.add_parser("sync")
ps.add_argument("--once", action="store_true", help="run one pass and exit")
ps.set_defaults(func=cmd_sync)
pq = sub.add_parser("query")
pqs = pq.add_subparsers(dest="what", required=True)
pqs.add_parser("summary")
pe = pqs.add_parser("entries")
pe.add_argument("--days", type=int, default=30)
pd = pqs.add_parser("daily")
pd.add_argument("--days", type=int, default=30)
pt = pqs.add_parser("tags")
pt.add_argument("--days", type=int, default=90)
pc = pqs.add_parser("correlate")
pc.add_argument("csv_path", help="CSV with 'day,value' rows (day=YYYY-MM-DD)")
pc.add_argument("--days", type=int, default=90)
pq.set_defaults(func=cmd_query)
return p
def main(argv=None):
args = build_parser().parse_args(argv)
return args.func(args) or 0
if __name__ == "__main__":
raise SystemExit(main())

23
mood/src/config.py Normal file
View File

@@ -0,0 +1,23 @@
"""Configuration for the mood archiver.
No credentials required: the source is a directly-readable SQLite file
(mood.alogins.net / container `moodtracker`), bind-mounted read-only into this
container. There is nothing to fetch from Vaultwarden for this service.
"""
import os
# Our own local archive.
DB_PATH = os.environ.get("MOOD_DB_PATH", "/data/mood_archive.sqlite")
# moodtracker's SQLite file, read-only bind mount (see docker-compose.yml).
SOURCE_DB_PATH = os.environ.get("MOOD_SOURCE_DB_PATH", "/source/moodtracker/mood.db")
# Re-check this many already-synced ids on every run (cheap safety net in case
# moodtracker ever grows an edit capability; it currently only supports
# insert + delete, so this is mostly a no-op today).
OVERLAP_ROWS = int(os.environ.get("MOOD_OVERLAP_ROWS", "3"))
# Seconds between automatic sync cycles when run as a long-lived service.
# Mood entries are logged manually a few times a week at most; hourly is far
# more than enough and the read is essentially free (local SQLite file).
SYNC_INTERVAL_SECONDS = int(os.environ.get("MOOD_SYNC_INTERVAL_SECONDS", "3600"))

34
mood/src/mood_source.py Normal file
View File

@@ -0,0 +1,34 @@
"""Reader for moodtracker's own SQLite file (the mood.alogins.net source DB).
moodtracker's schema (see /home/alvis/moodtracker/app.py):
CREATE TABLE entries (
id INTEGER PRIMARY KEY AUTOINCREMENT,
ts TEXT NOT NULL,
mood INTEGER NOT NULL,
tags TEXT NOT NULL, -- JSON array, e.g. '["sad","tired"]'
note TEXT,
affirmation TEXT
)
We open it read-only (URI mode=ro) so this archiver can never corrupt or lock
the live app's database.
"""
import sqlite3
def connect_source(path):
"""Read-only connection to the moodtracker SQLite file."""
conn = sqlite3.connect(f"file:{path}?mode=ro", uri=True)
conn.row_factory = sqlite3.Row
return conn
def fetch_entries_since(source_conn, since_id=0, limit=100000):
"""Entries with id > since_id, oldest first."""
rows = source_conn.execute(
"""SELECT id, ts, mood, tags, note, affirmation
FROM entries WHERE id > ? ORDER BY id ASC LIMIT ?""",
(since_id, limit),
).fetchall()
return [dict(r) for r in rows]

202
mood/src/store.py Normal file
View File

@@ -0,0 +1,202 @@
"""SQLite storage layer: schema init, idempotent upserts, and read queries.
All writes are UPSERTs keyed on (source, source_id), so re-running a sync over
an overlapping id range is a no-op rather than a duplicate. Reads back the
Adolf query CLI (`cli.py query ...`).
"""
import os
import sqlite3
from datetime import datetime, timezone
SCHEMA_PATH = os.path.join(os.path.dirname(os.path.dirname(__file__)), "schema.sql")
def _now_iso():
return datetime.now(tz=timezone.utc).isoformat()
def connect(db_path):
os.makedirs(os.path.dirname(os.path.abspath(db_path)), exist_ok=True)
conn = sqlite3.connect(db_path)
conn.row_factory = sqlite3.Row
return conn
def init_db(conn):
with open(SCHEMA_PATH) as f:
conn.executescript(f.read())
conn.commit()
# --- writes ---------------------------------------------------------------
def upsert_entries(conn, rows, source="moodtracker"):
now = _now_iso()
n = 0
for r in rows:
conn.execute(
"""
INSERT INTO mood_entries
(source, source_id, ts, mood, tags, note, affirmation, ingested_at)
VALUES (?,?,?,?,?,?,?,?)
ON CONFLICT(source, source_id) DO UPDATE SET
ts=excluded.ts,
mood=excluded.mood,
tags=excluded.tags,
note=excluded.note,
affirmation=excluded.affirmation,
ingested_at=excluded.ingested_at
""",
(
source, r["id"], r["ts"], r["mood"], r.get("tags", "[]"),
r.get("note"), r.get("affirmation"), now,
),
)
n += 1
conn.commit()
return n
# --- sync cursor & run audit ---------------------------------------------
def get_last_synced_id(conn, stream_key):
row = conn.execute(
"SELECT last_synced_id FROM sync_state WHERE stream_key=?", (stream_key,)
).fetchone()
return row["last_synced_id"] if row else 0
def set_sync_state(conn, stream_key, last_synced_id, status="ok", error=None):
conn.execute(
"""
INSERT INTO sync_state (stream_key, last_synced_id, last_run_at, last_status, last_error)
VALUES (?,?,?,?,?)
ON CONFLICT(stream_key) DO UPDATE SET
last_synced_id=MAX(sync_state.last_synced_id, excluded.last_synced_id),
last_run_at=excluded.last_run_at,
last_status=excluded.last_status,
last_error=excluded.last_error
""",
(stream_key, last_synced_id, _now_iso(), status, error),
)
conn.commit()
def start_run(conn):
cur = conn.execute(
"INSERT INTO ingest_runs (started_at, status) VALUES (?, 'running')", (_now_iso(),)
)
conn.commit()
return cur.lastrowid
def finish_run(conn, run_id, status, entries=0, error=None):
conn.execute(
"""UPDATE ingest_runs SET finished_at=?, status=?, entries_upserted=?, error=?
WHERE id=?""",
(_now_iso(), status, entries, error, run_id),
)
conn.commit()
# --- reads (Adolf query tool + reports) -----------------------------------
def summary(conn):
"""Compact snapshot: count, coverage, freshness, last run."""
out = {}
out["entries"] = conn.execute("SELECT COUNT(*) c FROM mood_entries").fetchone()["c"]
span = conn.execute("SELECT MIN(ts) a, MAX(ts) b FROM mood_entries").fetchone()
out["coverage"] = {"earliest": span["a"], "latest": span["b"]}
out["avg_mood_all_time"] = conn.execute(
"SELECT ROUND(AVG(mood), 2) a FROM mood_entries"
).fetchone()["a"]
out["sync_state"] = [dict(r) for r in conn.execute(
"SELECT stream_key, last_synced_id, last_run_at, last_status FROM sync_state"
).fetchall()]
last = conn.execute(
"SELECT started_at, finished_at, status, entries_upserted FROM ingest_runs "
"ORDER BY id DESC LIMIT 1"
).fetchone()
out["last_run"] = dict(last) if last else None
return out
def recent_entries(conn, days=30, limit=200):
return [dict(r) for r in conn.execute(
"""SELECT source_id, ts, mood, tags, note, affirmation
FROM mood_entries WHERE ts >= datetime('now', ?)
ORDER BY ts DESC LIMIT ?""",
(f"-{int(days)} days", int(limit)),
).fetchall()]
def daily_mood(conn, days=30):
"""Average mood and entry count per calendar day (report #1: mood over time)."""
return [dict(r) for r in conn.execute(
"""SELECT date(ts) AS day, ROUND(AVG(mood), 2) AS avg_mood, COUNT(*) AS entries
FROM mood_entries WHERE ts >= datetime('now', ?)
GROUP BY day ORDER BY day DESC""",
(f"-{int(days)} days",),
).fetchall()]
def tag_breakdown(conn, days=90, min_count=2):
"""Average mood per tag (simple correlation: which tags co-occur with
higher/lower mood). Tags are stored as a JSON array per entry; this
unpacks them in Python since SQLite has no native JSON array explode
without the (not always compiled-in) json1 table-valued functions."""
import json
rows = conn.execute(
"SELECT mood, tags FROM mood_entries WHERE ts >= datetime('now', ?)",
(f"-{int(days)} days",),
).fetchall()
by_tag = {}
for r in rows:
try:
tags = json.loads(r["tags"]) or []
except (TypeError, ValueError):
tags = []
for t in tags:
by_tag.setdefault(t, []).append(r["mood"])
out = [
{"tag": t, "avg_mood": round(sum(v) / len(v), 2), "count": len(v)}
for t, v in by_tag.items()
if len(v) >= min_count
]
out.sort(key=lambda x: x["avg_mood"])
return out
# --- correlation hook (generic, no other Agap service wired) --------------
def correlate_with_series(conn, external_daily, days=90):
"""Pearson correlation between daily average mood and an arbitrary
externally-supplied daily series.
`external_daily` is a dict {'YYYY-MM-DD': float}. This is a deliberate
seam for future cross-source correlation (e.g. googlefit sleep/steps) —
it takes plain data, not a live connection to another service's DB, so
wiring a second source later is a one-line change at the call site
(build the dict from that source's own query CLI) and never requires
this service to know about the other service's schema or container.
Returns {'n': overlap_days, 'r': pearson_r_or_None, 'points': [...]}."""
mood_by_day = {
r["day"]: r["avg_mood"] for r in daily_mood(conn, days=days)
}
common_days = sorted(set(mood_by_day) & set(external_daily))
xs = [mood_by_day[d] for d in common_days]
ys = [external_daily[d] for d in common_days]
n = len(xs)
if n < 2:
return {"n": n, "r": None, "points": list(zip(common_days, xs, ys))}
mx, my = sum(xs) / n, sum(ys) / n
cov = sum((x - mx) * (y - my) for x, y in zip(xs, ys))
varx = sum((x - mx) ** 2 for x in xs)
vary = sum((y - my) ** 2 for y in ys)
r = cov / (varx ** 0.5 * vary ** 0.5) if varx > 0 and vary > 0 else None
return {
"n": n,
"r": round(r, 3) if r is not None else None,
"points": [{"day": d, "mood": x, "value": y} for d, x, y in zip(common_days, xs, ys)],
}

50
mood/src/sync.py Normal file
View File

@@ -0,0 +1,50 @@
"""Sync orchestration: read moodtracker's SQLite file -> idempotent upsert.
Single stream ('moodtracker_entries'), cursor = highest source_id ingested so
far. Each run re-checks a small overlap of already-synced ids (config.OVERLAP_ROWS)
as a cheap safety net, then upserts anything with id > cursor - overlap.
"""
import time
from . import config, store
from .mood_source import connect_source, fetch_entries_since
STREAM_KEY = "moodtracker_entries"
def run_sync(conn, source_db_path=None):
"""One sync pass. Returns a counts dict. Never raises — errors are
recorded in ingest_runs/sync_state and returned in totals['errors']."""
source_db_path = source_db_path or config.SOURCE_DB_PATH
run_id = store.start_run(conn)
totals = {"entries": 0, "errors": []}
try:
last = store.get_last_synced_id(conn, STREAM_KEY)
since = max(0, last - config.OVERLAP_ROWS)
source_conn = connect_source(source_db_path)
try:
rows = fetch_entries_since(source_conn, since)
finally:
source_conn.close()
n = store.upsert_entries(conn, rows)
totals["entries"] = n
max_id = max((r["id"] for r in rows), default=last)
store.set_sync_state(conn, STREAM_KEY, max(max_id, last))
store.finish_run(conn, run_id, "ok", entries=n)
except Exception as e: # noqa: BLE001 - isolate failures, keep the loop alive
totals["errors"].append(str(e))
store.set_sync_state(
conn, STREAM_KEY, store.get_last_synced_id(conn, STREAM_KEY),
status="error", error=str(e),
)
store.finish_run(conn, run_id, "error", entries=0, error=str(e))
return totals
def sync_loop(conn, source_db_path=None, interval=None, once=False):
interval = interval or config.SYNC_INTERVAL_SECONDS
while True:
yield run_sync(conn, source_db_path)
if once:
return
time.sleep(interval)

0
mood/tests/__init__.py Normal file
View File

112
mood/tests/test_store.py Normal file
View File

@@ -0,0 +1,112 @@
import json
import os
import sys
import tempfile
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from src import store
def _fresh_db():
path = os.path.join(tempfile.mkdtemp(), "t.sqlite")
conn = store.connect(path)
store.init_db(conn)
return conn
def _row(id_, ts, mood, tags, note="", affirmation=""):
return {"id": id_, "ts": ts, "mood": mood, "tags": json.dumps(tags),
"note": note, "affirmation": affirmation}
def test_upsert_idempotent():
conn = _fresh_db()
rows = [
_row(1, "2026-07-01T08:00:00+00:00", 4, ["calm"]),
_row(2, "2026-07-02T08:00:00+00:00", 2, ["sad", "tired"]),
]
store.upsert_entries(conn, rows)
store.upsert_entries(conn, rows) # re-run same rows
count = conn.execute("SELECT COUNT(*) c FROM mood_entries").fetchone()["c"]
assert count == 2 # no duplication despite double ingest
def test_upsert_updates_on_conflict():
conn = _fresh_db()
rows = [_row(1, "2026-07-01T08:00:00+00:00", 4, ["calm"], note="first")]
store.upsert_entries(conn, rows)
rows[0]["note"] = "corrected"
store.upsert_entries(conn, rows)
got = conn.execute("SELECT note FROM mood_entries WHERE source_id=1").fetchone()["note"]
assert got == "corrected"
def test_sync_state_high_water_mark():
conn = _fresh_db()
store.set_sync_state(conn, "moodtracker_entries", 5)
store.set_sync_state(conn, "moodtracker_entries", 3) # older cursor must not regress
assert store.get_last_synced_id(conn, "moodtracker_entries") == 5
store.set_sync_state(conn, "moodtracker_entries", 9)
assert store.get_last_synced_id(conn, "moodtracker_entries") == 9
def test_summary_and_daily_mood():
conn = _fresh_db()
rows = [
_row(1, "2026-07-01T08:00:00+00:00", 4, ["calm"]),
_row(2, "2026-07-01T20:00:00+00:00", 2, ["tired"]),
_row(3, "2026-07-02T08:00:00+00:00", 5, ["happy"]),
]
store.upsert_entries(conn, rows)
s = store.summary(conn)
assert s["entries"] == 3
assert s["coverage"]["earliest"] is not None
daily = store.daily_mood(conn, days=30)
by_day = {d["day"]: d for d in daily}
assert by_day["2026-07-01"]["entries"] == 2
assert by_day["2026-07-01"]["avg_mood"] == 3.0 # (4+2)/2
assert by_day["2026-07-02"]["avg_mood"] == 5.0
def test_tag_breakdown():
conn = _fresh_db()
rows = [
_row(1, "2026-07-01T08:00:00+00:00", 5, ["happy", "energetic"]),
_row(2, "2026-07-02T08:00:00+00:00", 1, ["sad", "tired"]),
_row(3, "2026-07-03T08:00:00+00:00", 2, ["tired"]),
]
store.upsert_entries(conn, rows)
tags = store.tag_breakdown(conn, days=90, min_count=2)
by_tag = {t["tag"]: t for t in tags}
assert by_tag["tired"]["count"] == 2
assert by_tag["tired"]["avg_mood"] == 1.5
assert "happy" not in by_tag # min_count=2 filters singletons
def test_correlate_with_series():
conn = _fresh_db()
rows = [
_row(1, "2026-07-01T08:00:00+00:00", 5, []),
_row(2, "2026-07-02T08:00:00+00:00", 4, []),
_row(3, "2026-07-03T08:00:00+00:00", 2, []),
_row(4, "2026-07-04T08:00:00+00:00", 1, []),
]
store.upsert_entries(conn, rows)
# perfectly correlated external series (e.g. "hours slept")
external = {
"2026-07-01": 8.0, "2026-07-02": 7.0,
"2026-07-03": 5.0, "2026-07-04": 4.0,
}
result = store.correlate_with_series(conn, external, days=30)
assert result["n"] == 4
assert result["r"] > 0.99 # near-perfect positive correlation
def test_correlate_too_few_points():
conn = _fresh_db()
store.upsert_entries(conn, [_row(1, "2026-07-01T08:00:00+00:00", 3, [])])
result = store.correlate_with_series(conn, {"2026-07-01": 5.0}, days=30)
assert result["n"] == 1
assert result["r"] is None

108
mood/tests/test_sync.py Normal file
View File

@@ -0,0 +1,108 @@
import json
import os
import sqlite3
import sys
import tempfile
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from src import store
from src.sync import run_sync
def _mock_moodtracker_db(entries):
"""Build a SQLite file with moodtracker's exact `entries` schema
(see /home/alvis/moodtracker/app.py init_db) and seed rows."""
path = os.path.join(tempfile.mkdtemp(), "mood.db")
conn = sqlite3.connect(path)
conn.execute("""
CREATE TABLE entries (
id INTEGER PRIMARY KEY AUTOINCREMENT,
ts TEXT NOT NULL,
mood INTEGER NOT NULL,
tags TEXT NOT NULL,
note TEXT,
affirmation TEXT
)
""")
for ts, mood, tags, note, aff in entries:
conn.execute(
"INSERT INTO entries (ts, mood, tags, note, affirmation) VALUES (?,?,?,?,?)",
(ts, mood, json.dumps(tags), note, aff),
)
conn.commit()
conn.close()
return path
def _archive_conn():
path = os.path.join(tempfile.mkdtemp(), "archive.sqlite")
conn = store.connect(path)
store.init_db(conn)
return conn
def test_sync_pulls_all_rows_first_run():
source = _mock_moodtracker_db([
("2026-07-01T08:00:00+00:00", 4, ["calm"], "n1", "a1"),
("2026-07-02T08:00:00+00:00", 2, ["sad", "tired"], "n2", ""),
])
conn = _archive_conn()
totals = run_sync(conn, source_db_path=source)
assert totals["errors"] == []
assert totals["entries"] == 2
assert conn.execute("SELECT COUNT(*) c FROM mood_entries").fetchone()["c"] == 2
def test_sync_is_idempotent_across_runs():
source = _mock_moodtracker_db([
("2026-07-01T08:00:00+00:00", 4, ["calm"], "", ""),
])
conn = _archive_conn()
run_sync(conn, source_db_path=source)
run_sync(conn, source_db_path=source) # nothing new, cursor unchanged
run_sync(conn, source_db_path=source)
assert conn.execute("SELECT COUNT(*) c FROM mood_entries").fetchone()["c"] == 1
def test_sync_picks_up_new_rows_incrementally():
path = _mock_moodtracker_db([
("2026-07-01T08:00:00+00:00", 4, ["calm"], "", ""),
])
conn = _archive_conn()
run_sync(conn, source_db_path=path)
assert conn.execute("SELECT COUNT(*) c FROM mood_entries").fetchone()["c"] == 1
# a new entry gets logged upstream between syncs
src_conn = sqlite3.connect(path)
src_conn.execute(
"INSERT INTO entries (ts, mood, tags, note, affirmation) VALUES (?,?,?,?,?)",
("2026-07-03T09:00:00+00:00", 5, json.dumps(["happy"]), "", ""),
)
src_conn.commit()
src_conn.close()
run_sync(conn, source_db_path=path)
assert conn.execute("SELECT COUNT(*) c FROM mood_entries").fetchone()["c"] == 2
def test_sync_records_error_when_source_missing():
conn = _archive_conn()
totals = run_sync(conn, source_db_path="/nonexistent/path/mood.db")
assert totals["errors"]
row = conn.execute(
"SELECT last_status, last_error FROM sync_state WHERE stream_key='moodtracker_entries'"
).fetchone()
assert row["last_status"] == "error"
assert row["last_error"]
def test_sync_never_writes_to_source_db():
"""The source connection is opened read-only; a failed write attempt
would raise, and run_sync should never attempt one in the first place."""
source = _mock_moodtracker_db([("2026-07-01T08:00:00+00:00", 3, [], "", "")])
before = os.path.getmtime(source)
conn = _archive_conn()
run_sync(conn, source_db_path=source)
after = os.path.getmtime(source)
assert before == after