focus-area now recomputes at most once per hour, and only if the task list
actually changed since the last compute.
- focus-area TTL: 43200s → 3600s; version bumped to 2.1.0
- computeAndStore hashes sorted task contents (MD5) and checks the stored
_task_hash in the existing snapshot; skips the ml-serving call when the
hash matches and the output isn't expired
- ml-serving injects _task_hash into the snapshot so the next cycle can compare
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each unique task title is now enriched by LiteLLM once and cached in the DB.
Subsequent agent compute cycles (every 12h) fetch the cache before calling
ml-serving; only new titles hit the tip-generator.
- DB: task_enrichments(content_hash PK, description, model, created_at)
- TS: fetchEnrichmentCache / persistEnrichments helpers in agent-outputs.ts;
enrichment_cache passed in compute request, new_enrichments persisted from response
- Python: AgentComputeRequest.enrichment_cache / AgentComputeResponse.new_enrichments;
AgentInput.enrichment_cache; _enrich_batch returns (descriptions, new_entries);
cluster_tasks returns (clusters, new_enrichments)
- FocusAreaAgent stashes new_enrichments in signals_snapshot under _new_enrichments;
compute_agent endpoint pops it before storing the snapshot
Closes part of #129
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Ported from taskpile experiments/clustering_eval (prompt v1, qwen2.5:1.5b).
The experiment showed ARI 0.22→0.77 and AUROC 0.76→0.91 on synthetic tasks
when embedding LLM-expanded descriptions instead of raw titles.
- Expand each task title via LiteLLM tip-generator before embedding
- Prefix with "clustering: " (nomic-embed-text task instruction prefix)
- Cache expansions in-memory by content hash within a compute cycle
- Falls back to raw title if enrichment fails; no change to fallback behaviour
Fixes#129
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The old code called Ollama's /api/embeddings one task at a time, which caused
silent fallback to project-based grouping when host.docker.internal:11434 was
unreachable from the ml-serving container.
- Switch to LiteLLM /embeddings (model alias "embedder") as primary path
- Batch all task contents in one request instead of N serial calls
- Fall back to Ollama /api/embed (updated to current API) when LITELLM_URL is absent
- Update tests to mock _embed_batch instead of the removed _embed
Fixes#123
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When fetchOrchestratorTip returned null (LiteLLM timeout, bad JSON, etc.)
the recommender silently fell back to randomPolicy, serving a raw Todoist
task with no rationale — explaining both reported symptoms.
- Remove randomPolicy/signalToCandidate; return 204 when orchestrator fails
so the UI shows "All clear" instead of a confusing Todoist task
- Pass recent_tip through the stack (frontend → POST /recommend →
fetchOrchestratorTip → ml/serving RecommendRequest → build_orchestrator_messages)
so after snooze the LLM is instructed not to repeat the snoozed content
Fixes#122
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The middleware was redirecting non-admins to /forbidden but /forbidden
wasn't excluded from the matcher, so the middleware ran again on that
page, saw a non-admin, and redirected again — infinite loop. Added
/forbidden to the pass-through list alongside /login.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
OAuth2 flow with all 11 Google Fitness scopes (activity, body, sleep,
heart rate, nutrition, location, blood glucose/pressure/temperature,
oxygen saturation, reproductive health). Stores access + refresh tokens;
auto-refreshes on expiry.
GoogleHealthSignalSource fetches steps, sleep sessions, active minutes,
calories, and heart rate from the Fit aggregate + sessions APIs. Signals
flow into both the tip orchestrator and the health-vitals pre-compute
agent, which generates prompt snippets about step progress, sleep
deficit, sedentary time, and elevated heart rate.
Signal.kind extended with 'health'; IntegrationProvider extended with
'google-health'. Agent compute signal mapping enriched to include source,
kind, and all features so health-vitals can filter its own signals.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Convert ml-serving from isolated MLflow runs to nested traces using
mlflow.start_span_no_context(). The recommend endpoint now emits a full
span tree: recommend (CHAIN) → build_context (TOOL), agent:* (AGENT) ×N,
llm_orchestrator (LLM). Compute and infer endpoints each emit a single span.
Supporting changes:
- mlflow-skinny>=3.1.0 added to requirements
- MLflow configured with --serve-artifacts + mlflow-artifacts:/ default root
for cross-container artifact proxy (spans now persist from ml-serving)
- --allowed-hosts extended to include mlflow:5000 (SDK includes port in Host)
- science_destiny slider wired through prompts.py and recommend endpoint
- Config page exposes science/destiny slider (0=data-driven, 100=intuitive)
- Tip page shows rationale inline on tap
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Small models (qwen2.5:1.5b) mirror the language of task title content
in the prompt. Adding an explicit English note to snippets that embed
raw task titles (focus-area, overdue-task) prevents language bleed.
Also added the instruction to the orchestrator system prompt and user
message as belt-and-suspenders.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Pass MLFLOW_ADMIN_PASSWORD as fallback password credential
- Set host_header='localhost' to satisfy MLflow's --allowed-hosts check
(MLflow rejects Host: mlflow but accepts Host: localhost)
- Default MLFLOW_TRACKING_URI to http://mlflow:5000 in compose so the
env_file value is not silently overridden to empty
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Logs one MLflow run per /recommend (params, token metrics, latency,
full prompt + tip as artifacts) and per /agents/{id}/compute and
/infer call (signals snapshot, inferred prefs, latency).
Tracing is a no-op when MLFLOW_TRACKING_URI is unset; ml-serving
starts and serves tips correctly without MLflow configured.
Refs #118 (M4: remove from production / move off critical path).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds invalidated_by: tuple[str, ...] to ProfileFeature, mirroring the
invalidatedBy bus subjects from registry.ts. Adds a test that parses the
TS source and asserts Python stays in sync — same drift-detection pattern
used for names and ttlSec.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- focus-area v2.0.0 completion in recent completions; remove from active work
- Update focus-area inferred params table row
- min_history gotcha: checked against events, not task_completions
- httpx trust_env=False rule for ml/ code
- Agent test command
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Inference framework table updated: all agents at v1.2.0 with full param list
- Documents UserHistory.task_completions and AgentInferRequest.task_completions
- Marks #112/114/115/116 complete in recent completions
- Active work updated: #78 closed, #61 and #97/#113 as next priorities
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds four InferredParams (all TTL=24h, min_history=50 except preferred_hour=10):
- quiet_start / quiet_end: longest contiguous below-baseline hour run (HH:MM)
- peak_hours: top-quartile done-event hours, sorted ascending
- tz: cold-start only ("UTC"); populated from auth provider, no inference function
compute() updated:
- in_quiet check (quiet window) takes precedence over peak hours
- in_peak emits "peak productivity hour" language when current hour is in peak_hours
- approaching peak (within 2h) surfaces for orchestrator timing
- tz surfaced in snippet header when not UTC
- snapshot adds peak_hours, in_quiet, in_peak, tz
- Agent bumped to v1.2.0
- 21 new tests: night-owl, early-bird, shift-worker, quiet/peak snippet rendering
- Fixed test_snapshot_keys in test_agents.py to include new snapshot fields
Closes#112
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces the coarse density-bucket window_days with three InferredParams (all TTL=24h):
- lookback_days: min window containing ≥30 done events, capped at 30d (min_history=5)
- weekly_cycle: per-DOW peak-to-mean strength list (min_history=21, ≥3 weeks of signal)
- daily_cycle: per-hour peak-to-mean strength list (min_history=14)
compute() renders cycle hints when strength > 0.5:
"User tends to complete tips on Tuesdays and Saturdays."
"User is most active around 8pm."
Legacy window_days pref key still accepted as a fallback.
- window_days pref renamed lookback_days; backward-compat fallback in compute()
- Agent bumped to v1.2.0
- 19 new tests: weekend-warrior, weekday-only, evening-person, no-pattern,
legacy compat, snippet rendering with strong/weak signals
Closes#116
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds two InferredParams (TTL=7d) computed from 28-day rolling daily done counts:
- baseline_completions_per_day: mean done events/day over the window
- stdev: stdev of daily counts (floored at 0.1 to avoid division by zero)
MomentumAgent.compute() now calculates a z-score from recent done events in
inp.feedback_history vs the inferred baseline. Snippet language switches to
z-score framing ("above your usual pace", "slowing down") when |z| >= 1.0,
falling back to engagement_trend labels when in the normal range.
- engagement_trend InferredParam preserved for backward compatibility
- momentum_window pref added (default 7, user-overridable)
- 14 new tests covering power user, casual user, returning-from-break, and
relative stdev comparison; engagement_trend tests updated for z-score priority
- Agent bumped to v1.2.0
Closes#114
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces snooze-rate heuristic with p50 of actual task lateness (completedAt − dueAt).
Adds project_realness inference: projects with chronic lateness get realness < 1 and
the agent softens its snippet language from "overdue" to "past target date".
- TaskCompletion added to UserHistory with lateness_days computed property
- _infer_lateness_tolerance: p50 of task_completions, clipped at 0, float
- _infer_project_realness: per-project median lateness normalised by global median
- Both InferredParams use 7d TTL; cold_start = 0.0 / {}
- AgentInferRequest accepts task_completions; endpoint wires them through
- 12 new tests covering punctual/chronic/mixed users and language softening
- Agent bumped to v1.2.0
Closes#115
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Backfills consent_given=1 rows into user_consents as data:core before
dropping the legacy columns. auth.ts now writes user_consents on signup;
POST /consent writes user_consents; admin/user routes cleaned of the old
fields. Migration is idempotent — DROP COLUMN is wrapped in try/catch so
it no-ops on fresh DBs that never had the columns.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
All four agents bumped to v1.1.0.
momentum (#114): infers engagement_trend ('up'|'stable'|'down') by comparing
done-rate in the last 7 days vs the prior 7 days. Agent surfaces the trend
in its snippet ("trending up — build on the momentum").
overdue-task (#115): infers lateness_tolerance_days (0/1/2) from snooze rate.
Agent now filters tasks against the tolerance so low-urgency users aren't
nagged about tasks that are only hours overdue.
recent-patterns (#116): infers window_days (7/14/30) from feedback event
density — sparse users get a wider window so the snippet isn't always empty.
focus-area (#113): no inferred params (project-level feedback linkage needed,
tracked under #78). preferred_areas pref was declared but ignored; agent now
honours it as a tiebreaker and mentions it in the snippet.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each agent now exports a module-level MANIFEST declaring id, version,
pref_schema, required_consents, ttl_sec, and silenced_in_contexts. The
registry surfaces both the agent and its manifest, and rejects on
mismatch so the two cannot drift.
ml/serving exposes GET /agents/registry; services/api proxies it as
GET /api/agents/registry with a 60s in-process cache so admin pageviews
don't hammer upstream. Failures aren't cached.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds user_preferences, user_consents, user_contexts and the tone /
tip_kinds_json columns on users. Backfills consent_given=1 rows into
user_consents as data:core; INSERT OR IGNORE keeps it idempotent and
respects later revocations.
Migration body moves to db/migrations.ts so tests can apply it to a
fresh in-memory handle without opening the prod DB on import.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Propose a shared substrate for per-user prefs, contexts, per-key
consents, and per-agent state so adding an agent stays a manifest
change. Updates CLAUDE.md, README, and architecture docs to reflect
the multi-agent pipeline (ADR-0013) and the registry direction.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Remove unused `httpx` import from bench.ts (package does not exist)
- Add explicit `IRouter` type on `router` in agent-outputs.ts and bench.ts
to resolve TS2742 portable-type errors
- Remove `docs` from .dockerignore so Dockerfile.admin can copy it into
the runner image (DOCS_ROOT=/app/docs is read at runtime by the admin)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Deletes shadowPolicies map, getShadowPolicies, setPolicyActive from
recommender.ts; removes /api/admin/policies routes from admin.ts; removes
getPolicies, togglePolicy, PolicyInfo from admin api.ts; removes the
policy toggle section from the ops page.
168 API tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
POST /recommend now calls ml/serving /recommend with pre-computed agent
snippets + task context instead of /generate + /score/egreedy/v2. Falls
back to a random signal candidate when ml/serving is unavailable.
Removes: remotePolicy, fetchLlmCandidates, sendRewardWithRetry,
candidateCache, pickPromptVersion. Feedback handler keeps inferReward +
tipFeedback writes for observability; reward delivery to the bandit is gone.
tipScores.policy is now 'orchestrator'; promptVersion is 'v4-orchestrator'.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Extracts computeAndStore() from the /agents/:agentId/compute route so it
can be called without an HTTP round-trip. startAgentPrecomputeScheduler()
runs every 15 min: fetches active users (tip view in 48h), runs all agents
in parallel per user, then purges outputs expired >24h. Agent IDs are
resolved from ml/serving /health at startup with a fallback hardcoded list.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ADR-0007 and ADR-0012 both superseded by ADR-0013 as of 2026-05-01.
UsersTable gains a truncated ID column for quick user identification.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds ml/agents/ — five specialised sub-agents (overdue_task, momentum,
time_of_day, recent_patterns, focus_area) each producing a prompt snippet
from user signals. A registry wires them up; the orchestrator prompt in
ml/serving/prompts.py synthesises their outputs into one tip via LiteLLM.
Also wires /api/agents route in the API and updates the Dockerfile to copy
the full ml/ tree with PYTHONPATH=/app so agent imports resolve correctly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Drop all four Airflow containers (db, init, webserver, scheduler) from the
mlops compose profile, leaving MLflow as the sole mlops service. Remove
AIRFLOW_* env vars, config fields, health-check entries, DAG trigger code
in admin/bench routes, the airflow_dag_run_id schema column, Airflow nav
links and DAG-run links in the admin UI, the two Airflow DAG files
(bench_dag.py, sim_dag.py), and all related docs/ADR references.
Simulations now run exclusively via the subprocess path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- simulate/page.tsx: remove launch form — simulations are triggered via
Airflow DAG, not the admin UI. Page now shows run history + links to
Airflow and MLflow only (#109)
- docs.ts: use DOCS_ROOT env var (fallback: ../../docs for local dev) so
the path works in Docker standalone where CWD is /app (#110)
- Dockerfile.admin: copy docs/ into the runner image at /app/docs and set
DOCS_ROOT=/app/docs so listAllDocs() finds the files at runtime (#110)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds a one-line purpose description under the Ops heading so it is clear
what the section is for (shadow policy toggles, signal replay, per-user
actions). Removes the duplicate "User-level actions" subsection whose
content is now covered by the header description.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Remove "Helpful"/"Not helpful" from action sheet — reward is inferred
from done/snooze/dismiss + dwell time; explicit sentiment buttons were
redundant and cluttered the UI (#100)
- Move "notify me" push subscription button to new /config page (#101)
- Add settings gear icon (bottom-right, fixed) on tip page linking to /config (#102)
- New /config page: push notification toggle + link to /connect integrations
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- prompts.py: sort tasks overdue-first → priority desc → age desc before
rendering into the LLM prompt (same ordering as ml/features/context.py)
- prompts.py: render User profile summary line (completion_rate, dismiss_rate,
preferred_hour) when profile_features are present
- main.py: add profile_features field to PromptContext; plumb from
GenerateRequest into the prompt builder via model_copy
- logging_config.py: drop add_logger_name processor (incompatible with
PrintLoggerFactory — caused test ordering failures)
- test_generate.py: 6 new tests covering sort order, profile rendering,
partial fields, empty profile, and end-to-end plumbing through /generate
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New DAG (`ml/pipelines/bench_dag.py`) with three linked tasks:
1. collect.py — generates candidates, logs to MLflow
2. export_for_judge — exports pending runs for Claude Code scoring
3. compare — generates leaderboard by (model, prompt) cell
Config via dag_run.conf supports all collect.py options (models, prompts,
n_tips, n_scenarios, temperature, experiment name, max_model_b).
New admin API endpoints (`services/api/src/routes/bench.ts`):
- GET /api/bench/experiments — list tip-bench-* experiments
- POST /api/bench/run — trigger DAG with custom config
- GET /api/bench/runs/:experiment — list runs in experiment
- GET /api/bench/leaderboard/:experiment — leaderboard by (model, prompt)
All endpoints require admin auth. Human judge (Claude Code) scores are
applied manually post-export; future enhancement: add webhook to DAG.
Admin UI can now trigger and monitor benchmarks from a dashboard panel.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Combines model evaluation (#93) and prompt A/B testing (#95) into one
experiment. Evaluates all (model × prompt × scenario) cells on the same
fixed contexts so quality differences are attributable.
Architecture:
- Phase A (collect.py): generates candidates per cell, logs to MLflow
with judge_pending=true. Rejects models >4B, uses keep_alive=0 for
RAM safety (no concurrent model weights in VRAM).
- Phase B (judge_cli.py): exports pending runs as JSON for Claude Code
to score per the rubric, then applies scores back to MLflow.
- Phase C (compare.py): leaderboard by (model, prompt) cell.
Rubric (tip-v1) defines 1–5 scales for relevance, actionability, tone,
plus format_ok and overlong flags. Composite = rel + act + tone +
2×format_ok − overlong. Rubric is self-describing and persisted in every
run so judges use consistent criteria across sessions.
Artifacts (prompts, candidates, raw responses) stored as MLflow tags
because the server uses a file:// backend not accessible via REST. Full
artifacts accessible in MLflow UI → run → Tags section.
Tested end-to-end on local machine:
- 4 models (qwen2.5:0.5b/1.5b, gemma3:1b, llama3.2:3b) ≤4B
- 3 prompts (v1, v2-mentor, v3-few-shot)
- 4 scenarios (4 personas × 2 time-slots)
- 48 cells total, all judged and ranked
Winner: qwen2.5:1.5b × v3-few-shot (composite=12.75).
Ready for integration into Airflow prompt_ab_eval DAG and admin UI.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- docker-compose: pass ML_SERVING_URL, MLFLOW_URL, AIRFLOW_URL + creds to api service
- docker-compose: pass NEXT_PUBLIC_MLFLOW_URL/AIRFLOW_URL to admin service
- docker-compose: replace wget healthcheck with node fetch (wget not in node image)
- docker-compose: enable Airflow basic_auth API backend; add MLflow pip dep for DAGs
- Dockerfiles: tighten layer caching, add .dockerignore
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add POST /api/auth/token — validates ADMIN_TOKEN env var, creates a 24h
session and sets the sid cookie so automated tools can access the admin
panel without Google OAuth. Admin login page gains a token input form.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- TS: pino + pino-http; every HTTP request log includes traceId from
W3C traceparent header (generated if absent); forwarded to ml/serving
on all /score, /generate, /reward, and /api/ml proxy calls
- Python: structlog JSON; FastAPI middleware binds trace_id via
contextvars so every log line within a request carries it
- Sentry: optional SENTRY_DSN init in both runtimes (no-op if unset)
- Replace all console.* calls across services/api with pino logger
- Update tests to spy on logger instead of console
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
