alvis/oO

Files

alvis faf44c18fc feat: ε-greedy v1 as active policy; dwell-time reward inference; offline sim framework

- Promote egreedy-v1 to active serving policy (ADR-0007): /score/egreedy + /reward/egreedy
  replaces linucb-v1 endpoints after offline sim shows +10.7% mean reward (−0.548 vs −0.606)
- Replace explicit helpful/not_helpful feedback with dwell-time inferred reward (inferReward):
  dismiss=−1.0, snooze=+0.1, done<15s=−0.3, done 15s–2min=+1.0, done 2–10min=+0.6, done>10min=+0.3
- Add ml/serving ε-greedy endpoints: /score/egreedy, /reward/egreedy, /stats/egreedy/{user_id}
  with d=7 feature vector (base 5 + sin/cos day-of-week encoding)
- Add offline simulation framework (ml/experiments/sim): rule/LLM/claude-code judges,
  two-phase score+reward, synthetic personas, task generator; results stored in sim_runs/sim_events
- Add /admin/simulations page: start runs, live-poll status, reward curve SVG, action/persona tables
- Fix egreedy day_of_week training skew: reward endpoint now uses actual dow instead of hardcoded 0
- Fix runner.py proxy bypass: httpx.Client(trust_env=False) for localhost ML calls
- Add dwellMs to TipFeedbackEvent contract and bus.test.ts fixture
- Schema: sim_runs, sim_events tables; tip_feedback gains dwell_ms, reward_milli columns
- ADR-0006: admin console framework; ADR-0007: egreedy-v1 policy selection rationale

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-04-16 07:44:37 +00:00

3.2 KiB

Raw Blame History

ADR-0006: Admin console framework — Next.js 15 + Tremor + shadcn/ui + embed specialist tools

Status

Accepted — 2026-04-15

Context

M1 ships a bandit-driven recommender, an event bus, and a live feedback loop. Without a cockpit to observe these systems, every model change ships blind. An admin console is needed to:

Observe — DAU/WAU, tip outcomes, reaction rates, LinUCB arm stats, feature distributions
Inspect — per-user identity, consents, integrations, reward history
Act — revoke tokens, replay signals, reset a per-user bandit, promote a policy
Audit — every operator action is logged

The team is two people. The stack is TypeScript/React/Tailwind. Any framework that forks the stack creates a context-switch tax and a second deployment surface.

Decision

App shell — `apps/admin`, Next.js 15, App Router

Same stack as apps/web. Reuses packages/shared-types, the Auth.js session cookie, and the API rewrite convention. Deployed at admin.o.alogins.net behind Caddy, port 3080 in dev.

UI libraries

Layer	Library	Reason
Charts / KPI	Tremor	Analytics-first React + Tailwind components (KPI cards, time-series, bar lists). Designed for dashboards, not bolted on.
CRUD primitives	shadcn/ui	Copy-paste Radix components; forms, dialogs, command palette. No version lock-in — code lives in-repo.
Heavy grids	TanStack Table v8	Sortable / paginated / virtualized tables for events, users, tips.
Extra charts	Recharts	Fallback where Tremor falls short (histograms, distributions).

Embed, don't rebuild

Specialized tooling is reverse-proxied into the admin shell, not reimplemented:

MLflow UI → /admin/models (Caddy sub-path proxy)
Grafana panels → /admin/infra (iframed or embedded panels)
Marimo notebooks → launch-out link from admin

This prevents reimplementing artifact browsers or graph renderers we'd never do as well.

AuthZ

profile.role column on the users table (values: 'user' | 'admin'). First admin seeded via ADMIN_SEED_EMAIL env var at startup. Admin-only gate in Next.js middleware checks the session and the role returned by GET /api/user/me. Every write action through the admin API is appended to an admin_actions audit log.

Rejected alternatives

Option	Rejected because
Retool / AppSmith	Admin logic leaves the repo; weak analytics affordances
Streamlit / Gradio	Python-first; splits the frontend stack; thin RBAC
React-admin / Refine.dev	Strong CRUD scaffolding, analytics views feel bolted on
Superset / Metabase as the admin surface	Excellent BI, poor operational writes; plan: adopt Superset in M4 for BI alongside batch pipelines

Consequences

One more Next.js app in the monorepo. Build/dev added to Turborepo.
Tremor + shadcn/ui are added as dependencies. shadcn components are copied into apps/admin/src/components/ui/ — no runtime version coupling.
MLflow and Grafana must be reachable from the Caddy reverse proxy; they are not embedded in the JS bundle.
admin_actions audit log grows unboundedly — needs a retention policy before M4.

3.2 KiB Raw Blame History