alvis
2d7cf217a9
Ship the scaffolding for #99 (phase B.3 of #81): - ml/serving: add /score/egreedy/v2, /reward/egreedy/v2, /stats/egreedy/v2 endpoints (D=12). New feature dims: completion/dismiss rates, mean dwell (clipped 10min), preferred-hour alignment (cosine, 1-dim), tip volume (log). Separate state file per user (_egreedy_v2.json). /reset clears v2 state too. - ADR-0012: documents D=7→12 dimension change, normalization choices, shadow rollout protocol, and promotion gate (offline sim win per ADR-0002). - recommender.ts: register egreedy-v2-shadow in shadow-policy map (disabled by default). When enabled, calls /score/egreedy/v2 fire-and-forget and publishes shadow:egreedy-v2-shadow serve signal. No reward to shadow — sim is the gate. - sim runner/personas: personas carry synthetic profile_features per persona; _call_score/_call_reward thread profile_features through (None-safe for v1/linucb). - 18 new Python tests; all 56 Python + 170 TS tests pass. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ml/
Python. Owns models, features, training, online scoring.
| Dir | Role | Phase |
|---|---|---|
serving/ |
FastAPI online scorer (/score, /generate) + LiteLLM gateway + prompt registry (prompts.py), called by recommender |
1–2 |
features/ |
context assembler (context.py): signals → PromptContext; Feast adapter later |
2 |
pipelines/ |
batch feature + training DAGs (Prefect/Airflow) | 4 |
registry/ |
MLflow-backed model registry integration | 4 |
experiments/ |
A/B assignment + multi-armed bandit policies | 4 |
notebooks/ |
research; never imported by production code | — |
Principles
- Every model has a model card in
registry/describing inputs, offline metrics, fairness checks, and rollout history. - Online inference must be stateless and < 50ms p99.
- Training reads from the offline feature store; serving reads from the online feature store; definitions are shared (no train/serve skew).
- Shadow deploys before any policy change that affects real users.
Profile-feature contract
User-level features (completion rate, preferred hour, tip volume…) are computed
by the TypeScript recommender and shipped to ml/serving on every /score and
/generate call as profile_features: dict | None. The Python mirror in
features/profile_schema.py documents the available names + dtypes — keep it
in sync with services/api/src/profile/registry.ts (a CI-style test asserts
the name sets match). See ADR-0011.
Prompt registry
serving/prompts.py keys tip-generation prompts by stable version string. Adding a new variant means adding an entry — no caller changes. Selection precedence: POST /generate body's prompt_version field → env DEFAULT_PROMPT_VERSION → "v1". The TypeScript recommender drives selection via TIP_PROMPT_VERSION (single value or comma-separated rotation); the version actually used flows back in the response and is persisted to tip_scores.prompt_version so the admin reward-analytics dashboard can bucket reactions per variant.