alvis
ffdf70733f
Issues closed: #86, #87, #88, #89, #90, #91, #79, #80, #82 infra: - docker-compose `ai` profile: Ollama + LiteLLM services - infra/litellm/litellm_config.yaml: tip-generator / embedder / judge aliases - .env.example: LITELLM_URL, LITELLM_MASTER_KEY, OLLAMA_URL ml/serving: - POST /generate: calls LiteLLM tip-generator alias, returns TipCandidate[] - JSON retry loop (2 retries with correction prompt on malformed response) - _parse_llm_json strips markdown fences ml/features: - context.py: build_context() assembles user signals → PromptContext (sorts overdue/high-priority tasks first for LLM prompt quality) shared-types: - TipKind, TipSource, TipCandidate types - Tip gains kind + rationale fields services/api: - recommender: 3-stage pipeline (assemble → score → serve) Stage 1: Todoist tasks + LLM candidates fetched in parallel Stage 2: egreedy bandit scores merged candidate pool Stage 3: serve + log with prompt_version, llm_model, tip_kind - tip_scores: prompt_version, llm_model, tip_kind columns + migrations - config: LITELLM_URL added - integrations: surface token_status in /integrations response tests: - ml/serving/tests/test_generate.py: 13 tests (retry, 502/503, fence variants) - ml/features/test_context.py: 9 tests (sorting, edge cases) - services/api recommender.unit.test.ts: 16 pure-function tests (inferReward, dueAgeDays) - services/api recommender.test.ts: 4 integration tests (tip_scores columns, LLM fallback) - shared-types: TipCandidate, rationale, full TipFeedback action set docs: - ADR-0008: LiteLLM AI gateway decision - overview.md: M2 pipeline description updated - ml/README.md: serving + features roles updated Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
20 lines
994 B
Markdown
20 lines
994 B
Markdown
# ml/
|
||
|
||
Python. Owns models, features, training, online scoring.
|
||
|
||
| Dir | Role | Phase |
|
||
|---|---|---|
|
||
| `serving/` | FastAPI online scorer (`/score`, `/generate`) + LiteLLM gateway, called by `recommender` | 1–2 |
|
||
| `features/` | context assembler (`context.py`): signals → `PromptContext`; Feast adapter later | 2 |
|
||
| `pipelines/` | batch feature + training DAGs (Prefect/Airflow) | 4 |
|
||
| `registry/` | MLflow-backed model registry integration | 4 |
|
||
| `experiments/` | A/B assignment + multi-armed bandit policies | 4 |
|
||
| `notebooks/` | research; never imported by production code | — |
|
||
|
||
## Principles
|
||
|
||
- Every model has a **model card** in `registry/` describing inputs, offline metrics, fairness checks, and rollout history.
|
||
- Online inference must be stateless and < 50ms p99.
|
||
- Training reads from the offline feature store; serving reads from the online feature store; definitions are shared (no train/serve skew).
|
||
- Shadow deploys before any policy change that affects real users.
|