chore: scaffold oO monorepo with architecture, roadmap, and module stubs

This commit is contained in:
2026-04-13 14:19:56 +00:00
commit cf4c7a0eb4
36 changed files with 494 additions and 0 deletions

21
.gitignore vendored Normal file
View File

@@ -0,0 +1,21 @@
node_modules/
dist/
build/
.next/
.turbo/
.nx/
*.log
.env
.env.local
.env.*.local
__pycache__/
*.pyc
.venv/
.mypy_cache/
.pytest_cache/
.ruff_cache/
.DS_Store
coverage/
*.sqlite
.idea/
.vscode/

81
CLAUDE.md Normal file
View File

@@ -0,0 +1,81 @@
# oO — Project Instructions
## What this is
**oO** is a recommendation system for personal tips. It collects signals across a user's life (tasks, habits, calendar, mood, context) to build a rich profile and deliver **one** perfectly-timed tip — an advice or a todo — that feels like magic.
The magic is the product. Precision + timing + minimalism. The UI shows a single black page with one tip. The complexity lives behind it.
## Prime directives
1. **Modular, service-oriented from day one.** Even the prototype. We will scale to mobile (iOS/Android), many integrations, multi-tenant ML. Shortcuts that bake in a monolith are not acceptable.
2. **Recommendation engine is the core.** Every other service feeds it or renders its output. Design schemas, event contracts, and APIs with that in mind.
3. **Python owns ML.** Everything training, features, serving for models is Python (FastAPI + PyTorch/scikit + MLflow/feast). Application services are TypeScript (Node, Next.js) unless there's a reason.
4. **OAuth-first for identity and integrations.** Never ask users for passwords or raw API keys when a delegated-auth flow exists. Store provider tokens encrypted, refresh transparently.
5. **Feel-of-magic over feature count.** When in doubt, ship fewer things, polished.
## Architecture (high level)
```
apps/ user-facing clients
web/ Next.js PWA — the first shipped client
mobile-ios/ Swift/SwiftUI (Phase 3)
mobile-android/ Kotlin/Compose (Phase 3)
services/ backend microservices (each independently deployable)
gateway/ API gateway + BFF (GraphQL or tRPC)
auth/ OAuth (Google, Apple, ...), sessions, JWT issuance
profile/ user profile, preferences, consents
integrations/ third-party connectors (Todoist first); token vault
recommender/ Python; serves the "one best tip" decision
events/ event bus ingress (Kafka/NATS) + signal store
notifier/ push/email/web delivery of tips
packages/ shared libraries
shared-types/ OpenAPI/proto-generated types
sdk-js/ client SDK used by web + mobile webviews
ui/ shared React components + design tokens
ml/ Python MLOps
pipelines/ training / batch feature pipelines (Airflow/Prefect)
features/ feature definitions (Feast-style)
registry/ model registry (MLflow) integration
experiments/ A/B testing framework + bandit policies
serving/ online inference service (FastAPI)
notebooks/ research only — not production
infra/ docker-compose, k8s manifests, terraform, CI
docs/ architecture notes, ADRs, API specs
```
## Contracts between services
- **Events** (Kafka/NATS) — source of truth for user signals. All integrations emit normalized events; the recommender reads them.
- **HTTP/gRPC** — synchronous request/response (gateway → services).
- **Shared schemas** live in `packages/shared-types`; generated from a single OpenAPI / proto source. Do not redefine types per service.
## Conventions
- Every service ships a `README.md`, a `Dockerfile`, and a `/health` endpoint.
- One PR = one concern. Commits follow conventional-commit prefixes (`feat:`, `fix:`, `chore:`, `docs:`, `refactor:`).
- ADRs go in `docs/adr/NNNN-title.md` for any decision that constrains future work.
- No secrets in repo. Local dev via `.env.local` (gitignored), prod via the server's secret store (Vaultwarden now; k8s secrets later).
## Definition of done (per feature)
1. Code + tests merged.
2. Service's `README.md` updated.
3. If it changes a contract → `shared-types` regenerated + consumers updated.
4. If it changes architecture → ADR added.
5. Deployable via `docker compose up` locally.
## Current phase
**Phase 0 — Prototype.** See `README.md` for the phase roadmap and `docs/architecture/` for diagrams. Work is tracked as Gitea milestones + issues on `alvis/oO`.
## What NOT to do
- Don't copy Todoist's data into our DB. Store the OAuth token; fetch on demand.
- Don't implement auth by hand. Use a library (NextAuth / Auth.js, Ory, or Clerk-compatible). We will self-host.
- Don't hardwire a recommender. The "random todo" v0 must live behind the same interface the real ML model will implement (`POST /recommend``{tip}`). Swap internals, keep contract.
- Don't build an admin UI before the user-facing black page is polished.

71
PLAN.md Normal file
View File

@@ -0,0 +1,71 @@
# Implementation plan
Step-by-step build order for Phase 0 (prototype) and the seams that make Phases 15 cheap.
The principle: **build the contracts first, stub the internals.** Every service should exist with a `/health` endpoint and a minimal real implementation of its interface before any service is "finished". This gives us an end-to-end walking skeleton from week one.
---
## Stage 0 — Foundations (days 13)
1. **Monorepo tooling.** pnpm workspaces for JS/TS; uv or poetry for Python; turbo or nx for build graph; pre-commit (lint, typecheck, format).
2. **Docker Compose dev env.** Postgres, NATS, MinIO (S3), Mailhog, all services wired with hot-reload.
3. **CI skeleton** (Gitea Actions): lint → typecheck → unit test → build → publish images.
4. **Secrets convention.** `.env.example` per service; prod secrets injected by orchestrator.
5. **Shared types package.** OpenAPI source → generated TS + Python clients.
Deliverable: `docker compose up` brings a green dashboard of `/health` endpoints.
## Stage 1 — Identity & session (days 47)
1. `services/auth`: Google OAuth2 (PKCE), session cookies, short-lived JWTs, refresh rotation. Library-backed (Auth.js or Ory Kratos + Hydra) — we do not roll our own.
2. `services/profile`: minimal `User` record; created on first sign-in.
3. `apps/web` sign-in page; gateway verifies JWT.
Exit check: a user can sign in and fetch their own profile.
## Stage 2 — Integrations framework (days 812)
1. `services/integrations` with a **Connector** interface:
- `begin_oauth(user) → redirect_url`
- `finish_oauth(code, state) → StoredCredential`
- `fetch_signals(user, since) → Event[]`
2. **Token vault**: column-level encryption (libsodium), key from env or KMS.
3. **Todoist connector** as the first concrete implementation.
4. Web "Connect" page: list of connectors, button per connector, callback handling.
Exit check: a user taps "Connect Todoist", completes the OAuth dance, and the integrations service can fetch their tasks on demand.
## Stage 3 — Recommender contract (days 1316)
1. `services/recommender` exposes `POST /recommend {user_id, context} → {tip}`.
2. Policy interface (`Policy.pick(user, candidates, context) → tip`).
3. **`RandomPolicy` v0** — fetches candidates from `integrations` (Todoist tasks), returns one uniformly at random.
4. Tip shape is provider-agnostic: `{id, kind: "todo"|"advice", title, body, source, deep_link, meta}`.
5. `apps/web` tip page: full black, one tip centered, tap = mark done → callback fires to integrations (complete Todoist task) + emits a feedback event.
Exit check: three-page prototype works end-to-end for one user.
## Stage 4 — Hardening the prototype (days 1720)
1. Error surfaces (Sentry), structured logs (pino / structlog), trace IDs across services.
2. Rate limits + retries on outbound API calls.
3. Integration tests: Playwright for the web flow, pact-style contract tests between services.
4. Deploy to a single VM via docker-compose + Caddy.
Exit check: Phase 0 milestone closed.
---
## Seams prepared for later phases (do not implement yet, but do not foreclose)
- **Event bus.** From day one, `integrations` and `recommender` speak through an async fn that today is an in-process call but will be NATS tomorrow. Keep the signature `(event: NormalizedEvent) → void`.
- **Feature store.** The recommender accepts a `context` blob; later, a feature service fills it. Do not inline feature lookups inside the policy.
- **Policy registry.** `PolicyFactory.get(name)` so A/B and bandit policies slot in without code changes to the gateway.
- **Python boundary.** Recommender is TS today, but its scoring function is isolated — moving to FastAPI in Phase 1 is a file move, not a refactor.
---
## Staffing assumption
Work is parallelizable across ~3 streams: **infra/platform**, **backend services**, **web app**. Each Gitea issue notes which stream and which phase (milestone) it belongs to.

126
README.md Normal file
View File

@@ -0,0 +1,126 @@
# oO
> One tip. Right now. Feels like magic.
oO learns who you are from the apps you already use and surfaces **one** perfectly-timed suggestion — an advice or a todo — on a black page. No feed. No dashboard. One tip.
---
## Why
Everyone has too many tasks, too many apps, too much noise. What people actually need is a single, well-chosen nudge at the right moment. oO is that nudge, powered by a recommendation engine that gets smarter the more of your life it sees.
## Product principles
1. **One thing at a time.** The UI is a black page with one tip. That's the product.
2. **We don't own your data, we understand it.** Connect your apps; we read what we need, when we need it.
3. **Magic requires craft.** Precision, timing, and restraint matter more than features.
4. **Private by default.** Tokens are encrypted, models are per-user, deletion is one click.
## Prototype scope (Phase 0)
Three pages. That's it.
| Page | What it does |
|------|--------------|
| **Sign in** | Google / Apple OAuth. No passwords. |
| **Connect** | A list of integrations. Tap "Todoist" → OAuth flow → token stored. |
| **Tip** | Black page. One tip. Tap to dismiss / done / snooze. |
Under the hood the "pick a tip" call already routes through a `recommender` service with a pluggable policy — so v0 is literally "random Todoist task" but every other version slots into the same contract.
---
## Architecture at a glance
```
┌──────────┐ OAuth ┌────────────┐
│ Web / │──────────▶│ auth │
│ Mobile │ └─────┬──────┘
│ client │ │ JWT
│ │ REST/GraphQL ▼
│ │────────▶┌───────────────┐
└──────────┘ │ gateway │──┬──▶ profile
└───────┬───────┘ ├──▶ integrations ──▶ Todoist / Google / ...
│ └──▶ recommender ──▶ ml/serving (Python)
┌───────────────┐
│ events │ ◀── integrations emit normalized events
│ (Kafka/NATS) │ ──▶ ml/pipelines (features, training)
└───────────────┘
```
More detail in [`docs/architecture/`](docs/architecture/) and decisions in [`docs/adr/`](docs/adr/).
## Monorepo layout
See [`CLAUDE.md`](CLAUDE.md) for the full tree and conventions.
```
apps/ web, ios, android
services/ gateway, auth, profile, integrations, recommender, events, notifier
packages/ shared-types, sdk-js, ui
ml/ pipelines, features, registry, experiments, serving
infra/ docker, k8s, terraform, ci
docs/ architecture, adr, api
```
---
## Roadmap
### Phase 0 — Prototype *(M0)*
Goal: a single user can sign in, connect Todoist, and see one random Todoist task on a black page.
- [ ] Monorepo scaffold, CI skeleton, docker-compose dev env
- [ ] `auth` service with Google OAuth
- [ ] `integrations/todoist` OAuth2 flow + encrypted token vault
- [ ] `recommender` service with `RandomPolicy` (v0)
- [ ] `apps/web` — three pages (sign-in, connect, tip)
- [ ] Deploy to a single VM via docker-compose
### Phase 1 — Real signal *(M1)*
Goal: the tip is picked, not drawn from a hat. Still Todoist-only.
- [ ] Event bus (NATS) + ingestion from Todoist sync API
- [ ] Feature store skeleton (Feast or homegrown) and the first five features (time-of-day, overdue count, task age, priority, project)
- [ ] `ml/serving` FastAPI scoring endpoint; `recommender` calls it
- [ ] `ContextualBanditPolicy` v1 (LinUCB) replacing `RandomPolicy`
- [ ] Tip feedback loop: user reactions (done / snooze / dismiss) become rewards
### Phase 2 — Multi-source user profile *(M2)*
Goal: oO knows more than tasks.
- [ ] Integrations: Google Calendar, Apple Health (web import), generic webhook
- [ ] Unified `Profile` model (identity, preferences, contexts, consents)
- [ ] Timing signals (location, idle, focus windows) via client-side probes
- [ ] Advice library (curated tips, not only todos) + mixing policy
### Phase 3 — Mobile & notifications *(M3)*
- [ ] iOS app (SwiftUI) with APNs push
- [ ] Android app (Compose) with FCM push
- [ ] `notifier` service with quiet-hours + per-channel rate limits
- [ ] Rich notifications that deep-link to the tip page
### Phase 4 — MLOps at scale *(M4)*
- [ ] Airflow/Prefect orchestrator for batch retrains
- [ ] MLflow model registry + shadow deploys
- [ ] Online `experiments` framework: A/B + multi-armed bandits as first-class
- [ ] Cohort analysis + cross-user collaborative features (opt-in)
- [ ] Model cards, fairness checks, drift monitoring
### Phase 5 — Production hardening *(M5)*
- [ ] SOC2-style controls, audit logging, token rotation
- [ ] k8s deploy + horizontal autoscaling
- [ ] Multi-region failover, PITR backups
- [ ] Public integration SDK so third parties can add sources
- [ ] Billing + subscription tiers
---
## Contributing
This repo is split into independent modules; most tickets belong to exactly one. Pick an issue, check its milestone (= phase), read the service's `README.md`, ship.
Conventions and per-service guidance live in [`CLAUDE.md`](CLAUDE.md).
## License
TBD.

View File

0
apps/mobile-ios/.gitkeep Normal file
View File

0
apps/web/.gitkeep Normal file
View File

15
apps/web/README.md Normal file
View File

@@ -0,0 +1,15 @@
# apps/web
Next.js PWA. Phase 0 scope: three pages.
| Route | Purpose |
|---|---|
| `/sign-in` | Google/Apple OAuth buttons. No form. |
| `/connect` | List of integrations (cards). Tap → OAuth. Tap connected card → disconnect. |
| `/` (tip) | Pure black background. One tip centered. Tap gestures: done / snooze / dismiss. |
## Design notes
- The tip page is the product. Treat it like a watch face.
- Zero chrome, no nav bar, no settings icon while a tip is showing. Long-press reveals actions.
- Offline-first: last tip is cached; reactions queued until reconnect.

View File

@@ -0,0 +1,15 @@
# ADR-0001: Polyglot monorepo, TS for apps, Python for ML
## Status
Accepted — 2026-04-13
## Context
We ship web and mobile clients, backend services, and ML training/serving. Splitting into many repos early creates cross-repo PRs for every contract change and hurts velocity.
## Decision
One monorepo, managed with pnpm workspaces for TS and uv/poetry for Python. Shared contracts live in `packages/shared-types` generated from OpenAPI. ML is Python; everything else is TS.
## Consequences
- One CI system, one versioning flow, atomic cross-service PRs.
- Requires disciplined boundaries: services must still be independently deployable.
- Tooling complexity: two package managers, two lint stacks. Acceptable given the ML/app split.

View File

@@ -0,0 +1,20 @@
# ADR-0002: Recommender as the stable contract, policy as a plugin
## Status
Accepted — 2026-04-13
## Context
v0 picks a random Todoist task. v1+ will use a contextual bandit, then learned rankers, then collaborative signals. If the HTTP contract and the candidate-generation path are coupled to today's "random", every change is a migration.
## Decision
`recommender` exposes `POST /recommend` as the one stable contract. Internally it has three seams:
1. **Candidate sources** — async functions that yield `TipCandidate`s from integrations, advice libraries, etc.
2. **Context assembler** — pulls features (today: inline; later: feature store).
3. **Policy**`Policy.pick(candidates, context) → tip`. Registered by name; selected per-request by the experiments framework (Phase 4) or a static config (now).
Swapping a policy never changes the contract or the client.
## Consequences
- v0 policy is `RandomPolicy`, trivially 50 lines.
- v1 moves scoring to `ml/serving` behind the same `Policy` interface (`RemotePolicy` wrapper).
- A/B is introduced without touching clients.

View File

@@ -0,0 +1,44 @@
# Architecture overview
## Guiding constraints
- The **recommendation decision** is the hot path. Every architectural choice should shorten the distance between a new signal and a better tip.
- Services are small and independently deployable, but we do **not** multiply services for its own sake. Split by team-of-ownership and by data lifecycle.
- Python for ML, TypeScript for applications, shared contracts regenerated from a single source of truth.
## Services
| Service | Language | Responsibility | Owns data |
|---|---|---|---|
| `gateway` | TS (Node) | BFF for web/mobile; auth-checking; request fan-out | — |
| `auth` | TS | OAuth (Google, Apple), sessions, token issuance | identities, sessions |
| `profile` | TS | user profile, preferences, consents | profiles |
| `integrations` | TS | third-party connectors, token vault, signal fetch | credentials, cursors |
| `events` | TS | event-bus ingress, normalization, durable log | signal store |
| `recommender` | TS | orchestration: candidates → policy → tip; feedback sink | tip history |
| `ml/serving` | Python | online scoring for policies/models | — (stateless) |
| `ml/pipelines` | Python | batch feature + training pipelines | feature store, models |
| `notifier` | TS | push/email delivery, quiet hours, dedupe | delivery log |
## Data boundaries
Each service owns its schema; no cross-service DB access. When `recommender` needs profile data, it calls `profile` (read model), not its DB.
## Event flow
```
connector (integrations) ──emit──▶ events ──▶ feature pipelines (ml)
└──▶ recommender (context assembly)
```
User reactions (done / snooze / dismiss) are events too. They close the loop as rewards for bandit/RL policies.
## Why these choices
- **NATS JetStream** over Kafka for Phase 1: lighter, single-binary, fits the "one VM" deployment. Swap to Kafka in Phase 4.
- **Postgres** everywhere for OLTP. Per-service schemas, not per-service instances in dev.
- **FastAPI + Pydantic** for ML serving — fast, typed, swappable runtime (ONNX, Triton) behind it.
- **Feast** for feature store when we get there; homegrown adapter until then (Phase 1 seam).
- **MLflow** for model registry; artifacts in MinIO/S3.
- **Auth.js or Ory** for identity — we will not write crypto.

0
infra/ci/.gitkeep Normal file
View File

0
infra/docker/.gitkeep Normal file
View File

0
infra/k8s/.gitkeep Normal file
View File

0
infra/terraform/.gitkeep Normal file
View File

19
ml/README.md Normal file
View File

@@ -0,0 +1,19 @@
# ml/
Python. Owns models, features, training, online scoring.
| Dir | Role | Phase |
|---|---|---|
| `serving/` | FastAPI online scorer (`/score`), called by `recommender` | 1 |
| `features/` | feature definitions + store adapter (Feast later) | 1 |
| `pipelines/` | batch feature + training DAGs (Prefect/Airflow) | 4 |
| `registry/` | MLflow-backed model registry integration | 4 |
| `experiments/` | A/B assignment + multi-armed bandit policies | 4 |
| `notebooks/` | research; never imported by production code | — |
## Principles
- Every model has a **model card** in `registry/` describing inputs, offline metrics, fairness checks, and rollout history.
- Online inference must be stateless and < 50ms p99.
- Training reads from the offline feature store; serving reads from the online feature store; definitions are shared (no train/serve skew).
- Shadow deploys before any policy change that affects real users.

0
ml/experiments/.gitkeep Normal file
View File

0
ml/features/.gitkeep Normal file
View File

0
ml/notebooks/.gitkeep Normal file
View File

0
ml/pipelines/.gitkeep Normal file
View File

0
ml/registry/.gitkeep Normal file
View File

0
ml/serving/.gitkeep Normal file
View File

0
packages/sdk-js/.gitkeep Normal file
View File

View File

0
packages/ui/.gitkeep Normal file
View File

13
services/README.md Normal file
View File

@@ -0,0 +1,13 @@
# services/
Backend microservices. Each directory is independently deployable, ships a `Dockerfile`, a `/health` endpoint, and its own `README.md` describing its contract.
| Dir | Role | Phase introduced |
|---|---|---|
| `gateway/` | BFF for clients; auth check; fan-out to services | 0 |
| `auth/` | OAuth (Google/Apple), sessions, JWT | 0 |
| `profile/` | user profile, preferences, consents | 0 |
| `integrations/` | third-party connectors + encrypted token vault (Todoist first) | 0 |
| `recommender/` | `POST /recommend` — policy-driven tip selection | 0 |
| `events/` | event bus ingress + durable signal store | 1 |
| `notifier/` | push/email/web delivery with quiet-hours | 3 |

0
services/auth/.gitkeep Normal file
View File

14
services/auth/README.md Normal file
View File

@@ -0,0 +1,14 @@
# auth
OAuth-based identity. **Do not roll your own crypto or session logic** — back this with Auth.js or Ory Kratos+Hydra.
## Responsibilities
- Google OAuth (Phase 0), Apple OAuth (Phase 0.5), extensible to others.
- Issue short-lived JWTs + rotating refresh tokens; HttpOnly cookies for web.
- Expose `GET /me` (who am I), `POST /logout`, OIDC-style `/.well-known` endpoints.
## Non-goals
- Password auth. Ever.
- User-profile data — that lives in `profile/`.

0
services/events/.gitkeep Normal file
View File

View File

View File

View File

@@ -0,0 +1,28 @@
# integrations
Third-party connectors and the token vault.
## Connector interface
```ts
interface Connector {
id: string // e.g. "todoist"
beginOAuth(user): Promise<{ redirectUrl, state }>
finishOAuth(code, state): Promise<StoredCredential>
fetchSignals(user, since?): AsyncIterable<NormalizedEvent>
// optional write-back, e.g. mark task done
act?(user, action): Promise<void>
}
```
## Token vault
- Credentials encrypted at rest (libsodium sealed box); key from env/KMS.
- Refresh handled transparently; consumers never see raw tokens.
- One row per `(user, provider)` with provider-specific `meta`.
## Roadmap
- Phase 0: **Todoist** (OAuth2, read tasks, complete task).
- Phase 2: Google Calendar, Apple Health (web import), generic webhook ingress.
- Phase 5: public SDK so third parties can ship connectors.

View File

View File

View File

View File

@@ -0,0 +1,27 @@
# recommender
The core of oO. Takes a user + a context, returns **one** tip.
## Contract
```
POST /recommend
{ user_id, context?: { time, timezone, client, ... } }
→ { tip: { id, kind: "todo"|"advice", title, body, source, deep_link, meta } }
POST /feedback
{ user_id, tip_id, reaction: "done"|"snooze"|"dismiss", at }
```
## Internals (stable seams)
- **Candidate sources** — pluggable async generators. v0: Todoist tasks via `integrations`. Later: advice library, calendar nudges, health prompts.
- **Context assembler** — merges request context with features (inline now, feature-store later).
- **Policy** — `Policy.pick(candidates, context) → tip`. Registered by name:
- `random` — v0 (Phase 0).
- `bandit.linucb` — v1 (Phase 1).
- `remote` — delegates to `ml/serving` FastAPI scorer (Phase 1+).
## Phase 0 goal
`RandomPolicy` only. The service, contract, and seams exist; the brain does not yet.