From bd3ea1b8b13038b11c61609da78727bc93a1fe71 Mon Sep 17 00:00:00 2001 From: alvis Date: Sat, 25 Apr 2026 16:53:20 +0000 Subject: [PATCH] =?UTF-8?q?docs(schema):=20update=20docs=20for=20#54=20?= =?UTF-8?q?=E2=80=94=20proto=20registry=20+=20buf=20CI=20gate?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - packages/shared-types/README.md: new — documents HTTP vs event surfaces, proto file layout, schema evolution rules, and how to run buf locally - ml/serving/README.md: note pydantic payload validation in consumer section - CLAUDE.md: replace "schema registry enforced when #54 lands" with the actual state; remove #54 from active-work list Co-Authored-By: Claude Sonnet 4.6 --- CLAUDE.md | 4 +-- ml/serving/README.md | 2 ++ packages/shared-types/README.md | 63 +++++++++++++++++++++++++++++++++ 3 files changed, 67 insertions(+), 2 deletions(-) create mode 100644 packages/shared-types/README.md diff --git a/CLAUDE.md b/CLAUDE.md index 081c9f3..24cf7c6 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -56,7 +56,7 @@ docs/ architecture notes, ADRs, API specs ## Contracts between modules - **HTTP** (OpenAPI, in `packages/shared-types/http/`) — synchronous request/response. In-process today; over the network once extracted. Signatures are identical. -- **Events** (Protocol Buffers, in `packages/shared-types/events/`) — durable signals + feedback. Today: in-process `Bus` with a `onPublish` bridge to NATS JetStream when `NATS_URL` is set (ADR-0010). The in-proc bus stays the source of truth — JetStream is the durable mirror that cross-process consumers (`ml/serving`, future feature pipelines) tail. Schema registry enforced in CI when #54 lands; until then payloads are JSON envelopes (ADR-0005). +- **Events** (Protocol Buffers, in `packages/shared-types/events/`) — durable signals + feedback. Today: in-process `Bus` with a `onPublish` bridge to NATS JetStream when `NATS_URL` is set (ADR-0010). The in-proc bus stays the source of truth — JetStream is the durable mirror that cross-process consumers (`ml/serving`, future feature pipelines) tail. Proto schemas (ADR-0005) live in `packages/shared-types/events/oo/events/v1/`; `buf lint` + `buf breaking` run in CI on every PR touching those files (`.gitea/workflows/buf-check.yaml`). - Do not redefine types per module. Regenerate from `shared-types`. ## Conventions @@ -100,7 +100,7 @@ Ollama and LiteLLM are **shared Agap services**, not oO services — they live i **M1 shipped. M2 (AI tips) in progress.** See `README.md` for the phase roadmap and `docs/architecture/` for diagrams. Work is tracked as Gitea milestones + issues on `alvis/oO`. -Active work: bandit promotion (#99 — offline sim + ADR-0012 pending) and M2 issues (#54 schema registry, #61 freshness SLAs, #78 signal abstraction, #93 model benchmark). +Active work: bandit promotion (#99 — offline sim + ADR-0012 pending) and M2 issues (#61 freshness SLAs, #78 signal abstraction, #93 model benchmark). ## What NOT to do diff --git a/ml/serving/README.md b/ml/serving/README.md index 729ff3d..81dc01b 100644 --- a/ml/serving/README.md +++ b/ml/serving/README.md @@ -41,6 +41,8 @@ On startup, `nats_consumer.py` registers two durable push consumers against NATS - `signals.task.synced` — writes `{last_sync_ts, task_count}` to `{STATE_DIR}/{user}_sync.json` - `signals.tip.feedback` — logged for observability; reward update happens via the HTTP path in the recommender +**Payload validation:** each message is validated against the pydantic models in `schemas.py` (mirroring `packages/shared-types/events/oo/events/v1/`). A `ValidationError` triggers a nak so the message is redelivered rather than silently dropped. + **Ack semantics:** explicit ack on success; nak for redelivery on error; dead-lettered after `NATS_MAX_DELIVER` attempts. **Disabled** when `NATS_URL` is unset (default in local dev without NATS). No import of `nats-py` occurs in that case. diff --git a/packages/shared-types/README.md b/packages/shared-types/README.md new file mode 100644 index 0000000..1a1dd1c --- /dev/null +++ b/packages/shared-types/README.md @@ -0,0 +1,63 @@ +# @oo/shared-types + +Canonical contracts for all inter-module communication. Two surfaces: + +| Surface | Format | Location | +|---------|--------|----------| +| HTTP (sync) | OpenAPI / TypeScript interfaces | `src/http/` | +| Events (async) | Protocol Buffers + TS interfaces | `src/events/`, `events/` | + +## HTTP types + +Hand-written TypeScript interfaces generated from OpenAPI specs. Imported by +`services/api`, `apps/web`, and `ml/serving` (Python hand-mirrors). + +| File | Types | +|------|-------| +| `src/http/tip.ts` | `TipCandidate`, `RecommendResponse`, `TipFeedback` | +| `src/http/auth.ts` | `SessionUser` | +| `src/http/integrations.ts` | `IntegrationsResponse`, `Integration` | +| `src/http/user.ts` | `UserProfile` | +| `src/http/signal.ts` | `Signal`, `SignalSource` | + +## Event types + +Protobuf schemas live in `events/oo/events/v1/`. TypeScript interfaces in +`src/events/index.ts` mirror the proto envelope and payload types. + +| Proto file | Messages | +|------------|----------| +| `envelope.proto` | `Envelope` (wraps every event) | +| `signals.proto` | `TaskSyncedPayload`, `TipServedPayload`, `TipFeedbackPayload`, `TipRewardFailedPayload` | +| `integration.proto` | `IntegrationTokenExpiredPayload` | + +**Schema evolution rules (ADR-0005):** +- Additive changes only within a version (new fields, new message types). +- Removed fields must be marked `reserved` — never reuse a field number. +- Breaking changes require a new package version (`oo.events.v2`) and a `schemaVersion` bump in the envelope. + +## Schema registry / CI gate + +`buf` enforces lint and breaking-change detection on every PR that touches `events/`: + +```bash +# Lint +buf lint events/ + +# Breaking-change check against main +buf breaking events/ --against '.git#branch=main,subdir=packages/shared-types/events' +``` + +Local shortcut: `./scripts/buf-check.sh` + +CI: `.gitea/workflows/buf-check.yaml` (requires a Gitea Actions runner). + +Install buf: `curl -sSfL https://github.com/bufbuild/buf/releases/latest/download/buf-Linux-x86_64 -o /usr/local/bin/buf && chmod +x /usr/local/bin/buf` + +## Contract + +`/health` — not applicable (library package, no process). + +**Extraction criteria** — always a shared library. Extract to a separate registry +service only when schema governance requires independent versioning and deployment +(e.g. external consumers, SLA divergence from the monorepo).