docs(schema): update docs for #54 — proto registry + buf CI gate
- packages/shared-types/README.md: new — documents HTTP vs event surfaces, proto file layout, schema evolution rules, and how to run buf locally - ml/serving/README.md: note pydantic payload validation in consumer section - CLAUDE.md: replace "schema registry enforced when #54 lands" with the actual state; remove #54 from active-work list Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -56,7 +56,7 @@ docs/ architecture notes, ADRs, API specs
|
||||
## Contracts between modules
|
||||
|
||||
- **HTTP** (OpenAPI, in `packages/shared-types/http/`) — synchronous request/response. In-process today; over the network once extracted. Signatures are identical.
|
||||
- **Events** (Protocol Buffers, in `packages/shared-types/events/`) — durable signals + feedback. Today: in-process `Bus` with a `onPublish` bridge to NATS JetStream when `NATS_URL` is set (ADR-0010). The in-proc bus stays the source of truth — JetStream is the durable mirror that cross-process consumers (`ml/serving`, future feature pipelines) tail. Schema registry enforced in CI when #54 lands; until then payloads are JSON envelopes (ADR-0005).
|
||||
- **Events** (Protocol Buffers, in `packages/shared-types/events/`) — durable signals + feedback. Today: in-process `Bus` with a `onPublish` bridge to NATS JetStream when `NATS_URL` is set (ADR-0010). The in-proc bus stays the source of truth — JetStream is the durable mirror that cross-process consumers (`ml/serving`, future feature pipelines) tail. Proto schemas (ADR-0005) live in `packages/shared-types/events/oo/events/v1/`; `buf lint` + `buf breaking` run in CI on every PR touching those files (`.gitea/workflows/buf-check.yaml`).
|
||||
- Do not redefine types per module. Regenerate from `shared-types`.
|
||||
|
||||
## Conventions
|
||||
@@ -100,7 +100,7 @@ Ollama and LiteLLM are **shared Agap services**, not oO services — they live i
|
||||
|
||||
**M1 shipped. M2 (AI tips) in progress.** See `README.md` for the phase roadmap and `docs/architecture/` for diagrams. Work is tracked as Gitea milestones + issues on `alvis/oO`.
|
||||
|
||||
Active work: bandit promotion (#99 — offline sim + ADR-0012 pending) and M2 issues (#54 schema registry, #61 freshness SLAs, #78 signal abstraction, #93 model benchmark).
|
||||
Active work: bandit promotion (#99 — offline sim + ADR-0012 pending) and M2 issues (#61 freshness SLAs, #78 signal abstraction, #93 model benchmark).
|
||||
|
||||
## What NOT to do
|
||||
|
||||
|
||||
@@ -41,6 +41,8 @@ On startup, `nats_consumer.py` registers two durable push consumers against NATS
|
||||
- `signals.task.synced` — writes `{last_sync_ts, task_count}` to `{STATE_DIR}/{user}_sync.json`
|
||||
- `signals.tip.feedback` — logged for observability; reward update happens via the HTTP path in the recommender
|
||||
|
||||
**Payload validation:** each message is validated against the pydantic models in `schemas.py` (mirroring `packages/shared-types/events/oo/events/v1/`). A `ValidationError` triggers a nak so the message is redelivered rather than silently dropped.
|
||||
|
||||
**Ack semantics:** explicit ack on success; nak for redelivery on error; dead-lettered after `NATS_MAX_DELIVER` attempts.
|
||||
|
||||
**Disabled** when `NATS_URL` is unset (default in local dev without NATS). No import of `nats-py` occurs in that case.
|
||||
|
||||
63
packages/shared-types/README.md
Normal file
63
packages/shared-types/README.md
Normal file
@@ -0,0 +1,63 @@
|
||||
# @oo/shared-types
|
||||
|
||||
Canonical contracts for all inter-module communication. Two surfaces:
|
||||
|
||||
| Surface | Format | Location |
|
||||
|---------|--------|----------|
|
||||
| HTTP (sync) | OpenAPI / TypeScript interfaces | `src/http/` |
|
||||
| Events (async) | Protocol Buffers + TS interfaces | `src/events/`, `events/` |
|
||||
|
||||
## HTTP types
|
||||
|
||||
Hand-written TypeScript interfaces generated from OpenAPI specs. Imported by
|
||||
`services/api`, `apps/web`, and `ml/serving` (Python hand-mirrors).
|
||||
|
||||
| File | Types |
|
||||
|------|-------|
|
||||
| `src/http/tip.ts` | `TipCandidate`, `RecommendResponse`, `TipFeedback` |
|
||||
| `src/http/auth.ts` | `SessionUser` |
|
||||
| `src/http/integrations.ts` | `IntegrationsResponse`, `Integration` |
|
||||
| `src/http/user.ts` | `UserProfile` |
|
||||
| `src/http/signal.ts` | `Signal`, `SignalSource` |
|
||||
|
||||
## Event types
|
||||
|
||||
Protobuf schemas live in `events/oo/events/v1/`. TypeScript interfaces in
|
||||
`src/events/index.ts` mirror the proto envelope and payload types.
|
||||
|
||||
| Proto file | Messages |
|
||||
|------------|----------|
|
||||
| `envelope.proto` | `Envelope` (wraps every event) |
|
||||
| `signals.proto` | `TaskSyncedPayload`, `TipServedPayload`, `TipFeedbackPayload`, `TipRewardFailedPayload` |
|
||||
| `integration.proto` | `IntegrationTokenExpiredPayload` |
|
||||
|
||||
**Schema evolution rules (ADR-0005):**
|
||||
- Additive changes only within a version (new fields, new message types).
|
||||
- Removed fields must be marked `reserved` — never reuse a field number.
|
||||
- Breaking changes require a new package version (`oo.events.v2`) and a `schemaVersion` bump in the envelope.
|
||||
|
||||
## Schema registry / CI gate
|
||||
|
||||
`buf` enforces lint and breaking-change detection on every PR that touches `events/`:
|
||||
|
||||
```bash
|
||||
# Lint
|
||||
buf lint events/
|
||||
|
||||
# Breaking-change check against main
|
||||
buf breaking events/ --against '.git#branch=main,subdir=packages/shared-types/events'
|
||||
```
|
||||
|
||||
Local shortcut: `./scripts/buf-check.sh`
|
||||
|
||||
CI: `.gitea/workflows/buf-check.yaml` (requires a Gitea Actions runner).
|
||||
|
||||
Install buf: `curl -sSfL https://github.com/bufbuild/buf/releases/latest/download/buf-Linux-x86_64 -o /usr/local/bin/buf && chmod +x /usr/local/bin/buf`
|
||||
|
||||
## Contract
|
||||
|
||||
`/health` — not applicable (library package, no process).
|
||||
|
||||
**Extraction criteria** — always a shared library. Extract to a separate registry
|
||||
service only when schema governance requires independent versioning and deployment
|
||||
(e.g. external consumers, SLA divergence from the monorepo).
|
||||
Reference in New Issue
Block a user