feat: TipCandidate shared schema — typed candidates through the bandit pipeline #89

New Issue

alvis · 2026-04-17T08:11:11Z

alvis commented

2026-04-17 08:11:11 +00:00

Goal

Define a first-class TipCandidate type that flows from the LLM generator through the bandit scorer to the recommender API.

Schema

// packages/shared-types/src/tip.ts
export interface TipCandidate {
  id: string;            // uuid, stable for dedup
  content: string;       // the tip text shown to the user
  kind: TipKind;         // task | advice | insight | reminder
  source: TipSource;     // llm | task_direct | fallback
  model?: string;        // e.g. "qwen2.5:7b" — null for task_direct
  prompt_version?: string; // e.g. "v2" — null for task_direct
  confidence?: number;   // 0–1, model self-reported
  task_id?: string;      // set when source=task_direct or kind=task
}

export type TipKind = "task" | "advice" | "insight" | "reminder";
export type TipSource = "llm" | "task_direct" | "fallback";

Changes

Add to packages/shared-types
Update ml/serving /score endpoints to accept + return TipCandidate[]
Update services/api/src/routes/recommender.ts to pipe TipCandidate through scoring
Update tip_scores table: add kind, source, model, prompt_version columns
Update admin reward analytics dashboard to group by these new columns

Why now

This schema is the seam between the LLM generator (#79) and the bandit scorer. Defining it first lets both sides develop independently. The bandit doesn't care whether a candidate came from an LLM or a task list — it just scores features.

## Goal Define a first-class `TipCandidate` type that flows from the LLM generator through the bandit scorer to the recommender API. ## Schema ```typescript // packages/shared-types/src/tip.ts export interface TipCandidate { id: string; // uuid, stable for dedup content: string; // the tip text shown to the user kind: TipKind; // task | advice | insight | reminder source: TipSource; // llm | task_direct | fallback model?: string; // e.g. "qwen2.5:7b" — null for task_direct prompt_version?: string; // e.g. "v2" — null for task_direct confidence?: number; // 0–1, model self-reported task_id?: string; // set when source=task_direct or kind=task } export type TipKind = "task" | "advice" | "insight" | "reminder"; export type TipSource = "llm" | "task_direct" | "fallback"; ``` ## Changes - Add to `packages/shared-types` - Update `ml/serving` `/score` endpoints to accept + return `TipCandidate[]` - Update `services/api/src/routes/recommender.ts` to pipe `TipCandidate` through scoring - Update `tip_scores` table: add `kind`, `source`, `model`, `prompt_version` columns - Update admin reward analytics dashboard to group by these new columns ## Why now This schema is the seam between the LLM generator (#79) and the bandit scorer. Defining it first lets both sides develop independently. The bandit doesn't care whether a candidate came from an LLM or a task list — it just scores features.

alvis added this to the M2 — AI tips + multi-source signals milestone 2026-04-17 08:11:11 +00:00

alvis referenced this issue

2026-04-17 08:11:30 +00:00

feat: LLM output validation + structured JSON retry for tip generation #90

alvis referenced this issue

2026-04-17 08:11:55 +00:00

feat: prompt versioning — track prompt_version + model in tip_scores #91

alvis referenced this issue

2026-04-17 08:12:37 +00:00

feat: LLM tip quality monitoring dashboard in admin #92

alvis closed this issue

2026-04-17 14:22:49 +00:00