feat(clustering): persistent enrichment cache in task_enrichments table

Each unique task title is now enriched by LiteLLM once and cached in the DB.
Subsequent agent compute cycles (every 12h) fetch the cache before calling
ml-serving; only new titles hit the tip-generator.

- DB: task_enrichments(content_hash PK, description, model, created_at)
- TS: fetchEnrichmentCache / persistEnrichments helpers in agent-outputs.ts;
  enrichment_cache passed in compute request, new_enrichments persisted from response
- Python: AgentComputeRequest.enrichment_cache / AgentComputeResponse.new_enrichments;
  AgentInput.enrichment_cache; _enrich_batch returns (descriptions, new_entries);
  cluster_tasks returns (clusters, new_enrichments)
- FocusAreaAgent stashes new_enrichments in signals_snapshot under _new_enrichments;
  compute_agent endpoint pops it before storing the snapshot

Closes part of #129

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-05-12 14:39:35 +00:00
parent 08d08ad7b0
commit 9ddeea6cac
9 changed files with 158 additions and 40 deletions

View File

@@ -149,6 +149,13 @@ export function runMigrations(handle: BetterSqlite3Database) {
CREATE INDEX IF NOT EXISTS idx_agent_outputs_user_agent_exp
ON agent_outputs(user_id, agent_id, expires_at DESC);
CREATE TABLE IF NOT EXISTS task_enrichments (
content_hash TEXT PRIMARY KEY,
description TEXT NOT NULL,
model TEXT NOT NULL DEFAULT 'tip-generator',
created_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS user_preferences (
user_id TEXT NOT NULL REFERENCES users(id),
scope TEXT NOT NULL,
@@ -208,6 +215,15 @@ export function runMigrations(handle: BetterSqlite3Database) {
`);
} catch { /* column already dropped — nothing to backfill */ }
// Backfill (issue #127): grant data:<provider> consent for every active integration token.
// Idempotent — INSERT OR IGNORE skips rows that already exist.
handle.exec(`
INSERT OR IGNORE INTO user_consents (user_id, consent_key, granted_at)
SELECT user_id, 'data:' || provider, connected_at
FROM integration_tokens
WHERE token_status = 'active'
`);
// Drop legacy consent columns (ADR-0014 step 8). Runs after the backfill above.
// Silently skips if already dropped (column not found error) or never existed (new DB).
for (const stmt of [

View File

@@ -189,6 +189,15 @@ export const agentOutputs = sqliteTable('agent_outputs', {
agentVersion: text('agent_version').notNull(), // bump to invalidate on logic changes
});
// Persistent cache for LLM-enriched task descriptions used by clustering.
// Keyed by MD5 of raw task content; avoids re-calling LiteLLM on every agent compute cycle.
export const taskEnrichments = sqliteTable('task_enrichments', {
contentHash: text('content_hash').primaryKey(),
description: text('description').notNull(),
model: text('model').notNull().default('tip-generator'),
createdAt: text('created_at').notNull(),
});
// Admin saved SQL queries.
export const savedQueries = sqliteTable('saved_queries', {
id: text('id').primaryKey(),