feat(recommender): LLM schema validation + hardcoded fallback tips on AI failure (#90)
Python (ml/serving):
- Validate tip item after JSON parse: non-empty content, valid kind
- Retry on schema failure with a targeted clarification prompt, same 2× retry budget
- JSON parse failures keep the existing retry suffix
TypeScript (recommender):
- Add TipSource 'fallback' to shared-types
- FALLBACK_TIPS: 12 general-purpose life tips (hardcoded, no DB read)
- fetchOrchestratorTip returns {ok} discriminated union instead of null
- On !res.ok or fetch error: serve a random fallback tip with rationale 'AI service issues'
- Update tests: 204 path removed; both failure cases now expect source='fallback'
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -293,6 +293,25 @@ _RETRY_SUFFIX_OBJ = (
|
||||
"Reply ONLY with the JSON object — no prose, no markdown fences."
|
||||
)
|
||||
|
||||
_RETRY_SUFFIX_SCHEMA = (
|
||||
"\n\nYour previous response parsed as JSON but was missing required fields. "
|
||||
'Reply ONLY with a JSON object containing "content" (non-empty string) and "kind" '
|
||||
'(one of: advice, task, insight, reminder) — no prose, no markdown fences.'
|
||||
)
|
||||
|
||||
_VALID_KINDS = {"advice", "task", "insight", "reminder"}
|
||||
|
||||
|
||||
def _validate_tip_item(item: dict) -> str | None:
|
||||
"""Return an error string if item fails schema, else None."""
|
||||
content = item.get("content", "")
|
||||
if not isinstance(content, str) or not content.strip():
|
||||
return "missing or empty 'content' field"
|
||||
kind = item.get("kind", "")
|
||||
if kind and kind not in _VALID_KINDS:
|
||||
return f"invalid kind '{kind}', must be one of {_VALID_KINDS}"
|
||||
return None
|
||||
|
||||
|
||||
@app.post("/agents/{agent_id}/compute", response_model=AgentComputeResponse)
|
||||
async def compute_agent(agent_id: str, req: AgentComputeRequest) -> AgentComputeResponse:
|
||||
@@ -504,11 +523,15 @@ async def recommend(req: RecommendRequest) -> RecommendResponse:
|
||||
text = text[4:]
|
||||
parsed = json.loads(text)
|
||||
item: dict = parsed[0] if isinstance(parsed, list) else parsed
|
||||
schema_err = _validate_tip_item(item)
|
||||
if schema_err:
|
||||
raise ValueError(schema_err)
|
||||
break
|
||||
except (json.JSONDecodeError, ValueError, IndexError) as exc:
|
||||
last_parse_error = str(exc)
|
||||
messages.append({"role": "assistant", "content": last_raw})
|
||||
messages.append({"role": "user", "content": _RETRY_SUFFIX_OBJ})
|
||||
is_schema_err = not isinstance(exc, json.JSONDecodeError)
|
||||
messages.append({"role": "user", "content": _RETRY_SUFFIX_SCHEMA if is_schema_err else _RETRY_SUFFIX_OBJ})
|
||||
else:
|
||||
_end_span(llm_span, status="ERROR")
|
||||
_end_span(root, status="ERROR")
|
||||
|
||||
Reference in New Issue
Block a user