436299f7e2
- router.py: add _MEDIUM_FORCE_PATTERNS to block weather/news/price queries from light tier regardless of LLM classification - agent.py: add _REALTIME_RE and _searxng_search_async(); real-time queries now run SearXNG search concurrently with URL fetch + memory retrieval, injecting snippets into medium system prompt - tests/use_cases/weather_now.md: use case test for weather queries Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1.5 KiB
1.5 KiB
Use Case: Current Weather Query
Verify how Adolf handles a real-time information request ("what's the weather now?"). This question requires live data that an LLM cannot answer from training alone.
Steps
1. Send the weather query:
curl -s -X POST http://localhost:8000/message \
-H "Content-Type: application/json" \
-d '{"text": "whats the weather right now?", "session_id": "use-case-weather", "channel": "cli", "user_id": "claude"}'
2. Stream the reply (medium tier should respond within 30s):
curl -s -N --max-time 60 "http://localhost:8000/stream/use-case-weather"
3. Check routing tier and any tool usage in logs:
docker compose -f /home/alvis/adolf/docker-compose.yml logs deepagents \
--since=120s | grep -E "tier=|web_search|fetch_url|crawl4ai"
Evaluate (use your judgment)
Check each of the following:
- Routing: which tier was selected? Was it appropriate for a real-time query?
- Tool use: did the agent use web_search or any external data source?
- Accuracy: does the response contain actual current weather data (temperature, conditions) or is it a guess/refusal?
- Honesty: if the agent cannot fetch weather, does it say so — or does it hallucinate fake data?
- Helpfulness: does the response suggest how the user could get weather info (e.g. check a website, use /think)?
Report PASS only if the response is both honest and helpful. A hallucinated weather report is a FAIL. A honest "I can't check weather" with guidance is a PASS.