AgapHost/openai/cognee-llm/README.md

cognee-llm (:8011)

OpenAI-compatible wrapper around the Kimi Code CLI (@moonshot-ai/kimi-code, home /root/.kimi-code), built for Cognee's batch/structured LLM calls. Opposite policy to kimi-agent:

• Stateless one-shot — fresh temp dir under /workspace/<uuid> per request, kimi -p <prompt> --output-format stream-json, no -r/-S resume, dir removed after every call (success or failure).
• Non-streaming — always returns a full chat.completion body, even if the caller sets stream: true.
• No media, no MCP — text-only prompt built from messages; no image persistence, no .mcp.json.
• Structured/low-temperature intent via prompt, not a sampling param — the CLI has no raw temperature knob (it's an agent loop, not a completions API), so determinism/JSON-only output is enforced with an instruction preamble prepended to the caller's system prompt.
• Bounded concurrency — MAX_CONCURRENCY = 3 in server.js, queued beyond that.

Endpoints: GET /v1/models (model id cognee-llm), POST /v1/chat/completions.

Own disposable in-container /workspace (no host bind mount — nothing here is meant to survive a request, let alone a container restart) + own cognee-llm-home volume (/root/.kimi-code), same Kimi subscription as kimi-agent/adolf-llm, separate volume so each wrapper's CLI state stays isolated.

Important: this should NOT be Cognee's default LLM backend

Per docs/SPIKE-FINDINGS.md gate 5 (P0 spike, empirically measured against a throwaway authed container):

• JSON output from the CLI is clean and schema-conformant when instructed — that part works.
• Latency is the blocker: ~5s fixed per-invocation floor (process spawn, config/credential load) even for a trivial call, ~22-24s for a realistic structured extraction call. Cognify issues one such call per chunk/entity-extraction step, so a batch of even a few dozen chunks reaches many minutes of wall time serialized.
• Every call is agentic (tool-call round trips are possible even for "just extract JSON" prompts), and hammering the single-seat Kimi subscription with concurrent batch CLI spawns risks rate-limiting/throttling that hasn't been (and shouldn't be) tested at scale.

Recommendation: default Cognee's LLM_API_BASE to a LiteLLM-routed model (judge/local qwen, per ARCHITECTURE.md §3.3's own stated fallback), not this wrapper. This service stays buildable/available as the optional, low-volume path (http://cognee-llm:8011/v1) — e.g. for experimentation or if a future need specifically wants Kimi-subscription-backed structured calls — but P4 should wire Cognee's default LLM to LiteLLM, not here.

Smoke test

cd /home/alvis/agap_git/openai
docker build -t cognee-llm:local ./cognee-llm
docker run --rm -d --name cognee-llm-smoke -p 18011:8011 cognee-llm:local
curl -s http://localhost:18011/v1/models
docker rm -f cognee-llm-smoke

A full /v1/chat/completions round-trip needs a kimi login-authed /root/.kimi-code volume (shared Kimi subscription) — not present in a bare smoke container, so that step is deferred to integration/P4 wiring.