openai: add cognee-llm stateless Kimi-CLI wrapper (:8011) [Adolf P3]
Stateless one-shot wrapper for Cognee batch cognify: fresh temp dir per request, no resume, non-streaming, text-only, bounded concurrency (3). Per SPIKE-FINDINGS gate 5, Cognee should default its LLM to LiteLLM; this is the optional low-volume path. New service + cognee-llm-home volume in compose. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LeqyaxJF2nbRXJtae2kNB2
This commit is contained in:
58
openai/cognee-llm/README.md
Normal file
58
openai/cognee-llm/README.md
Normal file
@@ -0,0 +1,58 @@
|
||||
# cognee-llm (:8011)
|
||||
|
||||
OpenAI-compatible wrapper around the Kimi Code CLI (`@moonshot-ai/kimi-code`, home
|
||||
`/root/.kimi-code`), built for Cognee's batch/structured LLM calls. **Opposite policy to
|
||||
`kimi-agent`**:
|
||||
|
||||
- **Stateless one-shot** — fresh temp dir under `/workspace/<uuid>` per request, `kimi -p
|
||||
<prompt> --output-format stream-json`, **no `-r`/`-S` resume**, dir removed after every call
|
||||
(success or failure).
|
||||
- **Non-streaming** — always returns a full `chat.completion` body, even if the caller sets
|
||||
`stream: true`.
|
||||
- **No media, no MCP** — text-only prompt built from `messages`; no image persistence, no
|
||||
`.mcp.json`.
|
||||
- **Structured/low-temperature intent via prompt, not a sampling param** — the CLI has no raw
|
||||
temperature knob (it's an agent loop, not a completions API), so determinism/JSON-only output
|
||||
is enforced with an instruction preamble prepended to the caller's system prompt.
|
||||
- **Bounded concurrency** — `MAX_CONCURRENCY = 3` in `server.js`, queued beyond that.
|
||||
|
||||
Endpoints: `GET /v1/models` (model id `cognee-llm`), `POST /v1/chat/completions`.
|
||||
|
||||
Own disposable in-container `/workspace` (no host bind mount — nothing here is meant to
|
||||
survive a request, let alone a container restart) + own `cognee-llm-home` volume
|
||||
(`/root/.kimi-code`), same Kimi subscription as `kimi-agent`/`adolf-llm`, separate volume so
|
||||
each wrapper's CLI state stays isolated.
|
||||
|
||||
## Important: this should NOT be Cognee's default LLM backend
|
||||
|
||||
Per `docs/SPIKE-FINDINGS.md` gate 5 (P0 spike, empirically measured against a throwaway authed
|
||||
container):
|
||||
|
||||
- JSON output from the CLI is clean and schema-conformant when instructed — that part works.
|
||||
- **Latency is the blocker**: ~5s fixed per-invocation floor (process spawn, config/credential
|
||||
load) even for a trivial call, ~22-24s for a realistic structured extraction call. Cognify
|
||||
issues one such call per chunk/entity-extraction step, so a batch of even a few dozen chunks
|
||||
reaches many minutes of wall time serialized.
|
||||
- Every call is agentic (tool-call round trips are possible even for "just extract JSON"
|
||||
prompts), and hammering the single-seat Kimi subscription with concurrent batch CLI spawns
|
||||
risks rate-limiting/throttling that hasn't been (and shouldn't be) tested at scale.
|
||||
|
||||
**Recommendation: default Cognee's `LLM_API_BASE` to a LiteLLM-routed model (`judge`/local
|
||||
qwen, per `ARCHITECTURE.md` §3.3's own stated fallback), not this wrapper.** This service stays
|
||||
buildable/available as the optional, low-volume path (`http://cognee-llm:8011/v1`) — e.g. for
|
||||
experimentation or if a future need specifically wants Kimi-subscription-backed structured
|
||||
calls — but P4 should wire Cognee's default LLM to LiteLLM, not here.
|
||||
|
||||
## Smoke test
|
||||
|
||||
```bash
|
||||
cd /home/alvis/agap_git/openai
|
||||
docker build -t cognee-llm:local ./cognee-llm
|
||||
docker run --rm -d --name cognee-llm-smoke -p 18011:8011 cognee-llm:local
|
||||
curl -s http://localhost:18011/v1/models
|
||||
docker rm -f cognee-llm-smoke
|
||||
```
|
||||
|
||||
A full `/v1/chat/completions` round-trip needs a `kimi login`-authed
|
||||
`/root/.kimi-code` volume (shared Kimi subscription) — not present in a bare smoke container,
|
||||
so that step is deferred to integration/P4 wiring.
|
||||
Reference in New Issue
Block a user