From fb93655636312e4a44718dd0cff0ff0e3cd88a21 Mon Sep 17 00:00:00 2001 From: alvis Date: Sun, 5 Jul 2026 13:19:18 +0000 Subject: [PATCH] =?UTF-8?q?cognee-llm:=20correct=20docs=20=E2=80=94=20it?= =?UTF-8?q?=20IS=20cognee's=20LLM=20backbone=20(Kimi=20path)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reverse the earlier 'default to LiteLLM' recommendation: per user intent, cognee runs its LLM on the Kimi subscription via cognee-llm (the reason the wrapper exists). Gate-5 latency is an accepted tradeoff; LiteLLM stays a documented fallback. Embeddings remain on LiteLLM nomic-embed. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01LeqyaxJF2nbRXJtae2kNB2 --- openai/cognee-llm/README.md | 29 +++++++++++++---------------- 1 file changed, 13 insertions(+), 16 deletions(-) diff --git a/openai/cognee-llm/README.md b/openai/cognee-llm/README.md index f901b0a..682e9cb 100644 --- a/openai/cognee-llm/README.md +++ b/openai/cognee-llm/README.md @@ -23,25 +23,22 @@ survive a request, let alone a container restart) + own `cognee-llm-home` volume (`/root/.kimi-code`), same Kimi subscription as `kimi-agent`/`adolf-llm`, separate volume so each wrapper's CLI state stays isolated. -## Important: this should NOT be Cognee's default LLM backend +## This IS Cognee's LLM backbone -Per `docs/SPIKE-FINDINGS.md` gate 5 (P0 spike, empirically measured against a throwaway authed -container): +By design, Cognee's LLM runs on the flat Kimi subscription through this wrapper — the whole +reason it exists — mirroring how `adolf-llm` backs the assistant. P4 wires cognee's +`LLM_ENDPOINT` → `http://cognee-llm:8011`, `LLM_MODEL` → `openai/cognee-llm`. -- JSON output from the CLI is clean and schema-conformant when instructed — that part works. -- **Latency is the blocker**: ~5s fixed per-invocation floor (process spawn, config/credential - load) even for a trivial call, ~22-24s for a realistic structured extraction call. Cognify - issues one such call per chunk/entity-extraction step, so a batch of even a few dozen chunks - reaches many minutes of wall time serialized. -- Every call is agentic (tool-call round trips are possible even for "just extract JSON" - prompts), and hammering the single-seat Kimi subscription with concurrent batch CLI spawns - risks rate-limiting/throttling that hasn't been (and shouldn't be) tested at scale. +**Accepted tradeoff (SPIKE-FINDINGS gate 5).** The CLI's JSON output is clean/schema-conformant, +but it's slower than a raw API: ~5s fixed per-invocation floor + ~22-24s for a realistic +structured-extraction call, and every call is agentic. Cognify issues one call per +chunk/entity-extraction step, so large batches serialize into minutes. To protect the +single-seat subscription, `MAX_CONCURRENCY = 3` bounds concurrent spawns. -**Recommendation: default Cognee's `LLM_API_BASE` to a LiteLLM-routed model (`judge`/local -qwen, per `ARCHITECTURE.md` §3.3's own stated fallback), not this wrapper.** This service stays -buildable/available as the optional, low-volume path (`http://cognee-llm:8011/v1`) — e.g. for -experimentation or if a future need specifically wants Kimi-subscription-backed structured -calls — but P4 should wire Cognee's default LLM to LiteLLM, not here. +**Documented fallback (not the default):** if cognify throughput ever becomes a real problem, +route cognee's LLM to a LiteLLM model instead (`ARCHITECTURE.md` §3.3) — see the commented block +in `cognee/cognee.env`. Embeddings already run on LiteLLM's `nomic-embed` regardless (embeddings +can't go through the agentic CLI). ## Smoke test