ai: migrate LLM backbone from Kimi CLI to Codex CLI

Retires the Moonshot/Kimi subscription in favour of the already-paid ChatGPT
plan. Both CLI wrappers now run `codex exec`; the kimi-agent container is gone.

adolf-llm + hindsight-llm:
- runKimi -> runCodex (`codex exec --json --skip-git-repo-check`), resume via
  `codex exec resume <thread_id>`.
- MCP moves from a per-session .mcp.json (a workaround for Kimi having no
  --mcp-config-file flag) to a $CODEX_HOME/config.toml generated once at
  startup from shared-mcp.json. Field translation is load-bearing:
  bearerTokenEnvVar -> bearer_token_env_var, enabledTools -> enabled_tools.
- approval_policy="never" + sandbox_mode required, or unattended turns block
  on an approval prompt nobody can answer.

kimi-agent removed. It was the ONLY large-tier deployment behind LiteLLM, so
deleting it outright would have silently degraded every large-tier request to
the local 4B model via the existing fallbacks. tier-large, the auto_router
complex-reasoning route and their fallbacks now point at the codex-backed
adolf-llm wrapper (model_name: codex-agent).

Three environment blockers fixed along the way:
- OpenAI geo-blocks this host (403 unsupported_country_region_territory).
  Both containers now egress via the host xray proxy, with NO_PROXY keeping
  MCP and *.alogins.net traffic off the tunnel.
- node:22-slim ships no system CA store; the Rust codex binary validates TLS
  against it, so every HTTPS call failed with a generic transport error while
  Node's own fetch worked. ca-certificates added to both images.
- `codex exec resume` rejects -C/--cd (plain `codex exec` accepts it), which
  broke follow-up turns while first turns succeeded.

Known regression: Kimi's managed-usage API has no Codex equivalent, so the
/usage route returns 501 and there is no quota probe for the codex model.
The two quota plugins degrade quietly to no output.

Also: stop tracking cognee.env (live LLM + JWT secrets) and gitignore it.
The secrets remain in earlier history and should be rotated.

Verified live: plain turn, SSE streaming, session resume, MCP tool call,
bearer-token MCP call, and completions through both LiteLLM routes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Y5QPagv4iun1ghpwM96Ff
This commit is contained in:
2026-08-01 06:13:27 +00:00
parent a27bae828a
commit 9094d71e2f
66 changed files with 653 additions and 851 deletions

11
ai/cognee-llm/Dockerfile Normal file
View File

@@ -0,0 +1,11 @@
FROM node:22-slim
RUN npm install -g @moonshot-ai/kimi-code
WORKDIR /workspace
COPY server.js /app/server.js
EXPOSE 8011
ENTRYPOINT ["node", "/app/server.js"]

62
ai/cognee-llm/README.md Normal file
View File

@@ -0,0 +1,62 @@
# cognee-llm (:8011)
> ⚠️ **SUPERSEDED — Adolf's memory is migrating Cognee → Hindsight (2026-07-13).**
> Hindsight runs its LLM on LiteLLM `:4000` / Ollama, so this bespoke Kimi-CLI
> wrapper is being **retired**, not ported (SPIKE gate 5 already concluded the
> extraction workload shouldn't sit on the Kimi seat). This service is decommissioned
> in migration task **H4**. Plan: `agap_git/adolf/HINDSIGHT-MIGRATION.md`. The doc
> below describes the outgoing Cognee stack, kept until H4 lands.
OpenAI-compatible wrapper around the Kimi Code CLI (`@moonshot-ai/kimi-code`, home
`/root/.kimi-code`), built for Cognee's batch/structured LLM calls. **Opposite policy to
`kimi-agent`**:
- **Stateless one-shot** — fresh temp dir under `/workspace/<uuid>` per request, `kimi -p
<prompt> --output-format stream-json`, **no `-r`/`-S` resume**, dir removed after every call
(success or failure).
- **Non-streaming** — always returns a full `chat.completion` body, even if the caller sets
`stream: true`.
- **No media, no MCP** — text-only prompt built from `messages`; no image persistence, no
`.mcp.json`.
- **Structured/low-temperature intent via prompt, not a sampling param** — the CLI has no raw
temperature knob (it's an agent loop, not a completions API), so determinism/JSON-only output
is enforced with an instruction preamble prepended to the caller's system prompt.
- **Bounded concurrency** — `MAX_CONCURRENCY = 3` in `server.js`, queued beyond that.
Endpoints: `GET /v1/models` (model id `cognee-llm`), `POST /v1/chat/completions`.
Own disposable in-container `/workspace` (no host bind mount — nothing here is meant to
survive a request, let alone a container restart) + own `cognee-llm-home` volume
(`/root/.kimi-code`), same Kimi subscription as `kimi-agent`/`adolf-llm`, separate volume so
each wrapper's CLI state stays isolated.
## This IS Cognee's LLM backbone
By design, Cognee's LLM runs on the flat Kimi subscription through this wrapper — the whole
reason it exists — mirroring how `adolf-llm` backs the assistant. P4 wires cognee's
`LLM_ENDPOINT` → `http://cognee-llm:8011`, `LLM_MODEL` → `openai/cognee-llm`.
**Accepted tradeoff (SPIKE-FINDINGS gate 5).** The CLI's JSON output is clean/schema-conformant,
but it's slower than a raw API: ~5s fixed per-invocation floor + ~22-24s for a realistic
structured-extraction call, and every call is agentic. Cognify issues one call per
chunk/entity-extraction step, so large batches serialize into minutes. To protect the
single-seat subscription, `MAX_CONCURRENCY = 3` bounds concurrent spawns.
**Documented fallback (not the default):** if cognify throughput ever becomes a real problem,
route cognee's LLM to a LiteLLM model instead (`ARCHITECTURE.md` §3.3) — see the commented block
in `cognee/cognee.env`. Embeddings already run on LiteLLM's `nomic-embed` regardless (embeddings
can't go through the agentic CLI).
## Smoke test
```bash
cd /home/alvis/agap_git/ai
docker build -t cognee-llm:local ./cognee-llm
docker run --rm -d --name cognee-llm-smoke -p 18011:8011 cognee-llm:local
curl -s http://localhost:18011/v1/models
docker rm -f cognee-llm-smoke
```
A full `/v1/chat/completions` round-trip needs a `kimi login`-authed
`/root/.kimi-code` volume (shared Kimi subscription) — not present in a bare smoke container,
so that step is deferred to integration/P4 wiring.

178
ai/cognee-llm/server.js Normal file
View File

@@ -0,0 +1,178 @@
const http = require('http');
const fs = require('fs');
const path = require('path');
const crypto = require('crypto');
const { spawn } = require('child_process');
const PORT = 8011;
const MODEL_ID = 'cognee-llm';
const TIMEOUT_MS = 5 * 60 * 1000; // one-shot structured calls; generous but bounded
// Bounded parallelism: SPIKE-FINDINGS.md gate 5 flagged the Kimi subscription as a
// single-seat, interactive-oriented plan — batch cognify must not hammer it with
// unbounded concurrent CLI spawns (rate-limit/throttle risk on a shared live account).
const MAX_CONCURRENCY = 3;
const WORKSPACE = '/workspace';
fs.mkdirSync(WORKSPACE, { recursive: true });
// The CLI has no raw sampling-temperature knob (it's an agent loop, not a
// completions API) — "low temperature" for structured extraction is enforced
// via an instruction preamble instead, prepended to whatever system prompt
// the caller (Cognee) supplies.
const STRUCTURED_SYSTEM_PREAMBLE = [
'You are a stateless structured-extraction engine.',
'This is a one-shot call with no memory of prior calls: do not reference earlier turns.',
'Respond deterministically and concisely. When asked for JSON, output raw JSON only',
'- no prose, no markdown code fences, no commentary before or after.',
].join(' ');
// --- message helpers ---------------------------------------------------------
// Text only, no media parts: this wrapper's policy is no-media/no-MCP, unlike
// adolf-llm which persists inbound images and lets the CLI's ReadMediaFile
// tool read them.
function textOf(msg) {
const c = msg.content;
if (Array.isArray(c)) return c.map(p => (typeof p.text === 'string' ? p.text : '')).join('\n');
return c == null ? '' : String(c);
}
function buildPrompt(messages) {
const systemParts = messages.filter(m => m.role === 'system').map(textOf);
const rest = messages.filter(m => m.role !== 'system');
const preamble = [STRUCTURED_SYSTEM_PREAMBLE, ...systemParts].join('\n\n');
const transcript = rest
.map(m => `${m.role === 'assistant' ? 'Assistant' : 'User'}: ${textOf(m)}`)
.join('\n\n');
return `${preamble}\n\n${transcript}`.trim();
}
// --- bounded concurrency queue -----------------------------------------------
let active = 0;
const queue = [];
function drain() {
if (queue.length && active < MAX_CONCURRENCY) queue.shift()();
}
function withSlot(fn) {
return new Promise((resolve, reject) => {
const run = () => {
active++;
fn().then(
v => { active--; drain(); resolve(v); },
e => { active--; drain(); reject(e); },
);
};
if (active < MAX_CONCURRENCY) run();
else queue.push(run);
});
}
// --- kimi invocation: stateless one-shot, no resume --------------------------
// Fresh temp dir per call, NO -r/-S session flag, discard the dir after.
// Returns the assembled text from --output-format stream-json:
// {"role":"assistant","content":"..."}
// (reuses the same parse core as kimi-agent/server.js's runKimi, minus the
// resume/session-id bookkeeping that wrapper needs and this one deliberately
// does not).
function runKimi({ prompt, cwd }) {
return new Promise((resolve, reject) => {
const args = ['-p', prompt, '--output-format', 'stream-json'];
const child = spawn('kimi', args, { cwd, timeout: TIMEOUT_MS });
let stdout = '';
let stderr = '';
child.stdout.on('data', d => { stdout += d; });
child.stderr.on('data', d => { stderr += d; });
child.on('error', reject);
child.on('close', code => {
const parts = [];
for (const line of stdout.split('\n')) {
const t = line.trim();
if (!t) continue;
let obj;
try { obj = JSON.parse(t); } catch { continue; }
if (obj.role === 'assistant' && obj.content) parts.push(obj.content);
}
const text = parts.join('').trim();
if (!text && code !== 0) {
reject(new Error(`kimi exited ${code}: ${stderr.slice(0, 2000)}`));
} else {
resolve(text);
}
});
});
}
async function handleTurn(messages) {
const prompt = buildPrompt(messages || []);
const reqId = crypto.randomUUID();
const dir = path.join(WORKSPACE, reqId);
fs.mkdirSync(dir, { recursive: true });
try {
return await withSlot(() => runKimi({ prompt, cwd: dir }));
} finally {
// Stateless one-shot: nothing about this call is meant to survive it, so
// the temp dir is discarded unconditionally, success or failure.
fs.rm(dir, { recursive: true, force: true }, () => {});
}
}
// --- OpenAI-compatible HTTP surface (non-streaming only) ---------------------
function completionBody(text) {
return {
id: `chatcmpl-${Date.now()}`,
object: 'chat.completion',
created: Math.floor(Date.now() / 1000),
model: MODEL_ID,
choices: [{
index: 0,
message: { role: 'assistant', content: text },
finish_reason: 'stop',
}],
usage: { prompt_tokens: 0, completion_tokens: 0, total_tokens: 0 },
};
}
const server = http.createServer((req, res) => {
if (req.method === 'GET' && req.url === '/v1/models') {
res.writeHead(200, { 'Content-Type': 'application/json' });
res.end(JSON.stringify({
object: 'list',
data: [{ id: MODEL_ID, object: 'model', owned_by: 'moonshot' }],
}));
return;
}
if (req.method === 'POST' && req.url === '/v1/chat/completions') {
let body = '';
req.on('data', d => { body += d; });
req.on('end', async () => {
let parsed;
try {
parsed = JSON.parse(body);
} catch {
res.writeHead(400, { 'Content-Type': 'application/json' });
res.end(JSON.stringify({ error: 'invalid JSON body' }));
return;
}
try {
const text = await handleTurn(parsed.messages || []);
// Non-streaming policy: always return the full body even if the
// caller sets stream:true. Cognee's batch cognify has no use for SSE,
// and a one-shot call has nothing to incrementally stream anyway.
res.writeHead(200, { 'Content-Type': 'application/json' });
res.end(JSON.stringify(completionBody(text)));
} catch (err) {
res.writeHead(500, { 'Content-Type': 'application/json' });
res.end(JSON.stringify({ error: String(err.message || err) }));
}
});
return;
}
res.writeHead(404, { 'Content-Type': 'application/json' });
res.end(JSON.stringify({ error: 'not found' }));
});
server.listen(PORT, () => console.log(`cognee-llm wrapper listening on :${PORT}`));

View File

@@ -0,0 +1,15 @@
# Intended service block for /home/alvis/agap_git/ai/docker-compose.yml.
# Not wired in yet (see P3 task note) — orchestrator merges this in and adds
# `cognee-llm-home` to the top-level `volumes:` section.
cognee-llm:
build: ./cognee-llm
container_name: cognee-llm
ports:
- "8011:8011"
volumes:
- cognee-llm-home:/root/.kimi-code
restart: unless-stopped
# Add to the top-level `volumes:` block:
# cognee-llm-home: