openai: deploy cognee + cognee-mcp memory service [Adolf P4]

Resolves the 4 P4 blockers and wires cognee/cognee-mcp into the openai
compose stack:

- qdrant: container was gone (data intact under /mnt/ssd/dbs/qdrant);
  brought back up, confirmed healthy on :6333.
- Embeddings: switched from a dead LiteLLM route to ollama directly
  (host.docker.internal:11436, nomic-embed-text, 768-dim), using cognee's
  dedicated OllamaEmbeddingEngine and its native /api/embed endpoint.
  Requires extra_hosts: host.docker.internal:host-gateway since ollama
  lives in a separate compose project.
- cognee-llm kimi auth: root cause was that cognee-llm had never been
  started, so its kimi-agent-home-equivalent volume didn't exist yet.
  Seeded cognee-llm-home from the already-authed kimi-agent-home volume
  (read-only copy of config/credentials/oauth/device_id); cognee-llm now
  serves real completions.
- mkdir'd cognee data/system dirs: confirmed present (done by user).

Also fixed three issues found only during a live end-to-end smoke test:
- VECTOR_DB_PROVIDER must be a real container env var, not just present in
  the mounted cognee.env — the qdrant adapter's sitecustomize.py
  registration hook reads os.environ directly, which pydantic-settings'
  env_file parsing never populates.
- Baked the Kuzu/Ladybug JSON extension into the cognee image. This
  deployment's egress to extension.ladybugdb.com is bandwidth-throttled to
  ~1.2 KB/s, so cognee's own runtime auto-download reliably timed out,
  leaving /health permanently unhealthy and graph queries failing. Fetched
  the ~827KB extension out-of-band (16-way parallel ranged GETs) and added
  it to the image via COPY.
- LLM_ENDPOINT needed an explicit /v1 suffix (litellm appends
  "/chat/completions" verbatim) and LLM_INSTRUCTOR_MODE=json_mode is
  required since cognee-llm's Kimi wrapper is a text-only pass-through with
  no real tool-calling support.

Verified with a full remember -> recall round trip through cognee-mcp's
MCP tool surface: stored a fact containing a codeword, recalled it via
GRAPH_COMPLETION search, got the exact codeword back. Exercises cognee-llm,
ollama embeddings, Qdrant, and Kuzu together.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LeqyaxJF2nbRXJtae2kNB2

openai/cognee/Dockerfile

@@ -0,0 +1,66 @@
+# Adolf P4 — cognee memory service.
+#
+# Base: official upstream image (do not hand-roll cognee itself). Adds ONE
+# thing upstream doesn't ship: Qdrant vector-store support. Qdrant is a
+# *community* adapter (separate PyPI package, not one of cognee's own
+# `[project.optional-dependencies]` extras — the image's own EXTRAS=
+# mechanism only installs cognee's own extras, so it can't pull this in).
+#
+# Version note: cognee-community-vector-adapter-qdrant's declared dependency
+# pin (both the PyPI release 0.2.4 -> cognee==0.5.6, and the unreleased
+# GitHub main 0.3.0 -> cognee==1.1.0) trails this image's cognee 1.2.2.
+# Installed with --no-deps (below) to avoid pip fighting that pin and
+# downgrading cognee. Verified compatible by direct import test on 2026-07-05:
+# both registry hooks the adapter calls (`use_vector_adapter`,
+# `use_dataset_database_handler` from cognee.infrastructure.databases.*)
+# exist unchanged in cognee 1.2.2, and a full container import of
+# cognee_community_vector_adapter_qdrant.register succeeds with no error
+# against this exact image. Not yet exercised against a live Qdrant round
+# trip (cognify + search) — do that once the LiteLLM LLM/embedder blockers
+# below are resolved, as a final confirmation.
+FROM cognee/cognee:1.2.2
+
+# qdrant-client is the adapter's one genuinely-missing runtime dependency
+# (starlette/instructor are already satisfied by cognee's own base deps).
+# Installed normally (with deps) since it's a fresh package, not a conflict.
+RUN /usr/local/bin/pip --python /app/.venv/bin/python install --no-cache-dir \
+    "qdrant-client>=1.18.0"
+
+# Pinned to a specific commit for reproducibility (no tagged release exists
+# yet compatible with our cognee version — see version note above).
+RUN /usr/local/bin/pip --python /app/.venv/bin/python install --no-cache-dir --no-deps \
+    "https://github.com/topoteretes/cognee-community/archive/52281288052970f57e533b9be75b64da9ac7c773.tar.gz#subdirectory=packages/vector/qdrant"
+
+# sitecustomize.py auto-imports at every Python interpreter start in this
+# venv. Gated on VECTOR_DB_PROVIDER so it's a no-op unless qdrant is actually
+# selected — this is the adapter's own documented registration call
+# (cognee-community-vector-adapter-qdrant README: "Import and register the
+# adapter in your code: from cognee_community_vector_adapter_qdrant import
+# register"), just run automatically instead of requiring a cognee source
+# edit to add the import.
+RUN printf '%s\n' \
+    'import os' \
+    'if os.environ.get("VECTOR_DB_PROVIDER") == "qdrant":' \
+    '    from cognee_community_vector_adapter_qdrant import register  # noqa: F401' \
+    > /app/.venv/lib/python3.12/site-packages/sitecustomize.py
+
+# Pre-installed Kuzu/Ladybug JSON extension (P4 deploy blocker fix, 2026-07-05).
+# cognee's graph adapter (cognee/infrastructure/databases/graph/ladybug/adapter.py)
+# always tries `LOAD EXTENSION JSON` on startup and on every /health graph check,
+# falling back to `INSTALL JSON` (a network download from
+# extension.ladybugdb.com) if not already cached at
+# ~/.lbdb/extension/<kuzu_version>/<platform>/json/libjson.lbug_extension. This
+# extension is required for recall/temporal-search graph queries — without it
+# cognee's /health reports "unhealthy" and graph queries that use JSON fail
+# with a Binder exception ("Extension: json ... has not been installed").
+#
+# This deployment's egress to extension.ladybugdb.com is severely
+# bandwidth-throttled (~1-1.2 KB/s per connection — confirmed via direct curl,
+# not a proxy/DNS block: TLS handshake and HTTP 200 succeed, the transfer
+# itself just crawls), so the runtime auto-download reliably times out before
+# the ~827KB file finishes, and every subsequent health check/query re-attempts
+# and fails the same way. Downloaded once out-of-band (16-way parallel ranged
+# GETs, ~846920 bytes, verified ELF shared object) and baked into the image
+# here so the container never needs to touch that host at runtime.
+COPY extensions/0.17.0/linux_amd64/json/libjson.lbug_extension \
+    /root/.lbdb/extension/0.17.0/linux_amd64/json/libjson.lbug_extension

openai/cognee/cognee.env

@@ -0,0 +1,141 @@
+# Adolf P4 — Cognee memory service config (mounted at /app/.env in the
+# `cognee` container; matches upstream's own docker-compose `.env` pattern).
+# cognee-mcp does NOT need this file — it runs in API mode (see
+# service-block.yml) and only ever talks HTTP to `cognee`, never touching
+# these DBs directly.
+
+ENV=local
+DEBUG=false
+LOG_LEVEL=INFO
+CORS_ALLOWED_ORIGINS=*
+
+###############################################################################
+# LLM — cognee runs on the Kimi subscription via the `cognee-llm` wrapper
+# (:8011, built in P3). This is the intended backbone: the whole reason
+# cognee-llm exists is to be cognee's LLM on the flat Kimi subscription (no
+# per-token cost), consistent with adolf-llm doing the same for the assistant.
+#
+# Tradeoff (SPIKE-FINDINGS gate 5, accepted): the agentic CLI adds latency
+# (~5s floor + ~22-24s/structured call) and runs on a single-seat subscription,
+# so batch cognify is slower than a raw API. cognee-llm bounds concurrency
+# (MAX_CONCURRENCY=3) to protect the account. If cognify throughput ever
+# becomes a problem, the LiteLLM route below is the documented fallback.
+#
+# Requires: `kimi login` seeded into the `cognee-llm-home` volume (same as
+# adolf-llm/kimi-agent).
+###############################################################################
+LLM_PROVIDER=openai
+LLM_MODEL=openai/cognee-llm
+# Must include /v1 — cognee's OpenAI-compatible LLM adapter passes this
+# straight through to litellm as api_base and litellm appends
+# "/chat/completions" verbatim (no path normalization). Without /v1 this hits
+# http://cognee-llm:8011/chat/completions, which 404s (cognee-llm only serves
+# /v1/chat/completions and /v1/models) — confirmed 2026-07-05 during the P4
+# smoke test (litellm.NotFoundError: Error code 404 - 'not found').
+LLM_ENDPOINT=http://cognee-llm:8011/v1
+LLM_API_KEY=sk-cognee-llm-local
+
+# Force instructor's plain JSON-in-content mode instead of its default
+# tool-calling mode. cognee-llm's Kimi CLI wrapper is a text-only pass-through
+# (no real OpenAI function/tool-calling support — it just returns
+# {"content": "..."}), so instructor's default mode for the "openai" provider
+# (tool-calling, since no explicit LLM_INSTRUCTOR_MODE means it never applies
+# json_schema_mode either) fails with "Instructor does not support multiple
+# tool calls, use List[Model] instead" — confirmed 2026-07-05 during the P4
+# smoke test. json_mode matches cognee-llm's own documented behavior
+# (STRUCTURED_SYSTEM_PREAMBLE: "When asked for JSON, output raw JSON only").
+LLM_INSTRUCTOR_MODE=json_mode
+
+# Fallback only (NOT the default) — route cognify's LLM to a LiteLLM model if
+# the Kimi CLI path is ever too slow under batch load. Requires a working
+# LiteLLM general model (fix judge's ANTHROPIC_API_KEY or a local qwen's port):
+#LLM_MODEL=openai/judge
+#LLM_ENDPOINT=http://litellm:4000
+
+###############################################################################
+# Embeddings — ollama directly (P4 blocker #1 resolution, per orchestrator:
+# "use ollama directly"). LiteLLM's `embedder` route was dead (port bug), so
+# rather than fix that indirection we go straight to ollama's own dedicated
+# embedding-engine implementation (OllamaEmbeddingEngine, verified present in
+# cognee 1.2.2's infra/databases/vector/embeddings/).
+#
+# Ollama lives in a SEPARATE compose project (not on this `openai` network),
+# reachable from containers only via host.docker.internal — hence
+# extra_hosts: host.docker.internal:host-gateway on the cognee service in
+# docker-compose.yml. Verified 2026-07-05: `curl host.docker.internal:11436`
+# from a throwaway container with that extra_hosts entry returns 200.
+#
+# EMBEDDING_ENDPOINT must be the FULL endpoint URL including path —
+# OllamaEmbeddingEngine POSTs directly to whatever EMBEDDING_ENDPOINT is (its
+# own default is "http://localhost:11434/api/embed"), unlike the
+# openai_compatible engine which appends its own path onto a base URL. Ollama's
+# native /api/embed (batch endpoint, not the singular /api/embeddings) returns
+# {"embeddings": [[...]]}; the engine handles that key. Tested directly against
+# :11436 with model nomic-embed-text -> 768-dim vector, confirmed working
+# before wiring this in.
+###############################################################################
+EMBEDDING_PROVIDER=ollama
+EMBEDDING_MODEL=nomic-embed-text
+EMBEDDING_ENDPOINT=http://host.docker.internal:11436/api/embed
+EMBEDDING_DIMENSIONS=768
+HUGGINGFACE_TOKENIZER=nomic-ai/nomic-embed-text-v1.5
+
+###############################################################################
+# Graph store — SPIKE-FINDINGS gate 4: Kuzu embedded, not Neo4j.
+# This is cognee's own default; listed explicitly for clarity.
+###############################################################################
+GRAPH_DATABASE_PROVIDER=kuzu
+GRAPH_DATASET_DATABASE_HANDLER=kuzu
+
+###############################################################################
+# Vector store — Qdrant (existing infra, :6333). Community adapter installed
+# via the custom Dockerfile in this directory (see comments there).
+###############################################################################
+VECTOR_DB_PROVIDER=qdrant
+VECTOR_DB_URL=http://qdrant:6333
+VECTOR_DB_KEY=
+VECTOR_DATASET_DATABASE_HANDLER=qdrant
+
+###############################################################################
+# Relational metadata DB (cognee's own bookkeeping, not the memory graph).
+###############################################################################
+DB_PROVIDER=sqlite
+DB_NAME=cognee_db
+
+###############################################################################
+# Storage paths — persisted under /mnt/ssd/dbs/cognee/ on the host (see
+# service-block.yml volume mounts to /data and /system).
+###############################################################################
+DATA_ROOT_DIRECTORY=/data
+SYSTEM_ROOT_DIRECTORY=/system
+
+###############################################################################
+# Single-user/single-agent posture. Adolf is one Matrix bot (SPIKE-FINDINGS
+# gate 4's own reasoning: no multi-tenant/concurrent-writer need at this
+# scale). Scoping happens at the *dataset* level (one dataset per OpenClaw
+# chat_id — see P4 report), not via cognee's own per-user auth/isolation
+# machinery, so we skip that machinery rather than bootstrap a default user
+# just to satisfy it.
+#
+# ENABLE_BACKEND_ACCESS_CONTROL=true (cognee's own default) would give each
+# (user, dataset) pair a fully isolated Kuzu+vector store, but *requires*
+# authentication (REQUIRE_AUTHENTICATION=false is ignored when this is true)
+# - extra machinery (default user bootstrap, token plumbing into cognee-mcp)
+# for no real benefit in a single-owner home deployment. With it off, all
+# datasets share one graph/vector backend; dataset_name/datasets filters on
+# remember/recall/forget still scope top-level data points per conversation,
+# with one documented caveat: GRAPH_COMPLETION search can traverse into
+# nodes from other datasets. Acceptable for one person's own conversation
+# threads; revisit (flip this flag + bootstrap a default user) if that
+# leakage ever matters.
+###############################################################################
+ENABLE_BACKEND_ACCESS_CONTROL=False
+REQUIRE_AUTHENTICATION=False
+
+# Only exercised if the above is ever flipped to true.
+FASTAPI_USERS_JWT_SECRET=059bd0fdd9cecc46d055cf589d4275bd34c0fb73543f286beff09da2c2d27b65
+FASTAPI_USERS_VERIFICATION_TOKEN_SECRET=7246494bb622c9c89417fbe0b94de6d7718f1338eb40dd370fb072873f921832
+FASTAPI_USERS_RESET_PASSWORD_TOKEN_SECRET=18ad75671edf003f0142aad124276268fa766e702ab6bdb71a75d1c71a688beb
+
+TOKENIZERS_PARALLELISM=false
+LITELLM_LOG=ERROR

openai/docker-compose.yml

@@ -181,6 +181,63 @@ services:
       - adolf-llm-home:/root/.kimi-code
     restart: unless-stopped
 
+  # cognee — Adolf's memory backend (P4). FastAPI + embedded Kuzu graph +
+  # Qdrant vectors. LLM via cognee-llm:8011 (Kimi CLI wrapper), embeddings via
+  # ollama directly (host.docker.internal:11436, separate compose project —
+  # hence extra_hosts below). Sole owner of the on-disk Kuzu/SQLite files
+  # under /mnt/ssd/dbs/cognee/ (Kuzu is not safe for concurrent multi-process
+  # access) — never run a second process against those files.
+  cognee:
+    build: ./cognee
+    container_name: cognee
+    restart: unless-stopped
+    environment:
+      # Real OS env var, not just the mounted .env file: the qdrant vector
+      # adapter's registration hook (cognee/Dockerfile's sitecustomize.py)
+      # gates on os.environ.get("VECTOR_DB_PROVIDER") at Python interpreter
+      # start, which only sees actual container env vars — pydantic-settings'
+      # env_file=".env" parsing (used for the rest of cognee.env) never
+      # populates os.environ itself. Without this, cognee raises
+      # "Unsupported vector database provider: qdrant" at startup even though
+      # cognee.env sets VECTOR_DB_PROVIDER=qdrant. Verified 2026-07-05.
+      - VECTOR_DB_PROVIDER=qdrant
+    volumes:
+      - ./cognee/cognee.env:/app/.env
+      - /mnt/ssd/dbs/cognee/data:/data
+      - /mnt/ssd/dbs/cognee/system:/system
+    extra_hosts:
+      - "host.docker.internal:host-gateway"
+    # Not published to the host — only cognee-mcp (same compose network)
+    # needs to reach it. Uncomment for local debugging:
+    # ports:
+    #   - "8000:8000"
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 40s
+
+  # cognee-mcp — thin MCP-to-HTTP proxy in API mode (API_URL=cognee:8000).
+  # Never opens the graph/vector files itself, so it's safe to run alongside
+  # `cognee` without a second writer on the same Kuzu database. Exposes 3
+  # tools: remember / recall / forget.
+  cognee-mcp:
+    image: cognee/cognee-mcp:1.2.2
+    container_name: cognee-mcp
+    restart: unless-stopped
+    environment:
+      - ENV=local
+      - LOG_LEVEL=INFO
+      - PYTHONUNBUFFERED=1
+      - TRANSPORT_MODE=http
+      - API_URL=http://cognee:8000
+      - MCP_ALLOWED_HOSTS=cognee-mcp:*
+    ports:
+      - "8001:8000"
+    depends_on:
+      - cognee
+
 volumes:
   kimi-agent-home:
   adolf-state:

openai/shared-mcp.json

@@ -0,0 +1,8 @@
+{
+  "mcpServers": {
+    "cognee": {
+      "type": "http",
+      "url": "http://cognee-mcp:8000/mcp"
+    }
+  }
+}