feat: MLOps external services, AI stack planning, admin MLOps hub
Infrastructure: - Add `mlops` compose profile: MLflow (basic-auth, /mlflow path) + Airflow (LocalExecutor, /airflow path) + airflow-db - infra/mlflow/basic_auth.ini for MLflow auth config - Caddy routes /mlflow* and /airflow* inside existing o.alogins.net block (see agap_git) - Dockerfile.admin: NEXT_PUBLIC_MLFLOW_URL / NEXT_PUBLIC_AIRFLOW_URL build args (default /mlflow, /airflow) Admin panel: - /admin/models: replace MLflow iframe with external link cards - /admin/experiments: replace LinUCB stats with MLOps hub (links to MLflow experiments/models + Airflow DAGs/datasets) - AdminShell: external nav links for MLflow ↗ and Airflow ↗ under MLOps section Docs & planning: - README: new AI stack section (Ollama/LiteLLM/OpenWebUI three-tier, tip generation pipeline, model aliases) - README: Phase 2 expanded with AI infra issues (#86-#93) and granular pipeline breakdown - README: Phase 4 expanded with LLM MLOps items (#94-#97) - CLAUDE.md: AI stack section, updated current phase (M1 shipped / M2 in progress), compose profiles, updated What NOT to do - docs/architecture/overview.md: AI stack section, updated decision flow diagram for Phase 2 LLM pipeline - ADR-0006: updated to reflect external services (path-based, not embedded) - Gitea issues #86-#97 created (M2: AI infra + pipeline; M4: LLM MLOps) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -28,15 +28,16 @@ Same stack as `apps/web`. Reuses `packages/shared-types`, the Auth.js session co
|
||||
| Heavy grids | **TanStack Table v8** | Sortable / paginated / virtualized tables for events, users, tips. |
|
||||
| Extra charts | **Recharts** | Fallback where Tremor falls short (histograms, distributions). |
|
||||
|
||||
### Embed, don't rebuild
|
||||
### Link out, don't embed
|
||||
|
||||
Specialized tooling is **reverse-proxied into the admin shell**, not reimplemented:
|
||||
Specialized MLOps tooling runs as **separate external services** with their own auth, linked from the admin shell — not embedded or reimplemented:
|
||||
|
||||
- **MLflow UI** → `/admin/models` (Caddy sub-path proxy)
|
||||
- **Grafana panels** → `/admin/infra` (iframed or embedded panels)
|
||||
- **MLflow** → `https://o.alogins.net/mlflow` — experiment tracking, model registry, artifact browser; own basic-auth for now; see M3 for SSO consolidation
|
||||
- **Airflow** → `https://o.alogins.net/airflow` — batch pipeline orchestration, dataset management; own web-auth for now
|
||||
- **Grafana panels** → `/admin/infra` (iframed panels) — infra metrics
|
||||
- **Marimo notebooks** → launch-out link from admin
|
||||
|
||||
This prevents reimplementing artifact browsers or graph renderers we'd never do as well.
|
||||
The admin shell links to these services; clicking them opens a new tab. The `/experiments` and `/models` admin pages are hub pages with direct links to the relevant MLflow/Airflow views.
|
||||
|
||||
### AuthZ
|
||||
|
||||
@@ -55,5 +56,7 @@ This prevents reimplementing artifact browsers or graph renderers we'd never do
|
||||
|
||||
- One more Next.js app in the monorepo. Build/dev added to Turborepo.
|
||||
- Tremor + shadcn/ui are added as dependencies. shadcn components are copied into `apps/admin/src/components/ui/` — no runtime version coupling.
|
||||
- MLflow and Grafana must be reachable from the Caddy reverse proxy; they are not embedded in the JS bundle.
|
||||
- MLflow (`o.alogins.net/mlflow*` → port 5000) and Airflow (`o.alogins.net/airflow*` → port 8080) are path-based routes in the existing `o.alogins.net` Caddy block, started via `docker compose --profile mlops up`.
|
||||
- Each service manages its own auth (MLflow: built-in basic-auth; Airflow: built-in web UI auth). M3 will consolidate both behind the shared OIDC provider.
|
||||
- The `NEXT_PUBLIC_MLFLOW_URL` and `NEXT_PUBLIC_AIRFLOW_URL` build args in `Dockerfile.admin` default to the production URLs; override for dev builds.
|
||||
- `admin_actions` audit log grows unboundedly — needs a retention policy before M4.
|
||||
|
||||
Reference in New Issue
Block a user