nexus/.planning/research/STACK.md

# Technology Stack: v1.7 Content Generation

**Project:** Nexus v1.7 — additive to v1.6 stack (see prior STACK.md for voice pipeline, Telegram bridge, ffmpeg-static, grammy, @ricky0123/vad-react)
**Researched:** 2026-04-04
**Scope:** NEW libraries only for v1.7 — presentations/video, image generation, diagram rendering, PDF generation, SVG icons, color/theme tools, social media assets
**Confidence:** MEDIUM-HIGH (Remotion HIGH via official docs; satori+resvg-js HIGH via official repos; mermaid already installed; ComfyUI client MEDIUM; culori MEDIUM via comparison sources)

---

## Context: What Is Already Installed

Do not re-add or re-research these — confirmed present in `server/package.json` or `ui/package.json`:

| Package | Location | Version | Relevant To v1.7 |
|---------|----------|---------|-----------------|
| `sharp ^0.34.5` | `server/` | 0.34.5 | SVG→PNG conversion, image compositing, social asset output |
| `mermaid ^11.12.0` | `ui/` | 11.12.0 | Client-side diagram rendering already works |
| `ffmpeg-static ^5.3.0` | `server/` | 5.3.0 | Video stitching, audio for Remotion renders |
| `zod ^3.24.2` | `server/` | 3.24.2 | Schema validation for content generation requests |
| `express ^5.1.0` | `server/` | 5.1.0 | API endpoints for all content generation routes |

The v1.7 work **adds** new rendering capabilities — it does not replace or duplicate any of the above.

---

## New Libraries by Feature Area

### 1. Presentations & Video Generation

**Packages:** `remotion`, `@remotion/bundler`, `@remotion/renderer`
**Version:** `^4.0.443` (latest — all Remotion packages must share the same version)
**Where it lives:** New workspace package `packages/content-renderer/` (isolated — Remotion uses its own webpack pipeline)

**Why Remotion:**
- React-native: slides/presentations are React components — agents generate TSX, Remotion renders to MP4 or PNG sequences
- SSR API (`@remotion/renderer`) works in Node.js without a browser UI; the Express server calls it directly
- Remotion bundles its own Chromium headless shell — no separate Chrome install on Mac Mini
- `ffmpeg-static` is already installed and Remotion detects it automatically (`ensureFfmpeg()`)
- Official Express render-server template exists: `remotion-dev/template-render-server` — proven integration pattern
- Mac M4/Apple Silicon: Remotion downloads a macOS arm64 Chromium binary; confirmed working

**Why NOT Remotion Lambda/Cloud:** This is a single-user Mac Mini deployment. No serverless. Local rendering only.

**Rendering flow:**
```
Agent generates TSX composition
  → POST /api/content/render-video
    → @remotion/bundler: bundle(entryPoint) → tmpDir
    → @remotion/renderer: renderMedia({ composition, serveUrl, outputLocation })
    → returns MP4 path in Nexus file system
```

**License note:** Remotion Skills (Claude Code integration) requires a commercial license for companies ≥4 employees. Nexus is single-user personal use — free tier applies.

**Confidence: HIGH** — Official SSR docs verified at remotion.dev/docs/ssr. Express template confirmed at github.com/remotion-dev/template-render-server.

---

### 2. Diagram Generation (Mermaid → SVG/PNG, Server-Side)

**Package:** `@mermaid-js/mermaid-cli`
**Version:** `^11.12.0`
**Where it lives:** `server/` — invoked as a programmatic API, not a CLI subprocess

**Why:** `mermaid ^11.12.0` is already installed in `ui/` for client-side rendering. For server-side rendering (agent → stored SVG/PNG file), the CLI package exposes a Node.js API:

```typescript
import { run } from "@mermaid-js/mermaid-cli";

await run(inputMmdFile, outputSvgFile, { outputFormat: "svg" });
```

**Why NOT headless-mermaid or puppeteer-based alternatives:**
- `headless-mermaid` is unmaintained (last update 2022)
- `@mermaid-js/mermaid-cli` is the official tool, maintained by the mermaid-js org, same version as the `mermaid` npm package
- It uses puppeteer internally but packages everything — no additional Chrome install needed

**Important:** `@mermaid-js/mermaid-cli` installs its own puppeteer with bundled Chromium. On Mac M4, this is a separate ~300MB download from Remotion's Chromium. Consider: if Remotion is already installed and its Chromium binary is available, pipe mermaid rendering through `@remotion/renderer` instead to share the Chromium binary. This is an implementation optimization, not a blocker.

**Confidence: MEDIUM** — Official npm package, same version as mermaid. Programmatic API (`run()`) confirmed in README.

---

### 3. PDF Generation (Reports, Invoices)

**Package:** `playwright-chromium`
**Version:** `^1.50.0` (do NOT use `^1.59.1` from root devDependencies — that's for e2e tests; install a separate `playwright-chromium` in `server/`)
**Where it lives:** `server/`

**Why Playwright over Puppeteer:**
- 2026 benchmark (macOS arm64, Node 22): Playwright is 42ms cold vs Puppeteer's 147ms cold, 3ms warm vs 48ms warm
- `playwright-chromium` installs only the Chromium binary — no Firefox or WebKit overhead
- TypeScript-native, no `@types/playwright` needed
- Actively maintained by Microsoft; Puppeteer-core is maintained by Google but Playwright has overtaken it for new projects as of 2025-2026

**Why NOT puppeteer-core:**
- Slower at every data point in 2026 benchmarks
- `playwright-chromium` provides an equally thin package (Chromium only) at better performance

**Why NOT @remotion/renderer for PDFs:**
- Remotion is optimized for video frame rendering, not document layout
- Playwright renders HTML/CSS directly via Chrome's print-to-PDF — correct page breaks, headers, footers, print stylesheet support

**PDF generation pattern:**
```typescript
import { chromium } from "playwright-chromium";

const browser = await chromium.launch({ headless: true });
const page = await browser.newPage();
await page.setContent(htmlString, { waitUntil: "networkidle" });
const pdf = await page.pdf({ format: "A4", printBackground: true });
await browser.close();
return pdf; // Buffer
```

**Mac M4 note:** `playwright-chromium` downloads a macOS arm64 Chromium binary via `npx playwright install chromium`. This is yet another Chromium binary. Consider sharing with Remotion's binary — implementation detail, investigate during Phase execution.

**Confidence: HIGH** — Playwright PDF generation confirmed at pptr.dev equivalent, benchmark sourced from pdf4.dev/blog/html-to-pdf-benchmark-2026 (March 2026, macOS arm64 run).

---

### 4. Social Media Images & OG Images (Satori + resvg-js)

**Packages:** `satori`, `resvg-js`
**Versions:** `satori ^0.26.0`, `resvg-js ^2.6.2`
**Where it lives:** `server/`

**Why satori:**
- Converts React JSX (HTML/CSS subset) to SVG — no browser needed, pure Node.js
- Used by `@vercel/og` internally — battle-tested for exactly this use case (OG images, social cards)
- Agents generate JSX layouts describing the social post design; satori renders to SVG
- Supports TTF/OTF/WOFF fonts — can embed the Nexus brand font

**Why resvg-js (paired with satori):**
- Converts the SVG output from satori → PNG via Rust bindings (napi-rs)
- No headless browser needed — pure native Node.js module
- `sharp ^0.34.5` is already installed and can do SVG→PNG too, but resvg-js handles text rendering more accurately when the SVG uses custom fonts embedded by satori

**When to use sharp vs resvg-js:**
- `sharp`: image compositing, resizing, format conversion (WebP, AVIF), photo manipulation
- `resvg-js`: satori SVG → PNG with correct font rendering

**Social media output flow:**
```
Agent generates JSX layout + text
  → satori(jsx, { width: 1200, height: 630, fonts: [...] }) → SVG string
  → resvg-js Resvg(svgString).render().asPng() → PNG Buffer
  → sharp(pngBuffer).resize(width, height).webp().toBuffer() → final asset
```

**Platform dimensions (built-in config):**
| Platform | Size |
|----------|------|
| OpenGraph | 1200×630 |
| Twitter/X card | 1200×628 |
| Instagram square | 1080×1080 |
| Instagram story | 1080×1920 |
| LinkedIn post | 1200×627 |

**Confidence: HIGH** — satori GitHub (vercel/satori) reviewed, resvg-js GitHub (thx/resvg-js) reviewed. Pattern confirmed in multiple tutorials. Versions confirmed via npm registry.

---

### 5. SVG Icon Generation

**No new library needed.** The existing stack already covers this:

- `sharp ^0.34.5`: can rasterize SVG to PNG at any resolution
- `resvg-js ^2.6.2` (added above): accurate SVG rendering with custom fonts
- Template-based SVG generation: agents produce SVG strings using a template library or string composition

**Approach:** Agents generate SVG markup directly (simple geometric shapes, paths, text). The server validates the SVG, stores it in the file system, and optionally exports PNG variants via `sharp`. No additional icon library needed.

**Why NOT `svg.js` or `svg-builder`:** These are authoring libraries for complex interactive SVG manipulation in the browser. For agent-generated icons, the agent produces the SVG string — the server only needs to validate and render it.

**Confidence: HIGH** — this is an implementation decision, not a library gap.

---

### 6. Theme & Palette Generator with WCAG Contrast

**Package:** `culori`
**Version:** `^4.0.2`
**Where it lives:** `server/` and optionally `ui/` (tree-shakeable ESM)

**Why culori over chroma-js:**
- 2026 community consensus: "OKLCH is the future of CSS color — use culori for any design-system work"
- culori supports the OKLCH color space natively; chroma-js 3.x has limited OKLCH support
- WCAG contrast calculation in culori is more accurate (proper relative luminance implementation)
- culori is fully ESM + CJS with tree-shaking; chroma-js 3.x is CJS-first with ESM wrapper
- Both support WCAG contrast ratio, but culori's implementation is described as "most accurate" in 2026 comparisons

**Why NOT tinycolor2:** Minimal WCAG support, no OKLCH, not maintained for design-system work.

**WCAG AA validation pattern:**
```typescript
import { wcagContrast, oklch, formatHex } from "culori";

function meetsWcagAA(fg: string, bg: string): boolean {
  return wcagContrast(fg, bg) >= 4.5; // AA for normal text
}

function generateAccessiblePalette(baseHex: string) {
  const base = oklch(baseHex);
  // Generate shades by adjusting lightness in OKLCH space
  return [0.95, 0.85, 0.70, 0.55, 0.40, 0.25, 0.15].map(l => ({
    hex: formatHex({ ...base, l }),
    contrast: wcagContrast(formatHex({ ...base, l }), "#ffffff"),
  }));
}
```

**Confidence: MEDIUM** — culori version verified via npm (4.0.2). WCAG accuracy claim sourced from pkgpulse.com comparison article (2026). Official culori docs reviewed.

---

### 7. Local Image Generation (Stable Diffusion / Flux)

**Package:** `@stable-canvas/comfyui-client`
**Version:** `^1.5.9`
**Where it lives:** `server/` — optional, only used when ComfyUI is detected running locally

**Why ComfyUI + this client:**
- ComfyUI is the dominant local Stable Diffusion frontend in 2026 (alongside AUTOMATIC1111/Forge)
- It exposes a REST + WebSocket API that this client wraps with TypeScript types
- On Mac M4 (Apple Silicon), ComfyUI runs via Metal/MPS backend — confirmed working with Flux.1 models
- The client is zero-dependency, MIT licensed, supports both REST (sync) and WebSocket (streaming progress) APIs

**Why NOT direct Stable Diffusion Python bindings:**
- Mac M4 has no CUDA; Python SD libraries that require CUDA don't work
- ComfyUI abstracts the backend (Metal/MPS on Mac) behind a standard REST API
- Nexus agents call the REST API — they don't care about the backend hardware

**Architecture:** ComfyUI runs as a separate process (not managed by Nexus). Nexus detects it via health check on `http://localhost:8188` at startup. If not running, image generation features degrade gracefully with a "ComfyUI not available" message.

**Integration pattern:**
```typescript
import { Client } from "@stable-canvas/comfyui-client";

const client = new Client({ api_host: "localhost:8188" });
await client.init(); // WebSocket handshake

const result = await client.enqueue(workflowJson, {
  progress: (p) => sendSSEProgress(p),
});
// result.images[0] is a Buffer
```

**Confidence: MEDIUM** — Client package confirmed on npm (1.5.9, MIT, zero deps). ComfyUI Mac M4 support sourced from offlinecreator.com (March 2026). WebSocket API pattern confirmed via GitHub README.

---

## Installation Summary

```bash
# packages/content-renderer/ — new workspace package for Remotion
# (Remotion needs its own webpack pipeline; isolate from main server build)
pnpm --filter @paperclipai/content-renderer add remotion @remotion/bundler @remotion/renderer

# server/ — diagram rendering (server-side mermaid)
pnpm --filter @paperclipai/server add @mermaid-js/mermaid-cli

# server/ — PDF generation
pnpm --filter @paperclipai/server add playwright-chromium
# After install:
npx playwright install chromium

# server/ — social images and OG cards (no-browser SVG→PNG pipeline)
pnpm --filter @paperclipai/server add satori resvg-js

# server/ — color/theme/palette with WCAG
pnpm --filter @paperclipai/server add culori

# server/ — local image generation via ComfyUI (optional, graceful degradation)
pnpm --filter @paperclipai/server add @stable-canvas/comfyui-client

# ui/ — culori for live preview (tree-shakeable ESM, safe to add to both)
pnpm --filter @paperclipai/ui add culori
```

---

## What NOT to Add

| Avoid | Why | Use Instead |
|-------|-----|-------------|
| `puppeteer-core` | Slower than Playwright in 2026 benchmarks (42ms vs 147ms cold on macOS arm64); Playwright has overtaken it | `playwright-chromium ^1.50.0` |
| `@remotion/lambda` | Serverless — Nexus is Mac Mini local-only, no AWS account needed | `@remotion/renderer` local rendering |
| `headless-mermaid` | Unmaintained since 2022 | `@mermaid-js/mermaid-cli ^11.12.0` (official) |
| `chroma-js` | Limited OKLCH support; culori more accurate for WCAG; community consensus favors culori for design-system work in 2026 | `culori ^4.0.2` |
| `tinycolor2` | Minimal WCAG support, no OKLCH, not suited for palette/design-system work | `culori ^4.0.2` |
| `canvas` (`node-canvas`) | C++ bindings, complicated install, replaced by `sharp` + `resvg-js` for server-side image ops | `sharp` (already installed) + `resvg-js` |
| `jimp` | Pure JS image processing, slow for production image generation workloads | `sharp ^0.34.5` (already installed, libvips-backed) |
| `@ffmpeg/ffmpeg` (WASM) | WASM FFmpeg is 10× slower than native binary; intended for browser/serverless | `ffmpeg-static ^5.3.0` (already installed) |
| `svg.js` / `svg-builder` | Browser authoring libraries; agents generate SVG strings directly | Agent-generated SVG + `sharp`/`resvg-js` for rendering |
| `@vercel/og` | Wraps satori+resvg but adds Vercel Edge Runtime constraints; unnecessary wrapper | `satori` + `resvg-js` directly |
| `pdf-lib` | Pure JS PDF creation — no HTML rendering. Correct for form-filling or programmatic PDF assembly, wrong for agent-generated HTML reports | `playwright-chromium` (HTML→PDF) |
| ComfyUI with cloud API (Replicate, etc.) | Nexus is air-gap friendly; Mikkel has local GPU (Mac M4) | `@stable-canvas/comfyui-client` pointing to `localhost:8188` |

---

## Alternatives Considered

| Recommended | Alternative | When to Use Alternative |
|-------------|-------------|-------------------------|
| `@remotion/renderer` (local) | `@remotion/lambda` | If you need parallel cloud rendering at scale — N/A for single-user Mac Mini |
| `playwright-chromium` | `puppeteer-core` | If project already has Puppeteer deeply integrated; for new work prefer Playwright |
| `@mermaid-js/mermaid-cli` (Node API) | `child_process.spawn("mmdc")` | If you need a fully isolated subprocess; Node API is cleaner for the server pattern |
| `culori` | `chroma-js ^3.2.0` | If project requires many color interpolation methods (chroma has a richer palette generation API); for WCAG-first design systems culori wins |
| `satori` + `resvg-js` | `playwright-chromium` for OG images | Playwright is simpler but adds another headless browser instance; satori is ~100× faster for simple card layouts with no JS |
| `@stable-canvas/comfyui-client` | Direct `fetch` to ComfyUI REST API | If ComfyUI API changes and client lags; raw fetch is always viable since ComfyUI API is stable JSON |

---

## Version Compatibility

| Package | Compatible With | Notes |
|---------|-----------------|-------|
| `remotion ^4.0.443` | Node.js >=18, React >=18, TypeScript >=5 | All `@remotion/*` packages must be pinned to the same version; `ffmpeg-static` already installed is detected automatically |
| `@mermaid-js/mermaid-cli ^11.12.0` | Node.js >=18, puppeteer (bundled) | Installs its own bundled Chromium (~300MB); version must match `mermaid ^11.12.0` already in `ui/` |
| `playwright-chromium ^1.50.0` | Node.js >=18, macOS arm64 | Separate Chromium binary from Remotion and mermaid-cli. Requires `npx playwright install chromium` post-install |
| `satori ^0.26.0` | Node.js >=16, ESM | Supports TTF/OTF/WOFF fonts only — WOFF2 is NOT supported. Subset fonts for performance |
| `resvg-js ^2.6.2` | Node.js >=14, macOS arm64 | Rust napi-rs binary; downloads macOS arm64 binary at install time |
| `culori ^4.0.2` | Node.js >=14, ESM + CJS | Dual package; safe in both `server/` (CJS/ESM mix) and `ui/` (Vite ESM) |
| `@stable-canvas/comfyui-client ^1.5.9` | Node.js >=16, zero deps | Requires ComfyUI running at localhost:8188; graceful degradation if not available |

**Chromium binary count warning:** v1.7 potentially installs THREE separate Chromium binaries:
1. `remotion` — `~/.cache/puppeteer/chrome/` or Remotion's own cache
2. `@mermaid-js/mermaid-cli` — puppeteer bundled binary
3. `playwright-chromium` — `~/.cache/ms-playwright/`

Total disk: ~900MB. For the Mac Mini (256GB+ SSD) this is acceptable. During implementation, investigate whether `@mermaid-js/mermaid-cli` can be pointed at Playwright's Chromium via `PUPPETEER_EXECUTABLE_PATH` to reduce redundancy.

---

## Content Generation Package Architecture

```
nexus/
├── packages/
│   └── content-renderer/          ← NEW workspace package
│       ├── package.json           (remotion, @remotion/bundler, @remotion/renderer)
│       ├── src/
│       │   ├── compositions/      (React TSX slide templates)
│       │   └── index.ts           (render() export)
│       └── tsconfig.json
│
├── server/
│   └── src/
│       └── content/
│           ├── diagram.ts         (@mermaid-js/mermaid-cli)
│           ├── pdf.ts             (playwright-chromium)
│           ├── social.ts          (satori + resvg-js + sharp)
│           ├── theme.ts           (culori)
│           ├── image.ts           (@stable-canvas/comfyui-client)
│           └── index.ts           (route registrations)
│
└── ui/
    └── src/
        └── content/
            └── ThemePreview.tsx   (culori for live WCAG preview)
```

**Why isolate Remotion in its own package:**
- Remotion uses its own webpack bundler (`@remotion/bundler`) that conflicts with Vite
- Keeping it in `packages/content-renderer/` prevents build pipeline interference
- The Express server imports only the render function, not the full webpack config

---

## Sources

- [Remotion official SSR docs](https://www.remotion.dev/docs/ssr) — `@remotion/renderer` Node.js API confirmed
- [Remotion Express render-server template](https://github.com/remotion-dev/template-render-server) — Express integration pattern
- [Remotion Chrome headless shell docs](https://www.remotion.dev/docs/miscellaneous/chrome-headless-shell) — Mac arm64 binary confirmed
- [remotion npm](https://www.npmjs.com/package/remotion) — version 4.0.443 confirmed
- [mermaid-cli GitHub](https://github.com/mermaid-js/mermaid-cli) — Node.js `run()` API, v11.12.0
- [playwright-chromium npm](https://www.npmjs.com/package/playwright-chromium) — Chromium-only package
- [PDF benchmark 2026](https://pdf4.dev/blog/html-to-pdf-benchmark-2026) — Playwright vs Puppeteer, macOS arm64, Node 22 (MEDIUM confidence — single benchmark source)
- [vercel/satori GitHub](https://github.com/vercel/satori) — JSX→SVG, Node.js support, font format constraints
- [satori npm](https://www.npmjs.com/package/satori) — version 0.26.0 confirmed
- [thx/resvg-js GitHub](https://github.com/thx/resvg-js) — SVG→PNG via Rust napi-rs, macOS arm64 support
- [resvg-js npm](https://www.npmjs.com/package/resvg-js) — version 2.6.2 (note: npm showed 0.1.97 for `resvg-js`; the canonical package may be `@resvg/resvg-js` — verify exact package name during implementation)
- [culori npm](https://www.npmjs.com/package/culori) — version 4.0.2 confirmed
- [culori vs chroma-js 2026](https://www.pkgpulse.com/blog/culori-vs-chroma-js-vs-tinycolor2-color-manipulation-javascript-2026) — OKLCH + WCAG accuracy comparison (MEDIUM confidence)
- [@stable-canvas/comfyui-client npm](https://www.npmjs.com/package/@stable-canvas/comfyui-client) — version 1.5.9, MIT, zero deps
- [Best local SD setup 2026](https://offlinecreator.com/blog/best-local-stable-diffusion-setup-2026) — ComfyUI Mac M4 support (MEDIUM confidence)

---

**Unresolved — verify during implementation:**
1. `resvg-js` package name: npm shows v0.1.97 for `resvg-js` but v2.6.2 for `@resvg/resvg-js`. Use `@resvg/resvg-js ^2.6.2` (the Rust napi-rs backed version).
2. Chromium binary deduplication: test whether `@mermaid-js/mermaid-cli` respects `PUPPETEER_EXECUTABLE_PATH` pointing to Playwright's Chromium binary to save ~300MB.
3. Remotion webpack vs Vite isolation: confirm that `packages/content-renderer/` with its own `tsconfig.json` and webpack bundler does not affect the root `pnpm build` pipeline.

---

*Stack research for: Nexus v1.7 Content Generation*
*Researched: 2026-04-04*
*Supersedes: v1.6 STACK.md entries remain valid — this file covers only v1.7 additions*