nexus/.planning/research/STACK.md

Technology Stack: v1.7 Content Generation

Project: Nexus v1.7 — additive to v1.6 stack (see prior STACK.md for voice pipeline, Telegram bridge, ffmpeg-static, grammy, @ricky0123/vad-react) Researched: 2026-04-04 Scope: NEW libraries only for v1.7 — presentations/video, image generation, diagram rendering, PDF generation, SVG icons, color/theme tools, social media assets Confidence: MEDIUM-HIGH (Remotion HIGH via official docs; satori+resvg-js HIGH via official repos; mermaid already installed; ComfyUI client MEDIUM; culori MEDIUM via comparison sources)

---

Context: What Is Already Installed

Do not re-add or re-research these — confirmed present in server/package.json or ui/package.json:

Package	Location	Version	Relevant To v1.7
sharp ^0.34.5	server/	0.34.5	SVG→PNG conversion, image compositing, social asset output
mermaid ^11.12.0	ui/	11.12.0	Client-side diagram rendering already works
ffmpeg-static ^5.3.0	server/	5.3.0	Video stitching, audio for Remotion renders
zod ^3.24.2	server/	3.24.2	Schema validation for content generation requests
express ^5.1.0	server/	5.1.0	API endpoints for all content generation routes

The v1.7 work adds new rendering capabilities — it does not replace or duplicate any of the above.

---

New Libraries by Feature Area

1. Presentations & Video Generation

Packages: remotion, @remotion/bundler, @remotion/renderer Version: ^4.0.443 (latest — all Remotion packages must share the same version) Where it lives: New workspace package packages/content-renderer/ (isolated — Remotion uses its own webpack pipeline)

Why Remotion:

• React-native: slides/presentations are React components — agents generate TSX, Remotion renders to MP4 or PNG sequences
• SSR API (@remotion/renderer) works in Node.js without a browser UI; the Express server calls it directly
• Remotion bundles its own Chromium headless shell — no separate Chrome install on Mac Mini
• ffmpeg-static is already installed and Remotion detects it automatically (ensureFfmpeg())
• Official Express render-server template exists: remotion-dev/template-render-server — proven integration pattern
• Mac M4/Apple Silicon: Remotion downloads a macOS arm64 Chromium binary; confirmed working

Why NOT Remotion Lambda/Cloud: This is a single-user Mac Mini deployment. No serverless. Local rendering only.

Rendering flow:

Agent generates TSX composition
  → POST /api/content/render-video
    → @remotion/bundler: bundle(entryPoint) → tmpDir
    → @remotion/renderer: renderMedia({ composition, serveUrl, outputLocation })
    → returns MP4 path in Nexus file system

License note: Remotion Skills (Claude Code integration) requires a commercial license for companies ≥4 employees. Nexus is single-user personal use — free tier applies.

Confidence: HIGH — Official SSR docs verified at remotion.dev/docs/ssr. Express template confirmed at github.com/remotion-dev/template-render-server.

---

2. Diagram Generation (Mermaid → SVG/PNG, Server-Side)

Package: @mermaid-js/mermaid-cli Version: ^11.12.0 Where it lives: server/ — invoked as a programmatic API, not a CLI subprocess

Why: mermaid ^11.12.0 is already installed in ui/ for client-side rendering. For server-side rendering (agent → stored SVG/PNG file), the CLI package exposes a Node.js API:

import { run } from "@mermaid-js/mermaid-cli";

await run(inputMmdFile, outputSvgFile, { outputFormat: "svg" });

Why NOT headless-mermaid or puppeteer-based alternatives:

• headless-mermaid is unmaintained (last update 2022)
• @mermaid-js/mermaid-cli is the official tool, maintained by the mermaid-js org, same version as the mermaid npm package
• It uses puppeteer internally but packages everything — no additional Chrome install needed

Important: @mermaid-js/mermaid-cli installs its own puppeteer with bundled Chromium. On Mac M4, this is a separate ~300MB download from Remotion's Chromium. Consider: if Remotion is already installed and its Chromium binary is available, pipe mermaid rendering through @remotion/renderer instead to share the Chromium binary. This is an implementation optimization, not a blocker.

Confidence: MEDIUM — Official npm package, same version as mermaid. Programmatic API (run()) confirmed in README.

---

3. PDF Generation (Reports, Invoices)

Package: playwright-chromium Version: ^1.50.0 (do NOT use ^1.59.1 from root devDependencies — that's for e2e tests; install a separate playwright-chromium in server/) Where it lives: server/

Why Playwright over Puppeteer:

• 2026 benchmark (macOS arm64, Node 22): Playwright is 42ms cold vs Puppeteer's 147ms cold, 3ms warm vs 48ms warm
• playwright-chromium installs only the Chromium binary — no Firefox or WebKit overhead
• TypeScript-native, no @types/playwright needed
• Actively maintained by Microsoft; Puppeteer-core is maintained by Google but Playwright has overtaken it for new projects as of 2025-2026

Why NOT puppeteer-core:

• Slower at every data point in 2026 benchmarks
• playwright-chromium provides an equally thin package (Chromium only) at better performance

Why NOT @remotion/renderer for PDFs:

• Remotion is optimized for video frame rendering, not document layout
• Playwright renders HTML/CSS directly via Chrome's print-to-PDF — correct page breaks, headers, footers, print stylesheet support

PDF generation pattern:

import { chromium } from "playwright-chromium";

const browser = await chromium.launch({ headless: true });
const page = await browser.newPage();
await page.setContent(htmlString, { waitUntil: "networkidle" });
const pdf = await page.pdf({ format: "A4", printBackground: true });
await browser.close();
return pdf; // Buffer

Mac M4 note: playwright-chromium downloads a macOS arm64 Chromium binary via npx playwright install chromium. This is yet another Chromium binary. Consider sharing with Remotion's binary — implementation detail, investigate during Phase execution.

Confidence: HIGH — Playwright PDF generation confirmed at pptr.dev equivalent, benchmark sourced from pdf4.dev/blog/html-to-pdf-benchmark-2026 (March 2026, macOS arm64 run).

---

4. Social Media Images & OG Images (Satori + resvg-js)

Packages: satori, resvg-js Versions: satori ^0.26.0, resvg-js ^2.6.2 Where it lives: server/

Why satori:

• Converts React JSX (HTML/CSS subset) to SVG — no browser needed, pure Node.js
• Used by @vercel/og internally — battle-tested for exactly this use case (OG images, social cards)
• Agents generate JSX layouts describing the social post design; satori renders to SVG
• Supports TTF/OTF/WOFF fonts — can embed the Nexus brand font

Why resvg-js (paired with satori):

• Converts the SVG output from satori → PNG via Rust bindings (napi-rs)
• No headless browser needed — pure native Node.js module
• sharp ^0.34.5 is already installed and can do SVG→PNG too, but resvg-js handles text rendering more accurately when the SVG uses custom fonts embedded by satori

When to use sharp vs resvg-js:

• sharp: image compositing, resizing, format conversion (WebP, AVIF), photo manipulation
• resvg-js: satori SVG → PNG with correct font rendering

Social media output flow:

Agent generates JSX layout + text
  → satori(jsx, { width: 1200, height: 630, fonts: [...] }) → SVG string
  → resvg-js Resvg(svgString).render().asPng() → PNG Buffer
  → sharp(pngBuffer).resize(width, height).webp().toBuffer() → final asset

Platform dimensions (built-in config):

Platform	Size
OpenGraph	1200×630
Twitter/X card	1200×628
Instagram square	1080×1080
Instagram story	1080×1920
LinkedIn post	1200×627

Confidence: HIGH — satori GitHub (vercel/satori) reviewed, resvg-js GitHub (thx/resvg-js) reviewed. Pattern confirmed in multiple tutorials. Versions confirmed via npm registry.

---

5. SVG Icon Generation

No new library needed. The existing stack already covers this:

• sharp ^0.34.5: can rasterize SVG to PNG at any resolution
• resvg-js ^2.6.2 (added above): accurate SVG rendering with custom fonts
• Template-based SVG generation: agents produce SVG strings using a template library or string composition

Approach: Agents generate SVG markup directly (simple geometric shapes, paths, text). The server validates the SVG, stores it in the file system, and optionally exports PNG variants via sharp. No additional icon library needed.

Why NOT svg.js or svg-builder: These are authoring libraries for complex interactive SVG manipulation in the browser. For agent-generated icons, the agent produces the SVG string — the server only needs to validate and render it.

Confidence: HIGH — this is an implementation decision, not a library gap.

---

6. Theme & Palette Generator with WCAG Contrast

Package: culori Version: ^4.0.2 Where it lives: server/ and optionally ui/ (tree-shakeable ESM)

Why culori over chroma-js:

• 2026 community consensus: "OKLCH is the future of CSS color — use culori for any design-system work"
• culori supports the OKLCH color space natively; chroma-js 3.x has limited OKLCH support
• WCAG contrast calculation in culori is more accurate (proper relative luminance implementation)
• culori is fully ESM + CJS with tree-shaking; chroma-js 3.x is CJS-first with ESM wrapper
• Both support WCAG contrast ratio, but culori's implementation is described as "most accurate" in 2026 comparisons

Why NOT tinycolor2: Minimal WCAG support, no OKLCH, not maintained for design-system work.

WCAG AA validation pattern:

import { wcagContrast, oklch, formatHex } from "culori";

function meetsWcagAA(fg: string, bg: string): boolean {
  return wcagContrast(fg, bg) >= 4.5; // AA for normal text
}

function generateAccessiblePalette(baseHex: string) {
  const base = oklch(baseHex);
  // Generate shades by adjusting lightness in OKLCH space
  return [0.95, 0.85, 0.70, 0.55, 0.40, 0.25, 0.15].map(l => ({
    hex: formatHex({ ...base, l }),
    contrast: wcagContrast(formatHex({ ...base, l }), "#ffffff"),
  }));
}

Confidence: MEDIUM — culori version verified via npm (4.0.2). WCAG accuracy claim sourced from pkgpulse.com comparison article (2026). Official culori docs reviewed.

---

7. Local Image Generation (Stable Diffusion / Flux)

Package: @stable-canvas/comfyui-client Version: ^1.5.9 Where it lives: server/ — optional, only used when ComfyUI is detected running locally

Why ComfyUI + this client:

• ComfyUI is the dominant local Stable Diffusion frontend in 2026 (alongside AUTOMATIC1111/Forge)
• It exposes a REST + WebSocket API that this client wraps with TypeScript types
• On Mac M4 (Apple Silicon), ComfyUI runs via Metal/MPS backend — confirmed working with Flux.1 models
• The client is zero-dependency, MIT licensed, supports both REST (sync) and WebSocket (streaming progress) APIs

Why NOT direct Stable Diffusion Python bindings:

• Mac M4 has no CUDA; Python SD libraries that require CUDA don't work
• ComfyUI abstracts the backend (Metal/MPS on Mac) behind a standard REST API
• Nexus agents call the REST API — they don't care about the backend hardware

Architecture: ComfyUI runs as a separate process (not managed by Nexus). Nexus detects it via health check on http://localhost:8188 at startup. If not running, image generation features degrade gracefully with a "ComfyUI not available" message.

Integration pattern:

import { Client } from "@stable-canvas/comfyui-client";

const client = new Client({ api_host: "localhost:8188" });
await client.init(); // WebSocket handshake

const result = await client.enqueue(workflowJson, {
  progress: (p) => sendSSEProgress(p),
});
// result.images[0] is a Buffer

Confidence: MEDIUM — Client package confirmed on npm (1.5.9, MIT, zero deps). ComfyUI Mac M4 support sourced from offlinecreator.com (March 2026). WebSocket API pattern confirmed via GitHub README.

---

Installation Summary

# packages/content-renderer/ — new workspace package for Remotion
# (Remotion needs its own webpack pipeline; isolate from main server build)
pnpm --filter @paperclipai/content-renderer add remotion @remotion/bundler @remotion/renderer

# server/ — diagram rendering (server-side mermaid)
pnpm --filter @paperclipai/server add @mermaid-js/mermaid-cli

# server/ — PDF generation
pnpm --filter @paperclipai/server add playwright-chromium
# After install:
npx playwright install chromium

# server/ — social images and OG cards (no-browser SVG→PNG pipeline)
pnpm --filter @paperclipai/server add satori resvg-js

# server/ — color/theme/palette with WCAG
pnpm --filter @paperclipai/server add culori

# server/ — local image generation via ComfyUI (optional, graceful degradation)
pnpm --filter @paperclipai/server add @stable-canvas/comfyui-client

# ui/ — culori for live preview (tree-shakeable ESM, safe to add to both)
pnpm --filter @paperclipai/ui add culori

---

What NOT to Add

Avoid	Why	Use Instead
puppeteer-core	Slower than Playwright in 2026 benchmarks (42ms vs 147ms cold on macOS arm64); Playwright has overtaken it	playwright-chromium ^1.50.0
@remotion/lambda	Serverless — Nexus is Mac Mini local-only, no AWS account needed	@remotion/renderer local rendering
headless-mermaid	Unmaintained since 2022	@mermaid-js/mermaid-cli ^11.12.0 (official)
chroma-js	Limited OKLCH support; culori more accurate for WCAG; community consensus favors culori for design-system work in 2026	culori ^4.0.2
tinycolor2	Minimal WCAG support, no OKLCH, not suited for palette/design-system work	culori ^4.0.2
canvas (node-canvas)	C++ bindings, complicated install, replaced by sharp + resvg-js for server-side image ops	sharp (already installed) + resvg-js
jimp	Pure JS image processing, slow for production image generation workloads	sharp ^0.34.5 (already installed, libvips-backed)
@ffmpeg/ffmpeg (WASM)	WASM FFmpeg is 10× slower than native binary; intended for browser/serverless	ffmpeg-static ^5.3.0 (already installed)
svg.js / svg-builder	Browser authoring libraries; agents generate SVG strings directly	Agent-generated SVG + sharp/resvg-js for rendering
@vercel/og	Wraps satori+resvg but adds Vercel Edge Runtime constraints; unnecessary wrapper	satori + resvg-js directly
pdf-lib	Pure JS PDF creation — no HTML rendering. Correct for form-filling or programmatic PDF assembly, wrong for agent-generated HTML reports	playwright-chromium (HTML→PDF)
ComfyUI with cloud API (Replicate, etc.)	Nexus is air-gap friendly; Mikkel has local GPU (Mac M4)	@stable-canvas/comfyui-client pointing to localhost:8188

---

Alternatives Considered

Recommended	Alternative	When to Use Alternative
@remotion/renderer (local)	@remotion/lambda	If you need parallel cloud rendering at scale — N/A for single-user Mac Mini
playwright-chromium	puppeteer-core	If project already has Puppeteer deeply integrated; for new work prefer Playwright
@mermaid-js/mermaid-cli (Node API)	child_process.spawn("mmdc")	If you need a fully isolated subprocess; Node API is cleaner for the server pattern
culori	chroma-js ^3.2.0	If project requires many color interpolation methods (chroma has a richer palette generation API); for WCAG-first design systems culori wins
satori + resvg-js	playwright-chromium for OG images	Playwright is simpler but adds another headless browser instance; satori is ~100× faster for simple card layouts with no JS
@stable-canvas/comfyui-client	Direct fetch to ComfyUI REST API	If ComfyUI API changes and client lags; raw fetch is always viable since ComfyUI API is stable JSON

---

Version Compatibility

Package	Compatible With	Notes
remotion ^4.0.443	Node.js >=18, React >=18, TypeScript >=5	All @remotion/* packages must be pinned to the same version; ffmpeg-static already installed is detected automatically
@mermaid-js/mermaid-cli ^11.12.0	Node.js >=18, puppeteer (bundled)	Installs its own bundled Chromium (~300MB); version must match mermaid ^11.12.0 already in ui/
playwright-chromium ^1.50.0	Node.js >=18, macOS arm64	Separate Chromium binary from Remotion and mermaid-cli. Requires npx playwright install chromium post-install
satori ^0.26.0	Node.js >=16, ESM	Supports TTF/OTF/WOFF fonts only — WOFF2 is NOT supported. Subset fonts for performance
resvg-js ^2.6.2	Node.js >=14, macOS arm64	Rust napi-rs binary; downloads macOS arm64 binary at install time
culori ^4.0.2	Node.js >=14, ESM + CJS	Dual package; safe in both server/ (CJS/ESM mix) and ui/ (Vite ESM)
@stable-canvas/comfyui-client ^1.5.9	Node.js >=16, zero deps	Requires ComfyUI running at localhost:8188; graceful degradation if not available

Chromium binary count warning: v1.7 potentially installs THREE separate Chromium binaries:

1. remotion — ~/.cache/puppeteer/chrome/ or Remotion's own cache
2. @mermaid-js/mermaid-cli — puppeteer bundled binary
3. playwright-chromium — ~/.cache/ms-playwright/

Total disk: ~900MB. For the Mac Mini (256GB+ SSD) this is acceptable. During implementation, investigate whether @mermaid-js/mermaid-cli can be pointed at Playwright's Chromium via PUPPETEER_EXECUTABLE_PATH to reduce redundancy.

---

Content Generation Package Architecture

nexus/
├── packages/
│   └── content-renderer/          ← NEW workspace package
│       ├── package.json           (remotion, @remotion/bundler, @remotion/renderer)
│       ├── src/
│       │   ├── compositions/      (React TSX slide templates)
│       │   └── index.ts           (render() export)
│       └── tsconfig.json
│
├── server/
│   └── src/
│       └── content/
│           ├── diagram.ts         (@mermaid-js/mermaid-cli)
│           ├── pdf.ts             (playwright-chromium)
│           ├── social.ts          (satori + resvg-js + sharp)
│           ├── theme.ts           (culori)
│           ├── image.ts           (@stable-canvas/comfyui-client)
│           └── index.ts           (route registrations)
│
└── ui/
    └── src/
        └── content/
            └── ThemePreview.tsx   (culori for live WCAG preview)

Why isolate Remotion in its own package:

• Remotion uses its own webpack bundler (@remotion/bundler) that conflicts with Vite
• Keeping it in packages/content-renderer/ prevents build pipeline interference
• The Express server imports only the render function, not the full webpack config

---

Sources

• Remotion official SSR docs — @remotion/renderer Node.js API confirmed
• Remotion Express render-server template — Express integration pattern
• Remotion Chrome headless shell docs — Mac arm64 binary confirmed
• remotion npm — version 4.0.443 confirmed
• mermaid-cli GitHub — Node.js run() API, v11.12.0
• playwright-chromium npm — Chromium-only package
• PDF benchmark 2026 — Playwright vs Puppeteer, macOS arm64, Node 22 (MEDIUM confidence — single benchmark source)
• vercel/satori GitHub — JSX→SVG, Node.js support, font format constraints
• satori npm — version 0.26.0 confirmed
• thx/resvg-js GitHub — SVG→PNG via Rust napi-rs, macOS arm64 support
• resvg-js npm — version 2.6.2 (note: npm showed 0.1.97 for resvg-js; the canonical package may be @resvg/resvg-js — verify exact package name during implementation)
• culori npm — version 4.0.2 confirmed
• culori vs chroma-js 2026 — OKLCH + WCAG accuracy comparison (MEDIUM confidence)
• @stable-canvas/comfyui-client npm — version 1.5.9, MIT, zero deps
• Best local SD setup 2026 — ComfyUI Mac M4 support (MEDIUM confidence)

---

Unresolved — verify during implementation:

1. resvg-js package name: npm shows v0.1.97 for resvg-js but v2.6.2 for @resvg/resvg-js. Use @resvg/resvg-js ^2.6.2 (the Rust napi-rs backed version).
2. Chromium binary deduplication: test whether @mermaid-js/mermaid-cli respects PUPPETEER_EXECUTABLE_PATH pointing to Playwright's Chromium binary to save ~300MB.
3. Remotion webpack vs Vite isolation: confirm that packages/content-renderer/ with its own tsconfig.json and webpack bundler does not affect the root pnpm build pipeline.

---

Stack research for: Nexus v1.7 Content Generation Researched: 2026-04-04 Supersedes: v1.6 STACK.md entries remain valid — this file covers only v1.7 additions