600 lines
30 KiB
Markdown
600 lines
30 KiB
Markdown
# Phase 44: Video & Presentations - Research
|
||
|
||
**Researched:** 2026-04-04
|
||
**Domain:** Remotion server-side rendering, pitch deck composition, SSE render progress
|
||
**Confidence:** HIGH
|
||
|
||
<user_constraints>
|
||
## User Constraints (from CONTEXT.md)
|
||
|
||
### Locked Decisions
|
||
None — discuss phase was skipped. All implementation choices are at Claude's discretion.
|
||
|
||
### Claude's Discretion
|
||
All implementation choices including package structure, composition layout, slide design patterns, render settings, and progress event granularity.
|
||
|
||
### Deferred Ideas (OUT OF SCOPE)
|
||
None specified.
|
||
</user_constraints>
|
||
|
||
<phase_requirements>
|
||
## Phase Requirements
|
||
|
||
| ID | Description | Research Support |
|
||
|----|-------------|------------------|
|
||
| PRES-01 | User can generate pitch deck presentations from a conversation | LLM generates slide JSON; Remotion composition renders slides via `Series`; job infra from Phase 40 handles submission |
|
||
| PRES-02 | System renders presentations via Remotion to interactive web or MP4 | `@remotion/player` for interactive web preview; `renderMedia()` with `codec: 'h264'` for MP4 export |
|
||
| PRES-03 | User can generate demo and explainer videos from conversation content | Same renderer, different composition (e.g., `VideoPresentation` vs `PitchDeck`); LLM generates narration script and slide sequence |
|
||
| PRES-04 | System shows render progress via SSE during video generation | `onProgress` callback in `renderMedia()` yields 0–1 progress; publish `content_job.progress` live event; extend shared constants and useContentJob hook |
|
||
</phase_requirements>
|
||
|
||
---
|
||
|
||
## Summary
|
||
|
||
Phase 44 adds Remotion-powered video and presentation generation to the Content Studio. The core pattern follows all prior content phases: the UI submits a job via `POST /api/companies/:id/content-jobs`, the server renders asynchronously, and SSE events report status. Remotion adds two complications that don't exist in any prior renderer:
|
||
|
||
First, Remotion uses webpack (`@remotion/bundler` uses rspack internally) which must not enter the Vite/tsc server context. Per the locked architecture decision in STATE.md, Remotion must live in `packages/content-renderer/` as a dedicated workspace package. The server imports this package at runtime via dynamic import, keeping webpack out of the server's tsc compilation and Vite's bundling.
|
||
|
||
Second, Remotion renders can take several minutes. The current SSE infrastructure only emits lifecycle events (`queued`, `running`, `done`, `failed`). For PRES-04, we need to add a `content_job.progress` event type to `packages/shared/src/constants.ts` and wire `renderMedia()`'s `onProgress` callback to publish it. The `useContentJob` hook and SSE route already support arbitrary payload fields — we just need the new event type and a progress field in the payload.
|
||
|
||
**Primary recommendation:** Create `packages/content-renderer/` as a standalone Node ESM package containing the Remotion composition definitions and a `renderPresentation()` export. The server's `presentation-renderer.ts` calls this package's export. Bundle once at server startup and cache the path in a module-level variable.
|
||
|
||
---
|
||
|
||
## Standard Stack
|
||
|
||
### Core
|
||
| Library | Version | Purpose | Why Standard |
|
||
|---------|---------|---------|--------------|
|
||
| remotion | 4.0.445 | Composition model, timing API (`useCurrentFrame`, `Series`, `Sequence`) | Core Remotion library; required by all Remotion packages |
|
||
| @remotion/bundler | 4.0.445 | `bundle()` — compiles composition to a webpack/rspack bundle | Required for server-side rendering workflow |
|
||
| @remotion/renderer | 4.0.445 | `renderMedia()`, `selectComposition()` — headless render to MP4/buffer | Required for server-side rendering workflow |
|
||
| @remotion/player | 4.0.445 | React `<Player>` component for interactive web preview | Required for PRES-02 interactive web output |
|
||
|
||
### Supporting
|
||
| Library | Version | Purpose | When to Use |
|
||
|---------|---------|---------|-------------|
|
||
| @remotion/compositor-linux-x64-gnu | 4.0.445 | Rust compositor binary for frame stitching/encoding on Linux x64 | Installed automatically as optional dependency of @remotion/renderer |
|
||
| react + react-dom | (already in workspace) | Remotion compositions are React components | peerDependency of remotion; reuse workspace versions |
|
||
|
||
### Alternatives Considered
|
||
| Instead of | Could Use | Tradeoff |
|
||
|------------|-----------|----------|
|
||
| packages/content-renderer/ workspace package | Inline in server/ | Workspace package prevents webpack from entering tsc/Vite context; inline import would corrupt tsc compilation |
|
||
| @remotion/renderer (renderMedia) | Playwright headless screenshot-to-video | renderMedia has native frame-accurate encoding; screenshot-based approach lacks timing primitives |
|
||
| Series + Sequence for slides | Custom timing logic | Series handles cumulative durationInFrames automatically; custom timing is error-prone |
|
||
|
||
**Installation (packages/content-renderer/):**
|
||
```bash
|
||
cd packages/content-renderer
|
||
pnpm add remotion @remotion/bundler @remotion/renderer @remotion/player react react-dom
|
||
```
|
||
|
||
**Installation (server/ — presenter renderer only needs the package's export):**
|
||
```bash
|
||
# No new server deps — server imports from packages/content-renderer via workspace
|
||
```
|
||
|
||
**Installation (ui/ — for @remotion/player interactive preview):**
|
||
```bash
|
||
cd ui
|
||
pnpm add @remotion/player
|
||
```
|
||
|
||
**Version verification (confirmed 2026-04-04):**
|
||
```bash
|
||
npm view remotion version # 4.0.445
|
||
npm view @remotion/renderer version # 4.0.445
|
||
npm view @remotion/bundler version # 4.0.445
|
||
npm view @remotion/player version # 4.0.445
|
||
```
|
||
|
||
---
|
||
|
||
## Architecture Patterns
|
||
|
||
### Recommended Project Structure
|
||
```
|
||
packages/content-renderer/ # NEW: webpack-isolated Remotion workspace package
|
||
├── package.json # type: "commonjs" (rspack/webpack requirement)
|
||
├── src/
|
||
│ ├── index.ts # Exports: renderPresentation(), BUNDLE_PATH singleton
|
||
│ ├── Root.tsx # registerRoot() entry — registers all compositions
|
||
│ ├── compositions/
|
||
│ │ ├── PitchDeck.tsx # Pitch deck composition (Series of SlideFrame)
|
||
│ │ └── DemoVideo.tsx # Demo/explainer video composition
|
||
│ └── components/
|
||
│ ├── SlideFrame.tsx # Individual slide with title, body, visual
|
||
│ └── TitleSlide.tsx # Opening title slide variant
|
||
│
|
||
server/src/services/renderers/
|
||
└── presentation-renderer.ts # NEW: calls packages/content-renderer renderPresentation()
|
||
|
||
server/src/services/
|
||
└── content-job-runner.ts # EXTEND: add "presentation" case
|
||
|
||
ui/src/components/
|
||
└── PresentationPanel.tsx # NEW: matches BrandKitPanel pattern
|
||
ui/src/pages/
|
||
└── ContentStudio.tsx # EXTEND: add "presentations" tab
|
||
```
|
||
|
||
### Pattern 1: Bundle Once at Startup
|
||
**What:** Call `bundle()` once when the server starts, store the returned path in a module-level variable, pass it to every `renderMedia()` call.
|
||
**When to use:** Always — multiple renders share the same bundle. Re-bundling per render is an anti-pattern.
|
||
**Example:**
|
||
```typescript
|
||
// packages/content-renderer/src/index.ts
|
||
// Source: https://www.remotion.dev/docs/bundle
|
||
import { bundle } from "@remotion/bundler";
|
||
import path from "path";
|
||
import { fileURLToPath } from "url";
|
||
|
||
const __dirname = path.dirname(fileURLToPath(import.meta.url));
|
||
|
||
let cachedBundlePath: string | null = null;
|
||
|
||
export async function getBundlePath(): Promise<string> {
|
||
if (cachedBundlePath) return cachedBundlePath;
|
||
cachedBundlePath = await bundle({
|
||
entryPoint: path.resolve(__dirname, "Root.tsx"),
|
||
// rspack: true // can enable for faster bundling
|
||
});
|
||
return cachedBundlePath;
|
||
}
|
||
```
|
||
|
||
### Pattern 2: Render with Progress SSE
|
||
**What:** `renderMedia()` accepts an `onProgress` callback that fires with percentage (0–1). We call `publishLiveEvent` with `content_job.progress` from inside that callback.
|
||
**When to use:** All video/MP4 renders. Progress is meaningful because renders can take several minutes.
|
||
**Example:**
|
||
```typescript
|
||
// server/src/services/renderers/presentation-renderer.ts
|
||
// Source: https://www.remotion.dev/docs/renderer/render-media
|
||
import { renderMedia, selectComposition } from "@remotion/renderer";
|
||
import { getBundlePath } from "@paperclipai/content-renderer";
|
||
import { publishLiveEvent } from "../live-events.js";
|
||
|
||
export async function renderPresentation(
|
||
input: Record<string, unknown>,
|
||
companyId: string,
|
||
jobId: string,
|
||
): Promise<RenderResult> {
|
||
const serveUrl = await getBundlePath();
|
||
const compositionId = input.videoType === "demo" ? "DemoVideo" : "PitchDeck";
|
||
|
||
const composition = await selectComposition({
|
||
serveUrl,
|
||
id: compositionId,
|
||
inputProps: input,
|
||
});
|
||
|
||
const { buffer } = await renderMedia({
|
||
composition,
|
||
serveUrl,
|
||
codec: "h264",
|
||
inputProps: input,
|
||
concurrency: 1, // cap concurrency — renders alongside LLM
|
||
onProgress: ({ progress }) => {
|
||
publishLiveEvent({
|
||
companyId,
|
||
type: "content_job.progress",
|
||
payload: { jobId, progress: Math.round(progress * 100) },
|
||
});
|
||
},
|
||
});
|
||
|
||
// ... build bundle and return RenderResult
|
||
}
|
||
```
|
||
|
||
### Pattern 3: Composition with Series
|
||
**What:** Each slide is a `Series.Sequence`; total `durationInFrames` is computed by summing slide durations.
|
||
**When to use:** Any multi-slide sequence where timing must be sequential without manual `from` calculations.
|
||
**Example:**
|
||
```typescript
|
||
// packages/content-renderer/src/compositions/PitchDeck.tsx
|
||
// Source: https://www.remotion.dev/docs/series
|
||
import { AbsoluteFill, Series } from "remotion";
|
||
import { SlideFrame } from "../components/SlideFrame";
|
||
|
||
const FRAMES_PER_SLIDE = 90; // 3 seconds at 30fps
|
||
|
||
export interface Slide {
|
||
title: string;
|
||
body: string;
|
||
accent: string;
|
||
}
|
||
|
||
interface PitchDeckProps {
|
||
slides: Slide[];
|
||
companyName: string;
|
||
}
|
||
|
||
export const PitchDeck: React.FC<PitchDeckProps> = ({ slides, companyName }) => (
|
||
<AbsoluteFill style={{ background: "#0f0f0f" }}>
|
||
<Series>
|
||
{slides.map((slide, i) => (
|
||
<Series.Sequence key={i} durationInFrames={FRAMES_PER_SLIDE}>
|
||
<SlideFrame slide={slide} slideIndex={i} total={slides.length} />
|
||
</Series.Sequence>
|
||
))}
|
||
</Series>
|
||
</AbsoluteFill>
|
||
);
|
||
```
|
||
|
||
### Pattern 4: Root Registration
|
||
**What:** `registerRoot()` must be called in the entry point file that `bundle()` points to.
|
||
**When to use:** Exactly once, in the entry file.
|
||
**Example:**
|
||
```typescript
|
||
// packages/content-renderer/src/Root.tsx
|
||
import { Composition } from "remotion";
|
||
import { registerRoot } from "remotion";
|
||
import { PitchDeck } from "./compositions/PitchDeck";
|
||
import { DemoVideo } from "./compositions/DemoVideo";
|
||
|
||
const FRAMES_PER_SLIDE = 90;
|
||
|
||
export const RemotionRoot: React.FC = () => (
|
||
<>
|
||
<Composition
|
||
id="PitchDeck"
|
||
component={PitchDeck}
|
||
width={1920}
|
||
height={1080}
|
||
fps={30}
|
||
durationInFrames={FRAMES_PER_SLIDE * 8} // default; overridden by calculateMetadata
|
||
defaultProps={{ slides: [], companyName: "" }}
|
||
/>
|
||
<Composition
|
||
id="DemoVideo"
|
||
component={DemoVideo}
|
||
width={1920}
|
||
height={1080}
|
||
fps={30}
|
||
durationInFrames={FRAMES_PER_SLIDE * 6}
|
||
defaultProps={{ slides: [], title: "" }}
|
||
/>
|
||
</>
|
||
);
|
||
|
||
registerRoot(RemotionRoot);
|
||
```
|
||
|
||
### Pattern 5: Interactive Web Preview with @remotion/player
|
||
**What:** `<Player>` renders the same React composition interactively in the browser without encoding.
|
||
**When to use:** The UI "interactive web" output mode for PRES-02.
|
||
**Example:**
|
||
```typescript
|
||
// ui/src/components/PresentationPanel.tsx (result section)
|
||
import { Player } from "@remotion/player";
|
||
import { PitchDeck } from "@paperclipai/content-renderer";
|
||
|
||
<Player
|
||
component={PitchDeck}
|
||
inputProps={bundle.inputProps}
|
||
durationInFrames={bundle.durationInFrames}
|
||
fps={30}
|
||
compositionWidth={1920}
|
||
compositionHeight={1080}
|
||
style={{ width: "100%", aspectRatio: "16/9" }}
|
||
controls
|
||
/>
|
||
```
|
||
|
||
**NOTE:** This requires `PitchDeck` to be importable from `content-renderer` by the Vite build. The composition components (no `bundle()` / `renderMedia()` — those are server-only) can be re-exported from the package with a UI-safe entrypoint that does NOT import `@remotion/bundler` or `@remotion/renderer`.
|
||
|
||
### Anti-Patterns to Avoid
|
||
- **Calling `bundle()` per render:** Triggers a full webpack compilation each time (~30s+). Bundle once at startup and cache the path.
|
||
- **Importing `@remotion/bundler` or `@remotion/renderer` in the server's tsc entry:** These packages pull in webpack. Add to `packages/content-renderer/` only; server imports via `@paperclipai/content-renderer` workspace reference.
|
||
- **Importing webpack-containing code in Vite context:** `@remotion/bundler` will break Vite if imported by any UI module. Composition components should be split into a UI-safe sub-export that only imports from `remotion` (not `@remotion/bundler`).
|
||
- **Setting `concurrency` too high:** Video rendering and LLM inference compete for CPU. Use `concurrency: 1` or `"50%"` max per the success criteria SC4.
|
||
- **Omitting `content_job.progress` from shared/constants.ts:** TypeScript will reject the event type unless it's added to the `LIVE_EVENT_TYPES` array.
|
||
|
||
---
|
||
|
||
## Don't Hand-Roll
|
||
|
||
| Problem | Don't Build | Use Instead | Why |
|
||
|---------|-------------|-------------|-----|
|
||
| Frame timing for multi-slide decks | Custom timing calculator | `Series` component | Series handles cumulative `from` calculation; manual calculation has off-by-one errors |
|
||
| Video encoding (H.264, WebM) | ffmpeg subprocess with raw frames | `renderMedia()` | Remotion handles frame rendering + encoding pipeline; direct ffmpeg needs frame extraction first |
|
||
| Webpack bundling of React compositions | Custom webpack config | `bundle()` | Remotion's bundler has composition-specific optimizations and correct env setup |
|
||
| Progress percentage calculation | Polling DB status | `renderMedia()` onProgress callback | onProgress fires per-frame with accurate float; polling is coarse |
|
||
| Browser management for render | Manual Puppeteer setup | Remotion auto-manages via `openBrowser()` / auto-download | Remotion downloads its own compatible Chromium (or reuses via browserExecutable); mismatched versions cause render failures |
|
||
|
||
**Key insight:** Remotion is a complete render pipeline. The only custom code needed is React composition components (what each slide looks like) and the LLM prompt that generates the `inputProps` JSON.
|
||
|
||
---
|
||
|
||
## Common Pitfalls
|
||
|
||
### Pitfall 1: Webpack/Rspack in tsc/Vite Context
|
||
**What goes wrong:** `@remotion/bundler` imports webpack which breaks tsc strict mode and crashes Vite's dev server.
|
||
**Why it happens:** pnpm workspace hoisting can make `@remotion/bundler` visible to the server package's tsc even if it's only listed as a dep of `content-renderer`.
|
||
**How to avoid:** Keep `@remotion/bundler` and `@remotion/renderer` ONLY in `packages/content-renderer/package.json`. Import from `@paperclipai/content-renderer` using a dynamic import in `presentation-renderer.ts`. Add `@remotion/bundler` to server tsconfig `exclude` if needed.
|
||
**Warning signs:** `tsc` errors referencing webpack types; Vite startup errors about `require` in webpack modules.
|
||
|
||
### Pitfall 2: Bundle Path Not Persisted Across Hot Reloads
|
||
**What goes wrong:** In dev mode (`tsx watch`), module-level variables reset on each hot reload, triggering a new `bundle()` call every time a file changes — O(30s) per save.
|
||
**Why it happens:** `tsx` re-executes module code on watch. A plain module-level `let cachedBundlePath` is cleared.
|
||
**How to avoid:** Write the bundle path to a temp file on first run and re-read it on subsequent startups. Or accept the re-bundle in dev (not blocking, just slow).
|
||
**Warning signs:** Server logs showing "Bundling..." on every file save.
|
||
|
||
### Pitfall 3: Missing `content_job.progress` in Shared Constants
|
||
**What goes wrong:** `publishLiveEvent` TypeScript call fails to compile because `"content_job.progress"` is not in the `LiveEventType` union.
|
||
**Why it happens:** `packages/shared/src/constants.ts` defines the exhaustive list. New event types must be added there.
|
||
**How to avoid:** Add `"content_job.progress"` to `LIVE_EVENT_TYPES` array in shared/constants.ts as the FIRST change in Wave 1.
|
||
**Warning signs:** TypeScript error `Argument of type '"content_job.progress"' is not assignable to parameter of type 'LiveEventType'`.
|
||
|
||
### Pitfall 4: useContentJob Progress Not Surfaced to UI
|
||
**What goes wrong:** `content_job.progress` events arrive via SSE but the `useContentJob` hook ignores the `progress` field in the payload (it reads `data.status` but not `data.progress`).
|
||
**Why it happens:** Current `statusToProgress()` maps lifecycle stages to coarse values (5/50/100). Fine-grained progress from Remotion needs explicit handling.
|
||
**How to avoid:** Extend the `status` SSE event handler in `useContentJob` to check `data.progress` (number 0–100) and update the `progress` state field directly when `data.status === "running"`.
|
||
**Warning signs:** Progress bar stays stuck at 50% throughout the entire render despite SSE events arriving.
|
||
|
||
### Pitfall 5: @remotion/player Import Pulls in Renderer in Vite
|
||
**What goes wrong:** Importing `PitchDeck` composition component from `@paperclipai/content-renderer` in the UI inadvertently imports code that depends on `@remotion/bundler`.
|
||
**Why it happens:** If `packages/content-renderer/src/index.ts` re-exports everything including `getBundlePath()` and `renderPresentation()`, Vite will try to bundle those imports and fail on webpack/Node internals.
|
||
**How to avoid:** Use a separate UI-safe sub-export in content-renderer package: `"./compositions"` exports only React components; `"."` (default) exports `renderPresentation` and `getBundlePath`. The UI imports from `@paperclipai/content-renderer/compositions`; the server imports from `@paperclipai/content-renderer`.
|
||
**Warning signs:** Vite build errors about `process.env`, `__dirname`, `require`, or webpack-related modules.
|
||
|
||
### Pitfall 6: Chromium Binary Conflict
|
||
**What goes wrong:** Remotion auto-downloads its own Chrome Headless Shell while Playwright's Chromium is already at `~/.cache/ms-playwright`. Disk space wasted, possible version conflict.
|
||
**Why it happens:** Remotion's default behavior is to auto-download a compatible browser.
|
||
**How to avoid:** Pass `browserExecutable` pointing to the existing Playwright Chromium binary at `~/.cache/ms-playwright/chromium-1217/chrome-linux64/chrome`. This is the same path that `resolveBrowserPath()` in `diagram-renderer.ts` already resolves.
|
||
**Warning signs:** Remotion downloading ~300MB "chrome-headless-shell" at first render.
|
||
|
||
---
|
||
|
||
## Code Examples
|
||
|
||
Verified patterns from official sources and codebase:
|
||
|
||
### Startup Bundle Initialization (server/src/index.ts)
|
||
```typescript
|
||
// Source: https://www.remotion.dev/docs/bundle (bundle once pattern)
|
||
// Add near server startup sequence, after DB init
|
||
|
||
let remotionBundlePath: string | null = null;
|
||
|
||
async function initRemotionBundle(): Promise<void> {
|
||
const { getBundlePath } = await import("@paperclipai/content-renderer");
|
||
remotionBundlePath = await getBundlePath();
|
||
logger.info({ path: remotionBundlePath }, "Remotion bundle ready");
|
||
}
|
||
|
||
// Call during startup (non-blocking, fire and forget or await):
|
||
void initRemotionBundle().catch((err) => {
|
||
logger.warn({ err }, "Remotion bundle init failed — video rendering unavailable");
|
||
});
|
||
```
|
||
|
||
### Presentation Renderer (server/src/services/renderers/presentation-renderer.ts)
|
||
```typescript
|
||
// Source: https://www.remotion.dev/docs/renderer/render-media
|
||
import type { RenderResult } from "./types.js";
|
||
import { publishLiveEvent } from "../live-events.js";
|
||
import { puterChatComplete } from "../puter-inference.js";
|
||
|
||
export async function renderPresentation(
|
||
input: Record<string, unknown>,
|
||
companyId: string,
|
||
jobId: string,
|
||
): Promise<RenderResult> {
|
||
const { getBundlePath, renderPresentationComposition } = await import(
|
||
"@paperclipai/content-renderer"
|
||
);
|
||
const result = await renderPresentationComposition({
|
||
serveUrl: await getBundlePath(),
|
||
input,
|
||
onProgress: (progress: number) => {
|
||
publishLiveEvent({
|
||
companyId,
|
||
type: "content_job.progress",
|
||
payload: { jobId, progress },
|
||
});
|
||
},
|
||
browserExecutable: resolveBrowserPath(), // reuse Playwright Chromium
|
||
});
|
||
return result;
|
||
}
|
||
```
|
||
|
||
### content_job.progress Event in Shared Constants (packages/shared/src/constants.ts)
|
||
```typescript
|
||
// Source: codebase constants.ts line 324
|
||
export const LIVE_EVENT_TYPES = [
|
||
"heartbeat.run.queued",
|
||
"heartbeat.run.status",
|
||
"heartbeat.run.event",
|
||
"heartbeat.run.log",
|
||
"agent.status",
|
||
"activity.logged",
|
||
"plugin.ui.updated",
|
||
"plugin.worker.crashed",
|
||
"plugin.worker.restarted",
|
||
"content_job.queued",
|
||
"content_job.running",
|
||
"content_job.progress", // ADD THIS
|
||
"content_job.done",
|
||
"content_job.failed",
|
||
] as const;
|
||
```
|
||
|
||
### useContentJob Progress Extension (ui/src/hooks/useContentJob.ts)
|
||
```typescript
|
||
// Source: codebase ui/src/hooks/useContentJob.ts
|
||
// Extend the status event handler to capture fine-grained progress:
|
||
es.addEventListener("status", (e: MessageEvent) => {
|
||
const data = JSON.parse(e.data as string) as {
|
||
status?: string;
|
||
progress?: number; // NEW: Remotion render progress 0–100
|
||
resultAssetId?: string | null;
|
||
errorMessage?: string | null;
|
||
};
|
||
const status = (data.status ?? "queued") as JobStatus;
|
||
const progress =
|
||
typeof data.progress === "number"
|
||
? data.progress // fine-grained from Remotion onProgress
|
||
: statusToProgress(status); // coarse fallback for non-video jobs
|
||
|
||
setState((prev) => ({
|
||
...prev,
|
||
status,
|
||
progress,
|
||
resultAssetId: data.resultAssetId ?? prev.resultAssetId,
|
||
errorMessage: data.errorMessage ?? prev.errorMessage,
|
||
}));
|
||
// ...
|
||
});
|
||
```
|
||
|
||
### PresentationBundle Type (server/src/services/renderers/types.ts)
|
||
```typescript
|
||
// Follow existing bundle type pattern in types.ts
|
||
export interface PresentationBundle {
|
||
type: "presentation-bundle";
|
||
presentationType: "pitch-deck" | "demo-video";
|
||
title: string;
|
||
slideCount: number;
|
||
durationInFrames: number;
|
||
fps: number;
|
||
mp4Base64: string; // H.264 video output
|
||
inputProps: Record<string, unknown>; // for Player re-render in UI
|
||
}
|
||
```
|
||
|
||
### content-job-runner.ts Extension
|
||
```typescript
|
||
// Source: codebase server/src/services/content-job-runner.ts
|
||
// Add "presentation" case to renderContent() switch:
|
||
case "presentation": {
|
||
const { renderPresentation } = await import("./renderers/presentation-renderer.js");
|
||
return renderPresentation(input, job.companyId, job.id);
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## State of the Art
|
||
|
||
| Old Approach | Current Approach | When Changed | Impact |
|
||
|--------------|------------------|--------------|--------|
|
||
| Webpack bundler | rspack (default in @remotion/bundler 4.x) | Remotion 4.0 | ~3x faster bundle compilation; same API |
|
||
| Manual Puppeteer browser management | `openBrowser()` + `browserExecutable` | Remotion 3.x+ | Reuse existing browser binary |
|
||
| Lambda-only SSR | Full Node.js API (`bundle()` + `renderMedia()`) | Remotion 2.x+ | Self-hosted rendering without cloud dependency |
|
||
| webpack 4 | webpack 5.105 (via @remotion/bundler) | Remotion 4.x | Faster, better tree-shaking |
|
||
|
||
**Deprecated/outdated:**
|
||
- `renderFrames()` + manual encoding: Replaced by the unified `renderMedia()` which handles both frame rendering and encoding in one call.
|
||
- `getCompositions()` for single render: Use `selectComposition()` instead — it evaluates only the target composition and throws if not found.
|
||
|
||
---
|
||
|
||
## Open Questions
|
||
|
||
1. **packages/content-renderer/ module system: ESM vs CJS**
|
||
- What we know: `@remotion/bundler` uses webpack/rspack internally; the `bundle()` function itself runs in Node. The rest of the server uses ESM (`"type": "module"`).
|
||
- What's unclear: Whether `@remotion/bundler` requires CommonJS module context or works from ESM. The rspack integration uses `require()` internally.
|
||
- Recommendation: Start with `"type": "commonjs"` in packages/content-renderer/package.json and use `.cjs` extension or omit `"type": "module"`. Export via named CJS exports; server's dynamic import handles the interop.
|
||
|
||
2. **Browser reuse: Playwright vs Remotion auto-download**
|
||
- What we know: Playwright Chromium is at `~/.cache/ms-playwright/chromium-1217/chrome-linux64/chrome`. Remotion's `browserExecutable` parameter accepts an absolute path.
|
||
- What's unclear: Whether the Playwright Chromium version (1217 = Chrome 127 area) is compatible with what Remotion 4.0.445 expects.
|
||
- Recommendation: Pass `browserExecutable` pointing to the existing Playwright Chromium. If Remotion rejects it (version mismatch), fall back to letting Remotion auto-download — but add a startup log warning about the extra disk use.
|
||
|
||
3. **Interactive web preview: share composition code with UI**
|
||
- What we know: `@remotion/player` works in Vite. The composition components themselves (just React + remotion primitives) can be imported by Vite safely.
|
||
- What's unclear: Whether a sub-export path like `@paperclipai/content-renderer/compositions` is the right pattern, or whether we just duplicate the slide components in the UI package.
|
||
- Recommendation: Use sub-exports in packages/content-renderer/package.json: `"./compositions": "./src/compositions/index.ts"`. This sub-export must NOT re-export anything from `@remotion/bundler` or `@remotion/renderer`.
|
||
|
||
---
|
||
|
||
## Environment Availability
|
||
|
||
| Dependency | Required By | Available | Version | Fallback |
|
||
|------------|------------|-----------|---------|----------|
|
||
| Node.js | renderMedia(), bundle() | Yes | v20.20.2 | — |
|
||
| Playwright Chromium | Remotion browser (via browserExecutable) | Yes | ~1217 (Chrome 127) | Remotion auto-downloads its own |
|
||
| ffmpeg | Video encoding (Remotion uses internally) | Yes (via ffmpeg-static) | bundled | — |
|
||
| @remotion/compositor-linux-x64-gnu | Remotion Rust compositor (auto-installed) | Not yet — must pnpm add | 4.0.445 | No fallback — required for Linux render |
|
||
| pnpm workspace | packages/content-renderer/ as workspace pkg | Yes | workspace configured | — |
|
||
|
||
**Missing dependencies with no fallback:**
|
||
- `@remotion/compositor-linux-x64-gnu`: Installed automatically as an optional dep of `@remotion/renderer`. Must run `pnpm install` after adding `@remotion/renderer` to the workspace package.
|
||
|
||
**Missing dependencies with fallback:**
|
||
- Playwright Chromium for Remotion: If version incompatibility, Remotion auto-downloads Chrome Headless Shell (~300MB).
|
||
|
||
---
|
||
|
||
## Validation Architecture
|
||
|
||
### Test Framework
|
||
| Property | Value |
|
||
|----------|-------|
|
||
| Framework | vitest 3.0.5 |
|
||
| Config file | server/vitest.config.ts |
|
||
| Quick run command | `pnpm --filter @paperclipai/server test --run src/__tests__/presentation-renderer.test.ts` |
|
||
| Full suite command | `pnpm --filter @paperclipai/server test --run` |
|
||
|
||
### Phase Requirements → Test Map
|
||
| Req ID | Behavior | Test Type | Automated Command | File Exists? |
|
||
|--------|----------|-----------|-------------------|-------------|
|
||
| PRES-01 | LLM generates slide JSON, job submitted successfully | unit | `pnpm --filter @paperclipai/server test --run src/__tests__/presentation-renderer.test.ts` | ❌ Wave 0 |
|
||
| PRES-02 | renderPresentation() returns mp4Base64 + inputProps for Player | unit | same | ❌ Wave 0 |
|
||
| PRES-03 | DemoVideo composition renders with narration script input | unit | same | ❌ Wave 0 |
|
||
| PRES-04 | onProgress callback fires; content_job.progress events published | unit | `pnpm --filter @paperclipai/server test --run src/__tests__/content-jobs-sse.test.ts` | ✅ extend |
|
||
|
||
### Sampling Rate
|
||
- **Per task commit:** `pnpm --filter @paperclipai/server test --run src/__tests__/presentation-renderer.test.ts`
|
||
- **Per wave merge:** `pnpm --filter @paperclipai/server test --run`
|
||
- **Phase gate:** Full suite green before `/gsd:verify-work`
|
||
|
||
### Wave 0 Gaps
|
||
- [ ] `server/src/__tests__/presentation-renderer.test.ts` — covers PRES-01, PRES-02, PRES-03 (mock `@paperclipai/content-renderer` and `puter-inference`)
|
||
- [ ] `packages/content-renderer/` — new workspace package with package.json, tsconfig.json, src/Root.tsx
|
||
- [ ] Add `"content_job.progress"` to `packages/shared/src/constants.ts` — unblocks PRES-04 TypeScript
|
||
|
||
*(No new test framework needed — vitest already in server/ and all existing patterns apply)*
|
||
|
||
---
|
||
|
||
## Sources
|
||
|
||
### Primary (HIGH confidence)
|
||
- https://www.remotion.dev/docs/bundle — bundle() API, caching pattern
|
||
- https://www.remotion.dev/docs/renderer/render-media — renderMedia() onProgress, concurrency, outputLocation
|
||
- https://www.remotion.dev/docs/renderer/select-composition — selectComposition() API
|
||
- https://www.remotion.dev/docs/series — Series + Series.Sequence for slide timing
|
||
- https://www.remotion.dev/docs/composition — Composition component, registerRoot
|
||
- https://www.remotion.dev/docs/renderer/open-browser — browserExecutable pattern, shared instance
|
||
- `npm view @remotion/renderer optionalDependencies` — confirmed Rust compositor packages
|
||
- Codebase: server/src/services/renderers/ — existing renderer patterns
|
||
- Codebase: server/src/services/live-events.ts — EventEmitter SSE infrastructure
|
||
- Codebase: packages/shared/src/constants.ts — LIVE_EVENT_TYPES (missing content_job.progress)
|
||
- Codebase: packages/db/src/schema/content_jobs.ts — no schema changes needed
|
||
|
||
### Secondary (MEDIUM confidence)
|
||
- https://www.remotion.dev/docs/ssr-node — Node SSR workflow (bundle once + render per request)
|
||
- https://www.remotion.dev/docs/sequence — Sequence timing model
|
||
|
||
### Tertiary (LOW confidence)
|
||
- Module system choice (ESM vs CJS for content-renderer): Based on webpack's CommonJS requirements and existing workspace patterns. Not directly documented by Remotion.
|
||
|
||
---
|
||
|
||
## Metadata
|
||
|
||
**Confidence breakdown:**
|
||
- Standard stack: HIGH — npm view confirmed all package versions live
|
||
- Architecture: HIGH — Remotion official docs + codebase pattern match is clear
|
||
- Pitfalls: HIGH for webpack isolation (State.md pre-identifies this); MEDIUM for browser reuse (version compatibility unverified)
|
||
- SSE progress: HIGH — LIVE_EVENT_TYPES gap confirmed in codebase, fix path is clear
|
||
|
||
**Research date:** 2026-04-04
|
||
**Valid until:** 2026-05-04 (Remotion 4.x is stable; patch versions won't affect API)
|