# Architecture Research **Domain:** Content generation integration — Nexus v1.7 **Researched:** 2026-04-04 **Confidence:** HIGH (based on direct codebase inspection of /opt/nexus) ## Standard Architecture ### System Overview ``` +---------------------------------------------------------------------------------+ | UI Layer (React/Vite) | | +------------------+ +------------------+ +--------------+ +----------------+ | | | ChatPanel | | ContentJobViewer | | ThemePreview | | DiagramRenderer| | | | (existing, | | (new) | | (new) | | (new, wraps | | | | minor extension)| | progress+result | | CSS vars | | mermaid dep) | | | +--------+---------+ +--------+---------+ +------+-------+ +-------+--------+ | | | | | | | +-----------|--------------------|--------------------|-----------------|------------+ | HTTP/SSE | HTTP/SSE | HTTP | (client-side) +-----------|--------------------|--------------------|-------------------------+ | | API Layer (Express) | | | +--------v----------------------------------------------v------------------+ | | | /api/companies/:id/content-jobs (new) | | | | /api/content-jobs/:id (new) | | | | /api/companies/:id/themes/generate (new) | | | +-------------------------------------------------------------------------+ | +---------------------------------------------------------------------------------+ | Service Layer (Node.js) | | +-------------------+ +------------------+ +---------------------------------+| | | contentJobService | | themeEngineService| | renderPipelineService || | | (new) | | (new) | | (new) || | | enqueue, status, | | palette gen, | | routes jobs to renderer adapters || | | list | | WCAG check,export | | || | +--------+----------+ +------------------+ +---------------+-----------------+| | | | | | +--------v----------------------------------------------------v--------------+ | | | Renderer Adapters (new, behind interface) | | | | +-------------+ +------------+ +----------+ +-----------+ +----------+ | | | | | Mermaid | | SVG | | Remotion | | PDF | | Image | | | | | | (isomorphic)| | (generator)| | (CLI) | | (Puppeteer| | (Sharp) | | | | | +-------------+ +------------+ +----------+ +-----------+ +----------+ | | | +-------------------------------------------------------------------------+ | +---------------------------------------------------------------------------------+ | Storage + Events Layer (existing, minimally extended) | | +------------------+ +--------------------+ +-----------------------------+ | | | StorageService | | publishLiveEvent | | assets table (existing) | | | | (existing) | | (existing +3 types)| | content_jobs table (new) | | | +------------------+ +--------------------+ +-----------------------------+ | +---------------------------------------------------------------------------------+ ``` ### Component Responsibilities | Component | Responsibility | Status | Notes | |-----------|----------------|--------|-------| | `contentJobService` | Queue and track async render jobs; emit live events on status change | New | Factory function, matches `chatService` pattern | | `renderPipelineService` | Route render requests to the correct renderer adapter | New | Strategy pattern over adapters | | `themeEngineService` | Palette generation, WCAG AA validation, CSS/JSON/Tailwind exports | New | Pure computation, no DB, deterministic | | `mermaidRendererAdapter` | Mermaid DSL string to SVG buffer, server-side | New | Uses `@mermaid-js/mermaid-isomorphic`; no Chromium needed | | `remotionRendererAdapter` | Invoke Remotion CLI subprocess, return MP4/WebM path | New | Subprocess; outputs go to storage namespace `generated/videos` | | `svgGeneratorAdapter` | Template-based SVG generation (icons, banners, placeholders) | New | No binary deps; pure string construction + existing sanitizer | | `pdfRendererAdapter` | HTML to PDF via Puppeteer (arm64 Chromium on M4) | New | Subprocess; Puppeteer arm64 works on Apple Silicon | | `imageProcessorAdapter` | Composite and resize via Sharp | Modified | Sharp already in `server/package.json`; extend for content use | | `placeholderService` | Manifest tracking for draft assets | Existing | Already implemented; optionally extend PlaceholderEntry with `contentJobId` | | `assetService` | CRUD for the `assets` table | Existing | Already handles `createdByAgentId`; use as-is | | `StorageService` | Provider-agnostic blob storage | Existing | Use `generated/` namespace prefix for all new content | | `publishLiveEvent` | SSE fan-out to UI subscribers | Existing | Extend `LIVE_EVENT_TYPES` with 3 new content job event types | | `ContentJobViewer` (UI) | Poll/stream job status; show progress, render result inline | New | Subscribes to SSE live events | | `DiagramRenderer` (UI) | Client-side Mermaid render using existing `mermaid` dep | New | `mermaid ^11.12.0` already in `ui/package.json` | | `ThemePreview` (UI) | Live palette preview via CSS custom properties | New | No server round-trip for preview | | `ContentGallery` (UI) | Workspace page showing all generated assets | New | Pagination via `assetService.list` | ## Recommended Project Structure New files follow existing monorepo conventions: factory functions, co-located types, no class syntax. ``` server/src/ ├── services/ │ ├── content-job.ts # contentJobService factory │ ├── render-pipeline.ts # renderPipelineService — adapter dispatch │ ├── theme-engine.ts # themeEngineService — pure palette computation │ └── renderers/ │ ├── index.ts # RendererAdapter interface + barrel │ ├── mermaid-renderer.ts # Mermaid DSL -> SVG (server-side isomorphic) │ ├── remotion-renderer.ts # Remotion CLI subprocess wrapper │ ├── svg-generator.ts # Template SVG (icons, placeholders, banners) │ └── pdf-renderer.ts # HTML -> PDF via Puppeteer ├── routes/ │ ├── content-jobs.ts # GET/POST /companies/:id/content-jobs │ └── themes.ts # POST /companies/:id/themes/generate └── types/ └── content.ts # Server-internal ContentJobType, ContentJobStatus packages/db/src/ ├── schema/ │ └── content_jobs.ts # NEW table (upstream-safe, no upstream equivalent) └── migrations/ └── NNNN_add_content_jobs.sql packages/shared/src/ ├── types/ │ └── content.ts # ContentJob, ContentJobStatus shared types └── constants.ts # LIVE_EVENT_TYPES extended (+3 content.job.* types) packages/ └── remotion-compositions/ # NEW workspace package ├── package.json └── src/ └── index.ts # Remotion composition definitions ui/src/ ├── components/ │ ├── ContentJobViewer.tsx # Job progress + result display │ ├── ContentJobCard.tsx # Compact job status card │ ├── DiagramRenderer.tsx # Mermaid client-side wrapper │ ├── ThemePreview.tsx # Live palette preview │ └── GeneratedAssetCard.tsx # Thumbnail + download + metadata └── pages/ └── ContentGallery.tsx # Gallery of generated assets per workspace ``` ### Structure Rationale - **`server/src/services/renderers/`**: Isolates binary-dependent adapters behind a shared `RendererAdapter` interface. New renderers plug in without touching the core job service. - **`content_jobs` table**: Separate from `assets`. A job tracks render lifecycle (queued to running to done/failed); on success it writes an `assets` row and records the `assetId`. This mirrors how `heartbeat_runs` tracks execution separately from its outputs. - **`packages/remotion-compositions/`**: Remotion compositions must be bundled ahead of time. Keeping them in a dedicated workspace package lets the bundle step run once at startup, not on every render request. - **Content as skills**: Skills (`company_skills`) are Markdown instruction files. Content type skills tell agents which `/api/companies/:id/content-jobs` endpoint to call and with what parameters. No new schema needed. ## Architectural Patterns ### Pattern 1: Async Job with SSE Progress **What:** Long-running renders (Remotion, Puppeteer PDF) run asynchronously. The service creates a `content_jobs` row with `status: "queued"`, immediately returns the job record, then spawns the renderer. Live events push progress to the UI over the existing SSE stream. **When to use:** Any render taking more than ~200ms: Remotion, PDF, large Mermaid diagrams. Fast operations (SVG generation, theme palette) can be synchronous HTTP. **Trade-offs:** One DB row per render. Adds durable history of what was generated. Acceptable at solo-user scale. **Example:** ```typescript // server/src/services/content-job.ts export function contentJobService(db: Db, storage: StorageService) { return { async enqueue(companyId: string, input: ContentJobInput): Promise { const [row] = await db .insert(contentJobs) .values({ companyId, type: input.type, params: input.params, status: "queued" }) .returning(); publishLiveEvent({ companyId, type: "content.job.started", payload: { jobId: row.id } }); // Non-blocking — kick off render renderPipelineService(storage).render(row) .then(async (result) => { await db.update(contentJobs) .set({ status: "done", assetId: result.assetId, completedAt: new Date() }) .where(eq(contentJobs.id, row.id)); publishLiveEvent({ companyId, type: "content.job.done", payload: { jobId: row.id, assetId: result.assetId } }); }) .catch(async (err) => { await db.update(contentJobs) .set({ status: "failed", errorMessage: String(err) }) .where(eq(contentJobs.id, row.id)); publishLiveEvent({ companyId, type: "content.job.failed", payload: { jobId: row.id } }); }); return toContentJob(row); }, }; } ``` ### Pattern 2: RendererAdapter Interface **What:** Each renderer implements a shared interface. `renderPipelineService` selects the adapter based on `ContentJobType`. Adding a new renderer requires only: (a) implement the interface, (b) register in the dispatch table. **When to use:** Every new content type. **Trade-offs:** Thin abstraction, no framework needed. Appropriate for the codebase size and single-user scale. **Example:** ```typescript // server/src/services/renderers/index.ts export interface RendererAdapter { type: ContentJobType; render( params: Record, storage: StorageService, companyId: string ): Promise<{ objectKey: string; contentType: string; byteSize: number }>; } ``` ### Pattern 3: Content Types as Skill Files **What:** A Mermaid-generation "skill" is a Markdown file in `company_skills` that instructs agents: "When asked for a diagram, call `POST /api/companies/:id/content-jobs` with `{type: 'mermaid', params: {dsl: '...'}}` and wait for `content.job.done` event." No new schema required. **When to use:** All content types — this is how they become installable skills. **Trade-offs:** Agent must know the API contract, included in the skill markdown. Works with all adapters (Claude Code, Hermes, Ollama) since skills are plain text. ### Pattern 4: Theme Engine as Pure Function, Preview Client-Side **What:** Theme generation is a pure computation: seed hex color in, palette object out. Preview injects CSS custom properties directly into the DOM — no server round-trip. Saving a theme stores the palette JSON via `StorageService`. **When to use:** Theme generation and live preview. **Trade-offs:** No server latency for preview feedback. The JSON is reusable for all export formats (CSS, Tailwind config, design tokens). ## Data Flow ### Async Content Job Request Flow ``` Agent / UI | v POST /api/companies/:id/content-jobs { type: "mermaid", params: { dsl: "graph TD..." } } | v contentJobService.enqueue() INSERT content_jobs WHERE status = "queued" publishLiveEvent("content.job.started") (non-blocking) -> renderPipelineService.render() | v (async) renderPipelineService selects MermaidRendererAdapter | v MermaidRendererAdapter.render() Mermaid DSL -> SVG Buffer (via @mermaid-js/mermaid-isomorphic) | v StorageService.putFile() objectKey: "{companyId}/generated/diagrams/2026/04/04/{uuid}-diagram.svg" | v assetService.create() INSERT assets (objectKey, contentType, byteSize, createdByAgentId) | v contentJobService callback UPDATE content_jobs SET status = "done", asset_id = ... publishLiveEvent("content.job.done", { jobId, assetId }) | v UI (SSE subscriber) ContentJobViewer receives event -> fetches asset URL -> renders preview inline ``` ### Theme Generation Flow (Synchronous) ``` User picks seed color | v ThemePreview component (no server round-trip — CSS custom properties injected directly into DOM) | v (on "Save Theme") POST /api/companies/:id/themes/generate { seedColor: "#4a90d9" } | v themeEngineService.generate() Compute palette (tints, shades, semantic tokens, WCAG AA checks) Returns palette JSON | v StorageService.putFile() objectKey: "{companyId}/themes/{uuid}-theme.json" contentType: "application/json" | v assetService.create() -> 201 { assetId, downloadUrl } ``` ### Mermaid Client-Side Fast Path ``` Agent sends message with ```mermaid code block | v ChatMarkdownMessage (existing component, minor extension) Detects ```mermaid fence | v DiagramRenderer (new component, wraps existing mermaid dep) Calls mermaid.render() client-side (mermaid ^11.12 already in ui/package.json) Displays SVG inline "Save as asset" button -> POST /api/companies/:id/content-jobs (server path) ``` ### State Transitions: content_jobs ``` queued -> running (renderPipelineService picks up job) running -> done (renderer returns, asset created) running -> failed (renderer throws, error recorded) ``` ## New vs Modified: Explicit Breakdown ### New (does not exist) | Artifact | Type | Purpose | |----------|------|---------| | `content_jobs` table | DB schema + migration | Track async render job lifecycle | | `contentJobService` | Server service | Enqueue, status, list jobs | | `renderPipelineService` | Server service | Route jobs to renderer adapters | | `themeEngineService` | Server service | Palette generation + WCAG validation | | `mermaidRendererAdapter` | Server renderer | Server-side Mermaid to SVG | | `remotionRendererAdapter` | Server renderer | Remotion CLI to MP4/WebM | | `svgGeneratorAdapter` | Server renderer | Template SVG generation | | `pdfRendererAdapter` | Server renderer | HTML to PDF via Puppeteer | | `content-jobs.ts` route | API route | Create and list content jobs | | `themes.ts` route | API route | Synchronous theme generation | | `packages/remotion-compositions/` | Workspace package | Remotion composition definitions | | `packages/shared/src/types/content.ts` | Shared type | `ContentJob`, `ContentJobStatus` | | `ContentJobViewer` | UI component | Job progress + result display | | `DiagramRenderer` | UI component | Client-side Mermaid wrapper | | `ThemePreview` | UI component | Live palette preview | | `GeneratedAssetCard` | UI component | Asset thumbnail + actions | | `ContentGallery` | UI page | Workspace content library | ### Modified (exists, needs extension) | Artifact | Change | Risk | |----------|--------|------| | `packages/shared/src/constants.ts` | Add 3 new `LIVE_EVENT_TYPES`: `content.job.started`, `content.job.done`, `content.job.failed` | LOW — additive only | | `server/src/app.ts` | Mount `contentJobRoutes` and `themeRoutes` | LOW — two lines | | `ChatMarkdownMessage` | Detect triple-backtick mermaid fence; render via `DiagramRenderer` | MEDIUM — existing component, test carefully | | `assetService` | Add `list(companyId, opts)` method for gallery pagination | LOW — new method, no schema change | | `packages/db/src/schema/index.ts` | Export `contentJobs` table | LOW | | `packages/db/src/index.ts` | Export `contentJobs` from db package | LOW | ## Data Model ### New Table: `content_jobs` No changes to existing tables. Standalone new table; upstream-safe because Paperclip has no content generation system. ```sql CREATE TABLE content_jobs ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), company_id UUID NOT NULL REFERENCES companies(id), type TEXT NOT NULL, -- 'mermaid' | 'remotion' | 'pdf' | 'svg' | 'theme' | 'image' status TEXT NOT NULL DEFAULT 'queued', -- 'queued' | 'running' | 'done' | 'failed' params JSONB NOT NULL DEFAULT '{}', -- renderer-specific input params asset_id UUID REFERENCES assets(id), -- set on success error_message TEXT, -- set on failure created_by_agent_id UUID REFERENCES agents(id), created_by_user_id TEXT, started_at TIMESTAMPTZ, completed_at TIMESTAMPTZ, created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW() ); CREATE INDEX content_jobs_company_status_idx ON content_jobs(company_id, status); CREATE INDEX content_jobs_company_created_idx ON content_jobs(company_id, created_at DESC); ``` ### Storage Namespaces (extends existing StorageService path conventions) ``` {companyId}/generated/diagrams/YYYY/MM/DD/{uuid}-{name}.svg {companyId}/generated/videos/YYYY/MM/DD/{uuid}-{name}.mp4 {companyId}/generated/pdfs/YYYY/MM/DD/{uuid}-{name}.pdf {companyId}/generated/images/YYYY/MM/DD/{uuid}-{name}.png {companyId}/generated/icons/YYYY/MM/DD/{uuid}-{name}.svg {companyId}/themes/{uuid}-theme.json {companyId}/placeholders/{uuid}-placeholder.svg ``` The `StorageService.putFile()` method already handles path construction from `namespace` + `originalFilename` + timestamp. Pass `namespace: "generated/diagrams"` etc. ## Integration Points ### Existing System Touch Points | Integration Point | How Content Gen Connects | Notes | |-------------------|--------------------------|-------| | `assetService` + `assets` table | Every rendered output creates an asset row | `createdByAgentId` already supported; agents get credit | | `StorageService` | All rendered blobs stored via existing `putFile()` | Use `generated/` namespace prefix; no service changes | | `publishLiveEvent` | Job lifecycle events push to SSE stream | Extend `LIVE_EVENT_TYPES` in `packages/shared/src/constants.ts` | | `ChatMarkdownMessage` | Inline diagram rendering; "save as asset" button | Mermaid already a UI dep; add `DiagramRenderer` wrapper | | `companySkills` + `skill-registry` | Content types as installable skill markdown files | No schema change; skills are text files agents read as context | | `placeholderService` | Placeholder assets tracked in PLACEHOLDERS.md manifest | Optionally extend `PlaceholderEntry` with `contentJobId` | | `hardwareService` | Detect if Remotion/Puppeteer can run | M4 Mac Mini: arm64 Chromium available, 24GB unified memory sufficient | | `companyId` scoping | All content jobs scoped to `companyId` | Consistent with every other resource in the system | | Agent task sessions | Agents invoke content APIs during task execution | Use `createdByAgentId`; same pattern as `documents`, `work-products` | ### External Dependencies (new server deps) | Dependency | Purpose | Platform Notes | |------------|---------|---------------| | `@mermaid-js/mermaid-isomorphic` | Server-side Mermaid to SVG | No Chromium needed; fast; preferred over Puppeteer for Mermaid | | `puppeteer` | HTML to PDF | ~300MB install; bundled arm64 Chromium works on M4; only add if PDF is a phase priority | | `remotion` (CLI) | Video/presentation render | Add as devDep in `remotion-compositions` package; CLI called via subprocess | Mermaid client-side and Sharp are already present. No changes needed for those paths. ## Scaling Considerations This is a Mac Mini M4 single-user deployment. Analysis focuses on resource contention, not user count. | Concern | Approach | |---------|----------| | Concurrent render jobs | Node.js event loop is safe for I/O. CPU-bound renders (Remotion, Puppeteer) spawn subprocesses, keeping the event loop responsive. | | Remotion render duration | Renders can take minutes. Never synchronous HTTP. Async job pattern + SSE progress is mandatory. | | Chromium memory (PDF/Puppeteer) | Puppeteer can use 500MB+ per render. Serialize PDF renders via an in-memory queue (one at a time). | | Storage growth | Generated content accumulates. Add `retention_days` field to `content_jobs`; implement a cleanup cron using the existing `cron.ts` service. | | Remotion bundle step | Bundle compositions once at server startup (or on demand). Never bundle on each render request — it takes 30-60s. | ## Build Order (Phase Dependencies) Dependencies flow from infrastructure upward to content types upward to UI. ``` Phase A: Core Infrastructure (unblocks everything) - Add content_jobs schema + migration (db package) - Extend LIVE_EVENT_TYPES with content.job.* (shared package) - Implement contentJobService (server) - Implement renderPipelineService stub (server) - Add API routes + app.ts mounts (server) Phase B: Fast Content Types (no heavy binary deps; validates pipeline end-to-end) - svgGeneratorAdapter (pure TypeScript; icons, placeholders) - mermaidRendererAdapter (@mermaid-js/mermaid-isomorphic; no Chromium) - themeEngineService (pure computation) - UI: DiagramRenderer, ThemePreview, ContentJobViewer Phase C: Client-Side Mermaid + Content Gallery - ChatMarkdownMessage extension (detect mermaid fence) - DiagramRenderer client-side component - ContentGallery page + assetService.list() - GeneratedAssetCard Phase D: Document Generation (introduces Puppeteer) - Add puppeteer to server deps - pdfRendererAdapter - PDF download flow in UI Phase E: Video / Presentations (introduces Remotion) - packages/remotion-compositions/ workspace package - remotionRendererAdapter (CLI subprocess) - Video playback in UI Phase F: Image Generation - imageProcessorAdapter using Sharp (banners, OG images, social cards) - imageGenerationAdapter interface (Stable Diffusion / cloud APIs — future) - Social media content generation Phase G: Content as Skills (no code, pure skill markdown) - Skill markdown files for each content type in company_skills - Agent-callable via existing skill system ``` ## Anti-Patterns ### Anti-Pattern 1: Rendering inside chatService **What people do:** Add Mermaid rendering to `chatService` or `documentService` because content requests arrive from chat. **Why it's wrong:** Couples unrelated concerns. Future content types (video, PDF) would bloat chatService and block upstream rebases. **Do this instead:** `chatService` calls `contentJobService.enqueue()`. Rendering is entirely separate. Chat is a trigger, not an owner. ### Anti-Pattern 2: Synchronous HTTP response for long renders **What people do:** `POST /render/remotion` holds the connection open for 2+ minutes while rendering. **Why it's wrong:** HTTP timeout (30s default on most proxies). No progress feedback. Retry hell. **Do this instead:** Return a `contentJobId` immediately with `202 Accepted`. Client subscribes to SSE `content.job.done` event. ### Anti-Pattern 3: One DB table per content type **What people do:** Add separate `diagrams`, `presentations`, `themes` tables. **Why it's wrong:** The existing `assets` table already handles typed binary blobs. The `content_jobs` table handles any render job regardless of output type. Fragmented schema multiplies migration surface. **Do this instead:** Use `content_jobs.type` to discriminate job types. Use `assets.content_type` to discriminate output format. One jobs table, one assets table. ### Anti-Pattern 4: Bypassing StorageService for renderer output **What people do:** Remotion adapter writes to `/tmp` and returns a filesystem path. **Why it's wrong:** Bypasses the provider abstraction (local disk vs S3), deduplication (sha256), the `assets` table, and download URL generation. **Do this instead:** Renderer writes output to a `Buffer`, passes to `StorageService.putFile()`, returns `objectKey`. Asset serving goes through the existing `/api` asset download route. ### Anti-Pattern 5: Modifying upstream DB tables **What people do:** Add a `generated_content_type` column to the existing `assets` table. **Why it's wrong:** Modifies upstream schema — migration conflict on next `git rebase upstream/master`. Violates the display-only fork constraint. **Do this instead:** Use the `content_jobs.asset_id` FK as the signal that an asset is generated. Query `content_jobs JOIN assets` to distinguish generated from uploaded. Keep `assets` table untouched. ### Anti-Pattern 6: Remotion bundle on every render request **What people do:** Call `bundle()` inside the render adapter on each job. **Why it's wrong:** Bundling takes 30-60s. Renders that should take 5s take 90s. **Do this instead:** Bundle once at server startup (or lazily on first render, cached). The `remotionRendererAdapter` calls `renderMedia()` against the pre-built bundle path. ## Sources - Direct inspection of `/opt/nexus` codebase (2026-04-04): - `server/src/services/` — factory function service patterns - `server/src/storage/` — `StorageService` / `StorageProvider` interfaces - `server/src/storage/service.ts` — `buildObjectKey()` namespace + path conventions - `server/src/services/live-events.ts` — SSE event bus (`publishLiveEvent`, `subscribeCompanyLiveEvents`) - `server/src/services/voice-pipeline.ts` — async subprocess service pattern - `server/src/services/placeholder-service.ts` — existing `PlaceholderEntry` manifest service - `server/src/services/assets.ts` — `assetService` factory (minimal; extend for listing) - `server/src/services/work-products.ts` — job/output separation pattern - `packages/db/src/schema/assets.ts` — existing `assets` table - `packages/db/src/schema/documents.ts` — document + revision pattern - `packages/shared/src/constants.ts` — `LIVE_EVENT_TYPES` (currently 9 types) - `server/src/app.ts` — route mounting conventions - `server/src/routes/voice.ts` — SSE streaming response pattern - `ui/package.json` — confirms `mermaid ^11.12.0` already installed - `server/package.json` — confirms `sharp`, `ffmpeg-static` already installed - Mermaid v11 isomorphic: https://mermaid.js.org/config/usage.html - Remotion CLI rendering: https://www.remotion.dev/docs/cli/render - `@mermaid-js/mermaid-isomorphic` for server-side rendering without a browser --- *Architecture research for: Nexus v1.7 Content Generation* *Researched: 2026-04-04*