From d72c065fc7b5f99cf2ffa23e0a82ebb638397d9a Mon Sep 17 00:00:00 2001 From: Nexus Dev Date: Wed, 1 Apr 2026 22:55:54 +0000 Subject: [PATCH] =?UTF-8?q?docs(25-file-system):=20create=20phase=20plan?= =?UTF-8?q?=20=E2=80=94=204=20plans=20in=203=20waves?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Plans cover FILE-01 through FILE-06: DB schema + shared types (wave 1), server file service + routes and UI file upload (wave 2, parallel), file preview components + full wiring (wave 3). Co-Authored-By: Claude Opus 4.6 (1M context) --- .planning/ROADMAP.md | 13 +- .planning/phases/25-file-system/25-00-PLAN.md | 311 +++++++++++++ .planning/phases/25-file-system/25-01-PLAN.md | 297 +++++++++++++ .planning/phases/25-file-system/25-02-PLAN.md | 420 ++++++++++++++++++ .planning/phases/25-file-system/25-03-PLAN.md | 326 ++++++++++++++ .../phases/25-file-system/25-RESEARCH.md | 76 ++++ 6 files changed, 1440 insertions(+), 3 deletions(-) create mode 100644 .planning/phases/25-file-system/25-00-PLAN.md create mode 100644 .planning/phases/25-file-system/25-01-PLAN.md create mode 100644 .planning/phases/25-file-system/25-02-PLAN.md create mode 100644 .planning/phases/25-file-system/25-03-PLAN.md create mode 100644 .planning/phases/25-file-system/25-RESEARCH.md diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 87cfff0c..ba68fe6d 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -73,7 +73,7 @@ Plans: **Success Criteria** (what must be TRUE): 1. The Brainstormer is the default agent when a user opens a new conversation; it greets the user and begins a structured questioning flow 2. After the user answers clarifying questions, the Brainstormer produces a formatted spec card with What / Why / Constraints / Success fields and action buttons (Send to PM, Edit, Save as Draft) - 3. When the user clicks "Send to PM," a handoff indicator appears in the chat showing "Brainstormer → PM" with the spec content + 3. When the user clicks "Send to PM," a handoff indicator appears in the chat showing "Brainstormer -> PM" with the spec content 4. The PM agent creates one or more Nexus issues from the spec; the user can see task IDs referenced in the PM's reply 5. When an Engineer or Generalist completes a task, a status update message appears in the relevant chat conversation **Plans:** 4/4 plans complete @@ -116,7 +116,14 @@ Plans: 5. When an agent generates a placeholder asset, `PLACEHOLDERS.md` is updated in the project directory; when the placeholder is replaced, the DB records the replacement chain and the manifest reflects the change 6. A file uploaded in a conversation linked to a project lives in `files/projects//`; a file from an unlinked conversation lives in `files/chat//`; the user can promote a chat file to project scope 7. Voice input is available when local AI is enabled: user can hold the record button, speak, see a transcription preview, and confirm to send -**Plans:** [To be planned] +**Plans:** 4 plans + +Plans: +- [ ] 25-00-PLAN.md — DB schema (chat_files + chat_file_references), shared types/validators, test stubs +- [ ] 25-01-PLAN.md — Server: chatFileService + chatFileRoutes (upload, download, list, references) +- [ ] 25-02-PLAN.md — UI: ChatInput file upload (drag-drop, paste, file picker), useChatFileUpload hook +- [ ] 25-03-PLAN.md — UI: ChatFilePreview/ChatFileCard components, ChatMessage/ChatPanel wiring + **UI hint**: yes ### Phase 26: PWA & Performance @@ -215,5 +222,5 @@ All 65 v1 requirements are mapped to exactly one phase. No orphans. | 22. Agent Streaming | v1.3 | 6/6 | Complete | 2026-04-01 | | 23. Brainstormer Flow | v1.3 | 4/4 | Complete | 2026-04-01 | | 24. Search, History & Branching | v1.3 | 4/4 | Complete | 2026-04-01 | -| 25. File System | v1.3 | 0/? | Not started | - | +| 25. File System | v1.3 | 0/4 | Planned | - | | 26. PWA & Performance | v1.3 | 0/? | Not started | - | diff --git a/.planning/phases/25-file-system/25-00-PLAN.md b/.planning/phases/25-file-system/25-00-PLAN.md new file mode 100644 index 00000000..5dc16488 --- /dev/null +++ b/.planning/phases/25-file-system/25-00-PLAN.md @@ -0,0 +1,311 @@ +--- +phase: 25-file-system +plan: 00 +type: execute +wave: 1 +depends_on: [] +files_modified: + - packages/db/src/schema/chat_files.ts + - packages/db/src/schema/chat_file_references.ts + - packages/db/src/schema/index.ts + - packages/db/src/migrations/0053_create_chat_files.sql + - packages/db/src/migrations/0054_create_chat_file_references.sql + - packages/shared/src/types/chat.ts + - packages/shared/src/validators/chat.ts + - packages/shared/src/index.ts + - server/src/__tests__/chat-file-service.test.ts + - server/src/__tests__/chat-file-routes.test.ts +autonomous: true +requirements: + - FILE-01 + - FILE-02 + - FILE-03 +must_haves: + truths: + - "chat_files table exists in Postgres with all required metadata columns" + - "chat_file_references table exists enabling cross-conversation file references" + - "Shared types and validators for file operations are exported from @paperclipai/shared" + artifacts: + - path: "packages/db/src/schema/chat_files.ts" + provides: "chat_files Drizzle schema table" + contains: "pgTable" + - path: "packages/db/src/schema/chat_file_references.ts" + provides: "chat_file_references Drizzle schema table" + contains: "pgTable" + - path: "packages/shared/src/types/chat.ts" + provides: "ChatFile and ChatFileReference types" + contains: "ChatFile" + key_links: + - from: "packages/db/src/schema/chat_files.ts" + to: "packages/db/src/schema/chat_conversations.ts" + via: "FK reference conversationId" + pattern: "chatConversations" + - from: "packages/db/src/schema/chat_file_references.ts" + to: "packages/db/src/schema/chat_files.ts" + via: "FK reference fileId" + pattern: "chatFiles" +--- + + +Create the database schema, shared types, validators, and test stubs for the chat file system. + +Purpose: Establish the data layer contracts that all subsequent plans (upload routes, UI previews) build against. +Output: Two new DB tables (chat_files, chat_file_references), shared TypeScript types, Zod validators, and test scaffolds. + + + +@$HOME/.claude/get-shit-done/workflows/execute-plan.md +@$HOME/.claude/get-shit-done/templates/summary.md + + + +@.planning/PROJECT.md +@.planning/ROADMAP.md +@.planning/STATE.md +@.planning/phases/25-file-system/25-RESEARCH.md + +@packages/db/src/schema/chat_messages.ts +@packages/db/src/schema/chat_conversations.ts +@packages/db/src/schema/assets.ts +@packages/db/src/schema/index.ts +@packages/shared/src/types/chat.ts +@packages/shared/src/validators/chat.ts +@server/src/__tests__/chat-routes.test.ts + + + + + + Task 1: Create chat_files and chat_file_references DB schema + migrations + + packages/db/src/schema/chat_files.ts, + packages/db/src/schema/chat_file_references.ts, + packages/db/src/schema/index.ts, + packages/db/src/migrations/0053_create_chat_files.sql, + packages/db/src/migrations/0054_create_chat_file_references.sql + + + packages/db/src/schema/chat_messages.ts, + packages/db/src/schema/chat_conversations.ts, + packages/db/src/schema/assets.ts, + packages/db/src/schema/index.ts, + packages/db/src/migrations/meta/_journal.json + + + Create `packages/db/src/schema/chat_files.ts` with a `chatFiles` pgTable containing: + - id: uuid PK defaultRandom + - companyId: uuid NOT NULL FK to companies.id + - conversationId: uuid FK to chatConversations.id (nullable — file may exist without conversation) + - messageId: uuid FK to chatMessages.id ON DELETE SET NULL (nullable — file may be uploaded before message is sent) + - filename: text NOT NULL (display filename, may differ from original) + - originalFilename: text NOT NULL (as uploaded) + - mimeType: text NOT NULL + - sizeBytes: integer NOT NULL + - objectKey: text NOT NULL (StorageService object key) + - sha256: text NOT NULL + - source: text NOT NULL (values: 'user_upload', 'agent_generated') + - category: text (values: 'image', 'document', 'code', 'other' — nullable, derived from mimeType) + - projectId: uuid FK to projects.id ON DELETE SET NULL (nullable — for dual scoping FILE-04) + - createdAt: timestamp with timezone NOT NULL defaultNow + - updatedAt: timestamp with timezone NOT NULL defaultNow + + Indexes (use object-syntax callback like existing chat schemas): + - chat_files_conversation_idx on (conversationId) + - chat_files_message_idx on (messageId) + - chat_files_company_created_idx on (companyId, createdAt) + - chat_files_project_idx on (projectId) + + Create `packages/db/src/schema/chat_file_references.ts` with a `chatFileReferences` pgTable: + - id: uuid PK defaultRandom + - fileId: uuid NOT NULL FK to chatFiles.id ON DELETE CASCADE + - conversationId: uuid NOT NULL FK to chatConversations.id ON DELETE CASCADE + - messageId: uuid FK to chatMessages.id ON DELETE SET NULL (nullable) + - createdAt: timestamp with timezone NOT NULL defaultNow + + Indexes: + - chat_file_refs_file_idx on (fileId) + - chat_file_refs_conversation_idx on (conversationId) + - chat_file_refs_message_idx on (messageId) + + Add exports to `packages/db/src/schema/index.ts`: + - export { chatFiles } from "./chat_files.js"; + - export { chatFileReferences } from "./chat_file_references.js"; + + Create migration SQL files: + - 0053_create_chat_files.sql: CREATE TABLE chat_files with all columns and indexes + - 0054_create_chat_file_references.sql: CREATE TABLE chat_file_references with all columns and indexes + + Update `packages/db/src/migrations/meta/_journal.json` to add entries for idx 53 and 54. + Copy the latest snapshot JSON file as a template for the new snapshots (0053_snapshot.json, 0054_snapshot.json) — but note that the migration system only strictly requires the SQL files and journal entries; skip snapshot creation if too complex. + + IMPORTANT: Follow existing index callback pattern from chat_messages.ts — use `(table) => ({})` object syntax, not array syntax. + + + cd /opt/nexus && grep -q "chatFiles" packages/db/src/schema/index.ts && grep -q "chatFileReferences" packages/db/src/schema/index.ts && grep -q "chat_files" packages/db/src/migrations/0053_create_chat_files.sql && grep -q "chat_file_references" packages/db/src/migrations/0054_create_chat_file_references.sql && echo "PASS" + + + - grep "chatFiles" packages/db/src/schema/index.ts returns a match + - grep "chatFileReferences" packages/db/src/schema/index.ts returns a match + - grep "pgTable" packages/db/src/schema/chat_files.ts returns a match + - grep "pgTable" packages/db/src/schema/chat_file_references.ts returns a match + - grep "CREATE TABLE" packages/db/src/migrations/0053_create_chat_files.sql returns a match + - grep "CREATE TABLE" packages/db/src/migrations/0054_create_chat_file_references.sql returns a match + - grep "company_id" packages/db/src/schema/chat_files.ts returns a match (dual scoping support) + - grep "project_id" packages/db/src/schema/chat_files.ts returns a match (FILE-04 support) + - grep "source" packages/db/src/schema/chat_files.ts returns a match + + + chat_files and chat_file_references Drizzle schemas exist with all columns, indexes, FKs. + Migration SQL files exist. Schema index re-exports both tables. + + + + + Task 2: Add shared types, validators, and test stubs + + packages/shared/src/types/chat.ts, + packages/shared/src/validators/chat.ts, + packages/shared/src/index.ts, + server/src/__tests__/chat-file-service.test.ts, + server/src/__tests__/chat-file-routes.test.ts + + + packages/shared/src/types/chat.ts, + packages/shared/src/validators/chat.ts, + packages/shared/src/index.ts, + server/src/__tests__/chat-routes.test.ts + + + Extend `packages/shared/src/types/chat.ts` — add these interfaces (append, do NOT modify existing interfaces): + + ```typescript + export interface ChatFile { + id: string; + companyId: string; + conversationId: string | null; + messageId: string | null; + filename: string; + originalFilename: string; + mimeType: string; + sizeBytes: number; + objectKey: string; + sha256: string; + source: "user_upload" | "agent_generated"; + category: "image" | "document" | "code" | "other" | null; + projectId: string | null; + createdAt: string; + updatedAt: string; + } + + export interface ChatFileReference { + id: string; + fileId: string; + conversationId: string; + messageId: string | null; + createdAt: string; + } + + export interface ChatFileUploadResponse { + file: ChatFile; + contentPath: string; + } + + export interface ChatFileListResponse { + items: ChatFile[]; + } + ``` + + Also extend `ChatMessage` by adding an optional `files` field: + ```typescript + // In ChatMessage interface, add: + files?: ChatFile[]; + ``` + + Extend `packages/shared/src/validators/chat.ts` — add Zod schemas: + ```typescript + export const uploadChatFileSchema = z.object({ + conversationId: z.string().uuid().optional(), + messageId: z.string().uuid().optional(), + source: z.enum(["user_upload", "agent_generated"]).default("user_upload"), + projectId: z.string().uuid().optional(), + }); + + export const createFileReferenceSchema = z.object({ + fileId: z.string().uuid(), + messageId: z.string().uuid().optional(), + }); + ``` + + Ensure new schemas and types are re-exported from `packages/shared/src/index.ts` — check existing export pattern in that file and add: + - The new type exports (ChatFile, ChatFileReference, ChatFileUploadResponse, ChatFileListResponse) + - The new validator exports (uploadChatFileSchema, createFileReferenceSchema) + + Create test stubs: + + `server/src/__tests__/chat-file-service.test.ts`: + ```typescript + import { describe, it } from "vitest"; + + describe("chatFileService", () => { + it.todo("creates a file record after upload"); + it.todo("lists files for a conversation"); + it.todo("lists files for a message"); + it.todo("creates a file reference in another conversation"); + it.todo("returns file with contentPath"); + }); + ``` + + `server/src/__tests__/chat-file-routes.test.ts`: + ```typescript + import { describe, it } from "vitest"; + + describe("chatFileRoutes", () => { + it.todo("POST /conversations/:id/files uploads a file and returns 201"); + it.todo("GET /conversations/:id/files lists files for conversation"); + it.todo("GET /files/:fileId/content serves file content"); + it.todo("POST /files/:fileId/references creates a cross-conversation reference"); + it.todo("rejects upload when file exceeds size limit"); + it.todo("rejects upload when content type is not allowed"); + }); + ``` + + + cd /opt/nexus && npx tsc --noEmit -p packages/shared/tsconfig.json 2>&1 | tail -5 && grep -q "ChatFile" packages/shared/src/types/chat.ts && grep -q "uploadChatFileSchema" packages/shared/src/validators/chat.ts && echo "PASS" + + + - grep "ChatFile" packages/shared/src/types/chat.ts returns a match + - grep "ChatFileReference" packages/shared/src/types/chat.ts returns a match + - grep "ChatFileUploadResponse" packages/shared/src/types/chat.ts returns a match + - grep "files?" packages/shared/src/types/chat.ts returns a match (optional files on ChatMessage) + - grep "uploadChatFileSchema" packages/shared/src/validators/chat.ts returns a match + - grep "createFileReferenceSchema" packages/shared/src/validators/chat.ts returns a match + - grep "it.todo" server/src/__tests__/chat-file-service.test.ts returns matches + - grep "it.todo" server/src/__tests__/chat-file-routes.test.ts returns matches + - TypeScript compilation of shared package succeeds + + + ChatFile and related types exported from shared. Zod validators for file upload and reference creation exported. + ChatMessage type has optional files array. Test stubs exist for service and routes. + + + + + + +- `packages/db/src/schema/chat_files.ts` exists with pgTable definition +- `packages/db/src/schema/chat_file_references.ts` exists with pgTable definition +- Both tables exported from schema index +- SQL migration files exist for both tables +- `ChatFile` type exported from shared +- `uploadChatFileSchema` validator exported from shared +- Test stubs have `.todo()` tests for service and routes +- TypeScript compiles without errors in shared package + + + +Database schema for file tracking is defined with all columns needed for FILE-01 (directory/storage), FILE-02 (metadata), FILE-03 (cross-references), and FILE-04 (dual scoping via projectId). Shared types provide the contract for Plans 01-03. + + + +After completion, create `.planning/phases/25-file-system/25-00-SUMMARY.md` + diff --git a/.planning/phases/25-file-system/25-01-PLAN.md b/.planning/phases/25-file-system/25-01-PLAN.md new file mode 100644 index 00000000..a56102c7 --- /dev/null +++ b/.planning/phases/25-file-system/25-01-PLAN.md @@ -0,0 +1,297 @@ +--- +phase: 25-file-system +plan: 01 +type: execute +wave: 2 +depends_on: ["25-00"] +files_modified: + - server/src/services/chat-files.ts + - server/src/routes/chat-files.ts + - server/src/routes/index.ts + - server/src/app.ts + - server/src/__tests__/chat-file-service.test.ts + - server/src/__tests__/chat-file-routes.test.ts +autonomous: true +requirements: + - FILE-04 + - FILE-05 +must_haves: + truths: + - "User can upload a file via POST /api/conversations/:id/files and get back a ChatFile with contentPath" + - "Uploaded file is stored on disk via StorageService and metadata persisted in chat_files table" + - "File content is served via GET /api/files/:fileId/content with correct MIME type" + - "Files for a conversation are listed via GET /api/conversations/:id/files" + - "Cross-conversation file reference is created via POST /api/files/:fileId/references" + artifacts: + - path: "server/src/services/chat-files.ts" + provides: "chatFileService with CRUD + reference operations" + exports: ["chatFileService"] + - path: "server/src/routes/chat-files.ts" + provides: "Express router for file upload, list, download, reference" + exports: ["chatFileRoutes"] + key_links: + - from: "server/src/routes/chat-files.ts" + to: "server/src/services/chat-files.ts" + via: "chatFileService(db) call" + pattern: "chatFileService" + - from: "server/src/routes/chat-files.ts" + to: "server/src/storage/types.ts" + via: "StorageService for putFile/getObject" + pattern: "storage\\.putFile" + - from: "server/src/app.ts" + to: "server/src/routes/chat-files.ts" + via: "api.use(chatFileRoutes(db, storageService))" + pattern: "chatFileRoutes" +--- + + +Implement the server-side file service and REST routes for uploading, listing, downloading, and referencing chat files. + +Purpose: Provide the backend that the UI (Plans 02+03) will call for file operations. +Output: chatFileService with DB operations, chatFileRoutes with Express endpoints, wired into app.ts. + + + +@$HOME/.claude/get-shit-done/workflows/execute-plan.md +@$HOME/.claude/get-shit-done/templates/summary.md + + + +@.planning/PROJECT.md +@.planning/ROADMAP.md +@.planning/phases/25-file-system/25-RESEARCH.md +@.planning/phases/25-file-system/25-00-SUMMARY.md + +@server/src/routes/assets.ts +@server/src/routes/chat.ts +@server/src/services/chat.ts +@server/src/services/assets.ts +@server/src/storage/service.ts +@server/src/storage/types.ts +@server/src/attachment-types.ts +@server/src/app.ts +@server/src/__tests__/chat-routes.test.ts +@packages/db/src/schema/chat_files.ts +@packages/db/src/schema/chat_file_references.ts +@packages/shared/src/types/chat.ts +@packages/shared/src/validators/chat.ts + + + + + +From packages/db/src/schema/chat_files.ts (created in Plan 00): +```typescript +export const chatFiles = pgTable("chat_files", { + id: uuid("id").primaryKey().defaultRandom(), + companyId: uuid("company_id").notNull(), + conversationId: uuid("conversation_id"), + messageId: uuid("message_id"), + filename: text("filename").notNull(), + originalFilename: text("original_filename").notNull(), + mimeType: text("mime_type").notNull(), + sizeBytes: integer("size_bytes").notNull(), + objectKey: text("object_key").notNull(), + sha256: text("sha256").notNull(), + source: text("source").notNull(), + category: text("category"), + projectId: uuid("project_id"), + createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(), + updatedAt: timestamp("updated_at", { withTimezone: true }).notNull().defaultNow(), +}); +``` + +From packages/shared/src/validators/chat.ts (created in Plan 00): +```typescript +export const uploadChatFileSchema = z.object({ + conversationId: z.string().uuid().optional(), + messageId: z.string().uuid().optional(), + source: z.enum(["user_upload", "agent_generated"]).default("user_upload"), + projectId: z.string().uuid().optional(), +}); +export const createFileReferenceSchema = z.object({ + fileId: z.string().uuid(), + messageId: z.string().uuid().optional(), +}); +``` + +From server/src/storage/types.ts: +```typescript +export interface StorageService { + provider: StorageProviderId; + putFile(input: PutFileInput): Promise; + getObject(companyId: string, objectKey: string): Promise; +} +``` + + + + + + Task 1: Create chatFileService with DB operations + + server/src/services/chat-files.ts, + server/src/__tests__/chat-file-service.test.ts + + + server/src/services/assets.ts, + server/src/services/chat.ts, + packages/db/src/schema/chat_files.ts, + packages/db/src/schema/chat_file_references.ts, + server/src/__tests__/chat-routes.test.ts + + + Create `server/src/services/chat-files.ts` exporting a `chatFileService(db: Db)` function that returns an object with these methods: + + 1. **create(companyId, data)** — Insert a row into chat_files. `data` includes: conversationId, messageId, filename, originalFilename, mimeType, sizeBytes, objectKey, sha256, source, category, projectId. Returns the inserted row. + + 2. **getById(id)** — Select a chat_files row by id. Returns row or null. + + 3. **listByConversation(conversationId, opts?)** — Select chat_files where conversationId matches, ordered by createdAt desc. Support optional limit (default 50). + + 4. **listByMessage(messageId)** — Select chat_files where messageId matches, ordered by createdAt asc. + + 5. **createReference(data)** — Insert a row into chat_file_references with fileId, conversationId, messageId. Returns the inserted row. + + 6. **listReferences(fileId)** — Select chat_file_references where fileId matches. + + 7. **attachToMessage(fileId, messageId)** — Update chat_files set messageId where id = fileId. Returns updated row. + + Helper: `deriveCategory(mimeType: string): string` — returns "image" for image/*, "code" for text/javascript, text/typescript, text/css, text/html, application/json, text/x-python, text/x-java, etc., "document" for application/pdf, text/plain, text/markdown, text/csv, "other" for everything else. + + Use Drizzle `eq`, `desc` from drizzle-orm. Import `chatFiles`, `chatFileReferences` from `@paperclipai/db`. + + Update `server/src/__tests__/chat-file-service.test.ts` — replace todo stubs with real tests using vi.mock for the db. Follow the same mock pattern as chat-routes.test.ts: + - Mock the db select/insert/update calls + - Test create returns inserted row + - Test getById returns null when not found + - Test deriveCategory for image, code, document, other mime types + + + cd /opt/nexus && npx vitest run server/src/__tests__/chat-file-service.test.ts --reporter=verbose 2>&1 | tail -20 + + + - grep "chatFileService" server/src/services/chat-files.ts returns a match + - grep "deriveCategory" server/src/services/chat-files.ts returns a match + - grep "chatFiles" server/src/services/chat-files.ts returns a match + - grep "chatFileReferences" server/src/services/chat-files.ts returns a match + - grep "attachToMessage" server/src/services/chat-files.ts returns a match + - vitest chat-file-service tests pass + + + chatFileService handles all DB operations for file CRUD, references, and message attachment. + Tests verify service methods work correctly with mocked DB. + + + + + Task 2: Create chatFileRoutes and wire into app.ts + + server/src/routes/chat-files.ts, + server/src/routes/index.ts, + server/src/app.ts, + server/src/__tests__/chat-file-routes.test.ts + + + server/src/routes/assets.ts, + server/src/routes/chat.ts, + server/src/app.ts, + server/src/routes/index.ts, + server/src/attachment-types.ts, + server/src/services/chat-files.ts + + + Create `server/src/routes/chat-files.ts` exporting `chatFileRoutes(db: Db, storage: StorageService): Router`: + + Use multer v2 memory storage (same pattern as assets.ts): + ```typescript + const fileUpload = multer({ + storage: multer.memoryStorage(), + limits: { fileSize: MAX_ATTACHMENT_BYTES, files: 1 }, + }); + ``` + + Endpoints: + + 1. **POST /conversations/:id/files** — Upload a file to a conversation + - Use the same `runSingleFileUpload` pattern from assets.ts + - Parse body metadata with `uploadChatFileSchema` (from req.body after multer) + - Validate content type with `isAllowedContentType` + - Call `storage.putFile({ companyId, namespace: "chat-files", originalFilename, contentType, body })` + - Resolve companyId by calling chatService(db).getConversation(conversationId) + - Call `chatFileService.create(companyId, { ...metadata, objectKey, sha256, sizeBytes, category: deriveCategory(mimeType) })` + - Return 201 with `{ file: , contentPath: "/api/files/${file.id}/content" }` + + 2. **GET /conversations/:id/files** — List files for a conversation + - assertBoard(req) + - Call `chatFileService.listByConversation(conversationId)` + - Return `{ items: files }` + + 3. **GET /files/:fileId/content** — Serve file content (download/preview) + - assertBoard(req) + - Get file by id, assert company access + - Pipe `storage.getObject(file.companyId, file.objectKey).stream` to response + - Set Content-Type, Content-Length, Content-Disposition (inline for images, attachment for others), Cache-Control, X-Content-Type-Options: nosniff + + 4. **POST /files/:fileId/references** — Create cross-conversation reference + - assertBoard(req) + - Parse body with createFileReferenceSchema + - Get file by id, assert company access + - Create reference row + - Return 201 with reference + + 5. **PATCH /files/:fileId** — Attach file to a message (set messageId) + - assertBoard(req) + - Parse { messageId } from body + - Call chatFileService.attachToMessage(fileId, messageId) + - Return updated file + + Wire into app: + - Add `export { chatFileRoutes } from "./chat-files.js";` to `server/src/routes/index.ts` + - In `server/src/app.ts`, import `chatFileRoutes` and add `api.use(chatFileRoutes(db, opts.storageService));` right after the `chatRoutes(db)` line (around line 160) + + Update `server/src/__tests__/chat-file-routes.test.ts` — replace todo stubs with tests: + - Mock chatFileService and storage + - Test POST /conversations/:id/files returns 201 with file + - Test GET /files/:fileId/content returns streamed content + - Test 400 on missing file field + - Test 422 on unsupported content type + - Follow the same createApp() pattern from chat-routes.test.ts but pass mock storage + + + cd /opt/nexus && npx vitest run server/src/__tests__/chat-file-routes.test.ts --reporter=verbose 2>&1 | tail -20 + + + - grep "chatFileRoutes" server/src/routes/chat-files.ts returns a match + - grep "chatFileRoutes" server/src/routes/index.ts returns a match + - grep "chatFileRoutes" server/src/app.ts returns a match + - grep "POST.*files" server/src/routes/chat-files.ts returns a match + - grep "content" server/src/routes/chat-files.ts returns a match (content download route) + - grep "multer" server/src/routes/chat-files.ts returns a match + - grep "storage.putFile" server/src/routes/chat-files.ts returns a match + - vitest chat-file-routes tests pass + + + File upload, list, download, and reference endpoints work. Routes are wired into Express app. + Tests verify upload returns 201, content is streamed, validation rejects bad input. + + + + + + +- `POST /api/conversations/:id/files` with multipart form data stores file and creates DB record +- `GET /api/conversations/:id/files` returns list of files for conversation +- `GET /api/files/:fileId/content` streams file content with correct MIME type +- `POST /api/files/:fileId/references` creates cross-conversation reference +- Routes are mounted in app.ts +- All tests pass + + + +Complete server-side file system: upload via multipart, download via streaming, list per conversation, cross-conversation references. FILE-04 (dual scoping via projectId) and FILE-05 (upload from chat) backend is ready for the UI to consume. + + + +After completion, create `.planning/phases/25-file-system/25-01-SUMMARY.md` + diff --git a/.planning/phases/25-file-system/25-02-PLAN.md b/.planning/phases/25-file-system/25-02-PLAN.md new file mode 100644 index 00000000..00e1f771 --- /dev/null +++ b/.planning/phases/25-file-system/25-02-PLAN.md @@ -0,0 +1,420 @@ +--- +phase: 25-file-system +plan: 02 +type: execute +wave: 2 +depends_on: ["25-00"] +files_modified: + - ui/src/api/chat.ts + - ui/src/components/ChatInput.tsx + - ui/src/components/ChatFileDropZone.tsx + - ui/src/hooks/useChatFileUpload.ts + - ui/src/components/ChatInput.test.tsx +autonomous: true +requirements: + - FILE-05 +must_haves: + truths: + - "User can drag-and-drop a file onto the chat input area and it uploads" + - "User can paste an image from clipboard into the chat input and it uploads" + - "User can click a button to select a file for upload" + - "Pending file uploads show as preview chips in the input area before sending" + - "Upload progress is visible while file is uploading" + artifacts: + - path: "ui/src/components/ChatFileDropZone.tsx" + provides: "Drop zone overlay and drag state management" + contains: "ChatFileDropZone" + - path: "ui/src/hooks/useChatFileUpload.ts" + provides: "Upload state, progress, and API calls" + exports: ["useChatFileUpload"] + - path: "ui/src/api/chat.ts" + provides: "uploadFile method on chatApi" + contains: "uploadFile" + key_links: + - from: "ui/src/components/ChatInput.tsx" + to: "ui/src/hooks/useChatFileUpload.ts" + via: "useChatFileUpload hook" + pattern: "useChatFileUpload" + - from: "ui/src/hooks/useChatFileUpload.ts" + to: "ui/src/api/chat.ts" + via: "chatApi.uploadFile" + pattern: "chatApi\\.uploadFile" + - from: "ui/src/components/ChatInput.tsx" + to: "ui/src/components/ChatFileDropZone.tsx" + via: "wrapping textarea in drop zone" + pattern: "ChatFileDropZone" +--- + + +Add file upload capabilities to ChatInput: drag-and-drop, clipboard paste, and file picker button. Files upload immediately and appear as pending chips in the input area. + +Purpose: Enable users to attach files to chat messages (FILE-05, INPUT-02, INPUT-03). +Output: ChatFileDropZone component, useChatFileUpload hook, extended chatApi, updated ChatInput. + + + +@$HOME/.claude/get-shit-done/workflows/execute-plan.md +@$HOME/.claude/get-shit-done/templates/summary.md + + + +@.planning/PROJECT.md +@.planning/ROADMAP.md +@.planning/phases/25-file-system/25-RESEARCH.md +@.planning/phases/25-file-system/25-00-SUMMARY.md + +@ui/src/components/ChatInput.tsx +@ui/src/api/chat.ts +@ui/src/components/ChatPanel.tsx +@packages/shared/src/types/chat.ts + + + + +POST /api/conversations/:id/files — multipart/form-data with field "file" + optional body fields +Response: { file: ChatFile, contentPath: string } + +GET /api/files/:fileId/content — streams file binary + + +```typescript +export interface ChatFile { + id: string; + filename: string; + originalFilename: string; + mimeType: string; + sizeBytes: number; + source: "user_upload" | "agent_generated"; + category: "image" | "document" | "code" | "other" | null; + createdAt: string; +} + +export interface ChatFileUploadResponse { + file: ChatFile; + contentPath: string; +} +``` + + +```typescript +interface ChatInputProps { + onSend: (content: string) => void; + isSubmitting?: boolean; + disabled?: boolean; + placeholder?: string; + agents?: Agent[]; + agentsLoading?: boolean; +} +``` + + + + + + Task 1: Add chatApi.uploadFile and create useChatFileUpload hook + + ui/src/api/chat.ts, + ui/src/hooks/useChatFileUpload.ts + + + ui/src/api/chat.ts, + ui/src/hooks/useStreamingChat.ts, + packages/shared/src/types/chat.ts + + + Extend `ui/src/api/chat.ts` — add an `uploadFile` method to the `chatApi` object: + + ```typescript + async uploadFile( + conversationId: string, + file: File, + opts?: { source?: string; projectId?: string }, + onProgress?: (percent: number) => void, + ): Promise { + const formData = new FormData(); + formData.append("file", file); + if (opts?.source) formData.append("source", opts.source); + if (opts?.projectId) formData.append("projectId", opts.projectId); + + // Use XMLHttpRequest for progress tracking + return new Promise((resolve, reject) => { + const xhr = new XMLHttpRequest(); + xhr.open("POST", `/api/conversations/${conversationId}/files`); + xhr.withCredentials = true; + + xhr.upload.onprogress = (e) => { + if (e.lengthComputable && onProgress) { + onProgress(Math.round((e.loaded / e.total) * 100)); + } + }; + + xhr.onload = () => { + if (xhr.status >= 200 && xhr.status < 300) { + resolve(JSON.parse(xhr.responseText)); + } else { + reject(new Error(`Upload failed: ${xhr.status}`)); + } + }; + + xhr.onerror = () => reject(new Error("Upload failed")); + xhr.send(formData); + }); + }, + ``` + + Also add `ChatFileUploadResponse` to the import from `@paperclipai/shared` at the top of the file. + + Create `ui/src/hooks/useChatFileUpload.ts`: + + ```typescript + import { useState, useCallback } from "react"; + import { chatApi } from "../api/chat"; + import type { ChatFile } from "@paperclipai/shared"; + + export interface PendingFile { + id: string; // temp client-side id before upload completes + file: File; // raw File object + name: string; + mimeType: string; + sizeBytes: number; + progress: number; // 0-100 + status: "uploading" | "done" | "error"; + uploadedFile?: ChatFile; // set when upload completes + contentPath?: string; + error?: string; + } + + export function useChatFileUpload(conversationId: string | null) { + const [pendingFiles, setPendingFiles] = useState([]); + + const addFile = useCallback(async (file: File) => { + if (!conversationId) return; + + const tempId = crypto.randomUUID(); + const pending: PendingFile = { + id: tempId, + file, + name: file.name, + mimeType: file.type || "application/octet-stream", + sizeBytes: file.size, + progress: 0, + status: "uploading", + }; + + setPendingFiles((prev) => [...prev, pending]); + + try { + const result = await chatApi.uploadFile( + conversationId, + file, + { source: "user_upload" }, + (percent) => { + setPendingFiles((prev) => + prev.map((p) => (p.id === tempId ? { ...p, progress: percent } : p)) + ); + }, + ); + + setPendingFiles((prev) => + prev.map((p) => + p.id === tempId + ? { ...p, status: "done", progress: 100, uploadedFile: result.file, contentPath: result.contentPath } + : p + ) + ); + } catch (err) { + setPendingFiles((prev) => + prev.map((p) => + p.id === tempId ? { ...p, status: "error", error: (err as Error).message } : p + ) + ); + } + }, [conversationId]); + + const removeFile = useCallback((tempId: string) => { + setPendingFiles((prev) => prev.filter((p) => p.id !== tempId)); + }, []); + + const clearCompleted = useCallback(() => { + setPendingFiles((prev) => prev.filter((p) => p.status !== "done")); + }, []); + + const completedFileIds = pendingFiles + .filter((p) => p.status === "done" && p.uploadedFile) + .map((p) => p.uploadedFile!.id); + + return { pendingFiles, addFile, removeFile, clearCompleted, completedFileIds }; + } + ``` + + + cd /opt/nexus && grep -q "uploadFile" ui/src/api/chat.ts && grep -q "useChatFileUpload" ui/src/hooks/useChatFileUpload.ts && grep -q "PendingFile" ui/src/hooks/useChatFileUpload.ts && echo "PASS" + + + - grep "uploadFile" ui/src/api/chat.ts returns a match + - grep "FormData" ui/src/api/chat.ts returns a match + - grep "XMLHttpRequest" ui/src/api/chat.ts returns a match (for progress) + - grep "useChatFileUpload" ui/src/hooks/useChatFileUpload.ts returns a match + - grep "PendingFile" ui/src/hooks/useChatFileUpload.ts returns a match + - grep "addFile" ui/src/hooks/useChatFileUpload.ts returns a match + - grep "progress" ui/src/hooks/useChatFileUpload.ts returns a match + + + chatApi.uploadFile sends multipart form data with progress tracking via XHR. + useChatFileUpload hook manages pending file state with upload/progress/done/error lifecycle. + + + + + Task 2: Create ChatFileDropZone and integrate into ChatInput + + ui/src/components/ChatFileDropZone.tsx, + ui/src/components/ChatInput.tsx, + ui/src/components/ChatInput.test.tsx + + + ui/src/components/ChatInput.tsx, + ui/src/components/ChatInput.test.tsx, + ui/src/hooks/useChatFileUpload.ts, + ui/src/lib/utils.ts + + + Create `ui/src/components/ChatFileDropZone.tsx`: + + A wrapper component that handles drag-and-drop state with visual feedback: + ```typescript + interface ChatFileDropZoneProps { + onFilesDropped: (files: File[]) => void; + disabled?: boolean; + children: React.ReactNode; + } + ``` + + Implementation: + - Track isDragOver state via onDragEnter/onDragLeave/onDragOver/onDrop + - Prevent default on all drag events + - On drop: extract files from e.dataTransfer.files, call onFilesDropped + - When isDragOver: show a semi-transparent overlay with dashed border and "Drop files here" text, using Tailwind classes that work across all themes (use bg-primary/10, border-primary) + - Use cn() from lib/utils for conditional classes + + Update `ui/src/components/ChatInput.tsx`: + + 1. Add new props to ChatInputProps: + ```typescript + onFilesPicked?: (files: File[]) => void; + pendingFiles?: PendingFile[]; + onRemoveFile?: (id: string) => void; + ``` + + 2. Import Paperclip icon from lucide-react (for file attach button). + + 3. Wrap the form content in ChatFileDropZone: + ```tsx + files.forEach(f => onFilesPicked?.([f]))} disabled={disabled}> + {/* existing form */} + + ``` + + 4. Handle paste events on the textarea: + ```typescript + function handlePaste(e: React.ClipboardEvent) { + const files = Array.from(e.clipboardData.files); + if (files.length > 0) { + e.preventDefault(); + onFilesPicked?.(files); + } + // If no files, allow default paste behavior (text) + } + ``` + Add `onPaste={handlePaste}` to the textarea. + + 5. Add a file attach button (Paperclip icon) to the left of the send button: + ```tsx + + ``` + + 6. Show pending file chips above the textarea when pendingFiles has items: + ```tsx + {pendingFiles && pendingFiles.length > 0 && ( +
+ {pendingFiles.map((pf) => ( +
+ {pf.status === "uploading" && ( + + )} + {pf.name} + {pf.status === "uploading" && ( + {pf.progress}% + )} + +
+ ))} +
+ )} + ``` + + 7. Update the onSend signature: when files are attached, the parent (ChatPanel) needs to know. Keep onSend as `(content: string) => void` but the parent will read completedFileIds from the hook. No signature change needed. + + Update `ui/src/components/ChatInput.test.tsx`: + - Add test: "renders file attach button" + - Add test: "calls onFilesPicked when file input changes" + - Add test: "shows pending file chips" + + + cd /opt/nexus && grep -q "ChatFileDropZone" ui/src/components/ChatFileDropZone.tsx && grep -q "onFilesPicked" ui/src/components/ChatInput.tsx && grep -q "handlePaste" ui/src/components/ChatInput.tsx && grep -q "Paperclip" ui/src/components/ChatInput.tsx && echo "PASS" + + + - grep "ChatFileDropZone" ui/src/components/ChatFileDropZone.tsx returns a match + - grep "onDragOver" ui/src/components/ChatFileDropZone.tsx returns a match + - grep "onDrop" ui/src/components/ChatFileDropZone.tsx returns a match + - grep "onFilesPicked" ui/src/components/ChatInput.tsx returns a match + - grep "handlePaste" ui/src/components/ChatInput.tsx returns a match + - grep "clipboardData" ui/src/components/ChatInput.tsx returns a match + - grep "Paperclip" ui/src/components/ChatInput.tsx returns a match + - grep "pendingFiles" ui/src/components/ChatInput.tsx returns a match + - grep "type=\"file\"" ui/src/components/ChatInput.tsx returns a match + + + ChatInput supports file upload via drag-and-drop (ChatFileDropZone), clipboard paste (onPaste handler), and file picker button (Paperclip icon with hidden input). Pending files appear as chips above the textarea with progress indicators. + + + + + + +- ChatFileDropZone renders overlay on drag-over +- ChatInput shows Paperclip button that opens file picker +- Pasting an image triggers onFilesPicked +- Pending file chips show name, progress, and remove button +- chatApi.uploadFile sends FormData with progress callback +- useChatFileUpload manages upload lifecycle + + + +All three file input methods work: drag-and-drop shows drop zone overlay and triggers upload, clipboard paste of images triggers upload, Paperclip button opens native file picker. Pending uploads show progress chips. FILE-05 UI is complete. + + + +After completion, create `.planning/phases/25-file-system/25-02-SUMMARY.md` + diff --git a/.planning/phases/25-file-system/25-03-PLAN.md b/.planning/phases/25-file-system/25-03-PLAN.md new file mode 100644 index 00000000..3a1f00a2 --- /dev/null +++ b/.planning/phases/25-file-system/25-03-PLAN.md @@ -0,0 +1,326 @@ +--- +phase: 25-file-system +plan: 03 +type: execute +wave: 3 +depends_on: ["25-01", "25-02"] +files_modified: + - ui/src/components/ChatFilePreview.tsx + - ui/src/components/ChatFileCard.tsx + - ui/src/components/ChatMessage.tsx + - ui/src/components/ChatPanel.tsx + - ui/src/components/ChatMessageList.tsx + - ui/src/hooks/useChatMessages.ts + - server/src/services/chat.ts +autonomous: true +requirements: + - FILE-06 +must_haves: + truths: + - "Images attached to messages render inline in the chat" + - "Code files show syntax-highlighted preview in chat" + - "Any attached file can be downloaded with one click" + - "Files sent by user appear as preview cards in the conversation" + - "File previews work across all three themes" + artifacts: + - path: "ui/src/components/ChatFilePreview.tsx" + provides: "Inline file preview: images, code, documents" + contains: "ChatFilePreview" + - path: "ui/src/components/ChatFileCard.tsx" + provides: "Downloadable file card with icon and metadata" + contains: "ChatFileCard" + key_links: + - from: "ui/src/components/ChatMessage.tsx" + to: "ui/src/components/ChatFilePreview.tsx" + via: "renders file previews when message has files" + pattern: "ChatFilePreview" + - from: "ui/src/components/ChatPanel.tsx" + to: "ui/src/hooks/useChatFileUpload.ts" + via: "useChatFileUpload wired to ChatInput" + pattern: "useChatFileUpload" + - from: "server/src/services/chat.ts" + to: "packages/db/src/schema/chat_files.ts" + via: "join files when loading messages" + pattern: "chatFiles" +--- + + +Create file preview components and wire the full file flow: upload in ChatInput -> display as previews in ChatMessage -> download with one click. + +Purpose: Complete the user-facing file experience (FILE-06) by rendering uploaded/generated files inline in chat with appropriate previews. +Output: ChatFilePreview, ChatFileCard components; ChatMessage renders files; ChatPanel orchestrates upload-to-message flow. + + + +@$HOME/.claude/get-shit-done/workflows/execute-plan.md +@$HOME/.claude/get-shit-done/templates/summary.md + + + +@.planning/PROJECT.md +@.planning/ROADMAP.md +@.planning/phases/25-file-system/25-RESEARCH.md +@.planning/phases/25-file-system/25-00-SUMMARY.md +@.planning/phases/25-file-system/25-01-SUMMARY.md +@.planning/phases/25-file-system/25-02-SUMMARY.md + +@ui/src/components/ChatMessage.tsx +@ui/src/components/ChatPanel.tsx +@ui/src/components/ChatMessageList.tsx +@ui/src/components/ChatInput.tsx +@ui/src/components/ChatMarkdownMessage.tsx +@ui/src/hooks/useChatMessages.ts +@ui/src/hooks/useChatFileUpload.ts +@ui/src/api/chat.ts +@server/src/services/chat.ts +@server/src/services/chat-files.ts +@packages/shared/src/types/chat.ts + + + + +```typescript +export interface ChatFile { + id: string; + filename: string; + originalFilename: string; + mimeType: string; + sizeBytes: number; + source: "user_upload" | "agent_generated"; + category: "image" | "document" | "code" | "other" | null; + createdAt: string; +} + +// ChatMessage now has: +files?: ChatFile[]; +``` + + +GET /api/files/:fileId/content — streams file binary with correct MIME type +PATCH /api/files/:fileId — attach file to message: { messageId } + + +```typescript +export interface PendingFile { + id: string; + file: File; + name: string; + mimeType: string; + sizeBytes: number; + progress: number; + status: "uploading" | "done" | "error"; + uploadedFile?: ChatFile; + contentPath?: string; +} + +export function useChatFileUpload(conversationId: string | null): { + pendingFiles: PendingFile[]; + addFile: (file: File) => Promise; + removeFile: (tempId: string) => void; + clearCompleted: () => void; + completedFileIds: string[]; +} +``` + + +```typescript +interface ChatMessageProps { + id?: string; + role: "user" | "assistant" | "system"; + content: string; + messageType?: string | null; + // ... other existing props +} +``` + + + + + + Task 1: Create ChatFilePreview and ChatFileCard components + + ui/src/components/ChatFilePreview.tsx, + ui/src/components/ChatFileCard.tsx + + + ui/src/components/ChatMarkdownMessage.tsx, + ui/src/components/ChatCodeBlock.tsx, + ui/src/components/ChatMessage.tsx, + packages/shared/src/types/chat.ts, + ui/src/lib/utils.ts + + + Create `ui/src/components/ChatFileCard.tsx`: + + A compact card for any file attachment with download capability: + ```typescript + interface ChatFileCardProps { + file: ChatFile; + contentPath: string; + } + ``` + + Implementation: + - Show file icon based on category (use lucide icons: ImageIcon for image, FileCode for code, FileText for document, File for other) + - Show filename (truncated), file size (human-readable: KB, MB), and mime type + - Download button (Download icon from lucide) — opens `/api/files/${file.id}/content` in new tab or triggers download via an anchor element with `download` attribute + - Use theme-aware classes: `bg-muted rounded-lg border border-border p-3` — works across all three themes + - Compact layout: icon + info on left, download button on right + + Create `ui/src/components/ChatFilePreview.tsx`: + + Renders the appropriate preview based on file category: + ```typescript + interface ChatFilePreviewProps { + file: ChatFile; + contentPath: string; + } + ``` + + Implementation: + - **Images** (category === "image"): Render `` tag with `src={contentPath}` (the server serves the binary). Use `max-h-[300px] rounded-lg object-contain` to constrain size. Add loading="lazy". Wrap in a clickable link to open full-size in new tab. + - **Code files** (category === "code"): Render a ChatFileCard (do NOT attempt to fetch and syntax-highlight inline — that would require an extra fetch and significant complexity). The existing syntax highlighting from Phase 21 is for markdown code blocks, not standalone files. Show filename with code icon and download button. + - **Documents** (category === "document" and mimeType starts with "application/pdf"): Render ChatFileCard with a PDF icon. Full PDF preview is complex (would need pdf.js); a card with download is sufficient for v1. + - **Other documents** (text/plain, text/markdown, text/csv): Render ChatFileCard. + - **Everything else**: Render ChatFileCard. + + Below the preview, always render ChatFileCard so there is a download button even for images. + + Helper function `formatFileSize(bytes: number): string` — returns "1.2 KB", "3.4 MB", etc. + + All styles must use Tailwind utility classes that respect the theme system (bg-muted, text-foreground, border-border, etc.). + + + cd /opt/nexus && grep -q "ChatFilePreview" ui/src/components/ChatFilePreview.tsx && grep -q "ChatFileCard" ui/src/components/ChatFileCard.tsx && grep -q "formatFileSize" ui/src/components/ChatFileCard.tsx && grep -q "Download" ui/src/components/ChatFileCard.tsx && echo "PASS" + + + - grep "ChatFilePreview" ui/src/components/ChatFilePreview.tsx returns a match + - grep "ChatFileCard" ui/src/components/ChatFileCard.tsx returns a match + - grep "img" ui/src/components/ChatFilePreview.tsx returns a match (inline image rendering) + - grep "Download" ui/src/components/ChatFileCard.tsx returns a match + - grep "formatFileSize" ui/src/components/ChatFileCard.tsx returns a match + - grep "bg-muted" ui/src/components/ChatFileCard.tsx returns a match (theme-aware styling) + - grep "contentPath" ui/src/components/ChatFilePreview.tsx returns a match + + + ChatFilePreview renders inline images for image files and ChatFileCard for all other types. + ChatFileCard shows file metadata with one-click download. All styles work across themes. + + + + + Task 2: Wire files into ChatMessage, ChatPanel, and server message loading + + ui/src/components/ChatMessage.tsx, + ui/src/components/ChatPanel.tsx, + ui/src/components/ChatMessageList.tsx, + ui/src/hooks/useChatMessages.ts, + server/src/services/chat.ts + + + ui/src/components/ChatMessage.tsx, + ui/src/components/ChatPanel.tsx, + ui/src/components/ChatMessageList.tsx, + ui/src/hooks/useChatMessages.ts, + server/src/services/chat.ts, + server/src/services/chat-files.ts, + ui/src/hooks/useChatFileUpload.ts, + ui/src/api/chat.ts + + + **Server: Include files when loading messages** + + Update `server/src/services/chat.ts` — modify `listMessages` to also load files for each message: + - After fetching messages, collect all message IDs + - Query `chatFiles` WHERE messageId IN (messageIds), ordered by createdAt asc + - Group files by messageId into a Map + - Attach `files` array to each message in the response + - Import `chatFiles` from `@paperclipai/db` and `inArray` from `drizzle-orm` + - IMPORTANT: Only add file data to the response — do NOT modify the database query structure. Use a second query to fetch files, then merge. + + Also update `addMessage` return: after inserting a message, return it with an empty `files: []` array for consistency. + + **UI: ChatMessage renders files** + + Update `ui/src/components/ChatMessage.tsx`: + 1. Add `files?: ChatFile[]` to ChatMessageProps (import ChatFile from @paperclipai/shared) + 2. After the message content rendering (ChatMarkdownMessage), if `files && files.length > 0`, render: + ```tsx +
+ {files.map((f) => ( + + ))} +
+ ``` + 3. Import ChatFilePreview from "./ChatFilePreview" + + **UI: ChatMessageList passes files through** + + Update `ui/src/components/ChatMessageList.tsx`: + - Ensure the `files` prop from each message object is passed to `` + - No structural changes needed if messages already spread all props + + **UI: ChatPanel orchestrates upload flow** + + Update `ui/src/components/ChatPanel.tsx`: + 1. Import `useChatFileUpload` from hooks + 2. Call `const { pendingFiles, addFile, removeFile, clearCompleted, completedFileIds } = useChatFileUpload(activeConversationId);` + 3. Pass to ChatInput: `pendingFiles={pendingFiles}`, `onRemoveFile={removeFile}`, `onFilesPicked={(files) => files.forEach(addFile)}` + 4. In the handleSend function, after sending the message and getting the messageId back, call `chatApi.attachFiles(completedFileIds, messageId)` to link uploaded files to the message. Then call `clearCompleted()`. + - Add a new method to chatApi: `attachFilesToMessage(fileIds: string[], messageId: string)` that calls `PATCH /api/files/:fileId` for each fileId with `{ messageId }`. + - Alternatively, do this in a simpler way: after message creation, call `chatApi.attachFilesToMessage` which makes parallel PATCH calls. + 5. Invalidate messages query after attaching files so the message re-renders with file previews. + + Add `attachFilesToMessage` to `ui/src/api/chat.ts`: + ```typescript + async attachFilesToMessage(fileIds: string[], messageId: string) { + await Promise.all( + fileIds.map((fileId) => + api.patch(`/files/${fileId}`, { messageId }) + ) + ); + }, + ``` + + + cd /opt/nexus && grep -q "ChatFilePreview" ui/src/components/ChatMessage.tsx && grep -q "useChatFileUpload" ui/src/components/ChatPanel.tsx && grep -q "attachFilesToMessage" ui/src/api/chat.ts && grep -q "chatFiles" server/src/services/chat.ts && echo "PASS" + + + - grep "ChatFilePreview" ui/src/components/ChatMessage.tsx returns a match + - grep "files" ui/src/components/ChatMessage.tsx returns a match + - grep "useChatFileUpload" ui/src/components/ChatPanel.tsx returns a match + - grep "addFile" ui/src/components/ChatPanel.tsx returns a match + - grep "pendingFiles" ui/src/components/ChatPanel.tsx returns a match + - grep "clearCompleted" ui/src/components/ChatPanel.tsx returns a match + - grep "attachFilesToMessage" ui/src/api/chat.ts returns a match + - grep "chatFiles" server/src/services/chat.ts returns a match (file loading in listMessages) + - grep "inArray" server/src/services/chat.ts returns a match (batch file query) + + + Full file flow works: User drops/pastes/picks file -> uploads with progress -> sends message -> files attached to message -> message renders with inline image previews and download cards. Server includes files when loading messages. + + + + + + +- Server listMessages returns messages with files array populated +- ChatMessage renders ChatFilePreview for each attached file +- Images show inline with constrained dimensions +- Non-image files show as downloadable ChatFileCard +- ChatPanel wires useChatFileUpload to ChatInput +- After sending a message with files, files are attached and visible +- File previews use theme-aware Tailwind classes + + + +End-to-end file flow: upload -> store -> attach to message -> render preview -> download. Images render inline. Code and documents render as cards with download buttons. All previews work across Catppuccin Mocha, Tokyo Night, and Catppuccin Latte themes. FILE-06 is complete. + + + +After completion, create `.planning/phases/25-file-system/25-03-SUMMARY.md` + diff --git a/.planning/phases/25-file-system/25-RESEARCH.md b/.planning/phases/25-file-system/25-RESEARCH.md new file mode 100644 index 00000000..9b6975df --- /dev/null +++ b/.planning/phases/25-file-system/25-RESEARCH.md @@ -0,0 +1,76 @@ +# Phase 25: File System — Codebase Research + +**Gathered:** 2026-04-01 +**Status:** Complete + +## Existing Patterns + +### Storage System +- **StorageService** in `server/src/storage/service.ts` handles file persistence via a provider abstraction (local disk or S3) +- **LocalDiskProvider** in `server/src/storage/local-disk-provider.ts` writes files to `{instanceRoot}/data/storage/` +- Object keys follow `{companyId}/{namespace}/{year}/{month}/{day}/{uuid}-{stem}{ext}` pattern +- Storage service computes SHA-256 hash, validates inputs, sanitizes filenames +- `PutFileInput` requires: companyId, namespace, originalFilename, contentType, body (Buffer) +- `PutFileResult` returns: provider, objectKey, contentType, byteSize, sha256, originalFilename + +### Asset Upload (existing reference) +- `server/src/routes/assets.ts` uses **multer v2** with memory storage for single file upload +- Pattern: `multer.memoryStorage()` + `limits: { fileSize: MAX_ATTACHMENT_BYTES, files: 1 }` +- Manual `runSingleFileUpload` wrapper around multer's middleware +- After upload: calls `storage.putFile()`, then `assetService.create()` to write DB row +- Content served via `GET /api/assets/:assetId/content` which pipes `storage.getObject()` stream +- `MAX_ATTACHMENT_BYTES` defaults to 10MB (from `server/src/attachment-types.ts`) +- Allowed content types configurable via `PAPERCLIP_ALLOWED_ATTACHMENT_TYPES` env var + +### Database Schema +- **Drizzle ORM** with PostgreSQL, migrations in `packages/db/src/migrations/` (currently at 0052) +- Schema files in `packages/db/src/schema/`, exported from `index.ts` +- Convention: `pgTable()` with `uuid("id").primaryKey().defaultRandom()`, timestamps with timezone +- Index callbacks use object syntax: `(table) => ({ indexName: index("idx_name").on(table.col) })` +- Chat tables: `chat_conversations`, `chat_messages`, `chat_message_bookmarks` +- `chat_messages` columns: id, conversationId, role, content, agentId, messageType, createdAt, updatedAt + +### Chat Routes +- `server/src/routes/chat.ts` — `chatRoutes(db: Db)` returns Express Router +- Mounted at `/api` in `server/src/app.ts` line 160: `api.use(chatRoutes(db))` +- Currently does NOT receive `storageService` — only `db` +- Uses `assertBoard(req)` + `assertCompanyAccess(req, companyId)` for auth +- Uses validators from `@paperclipai/shared` (zod schemas) +- Message creation via `svc.addMessage(conversationId, { role, content, agentId })` + +### Chat Service +- `server/src/services/chat.ts` — `chatService(db)` returns object with methods +- Methods: listConversations, createConversation, getConversation, updateConversation, addMessage, editMessage, truncateMessagesAfter, addSystemMessage, streamEcho, searchMessages, toggleBookmark, getBookmarks, branchConversation, listBranches, exportConversation + +### Shared Types +- `packages/shared/src/types/chat.ts` — ChatConversation, ChatMessage, ChatConversationListResponse, etc. +- ChatMessage has: id, conversationId, role, content, agentId, messageType, createdAt, updatedAt +- NO file/attachment fields exist on ChatMessage yet + +### UI Components +- `ChatInput.tsx` — textarea with slash commands and @mentions; `onSend(content: string)` callback (text only) +- `ChatMessage.tsx` — dispatches to specialized components by messageType (spec_card, handoff, etc.) +- `ChatPanel.tsx` — orchestrates conversation selection, message list, input, streaming, bookmarks +- `chatApi` in `ui/src/api/chat.ts` — API client using fetch-based `api` helper + +### Migration Pattern +- SQL files named `NNNN_descriptive_name.sql` +- Journal in `packages/db/src/migrations/meta/_journal.json` +- Snapshots in `packages/db/src/migrations/meta/NNNN_snapshot.json` +- Next migration index: 0053 + +## Key Design Decisions (Claude's Discretion) + +1. **Reuse existing StorageService** — Files will be stored via the same `StorageService` (local disk or S3) used by assets. This means files live in `{instanceRoot}/data/storage/` under company-namespaced object keys, NOT in a separate `files/` git repo. The git versioning requirement (FILE-09, FILE-10) is out of scope for this batch of requirements (FILE-01 through FILE-06 only). + +2. **New `chat_files` table** — A new Drizzle schema table tracking file metadata: id, companyId, conversationId, messageId, filename, originalFilename, mimeType, sizeBytes, objectKey (StorageService key), sha256, source (user_upload | agent_generated), createdAt. + +3. **New `chat_file_references` table** — Enables a file to be referenced from multiple messages/conversations without re-uploading the binary. + +4. **Extend chat routes** — Add upload endpoint to chat routes (`POST /api/conversations/:id/files`) rather than creating a separate file routes module. This keeps the chat domain cohesive. + +5. **Extend ChatMessage type** — Add optional `files` array to ChatMessage response (populated via join or separate query). Messages with files get `messageType: "file_attachment"` or files are included as metadata alongside normal messages. + +6. **ChatInput gets file drop zone** — Add drag-and-drop, paste, and button-based file selection to ChatInput. Files are uploaded immediately on drop/paste, and the pending file IDs are sent with the message. + +7. **ChatFilePreview component** — Renders inline image previews, syntax-highlighted code previews (reusing existing rehype-highlight from Phase 21), and download buttons for other file types.