nexus/.planning/phases/21-chat-foundation/21-01-PLAN.md
Mikkel Georgsen af211e6a39 [nexus] docs(21-chat-foundation): create phase plan
4 plans across 3 waves for Chat Foundation phase.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 12:52:49 +02:00

23 KiB

phase plan type wave depends_on files_modified autonomous requirements must_haves
21-chat-foundation 01 execute 1
packages/db/src/schema/chat_conversations.ts
packages/db/src/schema/chat_messages.ts
packages/db/src/schema/index.ts
packages/shared/src/types/chat.ts
packages/shared/src/validators/chat.ts
server/src/services/chat.ts
server/src/routes/chat.ts
server/src/routes/index.ts
server/src/app.ts
server/src/__tests__/chat-service.test.ts
server/src/__tests__/chat-routes.test.ts
true
HIST-01
HIST-05
HIST-06
CHAT-04
CHAT-05
CHAT-06
truths artifacts key_links
Conversations and messages are stored in PostgreSQL and survive server restarts
Multiple conversations exist per company, listed sorted by updatedAt DESC
First message on a conversation auto-generates a title from first 60 characters
Conversations can be soft-deleted, archived, and pinned via timestamp columns
Conversations are accessible from any device via the REST API
path provides contains
packages/db/src/schema/chat_conversations.ts chat_conversations Drizzle table definition export const chatConversations
path provides contains
packages/db/src/schema/chat_messages.ts chat_messages Drizzle table definition with cascade delete onDelete: "cascade"
path provides exports
packages/shared/src/types/chat.ts ChatConversation and ChatMessage TypeScript interfaces
ChatConversation
ChatMessage
path provides exports
packages/shared/src/validators/chat.ts Zod schemas for create/update conversation and message
createConversationSchema
createMessageSchema
updateConversationSchema
path provides exports
server/src/services/chat.ts chatService factory with CRUD operations
chatService
path provides exports
server/src/routes/chat.ts chatRoutes factory mounting conversation and message endpoints
chatRoutes
path provides min_lines
server/src/__tests__/chat-service.test.ts Service-level unit tests 80
path provides min_lines
server/src/__tests__/chat-routes.test.ts Route-level integration tests 60
from to via pattern
server/src/routes/chat.ts server/src/services/chat.ts chatService(db) factory call chatService(db)
from to via pattern
server/src/app.ts server/src/routes/chat.ts api.use(chatRoutes(db)) chatRoutes(db)
from to via pattern
packages/db/src/schema/index.ts packages/db/src/schema/chat_conversations.ts re-export export.*chatConversations.*chat_conversations
Create the database schema, shared types, service layer, and REST API for chat conversations and messages.

Purpose: Establishes the persistence and API foundation that all UI components in subsequent plans depend on. Without this, no conversation can be created, stored, or retrieved. Output: Two new Drizzle schema files, a migration, shared types + validators, service + route factories, and automated tests.

<execution_context> @$HOME/.claude/get-shit-done/workflows/execute-plan.md @$HOME/.claude/get-shit-done/templates/summary.md </execution_context>

@.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/phases/21-chat-foundation/21-RESEARCH.md

From packages/db/src/schema/documents.ts (reference pattern for new schema):

import { pgTable, uuid, text, timestamp, index } from "drizzle-orm/pg-core";
import { companies } from "./companies.js";

export const documents = pgTable(
  "documents",
  {
    id: uuid("id").primaryKey().defaultRandom(),
    companyId: uuid("company_id").notNull().references(() => companies.id),
    // ... columns
    createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
    updatedAt: timestamp("updated_at", { withTimezone: true }).notNull().defaultNow(),
  },
  (table) => ({
    companyUpdatedIdx: index("documents_company_updated_idx").on(table.companyId, table.updatedAt),
  }),
);

From server/src/routes/activity.ts (reference pattern for route factory):

export function activityRoutes(db: Db) {
  const router = Router();
  const svc = activityService(db);
  router.get("/companies/:companyId/activity", async (req, res) => {
    assertBoard(req);
    assertCompanyAccess(req, req.params.companyId!);
    const result = await svc.list(filters);
    res.json(result);
  });
  return router;
}

From server/src/tests/activity-routes.test.ts (reference test pattern):

const mockActivityService = vi.hoisted(() => ({
  list: vi.fn(),
  create: vi.fn(),
}));
vi.mock("../services/activity.js", () => ({
  activityService: () => mockActivityService,
}));
function createApp() {
  const app = express();
  app.use(express.json());
  app.use((req, _res, next) => {
    (req as any).actor = {
      type: "board", userId: "user-1", companyIds: ["company-1"], source: "session", isInstanceAdmin: false,
    };
    next();
  });
  app.use("/api", activityRoutes({} as any));
  app.use(errorHandler);
  return app;
}

From server/src/middleware/validate.ts:

export function validate(schema: ZodSchema) {
  return (req: Request, _res: Response, next: NextFunction) => {
    req.body = schema.parse(req.body);
    next();
  };
}

From server/src/app.ts (route mounting — line 158):

api.use(activityRoutes(db));
Task 1: DB schema, shared types, validators, and service with tests packages/db/src/schema/chat_conversations.ts, packages/db/src/schema/chat_messages.ts, packages/db/src/schema/index.ts, packages/shared/src/types/chat.ts, packages/shared/src/validators/chat.ts, server/src/services/chat.ts, server/src/__tests__/chat-service.test.ts packages/db/src/schema/documents.ts, packages/db/src/schema/index.ts, packages/db/src/schema/companies.ts, packages/db/src/schema/agents.ts, packages/shared/src/types/company.ts, packages/shared/src/validators/company.ts, server/src/services/documents.ts, server/src/__tests__/activity-routes.test.ts - Test: chatService(db).createConversation({ companyId, title }) inserts a row and returns it with id, companyId, title, createdAt, updatedAt - Test: chatService(db).listConversations(companyId, {}) returns items sorted by updatedAt DESC, excludes soft-deleted rows (deletedAt IS NOT NULL) - Test: chatService(db).listConversations with cursor returns only rows older than cursor - Test: chatService(db).listConversations returns hasMore=true when more rows exist beyond limit - Test: chatService(db).addMessage({ conversationId, role, content }) inserts message and bumps conversation.updatedAt - Test: chatService(db).addMessage on a conversation with title=null sets title to first 60 chars of content - Test: chatService(db).addMessage on a conversation with existing title does NOT overwrite title - Test: chatService(db).softDeleteConversation sets deletedAt timestamp - Test: chatService(db).archiveConversation sets archivedAt timestamp - Test: chatService(db).pinConversation sets pinnedAt timestamp - Test: chatService(db).unpinConversation clears pinnedAt to null - Test: chatService(db).updateConversation({ title }) updates title 1. Create `packages/db/src/schema/chat_conversations.ts`: - Table name: `chat_conversations` - Columns: `id` (uuid PK defaultRandom), `companyId` (uuid FK to companies.id, NOT NULL), `title` (text, nullable), `agentId` (uuid FK to agents.id onDelete "set null", nullable), `pinnedAt` (timestamp with tz, nullable), `archivedAt` (timestamp with tz, nullable), `deletedAt` (timestamp with tz, nullable), `createdAt` (timestamp with tz, NOT NULL, defaultNow), `updatedAt` (timestamp with tz, NOT NULL, defaultNow) - Indexes: `chat_conversations_company_updated_idx` on (companyId, updatedAt), `chat_conversations_company_deleted_idx` on (companyId, deletedAt) - Export: `export const chatConversations`
2. Create `packages/db/src/schema/chat_messages.ts`:
   - Table name: `chat_messages`
   - Columns: `id` (uuid PK defaultRandom), `conversationId` (uuid FK to chatConversations.id onDelete "cascade", NOT NULL), `role` (text NOT NULL — values: "user" | "assistant" | "system"), `content` (text NOT NULL), `agentId` (uuid, nullable — which agent produced this), `createdAt` (timestamp with tz, NOT NULL, defaultNow)
   - Index: `chat_messages_conversation_created_idx` on (conversationId, createdAt)
   - Export: `export const chatMessages`

3. Add to `packages/db/src/schema/index.ts` — append two lines:
   ```
   export { chatConversations } from "./chat_conversations.js";
   export { chatMessages } from "./chat_messages.js";
   ```

4. Run `pnpm db:generate` to generate migration SQL. Verify the generated SQL contains:
   - `CREATE TABLE "chat_conversations"` with all columns
   - `CREATE TABLE "chat_messages"` with `ON DELETE CASCADE` on conversation_id FK
   - Both index `CREATE INDEX` statements

5. Create `packages/shared/src/types/chat.ts`:
   ```typescript
   export interface ChatConversation {
     id: string;
     companyId: string;
     title: string | null;
     agentId: string | null;
     pinnedAt: string | null;
     archivedAt: string | null;
     deletedAt: string | null;
     createdAt: string;
     updatedAt: string;
   }

   export interface ChatMessage {
     id: string;
     conversationId: string;
     role: "user" | "assistant" | "system";
     content: string;
     agentId: string | null;
     createdAt: string;
   }

   export interface ChatConversationListResponse {
     items: ChatConversation[];
     hasMore: boolean;
   }
   ```

6. Create `packages/shared/src/validators/chat.ts`:
   ```typescript
   import { z } from "zod";

   export const createConversationSchema = z.object({
     title: z.string().max(200).optional(),
   });

   export const updateConversationSchema = z.object({
     title: z.string().max(200).optional(),
   });

   export const createMessageSchema = z.object({
     role: z.enum(["user", "assistant", "system"]),
     content: z.string().min(1),
     agentId: z.string().uuid().optional().nullable(),
   });
   ```

7. Create `server/src/services/chat.ts` following the `documentService` factory pattern:
   ```typescript
   export function chatService(db: Db) {
     return {
       async listConversations(companyId: string, opts: { cursor?: string; limit?: number }) { ... },
       async createConversation(companyId: string, data: { title?: string }) { ... },
       async getConversation(id: string) { ... },
       async updateConversation(id: string, data: { title?: string }) { ... },
       async softDeleteConversation(id: string) { ... },
       async archiveConversation(id: string) { ... },
       async unarchiveConversation(id: string) { ... },
       async pinConversation(id: string) { ... },
       async unpinConversation(id: string) { ... },
       async listMessages(conversationId: string, opts: { cursor?: string; limit?: number }) { ... },
       async addMessage(conversationId: string, data: { role: string; content: string; agentId?: string | null }) { ... },
     };
   }
   ```
   Key implementation details:
   - `listConversations`: filter `WHERE companyId = $1 AND deletedAt IS NULL`, order by `updatedAt DESC`, cursor-based pagination with `updatedAt < cursor`, limit defaults to 30 (max 100), return `{ items, hasMore }` where hasMore is `rows.length > limit` and items is `rows.slice(0, limit)`
   - `addMessage`: after inserting the message, run `UPDATE chat_conversations SET updated_at = now() WHERE id = $conversationId`. Also, if `conversation.title IS NULL`, set `title = content.slice(0, 60)` using `WHERE id = $conversationId AND title IS NULL` for idempotency
   - `softDeleteConversation`: `UPDATE chat_conversations SET deleted_at = now() WHERE id = $id`
   - `archiveConversation`: `UPDATE SET archived_at = now()`
   - `pinConversation`: `UPDATE SET pinned_at = now()`
   - `unpinConversation`: `UPDATE SET pinned_at = null`

8. Create `server/src/__tests__/chat-service.test.ts` using the `vi.mock` pattern from `activity-routes.test.ts`. Mock `@paperclipai/db` to provide a mock `db` object with chainable `.select().from().where().orderBy().limit()` and `.insert().values().returning()` and `.update().set().where()` methods. Test all behaviors listed in the `<behavior>` block above.
cd /Volumes/UsbNvme/repos/nexus && pnpm vitest run server/src/__tests__/chat-service.test.ts - packages/db/src/schema/chat_conversations.ts contains `export const chatConversations = pgTable("chat_conversations"` - packages/db/src/schema/chat_conversations.ts contains `pinnedAt: timestamp("pinned_at"` and `archivedAt: timestamp("archived_at"` and `deletedAt: timestamp("deleted_at"` - packages/db/src/schema/chat_messages.ts contains `onDelete: "cascade"` - packages/db/src/schema/chat_messages.ts contains `role: text("role").notNull()` - packages/db/src/schema/index.ts contains `export { chatConversations } from "./chat_conversations.js"` - packages/db/src/schema/index.ts contains `export { chatMessages } from "./chat_messages.js"` - packages/shared/src/types/chat.ts contains `export interface ChatConversation` - packages/shared/src/types/chat.ts contains `export interface ChatMessage` - packages/shared/src/validators/chat.ts contains `export const createConversationSchema` - packages/shared/src/validators/chat.ts contains `export const createMessageSchema` - server/src/services/chat.ts contains `export function chatService(db: Db)` - server/src/services/chat.ts contains `async addMessage(` - server/src/services/chat.ts contains `title IS NULL` or `isNull(chatConversations.title)` for idempotent title set - server/src/__tests__/chat-service.test.ts exits 0 All schema files, types, validators, and service exist. Service test suite passes with coverage of list, create, add message (with auto-title), soft-delete, archive, pin/unpin. Task 2: REST API routes and route tests server/src/routes/chat.ts, server/src/routes/index.ts, server/src/app.ts, server/src/__tests__/chat-routes.test.ts server/src/routes/activity.ts, server/src/routes/index.ts, server/src/routes/authz.ts, server/src/app.ts, server/src/__tests__/activity-routes.test.ts, server/src/services/chat.ts, packages/shared/src/validators/chat.ts - Test: GET /api/companies/:companyId/conversations returns 200 with { items: [], hasMore: false } when empty - Test: POST /api/companies/:companyId/conversations returns 201 with the created conversation object - Test: GET /api/conversations/:id returns 200 with conversation object - Test: PATCH /api/conversations/:id with { title: "new title" } returns 200 with updated conversation - Test: DELETE /api/conversations/:id returns 204 - Test: POST /api/conversations/:id/archive returns 200 - Test: POST /api/conversations/:id/unarchive returns 200 - Test: POST /api/conversations/:id/pin returns 200 - Test: POST /api/conversations/:id/unpin returns 200 - Test: GET /api/conversations/:id/messages returns 200 with { items: [], hasMore: false } - Test: POST /api/conversations/:id/messages with { role: "user", content: "hello" } returns 201 1. Create `server/src/routes/chat.ts`: ```typescript import { Router } from "express"; import type { Db } from "@paperclipai/db"; import { assertBoard, assertCompanyAccess } from "./authz.js"; import { chatService } from "../services/chat.js"; import { validate } from "../middleware/validate.js"; import { createConversationSchema, updateConversationSchema, createMessageSchema } from "@paperclipai/shared";
   export function chatRoutes(db: Db) {
     const router = Router();
     const svc = chatService(db);

     // GET /api/companies/:companyId/conversations
     router.get("/companies/:companyId/conversations", async (req, res) => {
       const companyId = req.params.companyId as string;
       assertCompanyAccess(req, companyId);
       const cursor = req.query.cursor as string | undefined;
       const limit = req.query.limit ? Number(req.query.limit) : undefined;
       const result = await svc.listConversations(companyId, { cursor, limit });
       res.json(result);
     });

     // POST /api/companies/:companyId/conversations
     router.post("/companies/:companyId/conversations", validate(createConversationSchema), async (req, res) => {
       assertBoard(req);
       const companyId = req.params.companyId as string;
       const conversation = await svc.createConversation(companyId, req.body);
       res.status(201).json(conversation);
     });

     // GET /api/conversations/:id
     router.get("/conversations/:id", async (req, res) => {
       assertBoard(req);
       const conversation = await svc.getConversation(req.params.id as string);
       if (!conversation) { res.status(404).json({ error: "Not found" }); return; }
       res.json(conversation);
     });

     // PATCH /api/conversations/:id
     router.patch("/conversations/:id", validate(updateConversationSchema), async (req, res) => {
       assertBoard(req);
       const conversation = await svc.updateConversation(req.params.id as string, req.body);
       res.json(conversation);
     });

     // DELETE /api/conversations/:id
     router.delete("/conversations/:id", async (req, res) => {
       assertBoard(req);
       await svc.softDeleteConversation(req.params.id as string);
       res.status(204).end();
     });

     // POST /api/conversations/:id/archive
     router.post("/conversations/:id/archive", async (req, res) => {
       assertBoard(req);
       const result = await svc.archiveConversation(req.params.id as string);
       res.json(result);
     });

     // POST /api/conversations/:id/unarchive
     router.post("/conversations/:id/unarchive", async (req, res) => {
       assertBoard(req);
       const result = await svc.unarchiveConversation(req.params.id as string);
       res.json(result);
     });

     // POST /api/conversations/:id/pin
     router.post("/conversations/:id/pin", async (req, res) => {
       assertBoard(req);
       const result = await svc.pinConversation(req.params.id as string);
       res.json(result);
     });

     // POST /api/conversations/:id/unpin
     router.post("/conversations/:id/unpin", async (req, res) => {
       assertBoard(req);
       const result = await svc.unpinConversation(req.params.id as string);
       res.json(result);
     });

     // GET /api/conversations/:id/messages
     router.get("/conversations/:id/messages", async (req, res) => {
       assertBoard(req);
       const cursor = req.query.cursor as string | undefined;
       const limit = req.query.limit ? Number(req.query.limit) : undefined;
       const result = await svc.listMessages(req.params.id as string, { cursor, limit });
       res.json(result);
     });

     // POST /api/conversations/:id/messages
     router.post("/conversations/:id/messages", validate(createMessageSchema), async (req, res) => {
       assertBoard(req);
       const message = await svc.addMessage(req.params.id as string, req.body);
       res.status(201).json(message);
     });

     return router;
   }
   ```

2. Add to `server/src/routes/index.ts`:
   ```
   export { chatRoutes } from "./chat.js";
   ```

3. In `server/src/app.ts`, add import `import { chatRoutes } from "./routes/chat.js";` near line 27 (after activityRoutes import), and add `api.use(chatRoutes(db));` after line 158 (after `api.use(activityRoutes(db));`).

4. Create `server/src/__tests__/chat-routes.test.ts` following the exact `activity-routes.test.ts` mock pattern:
   - Use `vi.hoisted` to create `mockChatService` with all methods as `vi.fn()`
   - `vi.mock("../services/chat.js", () => ({ chatService: () => mockChatService }))`
   - `createApp()` function that sets up express with JSON parsing, actor middleware (type: "board", companyIds: ["company-1"]), mounts `chatRoutes({} as any)` under `/api`, and adds `errorHandler`
   - Test all behaviors listed in `<behavior>` block
   - Also mock `@paperclipai/shared` validators if needed, or let them pass through (they are pure Zod schemas that work without mocking)
cd /Volumes/UsbNvme/repos/nexus && pnpm vitest run server/src/__tests__/chat-routes.test.ts - server/src/routes/chat.ts contains `export function chatRoutes(db: Db)` - server/src/routes/chat.ts contains `router.get("/companies/:companyId/conversations"` - server/src/routes/chat.ts contains `router.post("/conversations/:id/messages"` - server/src/routes/chat.ts contains `assertBoard(req)` on every mutating route - server/src/routes/chat.ts contains `assertCompanyAccess(req, companyId)` on the list endpoint - server/src/routes/index.ts contains `export { chatRoutes } from "./chat.js"` - server/src/app.ts contains `import { chatRoutes }` and `chatRoutes(db)` - server/src/__tests__/chat-routes.test.ts exits 0 All chat REST endpoints exist and respond with correct status codes. Route test suite passes. Routes are mounted in app.ts and exported from routes/index.ts. - `pnpm vitest run server/src/__tests__/chat-service.test.ts` passes - `pnpm vitest run server/src/__tests__/chat-routes.test.ts` passes - `pnpm db:generate` produces migration SQL with both tables and cascade FK - `grep -r "chatConversations\|chatMessages" packages/db/src/schema/index.ts` shows both exports

<success_criteria>

  • Two new DB tables (chat_conversations, chat_messages) with correct columns, indexes, and cascade FK
  • Shared types and Zod validators for all create/update operations
  • Service layer with full CRUD + auto-title on first message + updatedAt bump
  • REST API with 11 endpoints mounted in app.ts
  • All automated tests green </success_criteria>
After completion, create `.planning/phases/21-chat-foundation/21-01-SUMMARY.md`