# Phase 24: Search, History & Branching - Research

**Researched:** 2026-04-01
**Domain:** PostgreSQL full-text search, conversation branching data model, bookmark storage, export formatting, React command palette UI
**Confidence:** HIGH

## Summary

Phase 24 adds four capabilities to the existing chat system: full-text search across all messages (CHAT-07, PERF-04), bookmarking any message (CHAT-13), conversation branching from any point (CHAT-14), and conversation export (HIST-04). The project uses PostgreSQL 17 with Drizzle ORM, Express 5, React 19, and TanStack Query — all patterns established in Phases 21–23.

The biggest technical decision is how to implement full-text search performantly. The codebase already uses `ilike` for conversation title search and for issue search, but the requirement is sub-500ms search across 10,000+ messages. PostgreSQL's native `tsvector`/`to_tsvector` with a GIN index is the correct approach for this scale — `ILIKE '%term%'` requires a sequential table scan and will not meet PERF-04 at volume. The codebase has no existing `tsvector` columns, but the migration infrastructure fully supports adding one.

Conversation branching is architecturally the most complex item. The existing data model is flat (each message belongs to one conversation, each conversation is linear). Branching requires either (a) a new branch relationship that forks a conversation from a message point, creating a new conversation with a reference to the branch-point message, or (b) a tree structure inside a conversation. Option (a) — creating a new child conversation — aligns cleanly with existing patterns: a `parentConversationId` foreign key plus a `branchFromMessageId` on `chatConversations` lets the UI show both branches without changing message storage logic.

Bookmarks and export are straightforward additions. Bookmarks require a new `chat_message_bookmarks` table and a corresponding service method and route. Export is a server-side endpoint that queries all messages for a conversation and serialises them as Markdown or JSON.

**Primary recommendation:** Use PostgreSQL `tsvector` + GIN index for message search (add via migration); model branching as child conversations with `parentConversationId` + `branchFromMessageId` foreign keys; store bookmarks in a dedicated join table; export from a dedicated server route that streams a file download.

<user_constraints>
## User Constraints (from CONTEXT.md)

### Locked Decisions

None — discuss phase was skipped per workflow.skip_discuss.

### Claude's Discretion

All implementation choices are at Claude's discretion. Use ROADMAP phase goal, success criteria, and codebase conventions to guide decisions.

### Deferred Ideas (OUT OF SCOPE)

None — discuss phase skipped. Refer to ROADMAP phase description and success criteria.
</user_constraints>

<phase_requirements>
## Phase Requirements

The ROADMAP (authoritative) lists these requirements for Phase 24:

| ID | Description | Research Support |
|----|-------------|------------------|
| CHAT-07 | Full-text search across all conversations | PostgreSQL tsvector + GIN index; new `/companies/:id/messages/search` route |
| CHAT-13 | Message reactions / bookmarks: mark important messages for later reference | New `chat_message_bookmarks` table; toggle route; bookmark list UI in sidebar |
| CHAT-14 | Conversation branching: editing a mid-conversation message creates a branch; both branches are preserved | `parentConversationId` + `branchFromMessageId` columns on `chatConversations`; branch create route; branch selector UI in ChatPanel |
| HIST-04 | Conversation export: download as Markdown or JSON | Server-side export route returning file download; client-side trigger |
| PERF-04 | Full-text search returns results in under 500ms across 10,000+ messages | GIN index on `tsvector` column satisfies this at 10k+ scale; ILIKE does not |

Note: The additional_context block referenced HIST-07 through HIST-12, which do not exist in REQUIREMENTS.md. The canonical requirement list is from ROADMAP.md, confirmed above.
</phase_requirements>

---

## Standard Stack

### Core
| Library | Version | Purpose | Why Standard |
|---------|---------|---------|--------------|
| drizzle-orm | ^0.38.4 | DB queries, schema definition, migrations | Project ORM — all DB access uses Drizzle |
| drizzle-kit | ^0.31.9 | Migration generation | Project migration tool |
| postgres | ^3.4.5 | PostgreSQL driver | Project driver |
| zod | ^3.24.2 | Request validation schemas | Project validator |
| express | 5.1.0 | Routes and middleware | Project HTTP framework |
| @tanstack/react-query | 5.x | Server state, data fetching | Project state layer |
| cmdk | ^1.1.1 | Command palette primitives | Already installed; `CommandDialog` in `ui/src/components/ui/command.tsx` |
| lucide-react | ^0.574.0 | Icons | Project icon library |

### Supporting
| Library | Version | Purpose | When to Use |
|---------|---------|---------|-------------|
| sql (drizzle-orm) | ^0.38.4 | Raw SQL fragments in Drizzle queries | Needed for `to_tsvector`, `plainto_tsquery`, GIN index DDL |

### Alternatives Considered
| Instead of | Could Use | Tradeoff |
|------------|-----------|----------|
| PostgreSQL tsvector | ILIKE `%term%` | ILIKE is simpler but requires full table scan — fails PERF-04 at 10k+ messages |
| PostgreSQL tsvector | Dedicated search engine (Meilisearch, Typesense) | External service adds operational complexity; Postgres FTS is sufficient at this scale and is already available |
| Child-conversation branching | In-conversation tree (parentMessageId) | Tree structure requires rewriting the entire message list rendering; child conversations reuse all existing conversation display logic |

**Installation:** No new packages required. All libraries are already in the project.

---

## Architecture Patterns

### Recommended Project Structure

New files follow the established pattern from Phase 21–23:

```
packages/db/src/schema/
├── chat_conversations.ts        # ADD: parentConversationId, branchFromMessageId columns
├── chat_messages.ts             # ADD: tsvector column (generated, persisted)
├── chat_message_bookmarks.ts    # NEW: bookmark join table
└── index.ts                     # EXPORT: chat_message_bookmarks

packages/db/src/migrations/
├── 0050_<slug>.sql              # ADD parentConversationId + branchFromMessageId to chat_conversations
├── 0051_<slug>.sql              # ADD tsvector column + GIN index to chat_messages
└── 0052_<slug>.sql              # CREATE chat_message_bookmarks table

packages/shared/src/types/
└── chat.ts                      # ADD: ChatMessageSearchResult, ChatBookmark types

packages/shared/src/validators/
└── chat.ts                      # ADD: searchMessagesSchema

server/src/services/
└── chat.ts                      # ADD: searchMessages, toggleBookmark, getBookmarks,
                                 #      branchConversation, exportConversation

server/src/routes/
└── chat.ts                      # ADD: search route, bookmark routes, branch route, export route

server/src/__tests__/
├── chat-service.test.ts         # ADD: tests for new service methods
└── chat-routes.test.ts          # ADD: tests for new routes

ui/src/api/
└── chat.ts                      # ADD: searchMessages, toggleBookmark, getBookmarks,
                                 #      branchConversation, exportConversation

ui/src/hooks/
└── useChatSearch.ts             # NEW: React Query hook for message search
└── useChatBookmarks.ts          # NEW: hook for bookmark list

ui/src/components/
├── ChatSearchDialog.tsx         # NEW: full-text message search overlay
├── ChatMessageBookmark.tsx      # NEW: bookmark toggle button on messages
├── ChatBookmarkList.tsx         # NEW: filterable bookmark list panel
├── ChatBranchSelector.tsx       # NEW: branch picker shown when conversation has branches
└── ChatConversationItem.tsx     # MODIFY: show branch indicator when branchFromMessageId set
```

### Pattern 1: PostgreSQL Full-Text Search with tsvector

**What:** Store a pre-computed `tsvector` in `chat_messages` using a generated stored column. Index it with GIN. Query with `plainto_tsquery`.

**When to use:** Any search across `content` column at scale.

**How it works in PostgreSQL 17:**

```sql
-- Migration: add generated tsvector column + GIN index
ALTER TABLE "chat_messages"
  ADD COLUMN "content_search" tsvector
  GENERATED ALWAYS AS (to_tsvector('english', content)) STORED;

CREATE INDEX "chat_messages_content_search_idx"
  ON "chat_messages" USING GIN ("content_search");
```

**Query pattern in Drizzle (using `sql` tag):**

```typescript
// Source: drizzle-orm docs — sql template literal for raw expressions
import { sql, and, eq, desc } from "drizzle-orm";

async searchMessages(companyId: string, query: string, opts: { limit?: number }) {
  const limit = Math.min(opts.limit ?? 20, 50);
  const tsQuery = sql`plainto_tsquery('english', ${query})`;

  const rows = await db
    .select({
      messageId: chatMessages.id,
      conversationId: chatMessages.conversationId,
      content: chatMessages.content,
      role: chatMessages.role,
      createdAt: chatMessages.createdAt,
      conversationTitle: chatConversations.title,
      rank: sql<number>`ts_rank(${chatMessages.contentSearch}, ${tsQuery})`,
    })
    .from(chatMessages)
    .innerJoin(chatConversations, eq(chatMessages.conversationId, chatConversations.id))
    .where(
      and(
        eq(chatConversations.companyId, companyId),
        sql`${chatMessages.contentSearch} @@ ${tsQuery}`,
      ),
    )
    .orderBy(desc(sql`ts_rank(${chatMessages.contentSearch}, ${tsQuery})`))
    .limit(limit);

  return rows;
}
```

**Drizzle schema for the generated column:**

Drizzle ORM does not have a first-class API for `GENERATED ALWAYS AS ... STORED` columns as of 0.38.x. The column must be added via raw SQL migration (not `drizzle-kit generate`) and declared as a `customType` or plain `sql` column in the schema for query use only. The safest approach:

1. Add the column via a hand-written migration SQL file.
2. Declare it in the Drizzle schema as a non-insertable column using `customType` or simply reference it via `sql` in queries without adding it to the Drizzle schema object (since it's never written by application code).

The cleanest pattern: declare a virtual query alias:

```typescript
// In chat_messages.ts schema — do NOT add contentSearch to insert types
// Reference it only in raw sql() expressions when querying
// The column exists in Postgres but Drizzle need not know its type for insertion
```

This avoids fighting Drizzle's type system for generated columns.

### Pattern 2: Conversation Branching via Child Conversations

**What:** When a user branches from message M in conversation C, create a new conversation C' that:
- Has `parentConversationId = C.id`
- Has `branchFromMessageId = M.id`
- Copies all messages from C up to and including M into C' (or references them)

**Recommended approach — copy messages up to branch point:**

Copy is simpler than reference because the existing `listMessages` and streaming logic requires messages to live in the same conversation. A reference approach would require `JOIN` on every message list query. Copying is O(n) at branch time and results in independent conversations — users can edit messages in branches without affecting each other.

```typescript
async branchConversation(parentConversationId: string, branchFromMessageId: string, companyId: string) {
  // 1. Get messages up to and including branch point, ordered chronologically
  const branchMsg = await db
    .select({ createdAt: chatMessages.createdAt })
    .from(chatMessages)
    .where(eq(chatMessages.id, branchFromMessageId));
  if (!branchMsg[0]) throw notFound("Branch message not found");

  const messagesUpToBranch = await db
    .select()
    .from(chatMessages)
    .where(
      and(
        eq(chatMessages.conversationId, parentConversationId),
        lte(chatMessages.createdAt, branchMsg[0].createdAt),
      ),
    )
    .orderBy(asc(chatMessages.createdAt));

  // 2. Create new conversation with branch metadata
  const [newConv] = await db
    .insert(chatConversations)
    .values({
      companyId,
      title: null,
      parentConversationId,
      branchFromMessageId,
    })
    .returning();

  // 3. Copy messages into new conversation
  if (messagesUpToBranch.length > 0) {
    await db.insert(chatMessages).values(
      messagesUpToBranch.map(({ id: _id, conversationId: _cid, ...rest }) => ({
        ...rest,
        conversationId: newConv!.id,
      })),
    );
  }

  return newConv!;
}
```

**Schema additions to `chat_conversations`:**

```typescript
parentConversationId: uuid("parent_conversation_id")
  .references(() => chatConversations.id, { onDelete: "set null" }),
branchFromMessageId: uuid("branch_from_message_id"),
// No FK on branchFromMessageId — message may have been deleted
```

**Listing branches:** Add `listBranches(conversationId)` service method that queries `WHERE parentConversationId = ?`.

### Pattern 3: Bookmarks as a Join Table

**What:** `chat_message_bookmarks` with `(userId, messageId)` or `(conversationId, messageId)`. Since the project uses board-level auth (not per-user), scope bookmarks to `companyId`.

**Schema:**

```typescript
export const chatMessageBookmarks = pgTable(
  "chat_message_bookmarks",
  {
    id: uuid("id").primaryKey().defaultRandom(),
    companyId: uuid("company_id").notNull().references(() => companies.id),
    messageId: uuid("message_id").notNull().references(() => chatMessages.id, { onDelete: "cascade" }),
    conversationId: uuid("conversation_id").notNull().references(() => chatConversations.id, { onDelete: "cascade" }),
    createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
  },
  (table) => ({
    companyMessageIdx: index("chat_bookmarks_company_message_idx").on(table.companyId, table.messageId),
    companyConvIdx: index("chat_bookmarks_company_conv_idx").on(table.companyId, table.conversationId),
  }),
);
```

**Toggle pattern:** Upsert-or-delete: if bookmark exists for `(companyId, messageId)`, delete it; otherwise insert. Return `{ bookmarked: boolean }`.

### Pattern 4: Export as Server Route

**What:** `GET /api/conversations/:id/export?format=markdown|json` returns a file download.

**Markdown format:**

```
# {conversation.title}
Exported: {date}

---

**{agentName}** ({timestamp})
{message.content}

---

**You** ({timestamp})
{message.content}
```

**JSON format:** Return the full `ChatMessageListResponse`-shaped object with all messages and conversation metadata.

**Route:**

```typescript
router.get("/conversations/:id/export", async (req, res) => {
  assertBoard(req);
  const format = req.query.format === "json" ? "json" : "markdown";
  const { content, filename } = await svc.exportConversation(req.params.id!, format);
  res.setHeader("Content-Disposition", `attachment; filename="${filename}"`);
  res.setHeader("Content-Type", format === "json" ? "application/json" : "text/markdown");
  res.send(content);
});
```

**Client-side trigger:** Use `window.location.href = url` or create a temporary `<a>` element with `download` attribute pointing to the API URL.

### Pattern 5: Chat Search Dialog — Cmd+K Routing

**Problem:** The existing `Cmd+K` handler opens `CommandPalette` (general app search). The ROADMAP success criterion says "Cmd+K opens a search overlay" for chat message search. These two handlers conflict.

**Resolution options:**

A. Add a "Search messages" item to the existing `CommandPalette` that opens a separate `ChatSearchDialog`.
B. When chat panel is open, `Cmd+K` opens `ChatSearchDialog` instead of `CommandPalette`.

**Recommendation: Option A.** The existing `CommandPalette` already intercepts `Cmd+K` globally. Adding a "Search chat messages" command item with a keyboard shortcut hint (e.g., `Cmd+Shift+F`) avoids handler conflicts and aligns with how CommandPalette is used for navigation. The `ChatSearchDialog` is a separate `CommandDialog` that can also be opened from the chat panel header via a search icon button.

**ChatSearchDialog uses existing `CommandDialog` + `CommandList` primitives from `ui/src/components/ui/command.tsx`.** Results are fetched via `useChatSearch` hook using TanStack Query (debounced, enabled when query length >= 2).

### Anti-Patterns to Avoid

- **ILIKE for message content search:** `ILIKE '%term%'` on `chat_messages.content` requires a full table scan. At 10,000+ messages this will not meet 500ms. Always use the GIN-indexed `tsvector` column.
- **In-place message tree for branching:** Adding `parentMessageId` to `chat_messages` requires rewriting all message list queries and the virtualised list rendering. Use child conversations instead.
- **Storing tsvector as a regular column:** The `tsvector` column must be a generated stored column that auto-updates when `content` changes. If you add it as a regular column, you must remember to update it on every `editMessage` call — a maintenance burden.
- **Using `drizzle-kit generate` for generated columns:** Drizzle Kit 0.31.x has incomplete support for PostgreSQL generated columns. Write the migration SQL by hand and keep the Drizzle schema declaration minimal (no `generatedAlwaysAs` — just reference via `sql` in queries).
- **Exporting all messages in memory for large conversations:** The export endpoint queries all messages without pagination. For very large conversations this could be slow. For Phase 24 this is acceptable (no limit specified); note it in code as a future streaming candidate.

---

## Don't Hand-Roll

| Problem | Don't Build | Use Instead | Why |
|---------|-------------|-------------|-----|
| Full-text tokenisation | Custom tokeniser, regex word split | PostgreSQL `to_tsvector('english', ...)` | Handles stemming, stop words, multilingual edge cases |
| Search ranking | Custom score function | `ts_rank()` / `ts_rank_cd()` | Built-in ranking weighted by proximity and frequency |
| Command palette UI | Custom modal + keyboard handler | `cmdk` via existing `CommandDialog` | Already installed, accessible, handles keyboard nav |
| Markdown serialization for export | Custom Markdown renderer | Plain string template — messages are already stored in Markdown | No library needed; just format with headings and separators |
| Bookmark toggle atomicity | Application-level check-then-insert | Upsert / INSERT ... ON CONFLICT DO NOTHING + DELETE | Race-condition safe |

---

## Common Pitfalls

### Pitfall 1: tsvector Not Updated on Message Edit
**What goes wrong:** `chat_messages.content_search` is a `GENERATED ALWAYS AS ... STORED` column in Postgres. Since it is a generated column, Postgres updates it automatically on `UPDATE`. This works correctly. No manual sync needed.
**Why it happens:** Developers familiar with application-managed FTS (triggers, background jobs) assume they must update the search column manually.
**How to avoid:** Use `GENERATED ALWAYS AS ... STORED` — Postgres handles updates transparently.
**Warning signs:** Tests that edit a message and then search for the new content fail to find it.

### Pitfall 2: Branch Isolation — Messages Must Be Copied, Not Shared
**What goes wrong:** If branch implementation uses a shared message table with a many-to-many join, `listMessages` must always filter by conversation. A shared model requires changing every downstream call. Streaming, edit, retry, and truncate all use `conversationId` as the primary scope.
**Why it happens:** Seems more storage-efficient to reference shared messages.
**How to avoid:** Copy messages on branch creation. Conversations stay independent. Total storage overhead is modest (messages are text, not binary).
**Warning signs:** `truncateMessagesAfter` called in a branched conversation deletes messages that belong to the parent.

### Pitfall 3: Cmd+K Conflict with Existing CommandPalette
**What goes wrong:** Two `document.addEventListener('keydown', ...)` handlers both match `Cmd+K`. Order is nondeterministic; one or both open.
**Why it happens:** The existing `useKeyboardShortcuts` hook handles `Cmd+K` globally and calls `onSearch()` which opens `CommandPalette`. If `ChatSearchDialog` adds its own `Cmd+K` handler, both fire.
**How to avoid:** Route all `Cmd+K` traffic through the existing `onSearch` hook callback. Add a "Search chat" item inside `CommandPalette`, or pass a separate shortcut (Cmd+Shift+F) to a chat-specific search trigger. Do not register a second `Cmd+K` handler.
**Warning signs:** `CommandPalette` and `ChatSearchDialog` both open simultaneously.

### Pitfall 4: Drizzle Schema Drift for Generated Columns
**What goes wrong:** `drizzle-kit generate` is run after adding hand-written migrations for the `tsvector` column. Drizzle Kit sees the schema out of sync and generates a migration that drops/re-adds columns.
**Why it happens:** The Drizzle schema (`chat_messages.ts`) does not declare `contentSearch` as a column, so Drizzle Kit has no record of it.
**How to avoid:** Do not add `contentSearch` to the Drizzle schema TypeScript file (or add it read-only with a `customType`). After adding the migration manually, snapshot the migration metadata to prevent Drizzle Kit from trying to reverse it. Alternatively, add the column to the schema as a `sql` custom type marked as not insertable — this aligns the snapshot without breaking type safety.
**Warning signs:** Running `pnpm db:generate` produces a migration that drops `content_search`.

### Pitfall 5: Export Route Missing Agent Names
**What goes wrong:** The export Markdown includes `agentId` UUIDs instead of human-readable agent names, because `chat_messages` only stores `agentId`, not the agent name.
**Why it happens:** Agent identity is resolved in the UI by joining `agentId` against the agents list. The export service needs to do the same join.
**How to avoid:** The `exportConversation` service method should join `chatMessages` with `agents` to resolve names, or accept an optional map from the caller.
**Warning signs:** Exported Markdown shows UUIDs like `(00000000-0000-0000-0000-000000000001)` as the speaker identity.

### Pitfall 6: Search Overlay Shows Stale Results After Message Edit
**What goes wrong:** After editing a message, a search for the old content still returns the message; searching for the new content does not find it.
**Why it happens:** TanStack Query caches the search result. The query key includes the search term but not the message `updatedAt`.
**How to avoid:** Invalidate `["chat", "search"]` queries whenever a message is edited: add `queryClient.invalidateQueries({ queryKey: ["chat", "search"] })` to the `handleEdit` callback in `ChatPanel`.

---

## Code Examples

### Adding tsvector Generated Column (Migration SQL)

```sql
-- 0051_add_message_search_vector.sql
ALTER TABLE "chat_messages"
  ADD COLUMN "content_search" tsvector
  GENERATED ALWAYS AS (to_tsvector('english', "content")) STORED;

CREATE INDEX "chat_messages_content_search_idx"
  ON "chat_messages" USING GIN ("content_search");
```

### Drizzle Search Query Pattern

```typescript
// server/src/services/chat.ts — searchMessages
import { sql, and, eq, isNull, desc } from "drizzle-orm";

async searchMessages(companyId: string, query: string, opts: { limit?: number }) {
  const limit = Math.min(opts.limit ?? 20, 50);
  const tsQuery = query.trim();
  if (!tsQuery) return { items: [] };

  const rows = await db
    .select({
      messageId: chatMessages.id,
      conversationId: chatMessages.conversationId,
      conversationTitle: chatConversations.title,
      content: chatMessages.content,
      role: chatMessages.role,
      agentId: chatMessages.agentId,
      createdAt: chatMessages.createdAt,
      rank: sql<number>`ts_rank("chat_messages"."content_search", plainto_tsquery('english', ${tsQuery}))`,
    })
    .from(chatMessages)
    .innerJoin(chatConversations, and(
      eq(chatMessages.conversationId, chatConversations.id),
      eq(chatConversations.companyId, companyId),
      isNull(chatConversations.deletedAt),
    ))
    .where(
      sql`"chat_messages"."content_search" @@ plainto_tsquery('english', ${tsQuery})`,
    )
    .orderBy(desc(sql`ts_rank("chat_messages"."content_search", plainto_tsquery('english', ${tsQuery}))`))
    .limit(limit);

  return { items: rows };
}
```

### useChatSearch Hook Pattern

```typescript
// ui/src/hooks/useChatSearch.ts
import { useQuery } from "@tanstack/react-query";
import { chatApi } from "../api/chat";

export function useChatSearch(companyId: string | null, query: string) {
  return useQuery({
    queryKey: ["chat", "search", companyId, query],
    queryFn: () => chatApi.searchMessages(companyId!, query),
    enabled: !!companyId && query.trim().length >= 2,
    placeholderData: (prev) => prev,
  });
}
```

### ChatMessage Bookmark Toggle

```typescript
// Toggling in ChatMessageActions — add onBookmark prop and Bookmark icon
<Button
  variant="ghost"
  size="icon"
  className="h-6 w-6"
  onClick={onBookmark}
  aria-label={isBookmarked ? "Remove bookmark" : "Bookmark message"}
>
  <Bookmark className={cn("h-3.5 w-3.5", isBookmarked && "fill-current")} />
</Button>
```

### Branch Conversation Service Skeleton

```typescript
async branchConversation(
  parentConversationId: string,
  branchFromMessageId: string,
  companyId: string,
) {
  const [branchMsg] = await db
    .select({ createdAt: chatMessages.createdAt })
    .from(chatMessages)
    .where(eq(chatMessages.id, branchFromMessageId));
  if (!branchMsg) throw notFound("Branch message not found");

  const messagesToCopy = await db
    .select()
    .from(chatMessages)
    .where(and(
      eq(chatMessages.conversationId, parentConversationId),
      lte(chatMessages.createdAt, branchMsg.createdAt),
    ))
    .orderBy(asc(chatMessages.createdAt));

  const [newConv] = await db
    .insert(chatConversations)
    .values({
      companyId,
      title: null,
      parentConversationId,
      branchFromMessageId,
    })
    .returning();

  if (messagesToCopy.length > 0) {
    await db.insert(chatMessages).values(
      messagesToCopy.map(({ id: _id, conversationId: _cid, ...rest }) => ({
        ...rest,
        conversationId: newConv!.id,
      })),
    );
  }

  return newConv!;
}
```

---

## State of the Art

| Old Approach | Current Approach | When Changed | Impact |
|--------------|------------------|--------------|--------|
| ILIKE for content search | tsvector + GIN (PostgreSQL FTS) | PostgreSQL 8.3 (2008) | Order-of-magnitude faster at scale |
| Manual tsvector maintenance via triggers | GENERATED ALWAYS AS ... STORED | PostgreSQL 12 (2019) | No triggers, auto-updated on UPDATE |
| cmdk v0.x (uncontrolled) | cmdk v1.x (controlled, `shouldFilter` prop) | cmdk 1.0 (2024) | Must set `shouldFilter={false}` when using server-side search to prevent cmdk's own client-side filter from re-filtering server results |

**Deprecated / outdated:**

- `plainto_tsquery` vs `websearch_to_tsquery`: `websearch_to_tsquery` (Postgres 11+) handles quoted phrases and `-exclusions` like a web search engine. For a basic first implementation, `plainto_tsquery` is simpler and correct. Upgrade to `websearch_to_tsquery` later if users need phrase search.

---

## Open Questions

1. **Branch UI placement**
   - What we know: `ChatConversationItem` shows conversation in the left column of ChatPanel. Branches would be child conversations visible in the same list with a visual indent or branch icon.
   - What's unclear: Whether branches should be nested under the parent in the conversation list or shown as peers with a parent link.
   - Recommendation: Show branches as indented items under the parent in `ChatConversationList`. Add a `parentConversationId` to `ChatConversationListItem` type and group by parent in the UI. Keep the server list endpoint flat (client-side grouping).

2. **Bookmark scope: company vs. per-user**
   - What we know: The project uses board-level (company-scoped) auth. There is no per-user identity surfaced in the chat service — `assertBoard` validates company access but does not expose `userId` to service methods in the current chat service signature.
   - What's unclear: Whether bookmarks should be shared across all users in a workspace or per-user.
   - Recommendation: Scope bookmarks to `companyId` for simplicity (shared bookmarks across the workspace). This matches how conversations are scoped. Per-user bookmarks can be added later when the user model is more prominent.

3. **Search result navigation**
   - What we know: Search results include `conversationId` and `messageId`. Clicking a result should navigate to that conversation and scroll to the message.
   - What's unclear: The existing `ChatPanel` has no mechanism to scroll to a specific message by ID.
   - Recommendation: Add a `scrollToMessageId` state to `ChatPanelContext`. When set, `ChatMessageList` uses the virtualiser's `scrollToIndex` method to jump to the message. Reset after scrolling. This follows the `nexus:focus-chat-search` custom event pattern already used for Cmd+K focus.

---

## Environment Availability

Step 2.6: No new external dependencies identified. PostgreSQL 17 is already the project's database. All npm packages are already installed.

---

## Validation Architecture

### Test Framework
| Property | Value |
|----------|-------|
| Framework | Vitest 3.2.4 |
| Config file | `server/vitest.config.ts` (node env), `ui/vitest.config.ts` (node env) |
| Quick run command | `pnpm --filter @paperclipai/server test run -- chat-service` |
| Full suite command | `pnpm test:run` |

### Phase Requirements → Test Map

| Req ID | Behavior | Test Type | Automated Command | File Exists? |
|--------|----------|-----------|-------------------|-------------|
| CHAT-07 | `searchMessages` returns ranked results for matching term | unit | `pnpm --filter @paperclipai/server test run -- chat-service` | ✅ (new describe block in existing file) |
| CHAT-07 | `GET /companies/:id/messages/search?q=term` returns 200 with results | integration | `pnpm --filter @paperclipai/server test run -- chat-routes` | ✅ (new describe block in existing file) |
| CHAT-13 | `toggleBookmark` inserts or removes bookmark row | unit | `pnpm --filter @paperclipai/server test run -- chat-service` | ✅ |
| CHAT-13 | `GET /companies/:id/bookmarks` returns bookmarked messages | integration | `pnpm --filter @paperclipai/server test run -- chat-routes` | ✅ |
| CHAT-14 | `branchConversation` creates new conversation with copied messages | unit | `pnpm --filter @paperclipai/server test run -- chat-service` | ✅ |
| CHAT-14 | `POST /conversations/:id/branch` returns 201 with new conversation | integration | `pnpm --filter @paperclipai/server test run -- chat-routes` | ✅ |
| HIST-04 | `exportConversation` returns correct Markdown structure | unit | `pnpm --filter @paperclipai/server test run -- chat-service` | ✅ |
| HIST-04 | `GET /conversations/:id/export?format=markdown` returns file download headers | integration | `pnpm --filter @paperclipai/server test run -- chat-routes` | ✅ |
| PERF-04 | GIN index query plan uses index scan (not seq scan) | manual/DB | `EXPLAIN ANALYZE ...` in Postgres | ❌ Wave 0 — manual verification |

### Sampling Rate
- **Per task commit:** `pnpm --filter @paperclipai/server test run -- chat-service`
- **Per wave merge:** `pnpm test:run`
- **Phase gate:** Full suite green before `/gsd:verify-work`

### Wave 0 Gaps
- [ ] New `describe("searchMessages")` block in `server/src/__tests__/chat-service.test.ts` — covers CHAT-07
- [ ] New `describe("toggleBookmark / getBookmarks")` block in `server/src/__tests__/chat-service.test.ts` — covers CHAT-13
- [ ] New `describe("branchConversation")` block in `server/src/__tests__/chat-service.test.ts` — covers CHAT-14
- [ ] New `describe("exportConversation")` block in `server/src/__tests__/chat-service.test.ts` — covers HIST-04
- [ ] New route-level describe blocks in `server/src/__tests__/chat-routes.test.ts` — covers all four route groups

---

## Project Constraints (from CLAUDE.md)

CLAUDE.md does not exist at `/opt/nexus/CLAUDE.md`. No additional project-level directives to document.

**Codebase conventions observed from prior phases:**

- Use `object-syntax (table) => ({})` for Drizzle index callbacks (not arrow-returning-object shorthand).
- Use `it.todo()` (not `it.skip()`) for Wave 0 test scaffolding.
- Use `@/lib/router` Link abstraction for navigation, not `react-router-dom` directly.
- Use `useToast()/pushToast()` for error toasts — not `sonner`.
- DB schema files are individual per table; exported from `packages/db/src/schema/index.ts`.
- Migrations are hand-numbered (`0050_`, `0051_`, ...) and journaled in `meta/_journal.json`.
- Shared types in `packages/shared/src/types/chat.ts`; validators in `packages/shared/src/validators/chat.ts`; both re-exported from `packages/shared/src/index.ts`.
- Custom window events (e.g. `nexus:focus-chat-search`) are the project pattern for decoupled cross-component communication.
- The `assertBoard(req)` + `assertCompanyAccess(req, companyId)` guard pattern is required on all chat routes.
- Service functions are factory functions `chatService(db)` returning a plain object — not classes.

---

## Sources

### Primary (HIGH confidence)
- Direct codebase inspection — all schema, service, route, and UI files read from `/opt/nexus/`
- `packages/db/src/schema/chat_conversations.ts` — confirmed existing columns
- `packages/db/src/schema/chat_messages.ts` — confirmed no existing tsvector
- `server/src/services/chat.ts` — confirmed ilike-only current search
- `server/src/routes/chat.ts` — confirmed route patterns
- `ui/src/components/CommandPalette.tsx` — confirmed cmdk usage and Cmd+K binding
- `ui/src/hooks/useKeyboardShortcuts.ts` — confirmed Cmd+K conflict point
- `.planning/ROADMAP.md` Phase 24 — canonical requirement list (CHAT-07, CHAT-13, CHAT-14, HIST-04, PERF-04)
- `.planning/codebase/STACK.md` — confirmed PostgreSQL 17, Drizzle 0.38.x, cmdk 1.1.1

### Secondary (MEDIUM confidence)
- PostgreSQL 12 documentation — GENERATED ALWAYS AS STORED columns
- PostgreSQL FTS documentation — tsvector, GIN indexes, plainto_tsquery, ts_rank

### Tertiary (LOW confidence, flag for validation)
- Drizzle ORM 0.38.x generated column support — my training data indicates incomplete support; verified indirectly by absence of `generatedAlwaysAs` usage in the codebase

---

## Metadata

**Confidence breakdown:**
- Standard stack: HIGH — all libraries confirmed present in codebase
- Architecture: HIGH — patterns derived directly from existing Phases 21–23 code
- Search implementation: HIGH — PostgreSQL FTS is well-established; tsvector generated column confirmed supported in PG17
- Drizzle generated column handling: MEDIUM — Drizzle Kit limitation confirmed by absence of existing usage; hand-written migration approach is the safe path
- Pitfalls: HIGH — derived from direct code inspection of conflict points

**Research date:** 2026-04-01
**Valid until:** 2026-05-01 (stable stack; no fast-moving dependencies)
