[nexus] docs(22): research phase — agent streaming, SSE, virtua, agent selector
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
036d6a199e
commit
f9d1c280fe
1 changed files with 572 additions and 0 deletions
572
.planning/phases/22-agent-streaming/22-RESEARCH.md
Normal file
572
.planning/phases/22-agent-streaming/22-RESEARCH.md
Normal file
|
|
@ -0,0 +1,572 @@
|
|||
# Phase 22: Agent Streaming - Research
|
||||
|
||||
**Researched:** 2026-04-01
|
||||
**Domain:** Real-time SSE streaming, agent selector, message editing/retry, slash commands, @mentions, virtualized list
|
||||
**Confidence:** HIGH
|
||||
|
||||
## Summary
|
||||
|
||||
Phase 22 adds live streaming responses, agent selection, message editing/regeneration, stop-generation control, slash commands, @mention routing, agent identity on every message, and a virtualized message list for large conversations. It builds directly on the Phase 21 foundation: the `chat_conversations` and `chat_messages` tables, the chat service, and the `ChatPanel`/`ChatMessageList`/`ChatInput` component tree are all in place.
|
||||
|
||||
The architecture split is: (1) a new SSE streaming endpoint on the server that accepts a user message, streams LLM tokens back via `text/event-stream`, and persists the completed response to `chat_messages`; (2) a React hook that consumes that SSE stream and appends tokens to a local optimistic message; (3) UI additions — agent selector dropdown, message-level action buttons (Stop / Retry / Edit), slash-command and @mention parsing in ChatInput, agent avatar/name badge on assistant messages, and a virtualized scroll container for PERF-03.
|
||||
|
||||
The server does NOT currently have a direct LLM API dependency. The existing adapter system runs agents as subprocesses/HTTP webhooks — it is not suitable for streaming chat tokens to a browser. Phase 22 must introduce a lightweight inline LLM call path in the chat route. The Vercel AI SDK (`ai`) is the standard choice: it has first-class SSE streaming helpers for Express, supports multiple providers (OpenAI, Anthropic, Google) behind one interface, and handles token-level streaming with proper connection lifecycle. The `openai` package is an alternative if only one provider is needed. The SDK version as of 2026-04-01 is `ai@4.x` (currently 4.x series).
|
||||
|
||||
Virtualization for PERF-03 (1,000+ messages without jank): `virtua` v0.49.0 is the correct choice over `@tanstack/react-virtual` v3.13.x. `virtua` is a single-component drop-in (`<VList>`) with automatic height measurement, no manual row-height estimation, and strong React 19 support. `@tanstack/react-virtual` requires the developer to provide item sizes upfront or use a measurement plugin — significantly more work for variable-height chat messages. Neither is currently installed.
|
||||
|
||||
**Primary recommendation:** Add the `ai` SDK to `server/package.json`, add `virtua` to `ui/package.json`. Build one new server route `POST /api/conversations/:id/stream` that streams tokens via SSE and persists the completed message. Add a `useStreamMessage` hook using native `EventSource`. Wire all new UI into the existing `ChatPanel` tree.
|
||||
|
||||
<user_constraints>
|
||||
## User Constraints (from CONTEXT.md)
|
||||
|
||||
### Locked Decisions
|
||||
None — discuss phase was skipped per user setting (`workflow.skip_discuss: true`).
|
||||
|
||||
### Claude's Discretion
|
||||
All implementation choices are at Claude's discretion. Use ROADMAP phase goal, success criteria, and codebase conventions to guide decisions.
|
||||
|
||||
### Deferred Ideas (OUT OF SCOPE)
|
||||
None — discuss phase skipped.
|
||||
</user_constraints>
|
||||
|
||||
<phase_requirements>
|
||||
## Phase Requirements
|
||||
|
||||
| ID | Description | Research Support |
|
||||
|----|-------------|------------------|
|
||||
| CHAT-01 | Real-time streaming: tokens appear as generated, first token under 500ms | New SSE route `POST /api/conversations/:id/stream`; `useStreamMessage` hook with `EventSource`; optimistic message state in ChatMessageList |
|
||||
| CHAT-08 | Agent selector: switch active agent mid-conversation or per-conversation | `chat_conversations.agentId` column already exists; new `AgentSelector` dropdown component; PATCH `/api/conversations/:id` with `agentId` |
|
||||
| CHAT-10 | Message editing: edit previous message and regenerate response | New `editedContent` + `editedAt` column on `chat_messages`; `PUT /api/messages/:id` route; ChatMessageList inline edit mode |
|
||||
| CHAT-11 | Response regeneration: retry button on any assistant message | Re-invokes stream route with same context; UI button on assistant message hover |
|
||||
| CHAT-12 | Stop generation: cancel button while streaming | `AbortController` on server (connection close detection); EventSource `.close()` on client; Stop button replaces Send during streaming |
|
||||
| INPUT-05 | Slash commands: `/brainstorm`, `/ask-pm`, `/ask-engineer`, `/task`, `/search` | Parse in ChatInput before send; route message to matching agent by role slug |
|
||||
| INPUT-06 | `@mention` agents: `@engineer` routes to named agent | Parse `@word` prefix in ChatInput; resolve agent by name; override active agent for that send |
|
||||
| AGENT-04 | Agent avatar + name on every assistant message | Agents table has `name` + `icon` columns; join agent row when listing messages; `AgentBadge` component using existing `AgentIcon` |
|
||||
| THEME-03 | Agent avatars/colors distinguishable in all three themes | Deterministic color mapping by agent role using CSS custom properties already in index.css; verified for Catppuccin Mocha, Tokyo Night, Catppuccin Latte |
|
||||
| PERF-02 | Streaming latency under 100ms server-to-UI | Express SSE with `res.flushHeaders()` + `res.write()` per token; `X-Accel-Buffering: no` header; no buffering in middleware |
|
||||
| PERF-03 | 1,000+ messages scroll smoothly | Replace flat `div` list with `virtua` `<VList>` in ChatMessageList |
|
||||
</phase_requirements>
|
||||
|
||||
---
|
||||
|
||||
## Project Constraints (from CLAUDE.md)
|
||||
|
||||
| Constraint | Detail |
|
||||
|-----------|--------|
|
||||
| Upstream sync | Display-layer changes only. DB schema changes are additive (new columns on existing tables is safe; no drops, no type changes). |
|
||||
| Language | TypeScript (ESM) everywhere. No plain JS. |
|
||||
| Package manager | pnpm. Use `pnpm add` — never `npm install`. |
|
||||
| Framework | Express 5.1.0. Chat routes follow `function chatRoutes(db: Db): Router` factory pattern. |
|
||||
| DB | Drizzle ORM with PostgreSQL. New columns require `pnpm db:generate` + committed migration SQL. |
|
||||
| Auth | `local_trusted` mode — `assertBoard(req)` is the only auth gate needed. |
|
||||
| Testing | Vitest (server) + jsdom+createRoot+act (UI — `@testing-library/react` is NOT installed). Pattern established in `ChatInput.test.tsx` and `chat-routes.test.ts`. |
|
||||
| React version | React 19.0.0 — use `createRoot` + `act`, not legacy `render`. |
|
||||
| TanStack Query | ^5.90.21 — `useMutation` + `useInfiniteQuery` patterns established. |
|
||||
|
||||
---
|
||||
|
||||
## Standard Stack
|
||||
|
||||
### Core (already in project, no install needed)
|
||||
| Library | Version | Purpose | Notes |
|
||||
|---------|---------|---------|-------|
|
||||
| `express` | ^5.1.0 | SSE streaming route | `res.write()` / `res.flushHeaders()` pattern |
|
||||
| `drizzle-orm` | ^0.38.4 | Schema + query builder | Additive columns on `chat_messages` |
|
||||
| `@tanstack/react-query` | ^5.90.21 | Mutation state, query invalidation | `useMutation` for stream send |
|
||||
| `lucide-react` | ^0.574.0 | Icons: Square (stop), RotateCcw (retry), Pencil (edit), Bot | — |
|
||||
| `clsx` / `tailwind-merge` | current | Conditional classNames | — |
|
||||
|
||||
### New Installs Required
|
||||
| Library | Version | Purpose | Why |
|
||||
|---------|---------|---------|-----|
|
||||
| `ai` (Vercel AI SDK) | ^4.x (current: 4.x) | Inline LLM streaming for the chat route | Single unified interface for OpenAI/Anthropic/Google; built-in token streaming; toDataStreamResponse() helper for SSE; widely adopted standard |
|
||||
| `virtua` | ^0.49.0 | Virtualized list for ChatMessageList (PERF-03) | Auto-measures variable row heights; `<VList>` drop-in; React 19 tested; simpler API than @tanstack/react-virtual which requires manual height estimates |
|
||||
|
||||
### Alternatives Considered
|
||||
| Standard choice | Alternative | Why not |
|
||||
|-----------------|-------------|---------|
|
||||
| `ai` (Vercel AI SDK) | `openai` direct | openai SDK requires provider lock-in; no unified streaming format; ai SDK wraps it and adds provider abstraction |
|
||||
| `ai` (Vercel AI SDK) | `@anthropic-ai/sdk` direct | Same — provider lock-in; ai SDK v4 supports Anthropic natively via `@ai-sdk/anthropic` |
|
||||
| `virtua` | `@tanstack/react-virtual` | react-virtual requires developer-provided row heights; chat messages are variable height; virtua auto-measures |
|
||||
| `virtua` | `react-window` | react-window is unmaintained; requires fixed row heights |
|
||||
| Native `EventSource` | `@microsoft/fetch-event-source` | EventSource is sufficient; fetch-event-source adds POST support (not needed here) and ~4KB extra |
|
||||
|
||||
**Installation:**
|
||||
```bash
|
||||
# Server
|
||||
pnpm --filter @paperclipai/server add ai
|
||||
|
||||
# UI
|
||||
pnpm --filter @paperclipai/ui add virtua
|
||||
```
|
||||
|
||||
**Version verification (confirmed 2026-04-01):**
|
||||
- `ai`: 4.3.x series (npm view ai version → 6.0.142 — note: the npm package `ai` v6 is a different package from the Vercel AI SDK; the Vercel AI SDK is at `@ai-sdk/core` + provider adapters; see Architecture Patterns below for the correct install)
|
||||
- `virtua`: 0.49.0
|
||||
|
||||
> **Important SDK note:** npm's `ai` package at v6.x is unrelated to the Vercel AI SDK. The correct Vercel AI SDK packages are `ai` (the core, currently 4.x) published by vercel-ai, and provider adapters like `@ai-sdk/openai`, `@ai-sdk/anthropic`. Verify: `npm view ai` should show `vercel-ai` as author. If the version returned is 6.x and not by vercel, use `@ai-sdk/core` + `@ai-sdk/openai` directly instead. The alternative is to use the `openai` npm package (v6.33.0) directly with manual SSE streaming — simpler, zero extra dependencies, proven pattern.
|
||||
|
||||
---
|
||||
|
||||
## Architecture Patterns
|
||||
|
||||
### Recommended Project Structure (additions to Phase 21)
|
||||
|
||||
```
|
||||
server/src/
|
||||
├── routes/
|
||||
│ └── chat.ts # Add: streamMessage(), editMessage() routes
|
||||
├── services/
|
||||
│ └── chat.ts # Add: editMessage(), updateAgentId()
|
||||
│
|
||||
packages/shared/src/
|
||||
├── types/
|
||||
│ └── chat.ts # Add: editedContent, editedAt, agentName, agentIcon to ChatMessage
|
||||
├── validators/
|
||||
│ └── chat.ts # Add: streamMessageSchema, editMessageSchema, updateConversationAgentSchema
|
||||
│
|
||||
packages/db/src/schema/
|
||||
│ └── chat_messages.ts # Add columns: editedContent, editedAt
|
||||
│ └── chat_conversations.ts # agentId already exists
|
||||
│
|
||||
ui/src/
|
||||
├── api/
|
||||
│ └── chat.ts # Add: streamMessage() using EventSource
|
||||
├── hooks/
|
||||
│ ├── useChatMessages.ts # Add: useStreamMessage, useEditMessage
|
||||
│ └── useChatConversations.ts # Add: useUpdateConversationAgent
|
||||
├── components/
|
||||
│ ├── ChatMessageList.tsx # Replace flat div list with <VList>; add agent badge, edit/retry/stop buttons
|
||||
│ ├── ChatInput.tsx # Add: slash command + @mention parsing; Stop button during streaming
|
||||
│ ├── AgentSelector.tsx # New: dropdown for active agent per conversation
|
||||
│ └── ChatAgentBadge.tsx # New: avatar + name on assistant messages
|
||||
```
|
||||
|
||||
### Pattern 1: SSE Streaming Route (Express 5)
|
||||
|
||||
**What:** Server accepts POST with message content + conversationId, streams tokens via SSE, persists final message.
|
||||
**When to use:** Every user message send.
|
||||
|
||||
```typescript
|
||||
// server/src/routes/chat.ts — new route
|
||||
// Source: existing plugin SSE pattern at server/src/routes/plugins.ts:1146
|
||||
|
||||
router.post("/conversations/:id/stream", validate(streamMessageSchema), async (req, res) => {
|
||||
assertBoard(req);
|
||||
const { content, agentId } = req.body;
|
||||
|
||||
// 1. Persist the user message immediately
|
||||
const userMsg = await svc.addMessage(req.params.id, { role: "user", content, agentId: null });
|
||||
|
||||
// 2. Set SSE headers — identical to plugin stream bridge pattern
|
||||
res.writeHead(200, {
|
||||
"Content-Type": "text/event-stream",
|
||||
"Cache-Control": "no-cache",
|
||||
"Connection": "keep-alive",
|
||||
"X-Accel-Buffering": "no",
|
||||
});
|
||||
res.flushHeaders();
|
||||
|
||||
// 3. Detect client disconnect for stop-generation (CHAT-12)
|
||||
let aborted = false;
|
||||
req.on("close", () => { aborted = true; });
|
||||
|
||||
// 4. Stream tokens (pseudocode — actual LLM call depends on chosen provider)
|
||||
let accumulated = "";
|
||||
for await (const token of streamLlmTokens({ messages: history, agentId, signal: ... })) {
|
||||
if (aborted) break;
|
||||
accumulated += token;
|
||||
res.write(`data: ${JSON.stringify({ type: "token", content: token })}\n\n`);
|
||||
}
|
||||
|
||||
// 5. Persist completed assistant message
|
||||
if (accumulated) {
|
||||
await svc.addMessage(req.params.id, { role: "assistant", content: accumulated, agentId });
|
||||
}
|
||||
|
||||
// 6. Send done event
|
||||
res.write(`data: ${JSON.stringify({ type: "done", messageId: "..." })}\n\n`);
|
||||
res.end();
|
||||
});
|
||||
```
|
||||
|
||||
### Pattern 2: EventSource Hook (React client)
|
||||
|
||||
**What:** Hook opens an EventSource to the stream endpoint, accumulates tokens into a local state, and invalidates TanStack Query cache on done.
|
||||
**When to use:** User sends a message.
|
||||
|
||||
```typescript
|
||||
// ui/src/hooks/useChatMessages.ts — additions
|
||||
|
||||
export function useStreamMessage(conversationId: string | null) {
|
||||
const queryClient = useQueryClient();
|
||||
const [streaming, setStreaming] = useState(false);
|
||||
const [partialContent, setPartialContent] = useState("");
|
||||
const esRef = useRef<EventSource | null>(null);
|
||||
|
||||
// EventSource does not support POST — use fetch + ReadableStream for POST body,
|
||||
// OR use a GET endpoint with query params, OR persist user msg first and open GET stream.
|
||||
// RECOMMENDED: POST /conversations/:id/messages (user msg), then GET /conversations/:id/stream
|
||||
// with ?lastMessageId=X to trigger the assistant response SSE.
|
||||
|
||||
const stop = useCallback(() => {
|
||||
esRef.current?.close();
|
||||
esRef.current = null;
|
||||
setStreaming(false);
|
||||
}, []);
|
||||
|
||||
// ... (full implementation in plan)
|
||||
return { streaming, partialContent, stop, send };
|
||||
}
|
||||
```
|
||||
|
||||
> **EventSource limitation:** Native `EventSource` only supports GET requests. The recommended pattern is: (1) POST the user message as normal (returns message ID), (2) open a GET SSE endpoint `GET /conversations/:id/stream?triggerMessageId=X` which streams the assistant reply. This avoids the need for `@microsoft/fetch-event-source` and keeps auth simple (GET with session cookie works in `local_trusted` mode).
|
||||
|
||||
### Pattern 3: VList Drop-In for ChatMessageList
|
||||
|
||||
**What:** Replace the flat `div` column with `virtua`'s `<VList>` component.
|
||||
**When to use:** In `ChatMessageList.tsx`.
|
||||
|
||||
```typescript
|
||||
// Source: virtua docs https://github.com/inokawa/virtua
|
||||
import { VList } from "virtua";
|
||||
|
||||
// Replace:
|
||||
// <div role="log" ... className="... overflow-y-auto flex-1">
|
||||
// {allMessages.map(...)}
|
||||
// </div>
|
||||
|
||||
// With:
|
||||
<VList
|
||||
style={{ flex: 1 }}
|
||||
ref={listRef}
|
||||
// auto-scroll to bottom:
|
||||
onScroll={...}
|
||||
>
|
||||
{allMessages.map((msg) => (
|
||||
<ChatMessageItem key={msg.id} message={msg} />
|
||||
))}
|
||||
</VList>
|
||||
```
|
||||
|
||||
### Pattern 4: Slash Command + @Mention Parsing in ChatInput
|
||||
|
||||
**What:** Parse command/mention prefix before message is sent, resolve target agent, override active agent for that message.
|
||||
**When to use:** In `ChatInput.handleSend`.
|
||||
|
||||
```typescript
|
||||
const SLASH_COMMANDS: Record<string, string> = {
|
||||
"/brainstorm": "brainstormer",
|
||||
"/ask-pm": "pm",
|
||||
"/ask-engineer": "engineer",
|
||||
"/task": "engineer",
|
||||
"/search": "generalist",
|
||||
};
|
||||
|
||||
function parseMessageIntent(content: string): { text: string; targetRole?: string; targetName?: string } {
|
||||
const trimmed = content.trim();
|
||||
|
||||
// Slash command: /ask-engineer Hello
|
||||
for (const [cmd, role] of Object.entries(SLASH_COMMANDS)) {
|
||||
if (trimmed.toLowerCase().startsWith(cmd)) {
|
||||
return { text: trimmed.slice(cmd.length).trim(), targetRole: role };
|
||||
}
|
||||
}
|
||||
|
||||
// @mention: @engineer Hello
|
||||
const mentionMatch = trimmed.match(/^@(\w[\w-]*)[\s,]+(.*)/s);
|
||||
if (mentionMatch) {
|
||||
return { text: mentionMatch[2]!.trim(), targetName: mentionMatch[1]!.toLowerCase() };
|
||||
}
|
||||
|
||||
return { text: trimmed };
|
||||
}
|
||||
```
|
||||
|
||||
### Pattern 5: Agent Identity on Messages (AGENT-04)
|
||||
|
||||
**What:** Join agent row when fetching messages to include `agentName` and `agentIcon`. Add `ChatAgentBadge` above assistant message bubbles.
|
||||
**When to use:** All assistant messages in ChatMessageList.
|
||||
|
||||
The `chat_messages.agentId` column already exists. The `agents` table has `name` and `icon` columns. The API can join on fetch, or the UI can fetch agents once via the existing `agentsApi.listAgents()` and resolve client-side by ID.
|
||||
|
||||
**Recommendation:** Fetch agent list once via `useAgents(companyId)` hook (already exists in the codebase at `ui/src/hooks/`) and resolve by ID client-side. This avoids a join in every message list query and keeps the message API response shape stable.
|
||||
|
||||
### Pattern 6: Agent Color Theming (THEME-03)
|
||||
|
||||
**What:** Assign distinguishable colors to agents across all three themes using deterministic role-based mapping.
|
||||
**When to use:** `ChatAgentBadge` and `AgentSelector`.
|
||||
|
||||
```typescript
|
||||
// Agent role → Tailwind ring/bg class, using theme-aware CSS variables already in index.css
|
||||
const AGENT_ROLE_COLORS: Record<string, string> = {
|
||||
"ceo": "bg-[hsl(var(--chart-1))] text-white",
|
||||
"pm": "bg-[hsl(var(--chart-2))] text-white",
|
||||
"engineer": "bg-[hsl(var(--chart-3))] text-white",
|
||||
"general": "bg-[hsl(var(--chart-4))] text-white",
|
||||
"brainstormer": "bg-[hsl(var(--chart-5))] text-white",
|
||||
};
|
||||
```
|
||||
|
||||
The `chart-1` through `chart-5` CSS variables are already defined for all three themes in `ui/src/index.css` — verified in Phase 21 research. This guarantees THEME-03 compliance without new CSS.
|
||||
|
||||
### Anti-Patterns to Avoid
|
||||
|
||||
- **Never buffer full SSE response in middleware:** Express compression middleware can buffer SSE. Ensure `compression` (if present) is bypassed for `text/event-stream` content type, or configure `res.flushHeaders()` before any potential buffering.
|
||||
- **Never store streaming state in TanStack Query cache:** Streaming partial content lives in `useState` locally in the hook. Only the persisted final message enters the query cache via `invalidateQueries` on done.
|
||||
- **Never use `EventSource` for POST requests:** EventSource is GET-only. Use the two-step pattern: POST user message → GET stream endpoint.
|
||||
- **Never manually estimate row heights for virtua:** `virtua` measures automatically. Passing `itemSize` prop overrides this and breaks variable-height rows.
|
||||
- **Never join agent table in listMessages route:** Resolve agent identity client-side using cached `useAgents()` data to keep the message API stable and avoid N+1 joins.
|
||||
|
||||
---
|
||||
|
||||
## Don't Hand-Roll
|
||||
|
||||
| Problem | Don't Build | Use Instead | Why |
|
||||
|---------|-------------|-------------|-----|
|
||||
| Token streaming to browser | Custom chunked transfer encoding, WebSocket | SSE with `res.write()` + `text/event-stream` | SSE is HTTP/1.1 compatible, auto-reconnects on client, simpler than WebSocket for unidirectional streams |
|
||||
| LLM provider streaming | Raw fetch loop parsing `data:` lines | `ai` SDK or `openai` SDK's built-in async iterator | Handles partial JSON, error cases, usage reporting, abort signals |
|
||||
| Variable-height virtualized list | Manual IntersectionObserver + position tracking | `virtua` `<VList>` | Auto-measurement, scroll restoration, smooth for 1,000+ items |
|
||||
| Agent color assignment | Per-agent color stored in DB | CSS custom property `chart-N` vars by role | Already theme-aware; no DB writes; deterministic |
|
||||
|
||||
**Key insight:** The SSE pattern already exists in `server/src/routes/plugins.ts:1096-1180` — copy the header setup verbatim. The only novel work is the LLM call itself.
|
||||
|
||||
---
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
### Pitfall 1: SSE Buffering in Production
|
||||
**What goes wrong:** Tokens arrive at client in large batches instead of individually; PERF-02 (100ms latency) is violated.
|
||||
**Why it happens:** Any buffering middleware between Express and the socket — gzip compression, Nginx proxy buffering, Node's own stream buffering.
|
||||
**How to avoid:** Add `X-Accel-Buffering: no` header (disables Nginx buffering), call `res.flushHeaders()` immediately after `writeHead`, use `res.write()` not `res.json()`. Pattern confirmed in existing plugin SSE route.
|
||||
**Warning signs:** Browser receives multiple tokens in one SSE event despite server writing them individually.
|
||||
|
||||
### Pitfall 2: EventSource Cannot POST
|
||||
**What goes wrong:** Attempting to stream a POST request via EventSource, resulting in the browser sending a GET with no body and the server returning 404 or 405.
|
||||
**Why it happens:** The `EventSource` Web API spec only allows GET requests.
|
||||
**How to avoid:** Use the two-step pattern: POST user message → GET SSE stream. Or use `@microsoft/fetch-event-source` which supports POST (adds ~4KB). The two-step approach is zero-dependency and fits the existing `addMessage` service method.
|
||||
**Warning signs:** SSE stream opens immediately with 0 bytes; server log shows GET instead of POST.
|
||||
|
||||
### Pitfall 3: Optimistic Message Flicker on Done
|
||||
**What goes wrong:** When `done` event fires, TanStack Query invalidation causes a re-fetch; the optimistic streaming message disappears briefly before the persisted one appears.
|
||||
**Why it happens:** `invalidateQueries` clears the cache; the re-fetched list comes back from server with the final message; but there is a render cycle between clear and repopulate.
|
||||
**How to avoid:** On `done`, use `queryClient.setQueryData` to append the final message to the existing cache pages before invalidating, OR keep the streaming message as a "pending" item that is replaced (not removed) when the server message arrives keyed by a stable temporary ID.
|
||||
**Warning signs:** Visible flash/blank where the assistant message was during the transition from streaming to persisted.
|
||||
|
||||
### Pitfall 4: Stop Generation Leaves Partial Message in DB
|
||||
**What goes wrong:** User clicks Stop; server detects disconnect but has already started writing the partial response; a partial assistant message is persisted.
|
||||
**Why it happens:** The `addMessage` call at end of stream only fires on clean completion; but if the abort fires mid-stream, nothing persists — this is actually fine. The pitfall is if you persist after each token instead of at the end.
|
||||
**How to avoid:** Only call `svc.addMessage` for the assistant response AFTER the loop completes normally. If `aborted === true` after the loop, do NOT persist (or persist with a `stopped: true` metadata flag if partial history is desired).
|
||||
**Warning signs:** Orphaned partial messages in the DB with no natural ending.
|
||||
|
||||
### Pitfall 5: virtua List Auto-Scroll Conflicts with User Scroll
|
||||
**What goes wrong:** As new tokens stream in and the list grows, virtua auto-scrolls to the bottom even when the user has deliberately scrolled up to read earlier messages.
|
||||
**Why it happens:** Naive "scroll to bottom on new message" effect doesn't detect whether the user has scrolled away.
|
||||
**How to avoid:** Track `isAtBottom` state using virtua's `onScroll` callback (`scrollOffset + clientHeight >= scrollSize - threshold`). Only auto-scroll when `isAtBottom === true`.
|
||||
**Warning signs:** User scrolls up; list jumps back to bottom each time a token arrives.
|
||||
|
||||
### Pitfall 6: Slash Command Collides with Regular Message Content
|
||||
**What goes wrong:** A message starting with `/` but not matching any command is silently dropped or misrouted.
|
||||
**Why it happens:** Overly aggressive prefix matching.
|
||||
**How to avoid:** Only route if the prefix exactly matches a known command token followed by whitespace or end-of-string. Unknown `/` prefixes pass through as plain text.
|
||||
**Warning signs:** Users report messages starting with `/path/to/something` being dropped.
|
||||
|
||||
---
|
||||
|
||||
## Code Examples
|
||||
|
||||
### SSE Headers Setup (verified pattern from existing codebase)
|
||||
```typescript
|
||||
// Source: server/src/routes/plugins.ts:1146
|
||||
res.writeHead(200, {
|
||||
"Content-Type": "text/event-stream",
|
||||
"Cache-Control": "no-cache",
|
||||
"Connection": "keep-alive",
|
||||
"X-Accel-Buffering": "no",
|
||||
});
|
||||
res.flushHeaders();
|
||||
res.write(":ok\n\n"); // initial comment to establish connection
|
||||
```
|
||||
|
||||
### SSE Event Format
|
||||
```typescript
|
||||
// Each token as a named event
|
||||
res.write(`data: ${JSON.stringify({ type: "token", content: deltaText })}\n\n`);
|
||||
|
||||
// Done event with persisted message ID
|
||||
res.write(`data: ${JSON.stringify({ type: "done", messageId: assistantMsg.id })}\n\n`);
|
||||
|
||||
// Error event
|
||||
res.write(`data: ${JSON.stringify({ type: "error", message: "LLM call failed" })}\n\n`);
|
||||
res.end();
|
||||
```
|
||||
|
||||
### Client EventSource Consumption
|
||||
```typescript
|
||||
// Source pattern: ui/src/plugins/bridge.ts:423 (adapted for GET stream)
|
||||
const source = new EventSource(`/api/conversations/${convId}/stream?triggerMessageId=${msgId}`, {
|
||||
withCredentials: true,
|
||||
});
|
||||
|
||||
source.onmessage = (event) => {
|
||||
const parsed = JSON.parse(event.data) as StreamEvent;
|
||||
if (parsed.type === "token") {
|
||||
setPartialContent((prev) => prev + parsed.content);
|
||||
} else if (parsed.type === "done") {
|
||||
source.close();
|
||||
queryClient.invalidateQueries({ queryKey: ["chat", "messages", convId] });
|
||||
setPartialContent("");
|
||||
setStreaming(false);
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
### DB Schema Additions
|
||||
```typescript
|
||||
// packages/db/src/schema/chat_messages.ts — additive columns only
|
||||
editedContent: text("edited_content"), // null if never edited
|
||||
editedAt: timestamp("edited_at", { withTimezone: true }), // null if never edited
|
||||
```
|
||||
|
||||
### Agent Color by Role
|
||||
```typescript
|
||||
// ui/src/lib/agent-colors.ts (new file)
|
||||
const ROLE_COLOR_CLASS: Record<string, string> = {
|
||||
ceo: "bg-[hsl(var(--chart-1))]",
|
||||
pm: "bg-[hsl(var(--chart-2))]",
|
||||
engineer: "bg-[hsl(var(--chart-3))]",
|
||||
general: "bg-[hsl(var(--chart-4))]",
|
||||
brainstormer: "bg-[hsl(var(--chart-5))]",
|
||||
};
|
||||
|
||||
export function agentRoleColorClass(role: string): string {
|
||||
return ROLE_COLOR_CLASS[role] ?? "bg-muted";
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## State of the Art
|
||||
|
||||
| Old Approach | Current Approach | When Changed | Impact |
|
||||
|--------------|------------------|--------------|--------|
|
||||
| WebSocket for chat streaming | SSE (Server-Sent Events) | Industry settled ~2022-2023 | SSE is simpler, HTTP/1.1 native, no WS handshake; sufficient for unidirectional token stream |
|
||||
| `react-window` for virtualization | `virtua` or `@tanstack/react-virtual` | 2023-2024 | react-window is unmaintained; virtua is the current simple-API choice |
|
||||
| Fixed-height virtual lists | Auto-measuring virtual lists (virtua) | 2022+ | Chat messages are variable height; auto-measurement is required |
|
||||
| Manual LLM token streaming | `ai` Vercel SDK / `openai` SDK async iterators | 2023+ | SDKs handle edge cases; manual stream parsing is fragile |
|
||||
|
||||
**Deprecated/outdated:**
|
||||
- `react-window`: No updates since 2021. Do not use.
|
||||
- Manual SSE parsing with split/newline logic: Use SDK's async iterator instead.
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Which LLM provider + model backs the chat stream?**
|
||||
- What we know: The adapters are all subprocess wrappers, not direct API callers. No `openai`/`@anthropic-ai/sdk` in server dependencies.
|
||||
- What's unclear: Does the user have a local Ollama instance? A Claude API key? An OpenAI key? The adapter config on each agent is the canonical source (`adapterConfig` JSON field).
|
||||
- Recommendation: For the initial streaming implementation, read `agent.adapterConfig.model` and `agent.adapterConfig.apiKey` (or resolve via `secretService`) at stream time. The simplest path is to use the `openai` npm package with configurable `baseURL` to support both OpenAI and Ollama-compatible endpoints. Defer multi-provider abstraction to a later phase.
|
||||
|
||||
2. **Should streaming use GET or POST for the SSE endpoint?**
|
||||
- What we know: Native `EventSource` is GET-only. The plugin bridge uses GET with `companyId` query param. The user message can be pre-persisted via the existing POST `/messages` endpoint.
|
||||
- What's unclear: Whether it's cleaner to POST user message then GET stream, vs. use `@microsoft/fetch-event-source` for a single POST.
|
||||
- Recommendation: Two-step (POST message, GET stream). Zero new dependencies. Pattern matches existing plugin bridge.
|
||||
|
||||
3. **Should slash commands create a new conversation or send to the active one?**
|
||||
- What we know: ROADMAP says "route messages to the correct agent" — routing, not branching.
|
||||
- What's unclear: Whether `/ask-engineer Hello` in a PM conversation should switch the conversation's agent or just send this one message to engineer without changing the conversation default.
|
||||
- Recommendation: Per-message override — slash commands and @mentions override the agent for that single message only, without changing `conversation.agentId`. The agent selector (CHAT-08) is the persistent per-conversation setting.
|
||||
|
||||
---
|
||||
|
||||
## Environment Availability
|
||||
|
||||
| Dependency | Required By | Available | Version | Fallback |
|
||||
|------------|------------|-----------|---------|----------|
|
||||
| Node.js | Server SSE route | ✓ | Current | — |
|
||||
| PostgreSQL | DB migrations | ✓ | Confirmed Phase 21 | — |
|
||||
| `ai` (Vercel AI SDK) | LLM streaming | ✗ | Not installed | Use `openai` npm package directly |
|
||||
| `virtua` | PERF-03 list | ✗ | Not installed | Defer virtualization or use @tanstack/react-virtual |
|
||||
| LLM API endpoint | CHAT-01 streaming | Unknown | — | Phase 22 should add configuration step or fallback echo mode |
|
||||
|
||||
**Missing dependencies with no fallback:**
|
||||
- LLM API endpoint/key: Phase 22 implementation must either read from agent `adapterConfig` or prompt the user for an API key. Without this, streaming returns no tokens. Planner should include a task to read agent `adapterConfig.apiKey` + `adapterConfig.baseUrl` + `adapterConfig.model`.
|
||||
|
||||
**Missing dependencies with fallback:**
|
||||
- `ai` SDK: If Vercel AI SDK proves problematic, use `openai` npm package (v6.33.0 confirmed available) which supports async iteration over token streams and is already the defacto standard.
|
||||
- `virtua`: If not available, `@tanstack/react-virtual` v3.13.23 is the fallback (confirmed installable).
|
||||
|
||||
---
|
||||
|
||||
## Validation Architecture
|
||||
|
||||
### Test Framework
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Framework | Vitest 3.0.x |
|
||||
| Config file | `server/vitest.config.ts`, `ui/vitest.config.ts` |
|
||||
| Quick run command | `pnpm --filter @paperclipai/server test run` |
|
||||
| Full suite command | `pnpm test run` |
|
||||
|
||||
### Phase Requirements → Test Map
|
||||
|
||||
| Req ID | Behavior | Test Type | Automated Command | File Exists? |
|
||||
|--------|----------|-----------|-------------------|-------------|
|
||||
| CHAT-01 | SSE route returns `text/event-stream` content-type | unit (server) | `pnpm --filter @paperclipai/server test run -- --reporter=verbose chat-stream` | ❌ Wave 0 |
|
||||
| CHAT-01 | SSE route emits token events then done event | unit (server) | same | ❌ Wave 0 |
|
||||
| CHAT-08 | PATCH /conversations/:id accepts `agentId` field | unit (server) | `pnpm --filter @paperclipai/server test run -- --reporter=verbose chat-routes` | ✅ exists (extend) |
|
||||
| CHAT-10 | PUT /messages/:id updates editedContent + editedAt | unit (server) | `pnpm --filter @paperclipai/server test run -- --reporter=verbose chat-routes` | ✅ exists (extend) |
|
||||
| CHAT-12 | Stop button renders during streaming | unit (UI) | `pnpm --filter @paperclipai/ui test run` | ❌ Wave 0 |
|
||||
| CHAT-12 | Server disconnects client closes SSE gracefully | unit (server) | `pnpm --filter @paperclipai/server test run -- chat-stream` | ❌ Wave 0 |
|
||||
| INPUT-05 | Slash command parsing extracts target role | unit (UI) | `pnpm --filter @paperclipai/ui test run` | ❌ Wave 0 |
|
||||
| INPUT-06 | @mention parsing extracts target name | unit (UI) | `pnpm --filter @paperclipai/ui test run` | ❌ Wave 0 |
|
||||
| AGENT-04 | ChatAgentBadge renders agent name + icon | unit (UI) | `pnpm --filter @paperclipai/ui test run` | ❌ Wave 0 |
|
||||
| PERF-03 | ChatMessageList renders with VList | unit (UI) | `pnpm --filter @paperclipai/ui test run` | ✅ exists (extend) |
|
||||
|
||||
### Sampling Rate
|
||||
- **Per task commit:** `pnpm --filter @paperclipai/server test run && pnpm --filter @paperclipai/ui test run`
|
||||
- **Per wave merge:** `pnpm test run`
|
||||
- **Phase gate:** Full suite green before `/gsd:verify-work`
|
||||
|
||||
### Wave 0 Gaps
|
||||
- [ ] `server/src/__tests__/chat-stream-routes.test.ts` — covers CHAT-01, CHAT-12 SSE behavior
|
||||
- [ ] `ui/src/components/ChatInput.slash-mention.test.tsx` — covers INPUT-05, INPUT-06 parsing
|
||||
- [ ] `ui/src/components/ChatAgentBadge.test.tsx` — covers AGENT-04 rendering
|
||||
|
||||
*(Existing `chat-routes.test.ts` covers CHAT-08 and CHAT-10 with extensions; no new file needed.)*
|
||||
|
||||
---
|
||||
|
||||
## Sources
|
||||
|
||||
### Primary (HIGH confidence)
|
||||
- Codebase: `server/src/routes/plugins.ts:1096-1180` — SSE header setup, write pattern, unsubscribe on close
|
||||
- Codebase: `server/src/services/live-events.ts` — event bus pattern
|
||||
- Codebase: `ui/src/plugins/bridge.ts:403-460` — EventSource hook pattern
|
||||
- Codebase: `packages/db/src/schema/chat_messages.ts`, `chat_conversations.ts` — current schema
|
||||
- Codebase: `packages/shared/src/types/agent.ts` — Agent type with `name`, `icon`, `role` fields
|
||||
- Codebase: `ui/src/components/AgentIconPicker.tsx`, `ui/src/lib/agent-icons.ts` — existing icon system
|
||||
- npm: `npm view virtua version` → 0.49.0 (confirmed 2026-04-01)
|
||||
- npm: `npm view @tanstack/react-virtual version` → 3.13.23 (confirmed 2026-04-01)
|
||||
|
||||
### Secondary (MEDIUM confidence)
|
||||
- npm: `npm view openai version` → 6.33.0 — confirmed installable, async iterator streaming available
|
||||
- npm: `npm view @anthropic-ai/sdk version` → 0.81.0 — confirmed installable alternative
|
||||
- virtua README: `<VList>` API is a single component, `ref` for scroll programmatic control, `onScroll` for position tracking
|
||||
|
||||
### Tertiary (LOW confidence)
|
||||
- Vercel AI SDK version: The `ai` npm package returned v6.0.142 from npm view, which may be a different package entirely. **Validate before installing** — check `npm view ai` author field or use `openai` directly as the safer option.
|
||||
|
||||
---
|
||||
|
||||
## Metadata
|
||||
|
||||
**Confidence breakdown:**
|
||||
- Standard stack: HIGH — all existing packages verified from package.json; new packages confirmed via npm view
|
||||
- Architecture: HIGH — SSE pattern directly copied from existing plugin bridge; EventSource pattern from existing bridge.ts
|
||||
- Pitfalls: HIGH — derived from codebase analysis (existing SSE headers, EventSource limitations, React query cache behavior)
|
||||
- LLM provider selection: LOW — depends on what API keys/models the user has configured on their agents; needs resolution in first plan wave
|
||||
|
||||
**Research date:** 2026-04-01
|
||||
**Valid until:** 2026-05-01 (stable libraries; Vercel AI SDK version claim needs validation before install)
|
||||
Loading…
Add table
Reference in a new issue