14 KiB
14 KiB
| phase | plan | type | wave | depends_on | files_modified | autonomous | requirements | must_haves | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 36-voice-pipeline-foundation | 03 | execute | 2 |
|
|
true |
|
|
Purpose: VPIPE-03 requires the voice pipeline to be callable from any transport via HTTP. VPIPE-06 requires dual output (spoken prose + full markdown) triggered by voiceMode=full_voice in the stream endpoint.
Output: server/src/routes/voice.ts with two endpoints, updated chat.ts with voiceMode wiring, cleaned chat-files.ts, updated app.ts mount.
<execution_context> @$HOME/.claude/get-shit-done/workflows/execute-plan.md @$HOME/.claude/get-shit-done/templates/summary.md </execution_context>
@.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/phases/36-voice-pipeline-foundation/36-RESEARCH.md @.planning/phases/36-voice-pipeline-foundation/36-01-SUMMARY.md @.planning/phases/36-voice-pipeline-foundation/36-02-SUMMARY.md From server/src/services/voice-pipeline.ts: ```typescript export function voicePipelineService(): { transcribe(buffer: Buffer, format: "webm" | "ogg" | "wav"): Promise<{ text: string; language?: string }>; synthesize(text: string, voiceId?: string): Promise; formatForVoice(text: string): string; }; ```From packages/shared/src/validators/chat.ts:
export const createMessageSchema = z.object({
role: z.enum(["user", "assistant", "system"]),
content: z.string().min(1).max(100_000),
agentId: z.string().uuid().optional(),
messageType: z.string().optional(),
voiceMode: z.enum(["text", "voice_input", "full_voice"]).optional(),
});
From server/src/routes/authz.ts:
export function assertBoard(req: Request) {
if (req.actor.type !== "board") throw forbidden("Board access required");
}
From server/src/attachment-types.ts:
export const MAX_ATTACHMENT_BYTES = ...
From server/src/app.ts (mount pattern, line ~164):
api.use(chatFileRoutes(db, opts.storageService));
api.use(nexusSettingsRoutes());
From server/src/routes/chat.ts (stream endpoint, lines 91-194):
router.post("/conversations/:id/stream", async (req, res) => {
assertBoard(req);
const { content, agentId } = req.body;
// ... builds messagesWithMemory array ...
// ... streams tokens ...
const message = await svc.addMessage(req.params.id!, {
role: "assistant",
content: fullContent.trim(),
agentId: agentId || undefined,
});
});
From server/src/routes/chat-files.ts (lines 297-386 to remove):
// POST /transcribe — the old endpoint with inline audioUpload multer, runAudioUpload helper,
// and whisper-cpp/openai-whisper cascade. This entire block (lines 297-386) is replaced by voice.ts.
2. Create `server/src/routes/voice.ts` (GREEN):
```typescript
import { Router } from "express";
import multer from "multer";
import { assertBoard } from "./authz.js";
import { voicePipelineService } from "../services/voice-pipeline.js";
import { MAX_ATTACHMENT_BYTES } from "../attachment-types.js";
export function voiceRoutes(): Router {
const router = Router();
const svc = voicePipelineService();
const audioUpload = multer({
storage: multer.memoryStorage(),
limits: { fileSize: MAX_ATTACHMENT_BYTES, files: 1 },
});
// POST /api/transcribe — transcribe uploaded audio via VoicePipelineService
router.post("/transcribe", async (req, res) => {
assertBoard(req);
await new Promise<void>((resolve, reject) =>
audioUpload.single("audio")(req, res, (err) => (err ? reject(err) : resolve()))
);
const file = (req as any).file as { buffer: Buffer; mimetype: string } | undefined;
if (!file) {
res.status(400).json({ error: "Missing audio field" });
return;
}
const fmt = file.mimetype.includes("ogg") ? "ogg"
: file.mimetype.includes("wav") ? "wav"
: "webm";
const result = await svc.transcribe(file.buffer, fmt);
res.json(result);
});
// POST /api/synthesize — synthesize text to speech via VoicePipelineService
router.post("/synthesize", async (req, res) => {
assertBoard(req);
const { text, voiceId } = req.body as { text?: string; voiceId?: string };
if (!text || typeof text !== "string") {
res.status(400).json({ error: "text is required" });
return;
}
const audioBuffer = await svc.synthesize(text, voiceId);
res.setHeader("Content-Type", "audio/wav");
res.send(audioBuffer);
});
return router;
}
```
2. Modify `server/src/routes/chat-files.ts` — remove old /transcribe endpoint:
- Delete lines 297-386: the `audioUpload` multer instance, `runAudioUpload` helper function, and the entire `router.post("/transcribe", ...)` handler
- The line `return router;` at line 388 should remain (it becomes the new end of the function)
- Remove the `multer` import ONLY if no other route in the file uses multer (check: the file upload routes at top of file use multer too, so keep the import)
3. Modify `server/src/app.ts` — mount voice routes:
- Add import at top with other route imports:
```typescript
import { voiceRoutes } from "./routes/voice.js";
```
- Add mount line after `api.use(nexusSettingsRoutes());` (after line 165):
```typescript
api.use(voiceRoutes());
```
<success_criteria> Voice pipeline is fully callable via HTTP (POST /api/transcribe, POST /api/synthesize). voiceMode flag propagates from client request through the stream endpoint to message persistence. Dual output prompt injected for full_voice mode. Old transcribe endpoint removed from chat-files.ts. All routes mounted and TypeScript compiles clean. </success_criteria>
After completion, create `.planning/phases/36-voice-pipeline-foundation/36-03-SUMMARY.md`