11 KiB
11 KiB
| phase | plan | type | wave | depends_on | files_modified | autonomous | requirements | must_haves | |||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 36-voice-pipeline-foundation | 01 | execute | 1 |
|
true |
|
|
Purpose: This is the keystone service for the entire v1.6 milestone. It encapsulates STT (Whisper), TTS (Piper), audio transcoding (ffmpeg), and dual-output formatting behind a clean factory function API.
Output: server/src/services/voice-pipeline.ts with transcribe(), synthesize(), formatForVoice() methods, plus unit tests.
<execution_context> @$HOME/.claude/get-shit-done/workflows/execute-plan.md @$HOME/.claude/get-shit-done/templates/summary.md </execution_context>
@.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/phases/36-voice-pipeline-foundation/36-RESEARCH.mdFrom server/src/services/nexus-settings.ts (factory function pattern):
export function nexusSettingsService() {
async function get(): Promise<NexusSettings> { ... }
async function set(patch: Partial<NexusSettings>): Promise<NexusSettings> { ... }
return { get, set };
}
From server/src/routes/chat-files.ts lines 297-386 (existing whisper cascade to extract and move):
// Writes raw WebM to temp file, then tries whisper-cpp --model base.en --no-timestamps
// Falls back to openai-whisper Python CLI
// Uses promisify(execFileCb) pattern from node:child_process
2. Create `server/src/__tests__/36-voice-pipeline.test.ts` with unit tests (RED phase):
- Mock `node:child_process` execFile and spawn
- Mock `ffmpeg-static` to return "/mock/ffmpeg" by default, null for the fail-fast test
- Test `transcodeToWav16k("webm")` verifies spawn is called with args ["-f", "webm", "-i", "pipe:0", "-ar", "16000", "-ac", "1", "-f", "wav", "pipe:1"]
- Test `transcribe(buffer, "webm")` verifies it calls transcodeToWav16k then whisper-cpp with `--language auto`
- Test `transcribe` whisper-cpp fallback to openai-whisper
- Test `synthesize("Hello world. How are you?")` verifies it splits into 2 sentences and calls piper execFile twice
- Test `synthesize` timeout rejects after 8000ms
- Test `formatForVoice("SPOKEN: Hello\n\nDETAILED: ## Hello\n**world**")` returns "Hello"
- Test `formatForVoice("## Hello\n**world**\n- item\n```code```")` returns "Hello\nworld\nitem\ncode" (markdown stripped)
- Test `formatForVoice("")` returns ""
- Test factory throws when ffmpegPath is null
3. Create `server/src/services/voice-pipeline.ts` (GREEN phase):
- Export `voicePipelineService()` factory function (matches nexus-settings pattern)
- Assert `ffmpegPath` is not null at construction time: `if (!ffmpegPath) throw new Error("ffmpeg-static binary not found on this platform");`
- `transcodeToWav16k(inputBuffer: Buffer, inputFormat: string): Promise<Buffer>` — uses `spawn(ffmpegPath, ["-f", inputFormat, "-i", "pipe:0", "-ar", "16000", "-ac", "1", "-f", "wav", "pipe:1"], { stdio: ["pipe", "pipe", "pipe"] })`; write inputBuffer to stdin, collect stdout chunks, resolve on close code 0, reject otherwise
- `withTimeout<T>(promise: Promise<T>, ms: number): Promise<T>` — `Promise.race([promise, new Promise<never>((_, reject) => setTimeout(() => reject(new Error("Timed out after ${ms}ms")), ms))])`
- `transcribe(buffer: Buffer, format: "webm" | "ogg" | "wav"): Promise<{ text: string; language?: string }>`:
1. If format !== "wav", call transcodeToWav16k(buffer, format)
2. Write WAV to temp file: `path.join(tmpdir(), "nexus-audio-" + Date.now() + ".wav")`
3. Try whisper-cpp: `execFile("whisper-cpp", ["--model", "base.en", "--file", tmpPath, "--no-timestamps", "--output-txt", "--language", "auto"], { timeout: 30000 })`
4. Parse language from whisper-cpp stdout if present; return `{ text: stdout.trim(), language }`
5. On failure, try openai-whisper: `execFile("whisper", [tmpPath, "--model", "base.en", "--output_format", "txt", "--output_dir", tmpdir()], { timeout: 60000 })`
6. On both failure, throw new Error("Whisper not available. Install whisper-cpp or openai-whisper for voice input.")
7. Cleanup temp file in `finally` block via `unlink(tmpPath).catch(() => {})`
- `synthesize(text: string, voiceId?: string): Promise<Buffer>`:
1. Split text into sentences: `text.split(/(?<=[.!?])\s+/).filter(s => s.length > 0)`
2. For each sentence, call piper via `withTimeout(execFile("piper", ["--model", voiceId || "en_US-lessac-medium", "--output-raw"], { timeout: 8000, maxBuffer: 10 * 1024 * 1024, input: sentence }), 8000)`
3. Concatenate all output buffers
4. On ENOENT (piper not found), throw new Error("Piper TTS not available. Install piper for voice output.")
- `formatForVoice(text: string): string`:
1. If empty, return ""
2. Check for `SPOKEN:` marker: `const spokenMatch = text.match(/SPOKEN:\s*([\s\S]*?)(?=\nDETAILED:|\n\n[A-Z]+:)/)`
3. If match found, return `spokenMatch[1].trim()`
4. Otherwise strip markdown: remove `# ` headings, `**` bold, `*` italic, triple backtick code fences (and lang identifier), `- ` and `* ` bullet prefixes, inline backticks
5. Collapse multiple blank lines, trim
- Return `{ transcribe, synthesize, formatForVoice }`
<success_criteria> VoicePipelineService is a working, tested factory function that downstream consumers (voice.ts routes in Plan 03, Telegram bridge in Phase 38) can import and call without additional setup. ffmpeg-static is installed. All unit tests pass. </success_criteria>
After completion, create `.planning/phases/36-voice-pipeline-foundation/36-01-SUMMARY.md`