982d75d0b3
- 34-02-SUMMARY.md: VoiceStep onboarding, 6-step wizard, PersonalAssistant with STT+TTS - STATE.md: plan advanced to last_plan (12/12 complete), metrics recorded, decisions added - ROADMAP.md: phase 34 marked Complete (2/2 summaries) - REQUIREMENTS.md: VOICE-03 marked complete
122 lines
5.4 KiB
Markdown
122 lines
5.4 KiB
Markdown
---
|
|
phase: 34-voice
|
|
plan: 02
|
|
subsystem: ui, onboarding
|
|
tags: [voice, onboarding, tts, stt, piper, whisper, personal-assistant]
|
|
|
|
# Dependency graph
|
|
requires:
|
|
- phase: 34-voice/34-01
|
|
provides: usePiperTts, TtsButton, VoiceRecordButton, voiceEnabled nexus-settings
|
|
- phase: 33-persistent-memory
|
|
provides: PersonalAssistant page, chatApi streaming
|
|
- phase: 32-multi-step-onboarding-wizard
|
|
provides: NexusOnboardingWizard 5-step base
|
|
|
|
provides:
|
|
- VoiceStep onboarding component with mic detection (enumerateDevices)
|
|
- 6-step NexusOnboardingWizard with voice opt-in at step 4
|
|
- voiceEnabled persisted in nexus-settings on workspace creation
|
|
- PersonalAssistant with VoiceRecordButton (STT) in input bar
|
|
- PersonalAssistant with TtsButton next to each assistant message (TTS)
|
|
|
|
affects: [ui/src/components/NexusOnboardingWizard.tsx, ui/src/pages/PersonalAssistant.tsx]
|
|
|
|
# Tech tracking
|
|
tech-stack:
|
|
added: []
|
|
patterns:
|
|
- "VoiceStep probes microphone with navigator.mediaDevices.enumerateDevices() — async, graceful fallback to false"
|
|
- "voiceEnabled captured in wizard state, persisted via updateNexusSettings after createWorkspace()"
|
|
- "TtsButton rendered inline in messages.map (no prop drilling) — pl-10 aligns under message bubble"
|
|
- "VoiceRecordButton appends transcription to textarea (not auto-send) — user reviews before sending"
|
|
|
|
key-files:
|
|
created:
|
|
- ui/src/components/onboarding/VoiceStep.tsx
|
|
modified:
|
|
- ui/src/components/NexusOnboardingWizard.tsx
|
|
- ui/src/pages/PersonalAssistant.tsx
|
|
|
|
key-decisions:
|
|
- "VoiceStep inserted at step 4; rootDir shifts to step 5, summary to step 6 — clean sequential numbering"
|
|
- "voiceEnabled persisted after mode save in createWorkspace() — non-blocking try/catch wrapper"
|
|
- "TtsButton rendered inline in messages.map rather than inside MessageBubble — avoids prop drilling usePiperTts through MessageBubble"
|
|
- "VoiceRecordButton appends (not replaces) transcription to textarea — user can combine typed + spoken input"
|
|
- "No TTS auto-prewarm on mount — triggered only on first TtsButton click to avoid unexpected WASM downloads"
|
|
|
|
requirements-completed: [VOICE-03]
|
|
|
|
# Metrics
|
|
duration: 4min
|
|
completed: 2026-04-03
|
|
tasks_completed: 2
|
|
files_changed: 3
|
|
---
|
|
|
|
# Phase 34 Plan 02: Voice Onboarding Step + PersonalAssistant Wire-up Summary
|
|
|
|
**VoiceStep onboarding component (mic detection, enable/skip) inserted as wizard step 4; VoiceRecordButton (STT) and TtsButton (TTS) wired into PersonalAssistant for full voice I/O**
|
|
|
|
## Performance
|
|
|
|
- **Duration:** ~4 min
|
|
- **Started:** 2026-04-03T22:38:32Z
|
|
- **Completed:** 2026-04-03T22:41:13Z
|
|
- **Tasks:** 2/2
|
|
- **Files modified:** 3
|
|
|
|
## Accomplishments
|
|
|
|
### Task 1: Create VoiceStep component and insert into NexusOnboardingWizard as step 4
|
|
|
|
- Created `ui/src/components/onboarding/VoiceStep.tsx` with:
|
|
- `navigator.mediaDevices.enumerateDevices()` mic probe with loading/available/unavailable states
|
|
- Two info cards (Mic/Whisper STT + Volume/Piper TTS) with conditional mic availability message
|
|
- Enable voice button (sets voiceEnabled = true, advances to step 5) and Skip button
|
|
- Updated `NexusOnboardingWizard.tsx`:
|
|
- Added `VoiceStep` import
|
|
- Added `voiceEnabled` state (default false, reset on close, persisted in `createWorkspace()`)
|
|
- Inserted step 4 (Voice) block with Back → step 3, Enable → step 5, Skip → step 5
|
|
- Shifted old step 4 (rootDir) to step 5 — "Review & finish" → step 6, Back → step 4 (voice), Skip to summary → step 6
|
|
- Shifted old step 5 (summary) to step 6 — Back → step 5
|
|
- Updated step indicator from `Step N of 4` to `Step N of 5` (summary shows "Summary" at step 6)
|
|
- Updated handleStartChat error message to reference step 5
|
|
|
|
### Task 2: Wire VoiceRecordButton and TtsButton into PersonalAssistant
|
|
|
|
- Added imports: `VoiceRecordButton`, `TtsButton`, `usePiperTts`
|
|
- Added `usePiperTts` hook call in component body — exposes `ttsStatus`, `ttsProgress`, `prewarm`, `speak`, `stop`
|
|
- Added `VoiceRecordButton` in the input bar between textarea and Send button:
|
|
- `onTranscription` callback appends transcribed text to textarea (does not auto-send)
|
|
- `disabled` passes through `isSending` state
|
|
- Added `TtsButton` next to each assistant message in `messages.map`:
|
|
- Wrapped message rendering in `<div>` to accommodate TtsButton below each bubble
|
|
- `pl-10` aligns button under message bubble (past avatar width)
|
|
- `-mt-1 mb-1` tucks close to message
|
|
- Only renders for messages with `role === "assistant"` and non-empty content
|
|
|
|
## Verification
|
|
|
|
- `grep -q "VoiceStep" ui/src/components/NexusOnboardingWizard.tsx` — PASS
|
|
- `grep -q "step === 6" ui/src/components/NexusOnboardingWizard.tsx` — PASS
|
|
- `grep -q "VoiceRecordButton" ui/src/pages/PersonalAssistant.tsx` — PASS
|
|
- `grep -q "TtsButton" ui/src/pages/PersonalAssistant.tsx` — PASS
|
|
- `grep -q "enumerateDevices" ui/src/components/onboarding/VoiceStep.tsx` — PASS
|
|
|
|
## Deviations from Plan
|
|
|
|
None — plan executed exactly as written.
|
|
|
|
## Known Stubs
|
|
|
|
None — all voice features are wired. TtsButton requires user to click for first download (intentional — avoids unexpected 50MB WASM download on page load).
|
|
|
|
## Commits
|
|
|
|
| Commit | Task | Description |
|
|
|--------|------|-------------|
|
|
| be8c1f67 | Task 1 | feat(34-02): create VoiceStep component and insert into NexusOnboardingWizard as step 4 |
|
|
| 263a19e4 | Task 2 | feat(34-02): wire VoiceRecordButton and TtsButton into PersonalAssistant |
|
|
|
|
## Self-Check: PASSED
|