mikkel/nexus
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
nexus/.planning/phases/37-web-chat-voice-ui/37-03-SUMMARY.md

5.4 KiB
Raw Blame History

phase plan subsystem tags requires provides affects tech-stack key-files key-decisions patterns-established requirements-completed duration completed
37-web-chat-voice-ui 03 ui
react
voice
audio
shadcn
lucide-react
localStorage
tailwind
phase provides
36-voice-pipeline-foundation POST /api/synthesize and POST /api/transcribe endpoints; voiceMode persisted via nexus-settings
phase provides
37-01 COOP/COEP headers for SharedArrayBuffer; VAD ONNX assets
ChatVoicePlayer
inline audio player with play/pause, auto-play, blob URL cleanup
ChatVoiceBadge
Voice badge + SPOKEN/DETAILED parsing + collapsible full markdown
VoiceModeToggle
three-pill Text/Voice In/Full Voice toggle with nexus-settings persistence
useVoiceMode
hook for reading/writing voiceMode via PATCH /api/nexus/settings
37-04
ChatMessage integration
ChatInput integration
added patterns
Blob URL lifecycle
createObjectURL on fetch response, revokeObjectURL in onEnded and cleanup
SPOKEN/DETAILED regex parsing for dual-format voice responses
Three-pill toggle pattern with active/inactive Tailwind classes + role="group" + aria-label
created modified
ui/src/components/ChatVoicePlayer.tsx
ui/src/components/ChatVoiceBadge.tsx
ui/src/components/VoiceModeToggle.tsx
ui/src/hooks/useVoiceMode.ts
useVoiceMode hook created in plan 37-03 (not 37-02) to unblock VoiceModeToggle — parallel execution
ChatVoicePlayer fetches audio on text change, not on play — pre-loads audio for smoother UX
Auto-play preference stored in localStorage (nexus:voice:autoplay), not nexus-settings — avoids server round-trip for fast UX
Pattern 1: blob URL cleanup — always revokeObjectURL in both onEnded and effect cleanup to prevent memory leaks
Pattern 2: voice badge renders differently per messageType — voice_input shows badge+text only; voice_full adds player+collapsible
WCHAT-04
WCHAT-05
WCHAT-06
2min 2026-04-04

Phase 37 Plan 03: Voice Output Components Summary

Inline audio player (ChatVoicePlayer), voice badge with collapsible markdown (ChatVoiceBadge), and three-pill mode toggle (VoiceModeToggle) — complete output-side voice UI

Performance

  • Duration: 2 min
  • Started: 2026-04-04T02:35:25Z
  • Completed: 2026-04-04T02:37:21Z
  • Tasks: 2
  • Files modified: 4

Accomplishments

  • ChatVoicePlayer: POST /api/synthesize → blob URL → native <audio> element with play/pause controls, auto-play support on mount, and proper blob URL revocation on end/unmount
  • ChatVoiceBadge: parses SPOKEN/DETAILED content format, renders Voice badge, spoken text, optional ChatVoicePlayer, and shadcn Collapsible for full markdown (voice_full only)
  • VoiceModeToggle: three pill buttons (Text / Voice In / Full Voice) with active/inactive Tailwind styling, PATCH /api/nexus/settings persistence, auto-play checkbox in full_voice mode

Task Commits

Each task was committed atomically:

  1. Task 1: Create ChatVoicePlayer and ChatVoiceBadge components - 601508bf (feat)
  2. Task 2: Create VoiceModeToggle three-pill component - 562e4451 (feat)

Plan metadata: (docs commit — see final commit)

Files Created/Modified

  • ui/src/components/ChatVoicePlayer.tsx — Inline audio player: synthesize fetch, play/pause, blob URL lifecycle
  • ui/src/components/ChatVoiceBadge.tsx — Voice badge + SPOKEN/DETAILED parsing + shadcn Collapsible for voice_full
  • ui/src/components/VoiceModeToggle.tsx — Three-pill mode toggle with nexus-settings persistence and auto-play checkbox
  • ui/src/hooks/useVoiceMode.ts — Hook: GET/PATCH /api/nexus/settings for voiceMode, loading state

Decisions Made

  • useVoiceMode hook created in this plan (37-03) to unblock VoiceModeToggle since plan 37-02 hadn't run in this worktree yet
  • Auto-play preference kept in localStorage (not nexus-settings) for zero-latency reads on component mount
  • ChatVoicePlayer pre-fetches audio on text prop change (not on user play click) for smoother playback experience

Deviations from Plan

Auto-fixed Issues

1. [Rule 3 - Blocking] Created useVoiceMode hook (plan 37-02 dependency)

  • Found during: Task 2 (VoiceModeToggle implementation)
  • Issue: VoiceModeToggle imports useVoiceMode, but the hook was not yet present in this worktree (37-02 is a parallel execution)
  • Fix: Created ui/src/hooks/useVoiceMode.ts implementing the exact interface specified in the plan context: { mode, setMode, isLoading } using GET/PATCH /api/nexus/settings
  • Files modified: ui/src/hooks/useVoiceMode.ts (created)
  • Verification: VoiceModeToggle imports and uses hook; all acceptance criteria pass
  • Committed in: 562e4451 (Task 2 commit)

Total deviations: 1 auto-fixed (1 blocking) Impact on plan: Essential for correct parallel execution. Implementation matches plan-specified interface exactly.

Issues Encountered

None — tasks executed cleanly once useVoiceMode was created.

User Setup Required

None - no external service configuration required.

Next Phase Readiness

  • All three voice output components are complete
  • ChatMessage needs to be updated to render <ChatVoiceBadge> for messageType voice_input/voice_full
  • ChatInput needs VoiceModeToggle added to toolbar row
  • These integrations are handled in plan 37-04 (ChatInput + ChatMessage integration)

Phase: 37-web-chat-voice-ui Completed: 2026-04-04