Commit graph

2186 commits

Author SHA1 Message Date
Nexus Dev
0cd6f2b8e1 docs(38-01): complete Telegram bridge core plan — telegramService + telegramRoutes 2026-04-04 03:15:26 +00:00
Nexus Dev
7142062c9b docs(38-03): complete Telegram onboarding step plan
- TelegramStep component with BotFather guided setup and token validation
- NexusOnboardingWizard updated to 7-step flow with Telegram at step 5
- ONBRD-03 requirement marked complete
2026-04-04 03:14:57 +00:00
Nexus Dev
3973fa08f7 feat(38-01): wire telegramService + telegramRoutes into app.ts
- Import telegramService, telegramRoutes, nexusSettingsService
- Mount /telegram routes under /api prefix
- Conditionally start Telegram bot on boot if telegramToken is configured
- Token route restarts bot after saving new token
2026-04-04 03:14:08 +00:00
Nexus Dev
48f708d908 feat(38-03): insert TelegramStep as step 5 in NexusOnboardingWizard
- Import TelegramStep component
- Insert Telegram step at position 5 (between Voice and Root Directory)
- Shift Root Directory from step 5 → step 6
- Shift Summary from step 6 → step 7
- Update step indicator from 'of 5' to 'of 6'
- Update summary indicator from step===6 to step===7
- Update all setStep() navigation callbacks accordingly
- Update error message referencing step 6 for root directory
2026-04-04 03:13:57 +00:00
Nexus Dev
8dbc7674a6 feat(38-01): install grammY, create telegramService + telegramRoutes
- Install grammy v2 for long polling Telegram bot
- telegramService: text relay handler, agent prefix, session map, deleteWebhook lifecycle
- telegramRoutes: POST /telegram/token (getMe validation), GET /telegram/status
- telegram.ts under 500 lines (187 lines)
2026-04-04 03:13:13 +00:00
Nexus Dev
d9d6e4f657 feat(38-03): create TelegramStep onboarding component
- BotFather numbered instructions (4-step setup guide)
- Token input with live validation via POST /api/telegram/token
- Success state showing connected bot username
- Error state with descriptive message
- Skip/Back/Next navigation; Next enabled only after validation
2026-04-04 03:12:31 +00:00
Nexus Dev
fa3529e784 docs(38): create 3 plans in 2 waves for Telegram bridge 2026-04-04 03:09:38 +00:00
Nexus Dev
46bad4cc60 docs(38): research Telegram bridge phase 2026-04-04 03:02:48 +00:00
Nexus Dev
b468921332 docs(38): auto-generated context (discuss skipped) 2026-04-04 02:53:46 +00:00
Nexus Dev
3bb3361a2b docs(phase-37): complete phase execution 2026-04-04 02:53:05 +00:00
Nexus Dev
b32e8029c0 fix(37): pass voiceMode in ChatPanel handleEdit path + add verification 2026-04-04 02:52:55 +00:00
Nexus Dev
c294277b84 docs(37-04): complete chat voice integration plan — voiceMode threading + VoiceMicButton wiring
- 37-04-SUMMARY.md created with full execution record
- STATE.md updated with decisions and session info
- ROADMAP.md copied from phase-37 branch
2026-04-04 02:47:33 +00:00
Nexus Dev
90efd342ef feat(37-04): wire VoiceMicButton, VoiceModeToggle, ChatVoiceBadge, voiceMode into chat UI
- ChatInput: replace VoiceRecordButton with VoiceMicButton (VAD-powered)
- ChatInput: add VoiceModeToggle above input when enableVoiceInput=true
- ChatMessage: add ChatVoiceBadge render for voice_input and voice_full messageTypes
- ChatMessage: auto-play reads from localStorage nexus:voice:autoplay key
- ChatPanel: import and call useVoiceMode, extract mode as voiceMode
- ChatPanel: pass voiceMode as third arg to all startStream calls (5 call sites)
2026-04-04 02:47:28 +00:00
Nexus Dev
39bfec7fd8 feat(37-04): add voiceMode to chatApi.postMessageAndStream + useStreamingChat.startStream
- postMessageAndStream data type extended with optional voiceMode field
- startStream signature updated: (userMessage, agentId?, voiceMode?)
- voiceMode forwarded into fetch body via postMessageAndStream call
2026-04-04 02:47:28 +00:00
Nexus Dev
b777ebe345 feat(37-03): VoiceModeToggle three-pill component + useVoiceMode hook
- VoiceModeToggle: Text / Voice In / Full Voice pills with active/inactive styling
- Auto-play checkbox in full_voice mode, persists to nexus:voice:autoplay in localStorage
- useVoiceMode: reads/writes voiceMode via PATCH /api/nexus/settings with loading state
  (deviation Rule 3: created missing blocking dependency for VoiceModeToggle)
2026-04-04 02:39:24 +00:00
Nexus Dev
45339bdac1 feat(37-03): ChatVoicePlayer + ChatVoiceBadge components
- ChatVoicePlayer: POST /api/synthesize, play/pause controls, autoPlay support, blob URL cleanup
- ChatVoiceBadge: Voice badge, SPOKEN/DETAILED parsing, collapsible full markdown for voice_full
2026-04-04 02:39:19 +00:00
Nexus Dev
9914699e3e docs(37-03): complete voice output components plan 2026-04-04 02:38:38 +00:00
Nexus Dev
fc422a2364 docs(37-02): complete voice recording components plan
- SUMMARY.md: encodeWav, useVadRecorder, useVoiceMode, VoiceWaveform, VoiceMicButton
- STATE.md: advanced to plan 3, 71% progress, added decisions
- ROADMAP.md: updated phase-37 progress (2/4 plans done)
- REQUIREMENTS.md: marked WCHAT-01..03, WCHAT-05 complete
2026-04-04 02:37:31 +00:00
Nexus Dev
bdb2f77075 feat(37-02): VoiceWaveform canvas component and VoiceMicButton
- VoiceWaveform: 80x32 canvas with Web Audio AnalyserNode (fftSize=64), 20 animated bars drawn from frequency data using --primary color
- VoiceMicButton: three visual states — idle (Mic icon), recording (VoiceWaveform + ring-2 ring-primary), processing (Loader2 animate-spin)
- All three states have correct aria-labels per UI spec copywriting contract
2026-04-04 02:36:07 +00:00
Nexus Dev
3676c9c349 feat(37-02): encodeWav utility, useVadRecorder + useVoiceMode hooks
- encodeWav: 44-byte WAV header encoder (RIFF/WAVE/fmt/data), PCM mono 16-bit
- useVadRecorder: wraps useMicVAD with startOnLoad:false, auto-stop on speech end, POSTs to /api/transcribe
- useVoiceMode: reads/writes voiceMode from GET/PATCH /api/nexus/settings with optimistic update
2026-04-04 02:35:27 +00:00
Nexus Dev
602bbdc7c6 docs(37-01): complete server prerequisites + VAD browser infrastructure plan
- Create 37-01-SUMMARY.md with task results, deviations, and self-check
- STATE.md: advance to plan 2, add 3 decisions, update progress to 57%
- ROADMAP.md: phase 37 in progress (1/4 plans complete)
- REQUIREMENTS.md: mark WCHAT-01, WCHAT-02, WCHAT-04 complete
2026-04-04 02:32:28 +00:00
Nexus Dev
fe74bcb00c feat(37-01): install VAD library, copy ONNX assets, configure Vite COOP/COEP headers
- Add @ricky0123/vad-react dependency to ui/package.json
- Add copy-vad-assets npm script for reproducible asset copying
- Copy vad.worklet.bundle.min.js, silero_vad_legacy.onnx, silero_vad_v5.onnx to ui/public/
- Add COOP/COEP headers to Vite dev server config (SharedArrayBuffer support in dev)
- Update pnpm lockfile
2026-04-04 02:31:54 +00:00
Nexus Dev
ee5538e5a4 feat(37-01): add COOP/COEP headers to Express server for SharedArrayBuffer support
- Add Cross-Origin-Opener-Policy: same-origin middleware before all routes
- Add Cross-Origin-Embedder-Policy: require-corp middleware before all routes
- Required for @ricky0123/vad-react (VAD uses SharedArrayBuffer internally)
2026-04-04 02:31:49 +00:00
Nexus Dev
29d02c2e10 docs(37-01): complete server prerequisites + VAD browser infrastructure plan
- Create 37-01-SUMMARY.md with task results and deviations
- Update STATE.md: advance to plan 2, add decisions, update progress to 57%
- Update ROADMAP.md: phase 37 in progress (1/4 plans complete)
- Mark WCHAT-01, WCHAT-02, WCHAT-04 complete in REQUIREMENTS.md
2026-04-04 02:26:50 +00:00
Nexus Dev
872b8fbd0a docs(37): create 4 plans in 3 waves for web chat voice UI 2026-04-04 02:17:14 +00:00
Nexus Dev
d4caf8f0da docs(37): phase research — VAD, COOP/COEP, component architecture 2026-04-04 02:07:19 +00:00
Nexus Dev
a010333787 docs(37): UI design contract for web-chat-voice-ui
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 01:53:32 +00:00
Nexus Dev
30708d38e5 docs(37): auto-generated context (discuss skipped) 2026-04-04 01:50:35 +00:00
Nexus Dev
2e210a4e6d docs(phase-36): complete phase execution 2026-04-04 01:49:24 +00:00
Nexus Dev
9813903270 fix(36): resolve TypeScript errors in voice-pipeline.ts (ffmpegPath cast, callback types) 2026-04-04 01:49:08 +00:00
Nexus Dev
d4db7ffffc docs(36-03): update STATE.md and REQUIREMENTS.md after plan completion 2026-04-04 01:41:38 +00:00
Nexus Dev
fd372eafbd feat(36-03): wire voiceMode through chat stream, mount voice routes, remove old transcribe
- server/src/routes/chat.ts: destructure voiceMode from req.body in stream endpoint
- server/src/routes/chat.ts: inject dual-output system prompt when voiceMode=full_voice (VPIPE-06)
- server/src/routes/chat.ts: persist voiceMode to messageType column (voice_full/voice_input)
- server/src/routes/chat-files.ts: remove old inline /transcribe endpoint (lines 297-386)
- server/src/app.ts: import and mount voiceRoutes() after nexusSettingsRoutes()
2026-04-04 01:41:32 +00:00
Nexus Dev
1150854762 feat(36-03): add voice HTTP routes with POST /transcribe and POST /synthesize
- Create server/src/routes/voice.ts with voiceRoutes() factory
- POST /transcribe: multer audio upload → VoicePipelineService.transcribe → JSON response
- POST /synthesize: text body → VoicePipelineService.synthesize → audio/wav response
- Both routes protected by assertBoard(req) auth check
- Create server/src/__tests__/36-voice-routes.test.ts with 5 passing tests
2026-04-04 01:41:32 +00:00
Nexus Dev
f991a11fa9 docs(36-03): complete voice HTTP routes plan — POST /transcribe + POST /synthesize + voiceMode wiring 2026-04-04 01:40:50 +00:00
Nexus Dev
d0d7a23af5 feat(36-02): extend nexus-settings schema with voiceMode, telegramToken, and binary paths
- Export VOICE_MODES constant and VoiceMode type from nexus-settings
- Export nexusSettingsSchema for testing
- Add voiceMode field with default 'text' to nexusSettingsSchema
- Add telegramToken optional field to nexusSettingsSchema
- Add piperBinaryPath and whisperBinaryPath optional fields
- Update fallback in get() to use nexusSettingsSchema.parse({}) for consistent defaults
- Add 5 passing tests for nexus-settings schema in 36-voice-schema.test.ts
2026-04-04 01:32:06 +00:00
Nexus Dev
b964c0e413 feat(36-02): add voiceMode field to createMessageSchema and ChatMessage interface
- Add VOICE_MODES constant and VoiceMode type to shared validators/chat.ts
- Extend createMessageSchema with optional voiceMode enum field
- Add voiceMode optional field to ChatMessage interface in types/chat.ts
- Add 36-voice-schema.test.ts with 6 passing tests for voiceMode validation
2026-04-04 01:32:06 +00:00
Nexus Dev
f8df254777 feat(36-01): VoicePipelineService with transcribe, synthesize, formatForVoice, transcodeToWav16k
- Install ffmpeg-static and @types/ffmpeg-static
- Create voice-pipeline.ts with voicePipelineService factory function
- transcodeToWav16k: pipes audio through ffmpeg at 16kHz mono WAV
- transcribe: whisper-cpp cascade with --language auto, falls back to openai-whisper
- synthesize: piper TTS with sentence chunking and 8s timeout via Promise.race
- formatForVoice: extracts SPOKEN marker or strips markdown as fallback
- Unit tests with mocked child_process (12 tests all passing)
2026-04-04 01:31:53 +00:00
Nexus Dev
826b455967 docs(36-01): complete VoicePipelineService plan 2026-04-04 01:30:38 +00:00
Nexus Dev
7d47463d5b docs(36-02): complete voice schema foundation plan
- Add 36-02-SUMMARY.md with task details and verification results
- Advance STATE.md to plan 2 of 3, 33% progress
- Update ROADMAP.md plan progress (1 of 3 summaries)
- Mark VPIPE-05 as complete in REQUIREMENTS.md
2026-04-04 01:25:24 +00:00
Nexus Dev
044e3dad54 feat(36-02): extend nexus-settings schema with voiceMode, telegramToken, and binary paths
- Export VOICE_MODES constant and VoiceMode type from nexus-settings
- Export nexusSettingsSchema for testing
- Add voiceMode field with default 'text' to nexusSettingsSchema
- Add telegramToken optional field to nexusSettingsSchema
- Add piperBinaryPath and whisperBinaryPath optional fields
- Update fallback in get() to use nexusSettingsSchema.parse({}) for consistent defaults
- Add 5 passing tests for nexus-settings schema in 36-voice-schema.test.ts
2026-04-04 01:23:46 +00:00
Nexus Dev
390034c76d feat(36-02): add voiceMode field to createMessageSchema and ChatMessage interface
- Add VOICE_MODES constant and VoiceMode type to shared validators/chat.ts
- Extend createMessageSchema with optional voiceMode enum field
- Add voiceMode optional field to ChatMessage interface in types/chat.ts
- Add 36-voice-schema.test.ts with 6 passing tests for voiceMode validation
2026-04-04 01:22:55 +00:00
Nexus Dev
edd7b17569 docs(36): create phase plan — 3 plans in 2 waves 2026-04-04 01:16:50 +00:00
Nexus Dev
eada6c44e5 docs(phase-36): add validation strategy 2026-04-04 01:11:40 +00:00
Nexus Dev
8788b70cb9 docs(36): research voice pipeline foundation 2026-04-04 01:10:58 +00:00
Nexus Dev
b18355bc47 docs(36): auto-generated context (discuss skipped) 2026-04-04 00:55:08 +00:00
Nexus Dev
68aa5ae052 docs: create milestone v1.6 roadmap (4 phases) 2026-04-04 00:53:37 +00:00
Nexus Dev
1cdcfac10b docs: define milestone v1.6 requirements 2026-04-04 00:25:20 +00:00
Nexus Dev
0c29013931 docs: complete project research 2026-04-03 23:53:14 +00:00
Nexus Dev
2dc450008d docs: start milestone v1.6 Voice Pipeline + Minimal Message Bridge 2026-04-03 23:31:55 +00:00
Nexus Dev
51eb2edf0b chore: complete v1.5 Smart Onboarding + Personal AI Assistant milestone
6 phases, 13 plans, 21 requirements.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 23:03:46 +00:00