Commit graph

2218 commits

Author SHA1 Message Date
Nexus Dev
1eaa6c4b3e docs(37): create 4 plans in 3 waves for web chat voice UI 2026-04-04 03:55:50 +00:00
Nexus Dev
fdc956c6a6 docs(37): phase research — VAD, COOP/COEP, component architecture 2026-04-04 03:55:50 +00:00
Nexus Dev
f2f381a3a2 docs(37): UI design contract for web-chat-voice-ui
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 03:55:50 +00:00
Nexus Dev
06085127af docs(37): auto-generated context (discuss skipped) 2026-04-04 03:55:50 +00:00
Nexus Dev
223916fcdb docs(phase-36): complete phase execution 2026-04-04 03:55:50 +00:00
Nexus Dev
b204d6318e fix(36): resolve TypeScript errors in voice-pipeline.ts (ffmpegPath cast, callback types) 2026-04-04 03:55:50 +00:00
Nexus Dev
f82fed7e73 docs(36-03): update STATE.md and REQUIREMENTS.md after plan completion 2026-04-04 03:55:50 +00:00
Nexus Dev
f23c4218aa feat(36-03): wire voiceMode through chat stream, mount voice routes, remove old transcribe
- server/src/routes/chat.ts: destructure voiceMode from req.body in stream endpoint
- server/src/routes/chat.ts: inject dual-output system prompt when voiceMode=full_voice (VPIPE-06)
- server/src/routes/chat.ts: persist voiceMode to messageType column (voice_full/voice_input)
- server/src/routes/chat-files.ts: remove old inline /transcribe endpoint (lines 297-386)
- server/src/app.ts: import and mount voiceRoutes() after nexusSettingsRoutes()
2026-04-04 03:55:50 +00:00
Nexus Dev
bf757509ef feat(36-03): add voice HTTP routes with POST /transcribe and POST /synthesize
- Create server/src/routes/voice.ts with voiceRoutes() factory
- POST /transcribe: multer audio upload → VoicePipelineService.transcribe → JSON response
- POST /synthesize: text body → VoicePipelineService.synthesize → audio/wav response
- Both routes protected by assertBoard(req) auth check
- Create server/src/__tests__/36-voice-routes.test.ts with 5 passing tests
2026-04-04 03:55:50 +00:00
Nexus Dev
e76c52f693 docs(36-03): complete voice HTTP routes plan — POST /transcribe + POST /synthesize + voiceMode wiring 2026-04-04 03:55:50 +00:00
Nexus Dev
9fcf27fed9 feat(36-02): extend nexus-settings schema with voiceMode, telegramToken, and binary paths
- Export VOICE_MODES constant and VoiceMode type from nexus-settings
- Export nexusSettingsSchema for testing
- Add voiceMode field with default 'text' to nexusSettingsSchema
- Add telegramToken optional field to nexusSettingsSchema
- Add piperBinaryPath and whisperBinaryPath optional fields
- Update fallback in get() to use nexusSettingsSchema.parse({}) for consistent defaults
- Add 5 passing tests for nexus-settings schema in 36-voice-schema.test.ts
2026-04-04 03:55:50 +00:00
Nexus Dev
8e564e8125 feat(36-02): add voiceMode field to createMessageSchema and ChatMessage interface
- Add VOICE_MODES constant and VoiceMode type to shared validators/chat.ts
- Extend createMessageSchema with optional voiceMode enum field
- Add voiceMode optional field to ChatMessage interface in types/chat.ts
- Add 36-voice-schema.test.ts with 6 passing tests for voiceMode validation
2026-04-04 03:55:50 +00:00
Nexus Dev
346b42dd73 feat(36-01): VoicePipelineService with transcribe, synthesize, formatForVoice, transcodeToWav16k
- Install ffmpeg-static and @types/ffmpeg-static
- Create voice-pipeline.ts with voicePipelineService factory function
- transcodeToWav16k: pipes audio through ffmpeg at 16kHz mono WAV
- transcribe: whisper-cpp cascade with --language auto, falls back to openai-whisper
- synthesize: piper TTS with sentence chunking and 8s timeout via Promise.race
- formatForVoice: extracts SPOKEN marker or strips markdown as fallback
- Unit tests with mocked child_process (12 tests all passing)
2026-04-04 03:55:50 +00:00
Nexus Dev
f7153db301 docs(36-01): complete VoicePipelineService plan 2026-04-04 03:55:50 +00:00
Nexus Dev
2fbd0dd06c docs(36-02): complete voice schema foundation plan
- Add 36-02-SUMMARY.md with task details and verification results
- Advance STATE.md to plan 2 of 3, 33% progress
- Update ROADMAP.md plan progress (1 of 3 summaries)
- Mark VPIPE-05 as complete in REQUIREMENTS.md
2026-04-04 03:55:50 +00:00
Nexus Dev
5ad8e2bee8 feat(36-02): extend nexus-settings schema with voiceMode, telegramToken, and binary paths
- Export VOICE_MODES constant and VoiceMode type from nexus-settings
- Export nexusSettingsSchema for testing
- Add voiceMode field with default 'text' to nexusSettingsSchema
- Add telegramToken optional field to nexusSettingsSchema
- Add piperBinaryPath and whisperBinaryPath optional fields
- Update fallback in get() to use nexusSettingsSchema.parse({}) for consistent defaults
- Add 5 passing tests for nexus-settings schema in 36-voice-schema.test.ts
2026-04-04 03:55:50 +00:00
Nexus Dev
14e059862b feat(36-02): add voiceMode field to createMessageSchema and ChatMessage interface
- Add VOICE_MODES constant and VoiceMode type to shared validators/chat.ts
- Extend createMessageSchema with optional voiceMode enum field
- Add voiceMode optional field to ChatMessage interface in types/chat.ts
- Add 36-voice-schema.test.ts with 6 passing tests for voiceMode validation
2026-04-04 03:55:49 +00:00
Nexus Dev
3a90d7f5c7 docs(36): create phase plan — 3 plans in 2 waves 2026-04-04 03:55:49 +00:00
Nexus Dev
fc3f1510b9 docs(phase-36): add validation strategy 2026-04-04 03:55:49 +00:00
Nexus Dev
0736541a91 docs(36): research voice pipeline foundation 2026-04-04 03:55:49 +00:00
Nexus Dev
0dfd0cbac5 docs(36): auto-generated context (discuss skipped) 2026-04-04 03:55:49 +00:00
Nexus Dev
b17280ce66 docs: create milestone v1.6 roadmap (4 phases) 2026-04-04 03:55:49 +00:00
Nexus Dev
1636694728 docs: define milestone v1.6 requirements 2026-04-04 03:55:49 +00:00
Nexus Dev
f6da67ecf4 docs: complete project research 2026-04-04 03:55:49 +00:00
Nexus Dev
5ea3f2d6b5 docs: start milestone v1.6 Voice Pipeline + Minimal Message Bridge 2026-04-04 03:55:49 +00:00
Nexus Dev
285bf585be chore: complete v1.5 Smart Onboarding + Personal AI Assistant milestone
6 phases, 13 plans, 21 requirements.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 03:55:49 +00:00
Nexus Dev
2a396ebc70 docs(phase-35): complete npx buildthis CLI phase 2026-04-04 03:55:49 +00:00
Nexus Dev
1262e4a243 feat(35-01): buildthis CLI package — hardware detection + bootstrap
Standalone npm package at packages/buildthis/. Probes running Nexus
instance, opens browser if found, guides install with hardware-aware
provider recommendations if not. 14 tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 03:55:49 +00:00
Nexus Dev
9bbcff995d docs(35-01): complete buildthis CLI package plan 2026-04-04 03:55:49 +00:00
Nexus Dev
c65fa168e6 docs(35): create phase plan for npx buildthis CLI
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 03:55:49 +00:00
Nexus Dev
fef0b48771 docs(35): research npx buildthis CLI phase domain
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 03:55:49 +00:00
Nexus Dev
4cd06b2623 docs(35): auto-generated context (discuss skipped) 2026-04-04 03:55:49 +00:00
Nexus Dev
1b22667302 docs(phase-34): complete voice phase 2026-04-04 03:55:49 +00:00
Nexus Dev
c7974fa67c feat(34-02): voice onboarding step + PersonalAssistant voice wiring
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 03:55:49 +00:00
Nexus Dev
982d75d0b3 docs(34-02): complete voice onboarding step + PersonalAssistant wire-up plan
- 34-02-SUMMARY.md: VoiceStep onboarding, 6-step wizard, PersonalAssistant with STT+TTS
- STATE.md: plan advanced to last_plan (12/12 complete), metrics recorded, decisions added
- ROADMAP.md: phase 34 marked Complete (2/2 summaries)
- REQUIREMENTS.md: VOICE-03 marked complete
2026-04-04 03:55:49 +00:00
Nexus Dev
2b568a0f5d docs(34-01): complete voice foundation plan — chatFileRoutes, usePiperTts, TtsButton, voiceEnabled
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 03:55:49 +00:00
Nexus Dev
847f316319 feat(34-01): create usePiperTts hook and TtsButton component with piper-tts-web
- Install @mintplex-labs/piper-tts-web as UI dependency
- Create usePiperTts hook with prewarm/speak/stop/status/progress (VOICE-01, VOICE-02)
- tts.stored() checks IndexedDB cache to skip re-download
- tts.download() with progress callback for visible download progress
- tts.predict() returns WAV blob URL for CPU-safe WASM synthesis
- Create TtsButton component showing download progress during prewarm
- TtsButton shows Volume2/VolumeX icons for idle/speaking states
2026-04-04 03:55:49 +00:00
Nexus Dev
36746ed17b feat(34-01): register chatFileRoutes + nexusSettingsRoutes in app.ts, add voiceEnabled to nexus-settings
- Add chatFileRoutes(db, storageService) after assistantHandoffRoutes (inside boardMutationGuard)
- Add nexusSettingsRoutes() after chatFileRoutes
- Extend nexusSettingsSchema with voiceEnabled: z.boolean().default(false)
- Update default return values in nexusSettingsService.get() to include voiceEnabled: false
- Add voiceEnabled?: boolean to NexusSettings client interface in hardware.ts
2026-04-04 03:55:49 +00:00
Nexus Dev
c0710c7100 docs(34-voice): create phase plan 2026-04-04 03:55:49 +00:00
Nexus Dev
1d8f1c5912 docs(34): research phase voice domain 2026-04-04 03:55:49 +00:00
Nexus Dev
784d09d929 docs(34): auto-generated context (discuss skipped) 2026-04-04 03:55:49 +00:00
Nexus Dev
1c29517514 docs(phase-33): complete persistent memory phase 2026-04-04 03:55:49 +00:00
Nexus Dev
222d00c57f feat(33-03): real AI streaming with memory injection + assistant handoff
Replace streamEcho with Puter proxy AI call, inject memory facts as
system message, append memory after each turn. Assistant-to-PM handoff
creates new conversation with context summary.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 03:55:49 +00:00
Nexus Dev
2dcb24b9ce docs(33-03): complete AI streaming + handoff plan — summary, state, roadmap updated
Real AI streaming with memory injection, SSE format fix, assistant-to-PM handoff route,
wired UI button. All 24 tests pass. Phase 33 complete.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 03:55:49 +00:00
Nexus Dev
7bb72a5a2f feat(33-01,33-02): memory service + sanitizer, personal assistant page
33-01: memory-sanitizer, assistant-memory service, REST routes, 17 tests
33-02: useNexusMode hook, PersonalAssistantPage, sidebar nav, route wiring

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 03:55:49 +00:00
Nexus Dev
f6caca9458 docs(33-02): complete personal-assistant-page plan — summary, state, roadmap updated 2026-04-04 03:55:49 +00:00
Nexus Dev
1d222e3e8a docs(33-01): complete persistent-memory memory-service plan
- SUMMARY.md with 17 passing tests, 2 deviations documented
- STATE.md advanced to plan 2, progress 80%
- ROADMAP.md updated: Phase 33 1/3 plans complete
- REQUIREMENTS.md: ASST-01 and ASST-02 marked complete
2026-04-04 03:55:49 +00:00
Nexus Dev
664392e9eb docs(33): create phase plan — memory service, personal assistant page, AI streaming + handoff 2026-04-04 03:55:49 +00:00
Nexus Dev
2efed8797e docs(33): research phase persistent memory domain 2026-04-04 03:55:49 +00:00
Nexus Dev
9ff01aeda9 docs(33): auto-generated context (discuss skipped) 2026-04-04 03:55:49 +00:00