Commit graph

2265 commits

Author SHA1 Message Date
Nexus Dev
a01c28dff2 feat: Phase 40 — Job Infrastructure (content jobs, SSE events, namespaced storage) 2026-04-05 09:55:08 +00:00
Nexus Dev
87272b79fc docs: create milestone v1.7 roadmap (6 phases) 2026-04-04 11:48:53 +00:00
Nexus Dev
cc569b4cd6 docs: define milestone v1.7 requirements 2026-04-04 11:24:04 +00:00
Nexus Dev
e4a103cd9b docs: complete project research 2026-04-04 04:25:21 +00:00
Nexus Dev
0abf30b8c1 docs: start milestone v1.7 Content Generation 2026-04-04 04:02:58 +00:00
Nexus Dev
f85cab192a [nexus] fix: resolve rebase artifacts — duplicate function, missing import, stale SDK types 2026-04-04 03:58:39 +00:00
Nexus Dev
3abe91ab43 chore: complete v1.6 Voice Pipeline + Minimal Message Bridge milestone 2026-04-04 03:55:50 +00:00
Nexus Dev
bf5c69eeb1 docs: milestone v1.6 audit — 23/23 requirements passed 2026-04-04 03:55:50 +00:00
Nexus Dev
66b19541a6 docs(phase-39): complete phase execution 2026-04-04 03:55:50 +00:00
Nexus Dev
af420780bf docs(39-02): complete voice hardware detection plan
- Add 39-02-SUMMARY.md with task results and self-check
- Update STATE.md progress (92%), decisions, session
- Update ROADMAP.md phase 39 progress (2 plans, 1 summary)
- Mark requirements ONBRD-01, ONBRD-02 complete
2026-04-04 03:55:50 +00:00
Nexus Dev
8f97e69184 feat(39-02): VoiceStep hardware-aware UI with conditional enable/skip
- Add VoiceCapability interface to ui/src/api/hardware.ts
- Export VoiceCapability type from useHardwareInfo.ts
- VoiceStep accepts voiceCapability prop, renders conditionally
- Insufficient hardware: shows capability note with skip-only button
- Binaries present: shows green checkmarks next to STT/TTS labels
- Missing binaries on sufficient hardware: shows install note, dimmed Enable
- NexusOnboardingWizard passes voiceCapability from hardware probe to VoiceStep
2026-04-04 03:55:50 +00:00
Nexus Dev
25e3eda0b9 feat(39-02): voice capability probe in hardware service
- Add VoiceCapability interface with whisperAvailable, piperAvailable, voiceTierSufficient
- Extend HardwareInfo with voiceCapability field
- Add detectVoiceCapability() probing whisper-cpp/whisper and piper with 2s timeout each
- voiceTierSufficient: true for apple_silicon/gpu, or cpu_only with >= 4GB free RAM
- Wrap voice probe in 3s timeout to avoid slowing hardware detection
- Route automatically includes voiceCapability via existing HardwareInfo return
2026-04-04 03:55:50 +00:00
Nexus Dev
0f46d9b3bd test(39-02): add failing tests for voice capability detection 2026-04-04 03:55:50 +00:00
Nexus Dev
de544a6dde docs(39-01): complete sentence-buffered TTS streaming + multi-language synthesis plan 2026-04-04 03:55:50 +00:00
Nexus Dev
9dc00f3a88 feat(39-01): ChatVoicePlayer sentence-buffered streaming playback
- Add streaming prop (default true) to ChatVoicePlayerProps
- Connect to POST /api/synthesize/stream via fetch + ReadableStream
- Parse SSE lines manually from response body stream
- First sentence audio begins playing as soon as first chunk arrives
- Subsequent sentences auto-play in sequence from audioQueue
- Show 'Sentence N of M' progress indicator during streaming playback
- Dot progress bar shows completed vs pending sentences
- Falls back to full-fetch mode on stream error or streaming=false
- Clean up all object URLs on unmount or new text
2026-04-04 03:55:50 +00:00
Nexus Dev
22beb245f2 feat(39-01): sentence-buffered TTS streaming + multi-language synthesis
- Export splitSentences() with title-abbreviation protection (Dr., Mr. etc.)
- Add synthesizeSentenceStream() AsyncGenerator yielding per-sentence audio chunks
- Add synthesizeMultiLang() synthesizing same text in N voices via Promise.all
- Add POST /api/synthesize/stream SSE endpoint with base64 audio per sentence
- Add POST /api/synthesize/multi-lang returning array of voiceId+audio pairs
- Existing POST /api/synthesize unchanged (backward compatible)
2026-04-04 03:55:50 +00:00
Nexus Dev
6be251a9fb test(39-01): add failing tests for sentence streaming and multi-lang synthesis 2026-04-04 03:55:50 +00:00
Nexus Dev
29f9345e9a docs: update STATE.md for phase 39 start 2026-04-04 03:55:50 +00:00
Nexus Dev
3690a47fd5 docs(39): create phase plan for voice polish
Two plans in wave 1 (parallel):
- 39-01: Sentence-buffered TTS streaming + multi-language synthesis (VPIPE-07, VPIPE-08)
- 39-02: Onboarding voice hardware capability probe (ONBRD-01, ONBRD-02)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 03:55:50 +00:00
Nexus Dev
b54130d991 docs(39): auto-generated context (discuss skipped) 2026-04-04 03:55:50 +00:00
Nexus Dev
006cc44d85 docs(phase-38): complete phase execution 2026-04-04 03:55:50 +00:00
Nexus Dev
262af05116 docs(38-02): complete Telegram voice handling plan — OGG download + Whisper STT + Piper TTS reply 2026-04-04 03:55:50 +00:00
Nexus Dev
6435ccb1c4 feat(38-02): add voice message handling + TTS reply to Telegram bridge
- Refactor text relay into shared relayToAgent() used by both text/voice handlers
- Add bot.on('message:voice') handler: send 'Transcribing...' immediately, process async
- Download OGG from Telegram CDN via ctx.getFile() + fetch, transcribe via voicePipelineService
- Synthesize agent responses to OGG Opus via transcodeToOggOpus() and ctx.replyWithVoice()
- TTS failure degrades gracefully (text reply already sent, voice is bonus)
- telegram.ts stays at 322 lines (under 500-line TGRAM-06 constraint)
2026-04-04 03:55:50 +00:00
Nexus Dev
2edb205f57 docs(38-01): complete Telegram bridge core plan — telegramService + telegramRoutes 2026-04-04 03:55:50 +00:00
Nexus Dev
2467b54167 docs(38-03): complete Telegram onboarding step plan
- TelegramStep component with BotFather guided setup and token validation
- NexusOnboardingWizard updated to 7-step flow with Telegram at step 5
- ONBRD-03 requirement marked complete
2026-04-04 03:55:50 +00:00
Nexus Dev
34bfbe06e1 feat(38-01): wire telegramService + telegramRoutes into app.ts
- Import telegramService, telegramRoutes, nexusSettingsService
- Mount /telegram routes under /api prefix
- Conditionally start Telegram bot on boot if telegramToken is configured
- Token route restarts bot after saving new token
2026-04-04 03:55:50 +00:00
Nexus Dev
ed21eb339c feat(38-03): insert TelegramStep as step 5 in NexusOnboardingWizard
- Import TelegramStep component
- Insert Telegram step at position 5 (between Voice and Root Directory)
- Shift Root Directory from step 5 → step 6
- Shift Summary from step 6 → step 7
- Update step indicator from 'of 5' to 'of 6'
- Update summary indicator from step===6 to step===7
- Update all setStep() navigation callbacks accordingly
- Update error message referencing step 6 for root directory
2026-04-04 03:55:50 +00:00
Nexus Dev
69269fea14 feat(38-01): install grammY, create telegramService + telegramRoutes
- Install grammy v2 for long polling Telegram bot
- telegramService: text relay handler, agent prefix, session map, deleteWebhook lifecycle
- telegramRoutes: POST /telegram/token (getMe validation), GET /telegram/status
- telegram.ts under 500 lines (187 lines)
2026-04-04 03:55:50 +00:00
Nexus Dev
713e92be0f feat(38-03): create TelegramStep onboarding component
- BotFather numbered instructions (4-step setup guide)
- Token input with live validation via POST /api/telegram/token
- Success state showing connected bot username
- Error state with descriptive message
- Skip/Back/Next navigation; Next enabled only after validation
2026-04-04 03:55:50 +00:00
Nexus Dev
9959d1b77e docs(38): create 3 plans in 2 waves for Telegram bridge 2026-04-04 03:55:50 +00:00
Nexus Dev
4073625cb0 docs(38): research Telegram bridge phase 2026-04-04 03:55:50 +00:00
Nexus Dev
1d64140575 docs(38): auto-generated context (discuss skipped) 2026-04-04 03:55:50 +00:00
Nexus Dev
0974a8ba27 docs(phase-37): complete phase execution 2026-04-04 03:55:50 +00:00
Nexus Dev
35be67d019 fix(37): pass voiceMode in ChatPanel handleEdit path + add verification 2026-04-04 03:55:50 +00:00
Nexus Dev
54d1f02d9d docs(37-04): complete chat voice integration plan — voiceMode threading + VoiceMicButton wiring
- 37-04-SUMMARY.md created with full execution record
- STATE.md updated with decisions and session info
- ROADMAP.md copied from phase-37 branch
2026-04-04 03:55:50 +00:00
Nexus Dev
7d3820a84f feat(37-04): wire VoiceMicButton, VoiceModeToggle, ChatVoiceBadge, voiceMode into chat UI
- ChatInput: replace VoiceRecordButton with VoiceMicButton (VAD-powered)
- ChatInput: add VoiceModeToggle above input when enableVoiceInput=true
- ChatMessage: add ChatVoiceBadge render for voice_input and voice_full messageTypes
- ChatMessage: auto-play reads from localStorage nexus:voice:autoplay key
- ChatPanel: import and call useVoiceMode, extract mode as voiceMode
- ChatPanel: pass voiceMode as third arg to all startStream calls (5 call sites)
2026-04-04 03:55:50 +00:00
Nexus Dev
9435458033 feat(37-04): add voiceMode to chatApi.postMessageAndStream + useStreamingChat.startStream
- postMessageAndStream data type extended with optional voiceMode field
- startStream signature updated: (userMessage, agentId?, voiceMode?)
- voiceMode forwarded into fetch body via postMessageAndStream call
2026-04-04 03:55:50 +00:00
Nexus Dev
8bf2a65a0b feat(37-03): VoiceModeToggle three-pill component + useVoiceMode hook
- VoiceModeToggle: Text / Voice In / Full Voice pills with active/inactive styling
- Auto-play checkbox in full_voice mode, persists to nexus:voice:autoplay in localStorage
- useVoiceMode: reads/writes voiceMode via PATCH /api/nexus/settings with loading state
  (deviation Rule 3: created missing blocking dependency for VoiceModeToggle)
2026-04-04 03:55:50 +00:00
Nexus Dev
6b60f42a25 feat(37-03): ChatVoicePlayer + ChatVoiceBadge components
- ChatVoicePlayer: POST /api/synthesize, play/pause controls, autoPlay support, blob URL cleanup
- ChatVoiceBadge: Voice badge, SPOKEN/DETAILED parsing, collapsible full markdown for voice_full
2026-04-04 03:55:50 +00:00
Nexus Dev
a50daa2129 docs(37-03): complete voice output components plan 2026-04-04 03:55:50 +00:00
Nexus Dev
c85a5016ac docs(37-02): complete voice recording components plan
- SUMMARY.md: encodeWav, useVadRecorder, useVoiceMode, VoiceWaveform, VoiceMicButton
- STATE.md: advanced to plan 3, 71% progress, added decisions
- ROADMAP.md: updated phase-37 progress (2/4 plans done)
- REQUIREMENTS.md: marked WCHAT-01..03, WCHAT-05 complete
2026-04-04 03:55:50 +00:00
Nexus Dev
21ecf23d9a feat(37-02): VoiceWaveform canvas component and VoiceMicButton
- VoiceWaveform: 80x32 canvas with Web Audio AnalyserNode (fftSize=64), 20 animated bars drawn from frequency data using --primary color
- VoiceMicButton: three visual states — idle (Mic icon), recording (VoiceWaveform + ring-2 ring-primary), processing (Loader2 animate-spin)
- All three states have correct aria-labels per UI spec copywriting contract
2026-04-04 03:55:50 +00:00
Nexus Dev
0d0b17c8a0 feat(37-02): encodeWav utility, useVadRecorder + useVoiceMode hooks
- encodeWav: 44-byte WAV header encoder (RIFF/WAVE/fmt/data), PCM mono 16-bit
- useVadRecorder: wraps useMicVAD with startOnLoad:false, auto-stop on speech end, POSTs to /api/transcribe
- useVoiceMode: reads/writes voiceMode from GET/PATCH /api/nexus/settings with optimistic update
2026-04-04 03:55:50 +00:00
Nexus Dev
16371f01f5 docs(37-01): complete server prerequisites + VAD browser infrastructure plan
- Create 37-01-SUMMARY.md with task results, deviations, and self-check
- STATE.md: advance to plan 2, add 3 decisions, update progress to 57%
- ROADMAP.md: phase 37 in progress (1/4 plans complete)
- REQUIREMENTS.md: mark WCHAT-01, WCHAT-02, WCHAT-04 complete
2026-04-04 03:55:50 +00:00
Nexus Dev
de1287fba3 feat(37-01): install VAD library, copy ONNX assets, configure Vite COOP/COEP headers
- Add @ricky0123/vad-react dependency to ui/package.json
- Add copy-vad-assets npm script for reproducible asset copying
- Copy vad.worklet.bundle.min.js, silero_vad_legacy.onnx, silero_vad_v5.onnx to ui/public/
- Add COOP/COEP headers to Vite dev server config (SharedArrayBuffer support in dev)
- Update pnpm lockfile
2026-04-04 03:55:50 +00:00
Nexus Dev
c42a64b7a2 feat(37-01): add COOP/COEP headers to Express server for SharedArrayBuffer support
- Add Cross-Origin-Opener-Policy: same-origin middleware before all routes
- Add Cross-Origin-Embedder-Policy: require-corp middleware before all routes
- Required for @ricky0123/vad-react (VAD uses SharedArrayBuffer internally)
2026-04-04 03:55:50 +00:00
Nexus Dev
904be2e0ef docs(37-01): complete server prerequisites + VAD browser infrastructure plan
- Create 37-01-SUMMARY.md with task results and deviations
- Update STATE.md: advance to plan 2, add decisions, update progress to 57%
- Update ROADMAP.md: phase 37 in progress (1/4 plans complete)
- Mark WCHAT-01, WCHAT-02, WCHAT-04 complete in REQUIREMENTS.md
2026-04-04 03:55:50 +00:00
Nexus Dev
1eaa6c4b3e docs(37): create 4 plans in 3 waves for web chat voice UI 2026-04-04 03:55:50 +00:00
Nexus Dev
fdc956c6a6 docs(37): phase research — VAD, COOP/COEP, component architecture 2026-04-04 03:55:50 +00:00
Nexus Dev
f2f381a3a2 docs(37): UI design contract for web-chat-voice-ui
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 03:55:50 +00:00