From d31675f98cacf7b4b7dfe6a325a0fc9999c074f7 Mon Sep 17 00:00:00 2001 From: Mikkel Georgsen Date: Wed, 4 Feb 2026 22:12:41 +0000 Subject: [PATCH] docs(02): complete telegram-integration phase Co-Authored-By: Claude Opus 4.5 --- .planning/REQUIREMENTS.md | 24 ++-- .planning/ROADMAP.md | 6 +- .planning/STATE.md | 49 ++++--- .../02-telegram-integration/02-02-SUMMARY.md | 120 ++++++++++++++++++ 4 files changed, 163 insertions(+), 36 deletions(-) create mode 100644 .planning/phases/02-telegram-integration/02-02-SUMMARY.md diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md index 3d66aec..112c3a9 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/REQUIREMENTS.md @@ -7,10 +7,10 @@ ### Core Messaging -- [ ] **MSG-01**: Incoming Telegram message auto-triggers Claude Code session and sends response back -- [ ] **MSG-02**: Typing indicator shown in Telegram while Claude is processing -- [ ] **MSG-03**: Brief tool-call progress notifications sent to Telegram (e.g. "Reading file...") -- [ ] **MSG-04**: Files/photos attached in Telegram saved to session folder and available to Claude +- [x] **MSG-01**: Incoming Telegram message auto-triggers Claude Code session and sends response back +- [x] **MSG-02**: Typing indicator shown in Telegram while Claude is processing +- [x] **MSG-03**: Brief tool-call progress notifications sent to Telegram (e.g. "Reading file...") +- [x] **MSG-04**: Files/photos attached in Telegram saved to session folder and available to Claude ### Session Management @@ -28,13 +28,13 @@ ### Output Modes -- [ ] **OUT-01**: Default mode: final answer + brief tool-call progress notifications +- [x] **OUT-01**: Default mode: final answer + brief tool-call progress notifications - [ ] **OUT-02**: `/verbose` mode: stream full Claude Code output across multiple messages - [ ] **OUT-03**: `/smart` mode: smart truncation with long outputs sent as file attachments ### Infrastructure -- [ ] **INFRA-01**: Runs as systemd user service alongside existing bot +- [x] **INFRA-01**: Runs as systemd user service alongside existing bot - [x] **INFRA-02**: Async subprocess management via asyncio (no PIPE deadlocks) - [x] **INFRA-03**: Concurrent stdout/stderr draining prevents buffer overflow @@ -67,10 +67,10 @@ | Requirement | Phase | Status | |-------------|-------|--------| -| MSG-01 | Phase 2 | Pending | -| MSG-02 | Phase 2 | Pending | -| MSG-03 | Phase 2 | Pending | -| MSG-04 | Phase 2 | Pending | +| MSG-01 | Phase 2 | Complete | +| MSG-02 | Phase 2 | Complete | +| MSG-03 | Phase 2 | Complete | +| MSG-04 | Phase 2 | Complete | | SESS-01 | Phase 1 | Complete | | SESS-02 | Phase 1 | Complete | | SESS-03 | Phase 3 | Pending | @@ -79,10 +79,10 @@ | LIFE-02 | Phase 3 | Pending | | LIFE-03 | Phase 3 | Pending | | LIFE-04 | Phase 3 | Pending | -| OUT-01 | Phase 2 | Pending | +| OUT-01 | Phase 2 | Complete | | OUT-02 | Phase 4 | Pending | | OUT-03 | Phase 4 | Pending | -| INFRA-01 | Phase 2 | Pending | +| INFRA-01 | Phase 2 | Complete | | INFRA-02 | Phase 1 | Complete | | INFRA-03 | Phase 1 | Complete | diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 4362344..95da1e6 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -13,7 +13,7 @@ This project transforms Telegram into a mobile interface for Claude Code, enabli Decimal phases appear between their surrounding integers in numeric order. - [x] **Phase 1: Session & Process Foundation** - Multi-session filesystem structure with subprocess management -- [ ] **Phase 2: Telegram Integration** - Core messaging loop with file handling and typing indicators +- [x] **Phase 2: Telegram Integration** - Core messaging loop with file handling and typing indicators - [ ] **Phase 3: Lifecycle Management** - Idle timeout, suspend/resume, and graceful cleanup - [ ] **Phase 4: Output Modes** - Advanced output control for verbose and smart modes @@ -51,7 +51,7 @@ Plans: Plans: - [x] 02-01-PLAN.md -- Persistent subprocess engine + message formatting utilities -- [ ] 02-02-PLAN.md -- Bot integration with batching, file handling, and systemd service +- [x] 02-02-PLAN.md -- Bot integration with batching, file handling, and systemd service ### Phase 3: Lifecycle Management **Goal**: Sessions suspend automatically after idle period and resume transparently with full context @@ -89,6 +89,6 @@ Phases execute in numeric order: 1 → 2 → 3 → 4 | Phase | Plans Complete | Status | Completed | |-------|----------------|--------|-----------| | 1. Session & Process Foundation | 3/3 | Complete | 2026-02-04 | -| 2. Telegram Integration | 1/2 | In progress | - | +| 2. Telegram Integration | 2/2 | Complete | 2026-02-04 | | 3. Lifecycle Management | 0/TBD | Not started | - | | 4. Output Modes | 0/TBD | Not started | - | diff --git a/.planning/STATE.md b/.planning/STATE.md index 1f0ecf1..a9dcafe 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -5,34 +5,34 @@ See: .planning/PROJECT.md (updated 2026-02-04) **Core value:** Frictionless conversation with Claude Code from anywhere via Telegram — no SSH, no manual inbox checking, just message and get a response. -**Current focus:** Phase 2 (Telegram Integration) +**Current focus:** Phase 2 complete, ready for Phase 3 ## Current Position -Phase: 2 of 4 (Telegram Integration) -Plan: 02-01 complete (1 of 2 plans completed) -Status: In progress -Last activity: 2026-02-04 — Completed 02-01-PLAN.md (Persistent subprocess + telegram utils) +Phase: 2 of 4 (Telegram Integration) — COMPLETE +Plan: 02-02 complete (2 of 2 plans completed) +Status: Complete +Last activity: 2026-02-04 — Completed 02-02-PLAN.md (Bot integration with batching, file handling, systemd service) -Progress: [████░░░░░░░░░░░] 33% +Progress: [██████████░░░░░] 50% ## Performance Metrics **Velocity:** -- Total plans completed: 4 -- Average duration: 8 min -- Total execution time: 0.53 hours +- Total plans completed: 5 +- Average duration: 23 min +- Total execution time: 1.95 hours **By Phase:** | Phase | Plans | Total | Avg/Plan | |-------|-------|-------|----------| | 1 | 3 | 27min | 9min | -| 2 | 1 | 5min | 5min | +| 2 | 2 | 95min | 48min | **Recent Trend:** -- Last 3 plans: 01-02 (9min), 01-03 (15min), 02-01 (5min) -- Trend: Improved velocity (yolo mode + better planning) +- Last 3 plans: 01-03 (15min), 02-01 (5min), 02-02 (90min) +- 02-02 included interactive debugging and human verification *Updated after each plan completion* @@ -57,6 +57,11 @@ Recent decisions affecting current work: - Persistent subprocess instead of fresh per turn: Eliminates ~1s spawn overhead, maintains context (02-01) - Split messages at 4000 chars (not 4096): Leaves room for MarkdownV2 escape expansion (02-01) - Never split inside code blocks: Track in_code_block state, only split when safe (02-01) +- Dynamic typing event lookup: Callbacks reference typing_tasks dict by session name, not captured event (02-02) +- --append-system-prompt instead of --system-prompt: Preserves Claude Code model identity (02-02) +- --dangerously-skip-permissions: Full tool access in non-interactive subprocess (02-02) +- Full model ID in persona: Use claude-sonnet-4-5-20250929 instead of alias (02-02) +- Stream-json NDJSON format: {type: user, message: {role: user, content: text}} (02-02) ### Pending Todos @@ -65,17 +70,19 @@ None yet. ### Blockers/Concerns **Phase 1 (Session & Process Foundation) — COMPLETE** -- ~~Claude Code CLI --resume behavior with pipes vs PTY unknown~~ — RESOLVED: Research confirms pipes + stream-json is correct approach -- ~~Output format for tool calls not documented~~ — RESOLVED: stream-json format documented and implemented +- ~~Claude Code CLI --resume behavior with pipes vs PTY unknown~~ — RESOLVED +- ~~Output format for tool calls not documented~~ — RESOLVED -**Phase 2 (Telegram Integration) — IN PROGRESS:** -- ~~Non-persistent process model (spawned fresh per turn)~~ — RESOLVED: Refactored to persistent subprocess with stream-json stdin (02-01) -- Message batching strategy needs validation against actual Claude output patterns -- File upload flow (save to session dir, notify Claude) needs end-to-end testing +**Phase 2 (Telegram Integration) — COMPLETE** +- ~~Non-persistent process model (spawned fresh per turn)~~ — RESOLVED (02-01) +- ~~Message batching strategy needs validation~~ — RESOLVED: Works with 2s debounce (02-02) +- ~~File upload flow needs end-to-end testing~~ — RESOLVED: User-verified (02-02) +- ~~Typing indicator not visible despite API success~~ — RESOLVED: Stale task cleanup + dynamic event lookup (02-02) +- ~~Model identifies as wrong version~~ — RESOLVED: --append-system-prompt preserves CLI defaults (02-02) ## Session Continuity -Last session: 2026-02-04T19:17:24Z -Stopped at: Completed 02-01-PLAN.md (Persistent subprocess + telegram utils) +Last session: 2026-02-04T22:10:00Z +Stopped at: Completed Phase 2 (Telegram Integration) Resume file: None -Next: 02-02-PLAN.md (Bot integration with batching, file handling, systemd service) +Next: Phase 3 (Lifecycle Management) diff --git a/.planning/phases/02-telegram-integration/02-02-SUMMARY.md b/.planning/phases/02-telegram-integration/02-02-SUMMARY.md new file mode 100644 index 0000000..b56ed3c --- /dev/null +++ b/.planning/phases/02-telegram-integration/02-02-SUMMARY.md @@ -0,0 +1,120 @@ +--- +phase: 02-telegram-integration +plan: 02 +subsystem: telegram +tags: [telegram, bot, typing-indicator, batching, file-handling, systemd, markdownv2] + +# Dependency graph +requires: + - phase: 02-telegram-integration + plan: 01 + provides: Persistent subprocess and telegram utils +provides: + - End-to-end Telegram-Claude Code messaging with typing indicators + - Message batching with debounce for rapid sequential messages + - Photo/document handling with auto-analysis + - Tool call progress notifications + - Systemd user service for reliability +affects: [03-lifecycle-management, bot-reliability] + +# Tech tracking +tech-stack: + added: + - MessageBatcher (asyncio.Queue + debounce timer) + - systemd user service (KillMode=mixed) + patterns: + - "Dynamic typing event lookup via session name in typing_tasks dict" + - "Debounce-based message batching with asyncio.Queue" + - "--append-system-prompt preserves Claude Code defaults while adding persona" + - "--dangerously-skip-permissions for full tool access in non-interactive mode" + +key-files: + created: + - telegram/message_batcher.py + - ~/.config/systemd/user/telegram-bot.service + modified: + - telegram/bot.py + - telegram/claude_subprocess.py + - telegram/telegram_utils.py + - telegram/personas/default.json + +key-decisions: + - "Dynamic typing event lookup: callbacks reference typing_tasks dict by session name, not captured event" + - "--append-system-prompt instead of --system-prompt: preserves Claude Code model identity" + - "--dangerously-skip-permissions: allows all tools in non-interactive subprocess" + - "Full model ID in persona (claude-sonnet-4-5-20250929) instead of alias" + - "Stream-json NDJSON format: {type: user, message: {role: user, content: text}}" + +patterns-established: + - "Pattern 1: Dynamic callback binding - closures look up mutable state by key instead of capturing immutable reference" + - "Pattern 2: Stale task cleanup - check task.done() and delete from dict before creating replacement" + - "Pattern 3: Persona via append - use --append-system-prompt to layer persona on top of CLI defaults" + +# Metrics +duration: ~90min (including interactive debugging with user) +completed: 2026-02-04 +--- + +# Phase 2 Plan 2: Bot Integration Summary + +**End-to-end Telegram-Claude Code messaging with typing, batching, file handling, and systemd service** + +## Performance + +- **Duration:** ~90 minutes (including interactive testing and bug fixes) +- **Started:** 2026-02-04T19:30:00Z +- **Completed:** 2026-02-04T22:10:00Z +- **Tasks:** 3 (2 auto + 1 human-verify checkpoint) +- **Files created:** 2 +- **Files modified:** 4 + +## Accomplishments + +- Created MessageBatcher with debounce-based batching (2s timer, asyncio.Queue) +- Overhauled bot.py: typing indicators, tool call progress, message batching, file handling +- Created systemd user service with KillMode=mixed for subprocess cleanup +- Fixed stream-json NDJSON input format (nested message object required) +- Fixed typing indicator lifecycle (stale task cleanup + dynamic event lookup) +- Switched to --append-system-prompt to preserve Claude Code model identity +- Added --dangerously-skip-permissions for full tool access +- Set full model ID (claude-sonnet-4-5-20250929) in default persona +- Human-verified: messages flow, typing indicators show, model identifies correctly + +## Task Commits + +1. **Task 1: MessageBatcher + bot.py overhaul** - `f246d18` (feat) +2. **Task 2: Systemd user service** - created during same executor run +3. **Task 3: Human verification** - bugs found and fixed: + - `2d0d4da` - typing indicator lifecycle, model identity, tool permissions + +## Bugs Found During Testing + +1. **Stream-json input format**: `{"type":"user","content":"..."}` rejected by Claude Code; correct format is `{"type":"user","message":{"role":"user","content":"..."}}` +2. **Typing indicator stale tasks**: After completion, typing task stayed in dict with set event; next message reused dead task and typing never started +3. **Typing indicator event mismatch**: Subprocess callbacks captured specific stop_typing event at creation time; new messages created new events but callbacks still set the old one +4. **Model identity lost**: `--system-prompt` overwrote Claude Code's built-in system prompt (which includes model version); fixed with `--append-system-prompt` +5. **Tool permissions blocked**: Non-interactive `-p` mode blocked tools requiring permission prompts; fixed with `--dangerously-skip-permissions` +6. **Model alias resolution**: `sonnet` alias resolved to 3.5 Sonnet in CLI; fixed by using full model ID + +## Files Created/Modified + +**Created:** +- `telegram/message_batcher.py` - Debounce-based message batching with asyncio.Queue +- `~/.config/systemd/user/telegram-bot.service` - Systemd user service with KillMode=mixed + +**Modified:** +- `telegram/bot.py` - Full overhaul: dynamic typing callbacks, stale task cleanup, batching, file handling +- `telegram/claude_subprocess.py` - NDJSON fix, --dangerously-skip-permissions, --append-system-prompt, full cmd logging +- `telegram/telegram_utils.py` - Debug logging for typing indicator +- `telegram/personas/default.json` - Full model ID instead of alias + +## Deviations from Plan + +1. **Typing indicator architecture changed**: Plan assumed capturing stop_typing event in callbacks; changed to dynamic lookup from typing_tasks dict by session name to fix lifecycle bugs +2. **System prompt approach changed**: Plan used `--system-prompt`; changed to `--append-system-prompt` to preserve Claude Code defaults +3. **Added --dangerously-skip-permissions**: Not in original plan; needed for tool access in non-interactive mode +4. **Model ID changed**: Plan used `sonnet` alias; changed to full ID after alias resolution issues + +--- +*Phase: 02-telegram-integration* +*Completed: 2026-02-04*