docs(02-01): complete persistent subprocess + telegram utils plan

Tasks completed: 2/2
- Refactor ClaudeSubprocess to persistent process with stream-json I/O
- Create telegram_utils.py with message formatting and typing indicator

SUMMARY: .planning/phases/02-telegram-integration/02-01-SUMMARY.md

.planning/ROADMAP.md

@@ -50,7 +50,7 @@ Plans:
 **Plans:** 2 plans
 
 Plans:
-- [ ] 02-01-PLAN.md -- Persistent subprocess engine + message formatting utilities
+- [x] 02-01-PLAN.md -- Persistent subprocess engine + message formatting utilities
 - [ ] 02-02-PLAN.md -- Bot integration with batching, file handling, and systemd service
 
 ### Phase 3: Lifecycle Management
@@ -89,6 +89,6 @@ Phases execute in numeric order: 1 → 2 → 3 → 4
 | Phase | Plans Complete | Status | Completed |
 |-------|----------------|--------|-----------|
 | 1. Session & Process Foundation | 3/3 | Complete | 2026-02-04 |
-| 2. Telegram Integration | 0/2 | Not started | - |
+| 2. Telegram Integration | 1/2 | In progress | - |
 | 3. Lifecycle Management | 0/TBD | Not started | - |
 | 4. Output Modes | 0/TBD | Not started | - |

.planning/STATE.md

@@ -5,33 +5,34 @@
 See: .planning/PROJECT.md (updated 2026-02-04)
 
 **Core value:** Frictionless conversation with Claude Code from anywhere via Telegram — no SSH, no manual inbox checking, just message and get a response.
-**Current focus:** Phase 1 (Session & Process Foundation)
+**Current focus:** Phase 2 (Telegram Integration)
 
 ## Current Position
 
-Phase: 1 of 4 (Session & Process Foundation)
+Phase: 2 of 4 (Telegram Integration)
-Plan: 01-03 complete (3 of 3 plans completed)
+Plan: 02-01 complete (1 of 2 plans completed)
-Status: Phase 1 complete
+Status: In progress
-Last activity: 2026-02-04 — Completed 01-03-PLAN.md (Bot command integration)
+Last activity: 2026-02-04 — Completed 02-01-PLAN.md (Persistent subprocess + telegram utils)
 
-Progress: [███░░░░░░░░░░░░] 25%
+Progress: [████░░░░░░░░░░░] 33%
 
 ## Performance Metrics
 
 **Velocity:**
-- Total plans completed: 3
+- Total plans completed: 4
-- Average duration: 10 min
+- Average duration: 8 min
-- Total execution time: 0.50 hours
+- Total execution time: 0.53 hours
 
 **By Phase:**
 
 | Phase | Plans | Total | Avg/Plan |
 |-------|-------|-------|----------|
 | 1     | 3     | 27min | 9min     |
+| 2     | 1     | 5min  | 5min     |
 
 **Recent Trend:**
-- Last 3 plans: 01-01 (3min), 01-02 (9min), 01-03 (15min)
+- Last 3 plans: 01-02 (9min), 01-03 (15min), 02-01 (5min)
-- Trend: Increasing complexity (more orchestrator fixes)
+- Trend: Improved velocity (yolo mode + better planning)
 
 *Updated after each plan completion*
 
@@ -53,6 +54,9 @@ Recent decisions affecting current work:
 - Callback architecture: Decouple subprocess from session management via on_output/on_error/on_complete/on_status (01-02)
 - Sibling imports over package imports: Avoids shadowing pip telegram package (01-03)
 - Archive sessions with tar+pigz: Compression + cleanup to sessions_archive/ (01-03)
+- Persistent subprocess instead of fresh per turn: Eliminates ~1s spawn overhead, maintains context (02-01)
+- Split messages at 4000 chars (not 4096): Leaves room for MarkdownV2 escape expansion (02-01)
+- Never split inside code blocks: Track in_code_block state, only split when safe (02-01)
 
 ### Pending Todos
 
@@ -64,21 +68,14 @@ None yet.
 - ~~Claude Code CLI --resume behavior with pipes vs PTY unknown~~ — RESOLVED: Research confirms pipes + stream-json is correct approach
 - ~~Output format for tool calls not documented~~ — RESOLVED: stream-json format documented and implemented
 
-**Phase 2 (Persistent Processes) — BLOCKER:**
+**Phase 2 (Telegram Integration) — IN PROGRESS:**
-- **Non-persistent process model:** Current implementation spawns fresh `claude -p` per turn (~1s overhead each)
+- ~~Non-persistent process model (spawned fresh per turn)~~ — RESOLVED: Refactored to persistent subprocess with stream-json stdin (02-01)
-  - Design goal was persistent processes that suspend when switching sessions
-  - This gap affects response latency and user experience
-  - Must be addressed before Phase 3 integration
-  - Requires refactoring ClaudeSubprocess to lifecycle-manage process instances
-
-**Phase 3 (Telegram Integration):**
 - Message batching strategy needs validation against actual Claude output patterns
-- Optimal chunk split points require experimentation
+- File upload flow (save to session dir, notify Claude) needs end-to-end testing
-- Streaming responses / typing indicators needed for UX during long API waits
 
 ## Session Continuity
 
-Last session: 2026-02-04T18:00:00Z
+Last session: 2026-02-04T19:17:24Z
-Stopped at: Completed 01-03-PLAN.md (Bot command integration)
+Stopped at: Completed 02-01-PLAN.md (Persistent subprocess + telegram utils)
 Resume file: None
-Next: Phase 2 ready (persistent process model needed)
+Next: 02-02-PLAN.md (Bot integration with batching, file handling, systemd service)

.planning/phases/02-telegram-integration/02-01-SUMMARY.md

@@ -0,0 +1,148 @@
+---
+phase: 02-telegram-integration
+plan: 01
+subsystem: infra
+tags: [asyncio, subprocess, python, claude-code-cli, stream-json, telegram, markdown, typing-indicator]
+
+# Dependency graph
+requires:
+  - phase: 01-session-process-foundation
+    provides: Session management and subprocess foundations
+provides:
+  - Persistent Claude Code subprocess with stream-json I/O (eliminates ~1s spawn overhead per turn)
+  - Tool call progress notifications via on_tool_use callback
+  - Smart message splitting respecting MarkdownV2 code block boundaries
+  - MarkdownV2 escaping with code block awareness
+  - Typing indicator loop for long operations
+affects: [02-02, bot-integration, message-handling]
+
+# Tech tracking
+tech-stack:
+  added:
+    - telegram.constants.ChatAction (typing indicators)
+  patterns:
+    - "Persistent subprocess with NDJSON stdin streaming"
+    - "Stream-json event routing including tool_use events"
+    - "Smart message splitting at code block boundaries"
+    - "Context-sensitive MarkdownV2 escaping (preserves code blocks)"
+    - "Typing indicator loop with asyncio.Event cancellation"
+
+key-files:
+  created:
+    - telegram/telegram_utils.py
+  modified:
+    - telegram/claude_subprocess.py
+
+key-decisions:
+  - "Use persistent subprocess instead of fresh process per turn (eliminates ~1s overhead)"
+  - "Write NDJSON to stdin with drain() to prevent pipe buffer deadlock"
+  - "Persistent readers run for process lifetime, result events mark not busy but don't exit loop"
+  - "Parse content_block_start events for tool_use to enable progress notifications"
+  - "Split messages at 4000 chars (not 4096) to leave room for escape character expansion"
+  - "Never split inside code blocks (track in_code_block state while iterating lines)"
+  - "Escape MarkdownV2 special chars only outside code blocks (regex pattern to identify code regions)"
+
+patterns-established:
+  - "Pattern 1: Persistent stdin streaming - start() spawns process, send_message() writes NDJSON, readers run until process dies"
+  - "Pattern 2: Tool use notifications - Parse content_block_start with type=tool_use, extract name and input dict, pass to callback"
+  - "Pattern 3: Smart message splitting - Track code block state, split only when NOT in code block and would exceed limit"
+  - "Pattern 4: Code-aware escaping - Split by code regions (regex), escape only non-code text, rejoin"
+  - "Pattern 5: Typing loop - Re-send every 4s via asyncio.wait_for timeout until Event is set"
+
+# Metrics
+duration: 5min
+completed: 2026-02-04
+---
+
+# Phase 2 Plan 1: Persistent Subprocess + Telegram Utils Summary
+
+**Persistent Claude Code subprocess with stream-json I/O and Telegram message formatting utilities for bot integration**
+
+## Performance
+
+- **Duration:** 5 minutes
+- **Started:** 2026-02-04T19:12:28Z
+- **Completed:** 2026-02-04T19:17:24Z
+- **Tasks:** 2
+- **Files created:** 1
+- **Files modified:** 1
+
+## Accomplishments
+
+- Refactored ClaudeSubprocess from fresh-process-per-turn to persistent process model
+- Eliminated ~1s spawn overhead per message turn
+- Added stream-json stdin I/O for NDJSON message delivery to persistent process
+- Implemented tool_use event parsing and on_tool_use callback for progress notifications
+- Created telegram_utils.py with smart message splitting, MarkdownV2 escaping, and typing indicator loop
+- Smart splitting respects code block boundaries (never splits inside triple-backtick blocks)
+- MarkdownV2 escaping preserves code blocks while escaping 17 special characters outside code
+- Typing indicator loop re-sends every 4 seconds with clean asyncio.Event cancellation
+
+## Task Commits
+
+Each task was committed atomically:
+
+1. **Task 1: Refactor ClaudeSubprocess to persistent process** - `6a115a4` (refactor)
+2. **Task 2: Create telegram_utils.py** - `6b624d7` (feat)
+
+## Files Created/Modified
+
+**Created:**
+- `telegram/telegram_utils.py` - Message formatting utilities: split_message_smart (code-block-aware splitting), escape_markdown_v2 (context-sensitive escaping), typing_indicator_loop (4s re-send pattern)
+
+**Modified:**
+- `telegram/claude_subprocess.py` - Refactored to persistent subprocess with --input-format stream-json stdin, persistent stdout/stderr readers, tool_use event parsing, NDJSON message delivery via stdin.write + drain pattern
+
+## Decisions Made
+
+1. **Persistent process architecture**: Spawn Claude Code once with stream-json I/O, write NDJSON to stdin for each turn. Eliminates ~1s spawn overhead per message and maintains conversation context across turns.
+
+2. **Tool use callback design**: Extract tool name and raw input dict from content_block_start events, pass directly to callback. Bot layer (Plan 02) will format human-readable progress messages. Keeps subprocess layer format-agnostic.
+
+3. **Smart splitting strategy**: Track code block state (in/out of triple backticks), split only when NOT in code block and would exceed 4000 char limit. If code block itself exceeds limit, include it whole (Telegram handles overflow).
+
+4. **Escape only outside code**: Use regex to identify code regions (```...``` and `...`), escape MarkdownV2 special chars only in non-code text. Prevents breaking code syntax while properly escaping user-facing text.
+
+5. **4000 char split threshold**: Use 4000 instead of Telegram's 4096 limit to leave room for MarkdownV2 escape character expansion (each special char becomes 2 chars with backslash).
+
+6. **Typing loop cancellation pattern**: Use asyncio.wait_for with 4s timeout on Event.wait(). Clean cancellation when Event is set, automatic re-send on timeout. Pattern from PTB community best practices.
+
+## Deviations from Plan
+
+None - plan executed exactly as written.
+
+## Issues Encountered
+
+**Issue 1: ChatAction import path**
+- **Problem:** Initial import `from telegram import ChatAction` failed with ImportError
+- **Root cause:** python-telegram-bot v22+ moved ChatAction to telegram.constants
+- **Resolution:** Changed to `from telegram.constants import ChatAction`
+- **Impact:** Minor, caught during testing, fixed immediately
+
+## User Setup Required
+
+None - no external service configuration required.
+
+## Next Phase Readiness
+
+**Ready for Plan 02 (Bot Integration):**
+- ClaudeSubprocess provides persistent process with tool call notifications
+- telegram_utils provides all message formatting needed for bot layer
+- Smart splitting and escaping handle Claude's output for Telegram delivery
+- Typing indicator loop ready for integration with message handlers
+- stdin.write + drain pattern ensures no pipe deadlock
+
+**Integration points for Plan 02:**
+- Call `subprocess.start()` once per session
+- Pass `on_tool_use` callback to format and send progress messages
+- Use `split_message_smart()` on Claude output before sending to Telegram
+- Use `escape_markdown_v2()` if sending with MarkdownV2 parse mode
+- Wrap `subprocess.send_message()` with `typing_indicator_loop()` background task
+- Handle file uploads by saving to session directory
+- Implement message batching with debounce timer
+
+**No blockers or concerns.**
+
+---
+*Phase: 02-telegram-integration*
+*Completed: 2026-02-04*