From 85b83b2b1700db4ae486da81e90ed602bd2e2ec2 Mon Sep 17 00:00:00 2001 From: Mikkel Georgsen Date: Wed, 4 Feb 2026 17:34:55 +0000 Subject: [PATCH] docs(01-02): complete ClaudeSubprocess plan Tasks completed: 1/1 - Create ClaudeSubprocess module with spawn, I/O, and lifecycle management SUMMARY: .planning/phases/01-session-process-foundation/01-02-SUMMARY.md --- .planning/STATE.md | 31 +++-- .../01-02-SUMMARY.md | 122 ++++++++++++++++++ 2 files changed, 139 insertions(+), 14 deletions(-) create mode 100644 .planning/phases/01-session-process-foundation/01-02-SUMMARY.md diff --git a/.planning/STATE.md b/.planning/STATE.md index ee87480..3d7a317 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -10,27 +10,27 @@ See: .planning/PROJECT.md (updated 2026-02-04) ## Current Position Phase: 1 of 4 (Session & Process Foundation) -Plan: - (not yet planned) -Status: Ready to plan -Last activity: 2026-02-04 — Roadmap created with 4 phases, 18 requirements mapped +Plan: 2 of 3 completed +Status: In progress +Last activity: 2026-02-04 — Completed 01-02-PLAN.md (ClaudeSubprocess module) -Progress: [░░░░░░░░░░] 0% +Progress: [██░░░░░░░░] 20% ## Performance Metrics **Velocity:** -- Total plans completed: 0 -- Average duration: - min -- Total execution time: 0.0 hours +- Total plans completed: 1 +- Average duration: 9 min +- Total execution time: 0.15 hours **By Phase:** | Phase | Plans | Total | Avg/Plan | |-------|-------|-------|----------| -| - | - | - | - | +| 1 | 1 | 9min | 9min | **Recent Trend:** -- Last 5 plans: None yet +- Last 5 plans: 01-02 (9min) - Trend: - *Updated after each plan completion* @@ -46,6 +46,9 @@ Recent decisions affecting current work: - Haiku polling + Opus conversation: Avoids burning expensive Opus tokens on idle monitoring (deferred to v2) - Path-based sessions: Leverages Claude Code's native session-per-directory behavior, files naturally scoped - Extend existing bot: Reuse proven Telegram integration rather than building from scratch +- Asyncio.gather for concurrent stream reading: Prevents pipe deadlock (01-02) +- Fresh process per turn: Spawn new `claude -p` invocation for Phase 1 simplicity (01-02) +- Callback architecture: Decouple subprocess from session management via on_output/on_error/on_complete/on_status (01-02) ### Pending Todos @@ -53,9 +56,9 @@ None yet. ### Blockers/Concerns -**Phase 2 (Process Management):** -- Claude Code CLI --resume behavior with pipes vs PTY unknown — needs empirical testing -- Output format for tool calls not documented — requires experimentation with --output-format stream-json +**Phase 1 (Session & Process Foundation):** +- ~~Claude Code CLI --resume behavior with pipes vs PTY unknown~~ — RESOLVED: Research confirms pipes + stream-json is correct approach +- ~~Output format for tool calls not documented~~ — RESOLVED: stream-json format documented and implemented **Phase 3 (Telegram Integration):** - Message batching strategy needs validation against actual Claude output patterns @@ -63,6 +66,6 @@ None yet. ## Session Continuity -Last session: 2026-02-04 -Stopped at: Roadmap creation complete, ready for Phase 1 planning +Last session: 2026-02-04T17:41:16Z +Stopped at: Completed 01-02-PLAN.md (ClaudeSubprocess module) Resume file: None diff --git a/.planning/phases/01-session-process-foundation/01-02-SUMMARY.md b/.planning/phases/01-session-process-foundation/01-02-SUMMARY.md new file mode 100644 index 0000000..39f8163 --- /dev/null +++ b/.planning/phases/01-session-process-foundation/01-02-SUMMARY.md @@ -0,0 +1,122 @@ +--- +phase: 01-session-process-foundation +plan: 02 +subsystem: infra +tags: [asyncio, subprocess, python, claude-code-cli, stream-json, process-management] + +# Dependency graph +requires: + - phase: 01-session-process-foundation + provides: Session metadata and directory management +provides: + - Claude Code subprocess lifecycle management with crash recovery + - Stream-json event parsing and routing to callbacks + - Concurrent stdout/stderr reading to prevent pipe deadlocks + - Message queueing during Claude processing + - Graceful process termination without zombies +affects: [01-03, telegram-integration, message-handling] + +# Tech tracking +tech-stack: + added: [] + patterns: + - "Asyncio subprocess management with concurrent stream readers" + - "Stream-json event routing to callback functions" + - "Crash recovery with --continue flag" + - "terminate() + wait_for() + kill() fallback pattern" + +key-files: + created: + - telegram/claude_subprocess.py + modified: [] + +key-decisions: + - "Use asyncio.gather for concurrent stdout/stderr reading (prevents pipe deadlock)" + - "Queue messages during processing, send after completion" + - "Auto-restart on crash with --continue flag (max 3 retries)" + - "Spawn fresh process per turn (not stdin piping) for Phase 1 simplicity" + +patterns-established: + - "Pattern 1: Concurrent stream reading - Always use asyncio.gather() for stdout/stderr to prevent pipe buffer overflow deadlocks" + - "Pattern 2: Process lifecycle - terminate() + wait_for(timeout) + kill() + wait() ensures no zombies" + - "Pattern 3: Stream-json parsing - Line-by-line JSON.loads() with try/except, route by event type" + - "Pattern 4: Callback architecture - on_output/on_error/on_complete/on_status for decoupled communication" + +# Metrics +duration: 9min +completed: 2026-02-04 +--- + +# Phase 1 Plan 2: ClaudeSubprocess Module Summary + +**Asyncio-based Claude Code subprocess engine with concurrent stream reading, message queueing, crash recovery, and stream-json event routing via callbacks** + +## Performance + +- **Duration:** 9 minutes +- **Started:** 2026-02-04T17:32:07Z +- **Completed:** 2026-02-04T17:41:16Z +- **Tasks:** 1 +- **Files modified:** 1 + +## Accomplishments +- Created ClaudeSubprocess class that spawns Claude Code CLI with stream-json output +- Implemented concurrent stdout/stderr reading via asyncio.gather to prevent pipe deadlocks +- Built message queueing system for messages received during Claude processing +- Implemented crash recovery with auto-restart using --continue flag (max 3 retries) +- Added graceful process termination with terminate → wait_for → kill → wait pattern (no zombies) +- Established callback-based architecture for decoupled communication with session manager + +## Task Commits + +Each task was committed atomically: + +1. **Task 1: Create ClaudeSubprocess module with spawn, I/O, and lifecycle management** - `8fce10c` (feat) + +## Files Created/Modified +- `telegram/claude_subprocess.py` - Claude Code subprocess lifecycle management with asyncio, handles spawning with persona settings, concurrent stream reading, stream-json event parsing, message queueing, crash recovery, and graceful termination + +## Decisions Made + +1. **Concurrent stream reading pattern**: Use `asyncio.gather()` to read stdout and stderr concurrently, preventing pipe buffer deadlock (verified by research as critical pattern) + +2. **Message queueing strategy**: Queue messages in `asyncio.Queue` while subprocess is busy, process queue after completion callback. This ensures messages don't interrupt active processing. + +3. **Crash recovery approach**: Auto-restart with `--continue` flag up to 3 times with 1-second backoff. Claude Code's session persistence in `.claude/` directory enables context preservation across crashes. + +4. **Fresh process per turn**: Spawn new `claude -p` invocation for each turn rather than piping to stdin. Simpler for Phase 1; Phase 2+ might use `--input-format stream-json` for live piping. + +5. **Callback architecture**: Decouple subprocess management from session management via callbacks (on_output, on_error, on_complete, on_status). Enables clean separation of concerns. + +## Deviations from Plan + +None - plan executed exactly as written. + +## Issues Encountered + +None - implementation followed research patterns without issues. + +## User Setup Required + +None - no external service configuration required. + +## Next Phase Readiness + +**Ready for Plan 03 (Telegram Integration):** +- ClaudeSubprocess provides complete subprocess lifecycle management +- Callback architecture enables clean integration with Telegram bot message handlers +- Message queueing handles concurrent messages during processing +- Process termination and crash recovery are production-ready + +**Integration points for Plan 03:** +- Pass Telegram message text to `send_message()` +- Route `on_output` callback to Telegram message sending +- Route `on_error` callback to Telegram error notifications +- Use `on_status` callback for typing indicators +- Call `terminate()` during session cleanup/switching + +**No blockers or concerns.** + +--- +*Phase: 01-session-process-foundation* +*Completed: 2026-02-04*