homelab/.planning/codebase/ARCHITECTURE.md

Architecture

Analysis Date: 2026-02-04

Pattern Overview

Overall: Hub-and-spoke service orchestration with API-driven infrastructure management.

Key Characteristics:

• Centralized management container (VMID 102 - mgmt) coordinating all infrastructure
• Layered abstraction: CLI helpers → REST APIs → external services
• Event-driven notifications (Telegram bot bridges management layer to user)
• Credential-based authentication for all service integrations

Layers

Management Layer:

• Purpose: Orchestration and automation entry point for the homelab
• Location: /home/mikkel/homelab (git repository in mgmt container)
• Contains: CLI helper scripts (~/bin/*), Telegram bot, documentation
• Depends on: Remote SSH access to container/VM IP addresses, Proxmox API, service REST APIs
• Used by: Claude Code automation, Telegram bot commands, cron jobs

API Integration Layer:

• Purpose: Abstracts service APIs into simple CLI interfaces
• Location: ~/bin/ (pve, npm-api, dns, pbs, beszel, kuma, updates, telegram)
• Contains: Python and Bash wrappers around external service APIs
• Depends on: Proxmox API, Nginx Proxy Manager API, Technitium DNS API, PBS REST API, Beszel PocketBase, Uptime Kuma REST API, Telegram Bot API
• Used by: Telegram bot, CI/CD automation, interactive CLI usage

Service Layer:

• Purpose: Individual hosted services providing infrastructure capabilities
• Location: Distributed across containers (NPM, DNS, PBS, Dockge, Forgejo, etc.)
• Contains: Docker containers, LXC services, backup systems
• Depends on: PVE host networking, shared storage, external integrations
• Used by: API layer, end-user access via web UI or CLI

Data & Communication Layer:

• Purpose: State persistence and inter-service communication
• Location: Shared storage (~/stuff - ZFS bind mount), credential files (~/.config/*/credentials)
• Contains: Backup data, configuration files, Telegram inbox/images/files
• Depends on: PVE ZFS dataset, filesystem access
• Used by: All services, backup/restore operations

Data Flow

Infrastructure Query Flow (e.g., pve list):

1. User invokes CLI helper: ~/bin/pve list
2. Helper loads credentials from ~/.config/pve/credentials
3. Helper authenticates to Proxmox API at core.georgsen.dk:8006 using token auth
4. Proxmox returns cluster resource state (VMs/containers)
5. Helper formats and displays output to user

Service Management Flow (e.g., dns add myhost 10.5.0.50):

1. User invokes: ~/bin/dns add myhost 10.5.0.50
2. DNS helper loads credentials and authenticates to Technitium at 10.5.0.2:5380
3. Helper makes HTTP API call to add A record
4. Technitium stores in zone file and updates DNS records
5. Helper confirms success to user

Backup Status Flow (e.g., /pbs command in Telegram):

1. Telegram user sends /pbs command
2. Bot handler in telegram/bot.py executes ~/bin/pbs status
3. PBS helper SSH's to 10.5.0.6 as root
4. SSH command reads backup logs and GC status from PBS container
5. Helper formats human-readable output
6. Bot sends result back to Telegram chat (truncated to 4000 chars for Telegram API limit)

State Management:

• Credentials: Stored in ~/.config/*/credentials files (sourced at runtime)
• Telegram messages: Appended to telegram/inbox file for Claude to read
• Media uploads: Saved to telegram/images/ and telegram/files/ with timestamps
• Authorization: telegram/authorized_users file maintains allowlist of chat IDs

Key Abstractions

Helper Scripts (API Adapters):

• Purpose: Translate user intent into remote service API calls
• Examples: ~/bin/pve, ~/bin/dns, ~/bin/pbs, ~/bin/beszel, ~/bin/kuma
• Pattern: Load credentials → authenticate → execute command → format output
• Language: Mix of Python (pve, updates, telegram) and Bash (dns, pbs, beszel, kuma)

Telegram Bot:

• Purpose: Provides two-way interactive access to management functions
• Implementation: telegram/bot.py using python-telegram-bot library
• Pattern: Command handlers dispatch to helper scripts, results sent back to user
• Channels: Commands (e.g., /pbs), free-text messages saved to inbox, photos/files downloaded

Service Registry (Documentation):

• Purpose: Centralized reference for service locations and access patterns
• Implementation: homelab-documentation.md and CLAUDE.md
• Contents: IP addresses, ports, authentication methods, SSH targets, network topology

Entry Points

CLI Usage (Direct):

• Location: ~/bin/{helper} scripts
• Triggers: Manual invocation by user or cron jobs
• Responsibilities: Execute service operations, format output, validate inputs

Telegram Bot:

• Location: telegram/bot.py (systemd service: telegram-bot.service)
• Triggers: Telegram message or command from authorized user
• Responsibilities: Authenticate user, route command/message, execute via helper scripts, send response

Automation Scripts:

• Location: Potential cron jobs or scheduled tasks
• Triggers: Time-based scheduling
• Responsibilities: Execute periodic management tasks (e.g., backup checks, updates)

Manual Execution:

• Location: Interactive shell in mgmt container
• Triggers: User SSH session
• Responsibilities: Run helpers for ad-hoc infrastructure management

Error Handling

Strategy: Graceful degradation with informative messaging.

Patterns:

• CLI helpers return non-zero exit codes on failure (exception handling in Python, set -e in Bash)
• Timeout protection: Telegram bot commands have 30-second timeout (configurable per command)
• Service unavailability: Caught in try/except blocks, fall back to next option (e.g., pve tries LXC first, then QEMU)
• Credential failures: Load-time validation, clear error message if credentials file missing
• Network errors: SSH timeouts, API connection failures logged to stdout/stderr

Cross-Cutting Concerns

Logging:

• Telegram bot uses Python stdlib logging (INFO level, writes to systemd journal)
• CLI helpers write directly to stdout/stderr
• PBS helper uses SSH error output for remote command failures

Validation:

• Telegram bot validates hostnames (alphanumeric + dots + hyphens only) before ping
• DNS helper validates that name and IP are provided before API call
• PVE helper validates VMID is integer before API call

Authentication:

• Credentials stored in ~/.config/{service}/credentials as simple key=value files
• Sourced at runtime (Bash) or read at startup (Python)
• Token-based auth for Proxmox (no password in memory)
• Basic auth for DNS and other REST APIs (credentials URL-encoded if needed)
• Bearer token for Uptime Kuma (API key-based)

---

Architecture analysis: 2026-02-04