homelabby/.planning/phases/02-ai-pipeline/02-CONTEXT.md

Phase 2: AI Pipeline - Context

Gathered: 2026-04-10 Status: Ready for planning Mode: Auto-generated (autonomous mode)

## Phase Boundary

Users can submit 1-3 photos and receive a structured NetBox-ready record with AI-extracted specs, suggested category/tags, and a quality gate status reflecting confidence. This phase delivers the oMLX setup (or mocked interface), three-tier AI orchestrator, photo intake endpoint, and confidence-based quality gate wiring.

## Implementation Decisions

AI Tier Configuration

• Single go-openai client with configurable BaseURL per tier
• Tier 1: oMLX at http://localhost:8000/v1 (Gemma 4 E4B default)
• Tier 2: OpenRouter at https://openrouter.ai/api/v1 (research agent)
• Tier 3: OpenRouter (Opus for Lab Advisor — deferred to Phase 6)
• Config JSON drives tier routing — no code changes to swap providers

Photo Intake

• POST /api/intake accepts multipart/form-data with 1-3 photo files
• Photos encoded as base64 and sent to Gemma 4 vision endpoint
• AI extracts: serial number, model, manufacturer, specs, category, suggested tags
• Confidence score determines catalog_status: high → indexed, low → needs_research

Quick Add Mode

• Config flag enables skip-review flow for high-confidence items
• High confidence = no review screen, goes straight to NetBox CREATE

Development Strategy

• oMLX may not be installed on dev machine — use a mock AI client for unit tests
• Integration tests skip gracefully when oMLX unreachable
• Expose AIClient interface so production uses oMLX, tests use mock

Infrastructure

• PostgreSQL at postgresql://homelabby:homelabby_2024_secure@10.5.0.109:5432/homelabby (if needed)
• DragonFlyDB at redis://:nUq/IfoIQJf/kouckKHRQOk7vV0NwCuI@10.5.0.10:6379 (WAQ available)
• NetBox at http://10.5.0.130:8000/api (placeholder token — integration tests skip)
• SearXNG at http://10.5.0.129:8080/search (Phase 7 will use — stub only)
• oMLX target: http://localhost:8000/v1 (will be running on Mac Mini M4)

Claude's Discretion

All implementation details are at Claude's discretion. Use phase 1 artifacts (NetBox client, quality gate, HW-ID) as building blocks. Function calling for SearXNG is deferred to Phase 7.

<code_context>

Existing Code Insights

Reusable Assets from Phase 1

• internal/netbox/client.go — NetBox CRUD (ListDevices, GetDevice, PatchCustomFields)
• internal/netbox/hwid.go — AllocateNextHWID for intake ID assignment
• internal/netbox/custom_fields.go — BuildCustomFieldsPatch for writing AI results
• internal/netbox/tags.go — SyncTags for AI-suggested tag sync
• internal/inventory/quality_gate.go — CatalogStatus state machine
• internal/inventory/catalog_updater.go — UpdateCatalogStatus wires gate to NetBox
• internal/api/router.go — chi router (add POST /api/intake)
• internal/config/config.go — viper config (add AI provider config)
• internal/queue/waq.go — write-ahead queue for deferred NetBox writes

Established Patterns

• All files use package path git.georgsen.dk/hwlab
• Integration tests use t.Skip() when credentials unavailable
• Concrete values over references in all code
• chi HTTP handlers in internal/api/handlers/

Integration Points

• POST /api/intake → internal/api/handlers/intake.go (new)
• AI orchestrator → internal/ai/ (new package)
• Mock AI client for tests → internal/ai/mock.go

</code_context>

## Specific Ideas

• Use sashabaranov/go-openai as recommended in STACK.md
• AI config lives in ai_config.json (separate from main config.json for easy swap)
• Intake handler should use the write-ahead queue if NetBox unreachable
• Photo encoding: accept JPEG/PNG, convert to base64, send as vision message

## Deferred Ideas

• SearXNG function calling (Phase 7)
• Lab Advisor tier 3 (Phase 6)
• Natural language search (Phase 7)
• Actual Gemma 4 model tuning/fine-tuning
• React UI for intake (Phase 3)