From 5313dfb501d48c0f6300380278453b831238f102 Mon Sep 17 00:00:00 2001 From: Mikkel Georgsen Date: Fri, 10 Apr 2026 05:28:22 +0000 Subject: [PATCH] docs(02): auto-generated context (autonomous mode) --- .planning/phases/02-ai-pipeline/02-CONTEXT.md | 97 +++++++++++++++++++ 1 file changed, 97 insertions(+) create mode 100644 .planning/phases/02-ai-pipeline/02-CONTEXT.md diff --git a/.planning/phases/02-ai-pipeline/02-CONTEXT.md b/.planning/phases/02-ai-pipeline/02-CONTEXT.md new file mode 100644 index 0000000..0ceec02 --- /dev/null +++ b/.planning/phases/02-ai-pipeline/02-CONTEXT.md @@ -0,0 +1,97 @@ +# Phase 2: AI Pipeline - Context + +**Gathered:** 2026-04-10 +**Status:** Ready for planning +**Mode:** Auto-generated (autonomous mode) + + +## Phase Boundary + +Users can submit 1-3 photos and receive a structured NetBox-ready record with AI-extracted specs, suggested category/tags, and a quality gate status reflecting confidence. This phase delivers the oMLX setup (or mocked interface), three-tier AI orchestrator, photo intake endpoint, and confidence-based quality gate wiring. + + + + +## Implementation Decisions + +### AI Tier Configuration +- Single `go-openai` client with configurable BaseURL per tier +- Tier 1: oMLX at http://localhost:8000/v1 (Gemma 4 E4B default) +- Tier 2: OpenRouter at https://openrouter.ai/api/v1 (research agent) +- Tier 3: OpenRouter (Opus for Lab Advisor — deferred to Phase 6) +- Config JSON drives tier routing — no code changes to swap providers + +### Photo Intake +- POST /api/intake accepts multipart/form-data with 1-3 photo files +- Photos encoded as base64 and sent to Gemma 4 vision endpoint +- AI extracts: serial number, model, manufacturer, specs, category, suggested tags +- Confidence score determines catalog_status: high → indexed, low → needs_research + +### Quick Add Mode +- Config flag enables skip-review flow for high-confidence items +- High confidence = no review screen, goes straight to NetBox CREATE + +### Development Strategy +- oMLX may not be installed on dev machine — use a mock AI client for unit tests +- Integration tests skip gracefully when oMLX unreachable +- Expose `AIClient` interface so production uses oMLX, tests use mock + +### Infrastructure +- PostgreSQL at postgresql://homelabby:homelabby_2024_secure@10.5.0.109:5432/homelabby (if needed) +- DragonFlyDB at redis://:nUq/IfoIQJf/kouckKHRQOk7vV0NwCuI@10.5.0.10:6379 (WAQ available) +- NetBox at http://10.5.0.130:8000/api (placeholder token — integration tests skip) +- SearXNG at http://10.5.0.129:8080/search (Phase 7 will use — stub only) +- oMLX target: http://localhost:8000/v1 (will be running on Mac Mini M4) + +### Claude's Discretion +All implementation details are at Claude's discretion. Use phase 1 artifacts (NetBox client, quality gate, HW-ID) as building blocks. Function calling for SearXNG is deferred to Phase 7. + + + + +## Existing Code Insights + +### Reusable Assets from Phase 1 +- `internal/netbox/client.go` — NetBox CRUD (ListDevices, GetDevice, PatchCustomFields) +- `internal/netbox/hwid.go` — AllocateNextHWID for intake ID assignment +- `internal/netbox/custom_fields.go` — BuildCustomFieldsPatch for writing AI results +- `internal/netbox/tags.go` — SyncTags for AI-suggested tag sync +- `internal/inventory/quality_gate.go` — CatalogStatus state machine +- `internal/inventory/catalog_updater.go` — UpdateCatalogStatus wires gate to NetBox +- `internal/api/router.go` — chi router (add POST /api/intake) +- `internal/config/config.go` — viper config (add AI provider config) +- `internal/queue/waq.go` — write-ahead queue for deferred NetBox writes + +### Established Patterns +- All files use package path git.georgsen.dk/hwlab +- Integration tests use t.Skip() when credentials unavailable +- Concrete values over references in all code +- chi HTTP handlers in internal/api/handlers/ + +### Integration Points +- POST /api/intake → internal/api/handlers/intake.go (new) +- AI orchestrator → internal/ai/ (new package) +- Mock AI client for tests → internal/ai/mock.go + + + + +## Specific Ideas + +- Use sashabaranov/go-openai as recommended in STACK.md +- AI config lives in ai_config.json (separate from main config.json for easy swap) +- Intake handler should use the write-ahead queue if NetBox unreachable +- Photo encoding: accept JPEG/PNG, convert to base64, send as vision message + + + + +## Deferred Ideas + +- SearXNG function calling (Phase 7) +- Lab Advisor tier 3 (Phase 6) +- Natural language search (Phase 7) +- Actual Gemma 4 model tuning/fine-tuning +- React UI for intake (Phase 3) + +