mikkel/homelabby
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs: complete project research

Browse source
This commit is contained in:
Mikkel Georgsen 2026-04-09 23:35:26 +00:00
parent 1374c3bc3b
commit 5d7c5e90a2
5 changed files with 1244 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

388
.planning/research/ARCHITECTURE.md Normal file
View file

@ -0,0 +1,388 @@
# Architecture Research
**Domain:** AI-powered hardware inventory management with USB peripheral control
**Researched:** 2026-04-09
**Confidence:** HIGH (project stack is well-defined; patterns are established Go idioms)
## Standard Architecture
### System Overview
```
┌─────────────────────────────────────────────────────────────────────┐
│ Browser (React SPA) │
│ ┌────────────┐ ┌───────────────┐ ┌──────────────┐ │
│ │ Inventory │ │ Lab Advisor │ │ USB Testing │ │
│ │ Dashboard │ │ Chat View │ │ Workflow UI │ │
│ └─────┬──────┘ └──────┬────────┘ └──────┬───────┘ │
└────────┼────────────────┼────────────────────┼───────────────────────┘
│ │ │ HTTP/REST + SSE
┌────────▼────────────────▼────────────────────▼───────────────────────┐
│ Go Backend (Single Binary) │
│ │
│ ┌─────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ HTTP Layer │ │ AI Orch. │ │ USB Manager │ │
│ │ (Handlers) │ │ (3-tier │ │ (Serial/USB │ │
│ │ │ │ pipeline) │ │ devices) │ │
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │
│ │ │ │ │
│ ┌──────▼─────────────────▼──────────────────▼───────┐ │
│ │ Service Layer │ │
│ │ ┌──────────┐ ┌──────────┐ ┌──────────────────┐ │ │
│ │ │ Inventory │ │ Advisor │ │ Label / Testing │ │ │
│ │ │ Service │ │ Service │ │ Service │ │ │
│ │ └──────┬────┘ └────┬─────┘ └────────┬─────────┘ │ │
│ └─────────┼────────────┼──────────────────┼───────────┘ │
│ │ │ │ │
│ ┌─────────▼────────────▼──────────────────▼───────┐ │
│ │ Client / Adapter Layer │ │
│ │ ┌──────────┐ ┌──────────┐ ┌──────────────────┐│ │
│ │ │ NetBox │ │ AI │ │ SQLite (chat ││ │
│ │ │ Client │ │ Client │ │ history only) ││ │
│ │ └──────┬────┘ └────┬─────┘ └──────────────────┘│ │
│ └─────────┼────────────┼─────────────────────────────┘ │
└────────────┼────────────┼──────────────────────────────────────────────┘
│ │
┌───────▼───┐ ┌─────▼──────────────────────────────┐
│ NetBox │ │ AI Inference Layer │
│ LXC 130 │ │ ┌────────────┐ ┌───────────────┐ │
│ REST API │ │ │ oMLX │ │ OpenRouter │ │
└───────────┘ │ │ (Gemma 4 │ │ (Tier 2/3 │ │
│ │ local) │ │ remote) │ │
│ └────────────┘ └───────────────┘ │
└────────────────────────────────────────┘
```
### Component Responsibilities
| Component | Responsibility | Notes |
|-----------|----------------|-------|
| React SPA | All user interaction — intake, search, chat, USB workflows | Served as static files by Go; communicates via REST + SSE |
| HTTP Layer (Go handlers) | Route requests, validate input, marshal/unmarshal JSON | Thin — no business logic |
| AI Orchestrator | 3-tier pipeline: route task to local/remote model, aggregate results | Owns escalation logic, prompt construction |
| USB Manager | Own goroutines per device, serial protocol handling, event fan-out | Manages PRT Qutie, 3x Treedix, FNIRSI FNB58 |
| Inventory Service | Photo intake flow, NetBox record creation, quality gate transitions | The core domain service |
| Advisor Service | Chat session management, context assembly from NetBox, response streaming | Wraps AI Orchestrator with lab-context injection |
| Label/Testing Service | Print job construction, cable test sequencing, results capture | Coordinates USB Manager with inventory records |
| NetBox Client | All NetBox REST calls, custom field mapping, HW-XXXXX ID allocation | Single integration point — no direct NetBox calls elsewhere |
| AI Client | OpenAI-compatible interface pointed at oMLX or OpenRouter | Swappable base URL = swappable model tier |
| SQLite store | Chat history, config, local-only ephemeral state | NOT inventory data — NetBox owns that |
| oMLX (Mac Mini) | Local Gemma 4 inference via MLX, OpenAI-compatible endpoint | Tier 1 — always tried first |
| OpenRouter | Remote model routing to Claude Opus, GPT-4o, etc. | Tier 2/3 — escalated only |
| NetBox (LXC 130) | Sole inventory data store, REST API at 10.5.0.130:8000 | Source of truth |
| SearXNG (LXC 129) | Product research search API at 10.5.0.129:8080 | Called by AI Orchestrator during research agent tasks |
## Recommended Project Structure
```
hwlab/
├── cmd/
│ └── hwlab/
│ └── main.go # Wire-up only: init config, start server
├── internal/
│ ├── api/
│ │ ├── handlers/ # HTTP handlers — thin, delegate to services
│ │ │ ├── inventory.go
│ │ │ ├── advisor.go
│ │ │ └── usb.go
│ │ ├── middleware/ # Auth (if any), logging, CORS
│ │ └── router.go # Route registration
│ ├── ai/
│ │ ├── orchestrator.go # 3-tier routing, escalation logic
│ │ ├── client.go # OpenAI-compatible HTTP client (base URL configurable)
│ │ └── prompts/ # Prompt templates per task type
│ ├── inventory/
│ │ ├── service.go # Intake, quality gate, search
│ │ └── types.go # Domain types for inventory items
│ ├── advisor/
│ │ ├── service.go # Chat session, context assembly
│ │ └── history.go # SQLite persistence for chat
│ ├── usb/
│ │ ├── manager.go # Device lifecycle, goroutine-per-device
│ │ ├── printer/ # PRT Qutie protocol
│ │ ├── tester/ # Treedix cable tester protocol
│ │ └── powermeter/ # FNIRSI FNB58 protocol
│ ├── label/
│ │ └── service.go # QR code generation, print job construction
│ ├── netbox/
│ │ └── client.go # All NetBox REST calls, field mapping
│ ├── search/
│ │ └── client.go # SearXNG JSON API client
│ └── config/
│ └── config.go # App config (env/file), secrets
├── web/ # React SPA build output (embedded via go:embed)
│ └── dist/
├── frontend/ # React TypeScript source
│ ├── src/
│ │ ├── views/
│ │ ├── components/
│ │ └── api/ # Typed fetch wrappers
│ ├── package.json
│ └── vite.config.ts
└── Makefile # build, dev, test, embed-frontend
```
### Structure Rationale
- **internal/:** Go convention — prevents external import of application internals
- **internal/ai/:** Keeps all AI concerns (prompts, routing, client) co-located; orchestrator is the only caller of the AI client
- **internal/usb/:** Each device gets its own sub-package; USB Manager owns the goroutines and exposes typed channels upward
- **internal/netbox/:** Single client prevents NetBox calls from leaking into service or handler layers
- **web/dist/:** Embedded into the Go binary at build time via `go:embed` — single binary deployment, no separate static server needed
- **cmd/hwlab/:** Only wire-up code; no business logic in main
## Architectural Patterns
### Pattern 1: Single Binary with Embedded Frontend
**What:** The React SPA build output is embedded into the Go binary using `go:embed`. The Go HTTP server serves static assets on `/` and API routes on `/api/`.
**When to use:** Single-operator homelab tool where deployment simplicity matters more than independent frontend deploys.
**Trade-offs:** Rebuild binary to update frontend. Acceptable for solo use; would need rethinking for teams.
**Example:**
```go
//go:embed web/dist
var staticFiles embed.FS
mux.Handle("/", http.FileServerFS(staticFiles))
mux.Handle("/api/", apiRouter)
```
### Pattern 2: Goroutine-Per-USB-Device with Channel Fan-Out
**What:** Each USB device runs in a dedicated goroutine. The USB Manager holds typed channels for commands (in) and events (out). HTTP handlers send commands; SSE endpoint subscribes to event stream.
**When to use:** Any time you need concurrent, non-blocking I/O to physical devices without coordination complexity.
**Trade-offs:** Simple but requires careful channel sizing and shutdown sequencing via `context.Context`.
**Example:**
```go
type USBManager struct {
printer *PrinterDevice
testers [3]*TesterDevice
powerMeter *PowerMeterDevice
}
// Each device goroutine:
func (d *PrinterDevice) run(ctx context.Context, cmds <-chan PrintCmd, events chan<- DeviceEvent) {
for {
select {
case cmd := <-cmds:
// send to serial port
case <-ctx.Done():
return
}
}
}
```
### Pattern 3: Three-Tier AI Orchestrator with Eager Local, Lazy Remote
**What:** All AI tasks are attempted first with oMLX (Tier 1 — local Gemma 4). If confidence is below threshold or the task type explicitly requires it, the orchestrator escalates to OpenRouter (Tier 2/3 — Claude Opus or similar). The AI Client struct accepts a base URL, so switching tiers is a URL swap, not a code change.
**When to use:** Local inference is available but not always sufficient; cloud costs must be minimized.
**Trade-offs:** Adds latency on escalation paths. Confidence scoring for vision tasks (photo intake) needs calibration.
**Example:**
```go
type AIClient struct {
BaseURL string // "http://localhost:11434/v1" or "https://openrouter.ai/api/v1"
Model string
APIKey string
}
func (o *Orchestrator) Run(ctx context.Context, task Task) (Result, error) {
result, err := o.tier1.Complete(ctx, task)
if err != nil || result.Confidence < o.threshold {
return o.tier2.Complete(ctx, task)
}
return result, nil
}
```
### Pattern 4: NetBox as External State — Repository Pattern Wrapper
**What:** The `internal/netbox` package implements a typed repository interface over the NetBox REST API. Services call the repository; they never construct raw HTTP calls to NetBox. The repository maps between NetBox's JSON schema and HWLab domain types.
**When to use:** Any time an external service is the source of truth and its API shape should not leak into domain logic.
**Trade-offs:** Extra mapping layer. Worth it: if NetBox API changes, only `internal/netbox` needs updating.
## Data Flow
### Photo Intake Flow (Primary Flow)
```
User uploads photo
↓
POST /api/intake (handler)
↓
InventoryService.Intake(photo)
↓
AIOrchestrator.AnalyzeHardware(photo)
↓ (Tier 1: oMLX/Gemma 4 vision)
→ if low confidence → Tier 2: OpenRouter (Claude Opus vision)
↓
Structured item data (name, category, specs)
↓
AIOrchestrator.Research(item) [optional, if quality gate requires]
↓ (SearXNG search + AI synthesis)
↓
NetBoxClient.CreateItem(item)
↓ → NetBox LXC (REST POST)
↓
HW-XXXXX ID assigned
↓
LabelService.Print(item)
↓
USBManager.Printer.Print(label)
↓ → PRT Qutie (USB serial)
↓
SSE event → frontend: "intake complete, HW-XXXXX"
```
### Cable Testing Flow
```
User initiates test (selects item HW-XXXXX, selects tester port)
↓
POST /api/test (handler)
↓
TestingService.RunTest(itemID, testerIndex)
↓
USBManager.Tester[n].RunTest(cableType)
↓ → Treedix device (USB serial command)
↓
DeviceEvent (test result bytes)
↓
TestingService.ParseResult(bytes)
↓
NetBoxClient.UpdateItemField(itemID, "cable_test_result", result)
↓
SSE event → frontend: test result display
```
### Lab Advisor Chat Flow
```
User sends message
↓
POST /api/advisor/chat (handler, streams response)
↓
AdvisorService.Chat(sessionID, message)
↓
NetBoxClient.GetInventoryContext() [fetch relevant items for context]
↓
AIOrchestrator.Chat(systemPrompt + context + history + message)
↓ (Tier 3: OpenRouter/Claude Opus — always remote for advisor)
↓
Streamed tokens → SSE → frontend chat view
↓
AdvisorService.SaveMessage(sessionID, message, response) → SQLite
```
### Key Data Flow Rules
1. **NetBox is write-through:** Any state change to an inventory item goes to NetBox immediately — no local cache of inventory records.
2. **USB events are push, not poll:** Device goroutines push events onto channels; the SSE handler fans them out to connected browser clients.
3. **AI calls are always async from the user's perspective:** Handlers return immediately with a job ID; progress arrives via SSE.
4. **SQLite is append-only for chat:** No inventory data in SQLite. Chat history rows are never updated, only inserted and read.
## Build Order (Phase Dependencies)
The architecture has a clear dependency DAG that should drive phase sequencing:
```
Phase 1: Go scaffold + NetBox client + basic CRUD
↓ (NetBox client is a dependency of everything)
Phase 2: AI client + local oMLX integration + photo intake pipeline
↓ (AI pipeline needed before intake flow is useful)
Phase 3: React SPA + inventory dashboard (reads from NetBox via Go API)
↓ (frontend needs backend API to be stable)
Phase 4: USB Manager + Printer + label printing flow
↓ (USB hardware arrives 2026-04-13; serial protocols need reverse-engineering)
Phase 5: Cable tester integration + testing workflows
↓ (builds on USB Manager foundation)
Phase 6: Lab Advisor chat (OpenRouter, streaming SSE, SQLite history)
↓ (can be parallel with Phase 4/5 but needs frontend)
Phase 7: SearXNG research agent + quality gate automation
↓ (enhancement layer — all primitives exist by this point)
```
**Critical dependency:** The NetBox client and AI client must be stable before building services that depend on them. Building the NetBox client first (with a thin mock layer for tests) unblocks parallel work on AI and USB.
**USB hardware blocker:** USB device protocols (printer, testers, power meter) cannot be finalized until the hardware arrives (2026-04-13). Phase 4+ USB work should start with a stub/simulator interface that gets replaced once protocols are reverse-engineered.
## Anti-Patterns
### Anti-Pattern 1: Storing Inventory Data Locally
**What people do:** Cache NetBox responses in SQLite or an in-memory map for "performance."
**Why it's wrong:** Creates a second source of truth. NetBox records updated via other tools (direct API, NetBox UI) silently diverge from the local cache. Debugging inventory discrepancies is brutal.
**Do this instead:** Accept the latency of NetBox REST calls. If performance becomes an issue post-MVP, add a read-through cache with short TTL (30–60s) and an explicit invalidation hook.
### Anti-Pattern 2: USB Device Logic in HTTP Handlers
**What people do:** Open serial port in the HTTP handler, write bytes, read response synchronously, close port.
**Why it's wrong:** Serial devices are stateful (connection must persist), blocking (reads block the goroutine), and shared (multiple requests may target the same device). Handler-level serial access causes race conditions and hangs.
**Do this instead:** USB Manager owns all device connections. Handlers send typed commands on a channel and wait for a response on a reply channel (or SSE events for streaming results).
### Anti-Pattern 3: Mixing AI Tiers in Service Logic
**What people do:** Hard-code "use Claude for advisor, use Gemma for intake" calls in service layer code.
**Why it's wrong:** Ties business logic to specific model names. Swapping models (e.g., Gemma 4 → Gemma 4 26B) requires touching service code.
**Do this instead:** Services call the AI Orchestrator with a task type and confidence requirement. The Orchestrator owns model selection. Config file maps task types to model preferences.
### Anti-Pattern 4: Polling USB Devices from the Frontend
**What people do:** Frontend polls `GET /api/usb/status` every second to get device state.
**Why it's wrong:** Creates 1 HTTP request/second per connected client, adds 0–1s latency to test result display, and hammers the serial port with unnecessary reads.
**Do this instead:** Use Server-Sent Events (SSE). USB Manager emits events; Go SSE handler fans out to subscribed browser clients. Instant updates, one persistent connection per client.
## Integration Points
### External Services
| Service | Integration Pattern | Notes |
|---------|---------------------|-------|
| NetBox (LXC 130) | REST API via `internal/netbox` typed client | go-netbox official client or hand-rolled against v4 OpenAPI spec; needs custom fields for HW IDs and quality gate status |
| oMLX (local) | OpenAI-compatible `/v1/chat/completions` + `/v1/completions` with vision | Base URL: `http://localhost:PORT/v1`; Gemma 4 E4B confirmed fits in 16GB |
| OpenRouter | OpenAI-compatible API with `Authorization: Bearer` | Base URL: `https://openrouter.ai/api/v1`; model specified per request |
| SearXNG (LXC 129) | HTTP GET `http://10.5.0.129:8080/search?q=...&format=json` | No auth; called by AI Orchestrator during research tasks |
| PRT Qutie | USB serial (CDC-ACM or HID) | Protocol unknown until hardware arrives; need to sniff USB traffic from official macOS app |
| Treedix testers (x3) | USB serial (likely CDC-ACM) | 3 independent serial ports; USB Manager assigns by device serial number |
| FNIRSI FNB58 | USB HID or CDC-ACM | Power meter; likely streams measurement data continuously |
### Internal Boundaries
| Boundary | Communication | Notes |
|----------|---------------|-------|
| Handler → Service | Direct Go function call | Handlers are thin wrappers; no business logic |
| Service → AI Orchestrator | Direct Go function call; returns via channel for streaming | Orchestrator manages concurrency internally |
| Service → NetBox Client | Direct Go function call (sync); client handles HTTP retries | Client is the only thing that knows about NetBox |
| Service → USB Manager | Typed command channel (in), event channel (out) | Decouples HTTP request lifecycle from serial I/O timing |
| AI Orchestrator → AI Client | Direct Go function call; streaming via `io.Reader` | Client is a thin HTTP wrapper; Orchestrator does prompt assembly |
| HTTP Handler → SSE | Go channel per client connection, fan-out goroutine | SSE handler registers a channel, USB Manager / AI Orchestrator write to it |
| Frontend → Backend | REST JSON for commands, SSE for async results | No WebSocket — SSE is simpler and sufficient for unidirectional push |
## Scaling Considerations
This is a single-operator homelab tool. Scaling beyond one concurrent user is out of scope. The relevant concerns are:
| Concern | Approach |
|---------|----------|
| Memory (16GB shared with oMLX) | Gemma 4 E4B uses ~8–10GB; Go backend + SQLite uses ~100MB; leave headroom for 26B A4B if tested |
| oMLX inference latency | Tier 1 photo analysis: 2–5s expected. Use async + SSE, never block the HTTP response. |
| NetBox API latency | LXC on same network; expect <10ms. No caching needed initially. |
| USB serial reliability | Serial devices disconnect. USB Manager must handle reconnect gracefully without crashing. |
| Single binary size | go:embed of React build adds ~2–5MB. Acceptable. |
## Sources
- [go-netbox official Go client](https://github.com/netbox-community/go-netbox) — MEDIUM confidence (official but may lag NetBox v4.2 custom field API)
- [NetBox REST API Overview](https://netboxlabs.com/docs/netbox/integrations/rest-api/) — HIGH confidence
- [OpenAI Go library (official)](https://github.com/openai/openai-go) — HIGH confidence; base URL override works for any OpenAI-compatible server
- [go.bug.st/serial for USB serial](https://pkg.go.dev/github.com/bugst/go-serial) — HIGH confidence; standard library for serial in Go
- [Go modular monolith patterns](https://medium.com/@hocineelhadj/building-a-simple-backend-with-modular-monolith-architecture-in-go-e2ec7b59bc58) — MEDIUM confidence
- [Azure AI Agent Orchestration Patterns](https://learn.microsoft.com/en-us/azure/architecture/ai-ml/guide/ai-agent-design-patterns) — HIGH confidence for orchestration pattern naming and structure
- [Alex Edwards: Fat Service Pattern for Go](https://www.alexedwards.net/blog/the-fat-service-pattern) — HIGH confidence; well-regarded Go architecture blog
---
*Architecture research for: HWLab — AI-powered homelab hardware inventory management*
*Researched: 2026-04-09*

193
.planning/research/FEATURES.md Normal file
View file

@ -0,0 +1,193 @@
# Feature Research
**Domain:** Self-hosted AI-powered hardware inventory management (homelab)
**Researched:** 2026-04-09
**Confidence:** MEDIUM-HIGH (training knowledge + web research; AI intake patterns are novel)
## Feature Landscape
### Table Stakes (Users Expect These)
Features users assume exist. Missing these = product feels incomplete.
| Feature | Why Expected | Complexity | Notes |
|---------|--------------|------------|-------|
| Asset record CRUD | Fundamental inventory operation — without this there's no product | LOW | Delegated to NetBox via API; HWLab provides the UI layer |
| Unique asset identifiers | Every inventory tool assigns IDs; users expect stable references | LOW | HW-XXXXX scheme; NetBox asset_tag field |
| Asset search / filter | Users need to find items by name, serial, category | LOW | Table search + AI natural language; NetBox query API |
| Asset status tracking | "Available / In Use / Retired / Unknown" is baseline expectation | LOW | NetBox status field; surfaced in dashboard |
| Category / type classification | Users expect grouping by device type (server, cable, SFP, etc.) | LOW | NetBox device types + custom tags |
| Manufacturer + model fields | Every ITAM tool tracks make/model; users expect this | LOW | NetBox native fields, populated by AI intake |
| Serial number field | Required for warranty claims, RMA, deduplication | LOW | NetBox native field; extracted by AI from photo |
| Location / rack / site tracking | Physical placement is core to homelab inventory | MEDIUM | NetBox rack/location hierarchy; populated during intake |
| Attachment / photo storage | Attaching a photo to a record is expected in any modern ITAM tool | LOW | Photo stored in NetBox attachment; used as intake source |
| Audit trail / history | Who did what, when — expected in any system managing physical assets | MEDIUM | NetBox change log; surfaced in record detail |
| Export capability | Users expect to get their data out (CSV, JSON) | LOW | NetBox REST API; direct export via UI or API |
| Label / tag printing | Physical assets need physical labels — QR or barcode is standard | MEDIUM | PRT Qutie via USB; thermal ZPL or ESC/POS printing |
### Differentiators (Competitive Advantage)
Features that set the product apart. Not required, but valued.
| Feature | Value Proposition | Complexity | Notes |
|---------|-------------------|------------|-------|
| AI photo intake (zero-entry) | Upload photo → record created — eliminates manual data entry for 80–90% of items | HIGH | Gemma 4 via oMLX; multimodal vision + structured extraction |
| Three-tier AI escalation | Local-first (Gemma 4) → research agent (OpenRouter) → Lab Advisor (Opus) — cost-optimal and offline-capable | HIGH | Tier routing logic is the novel architectural piece |
| Lab Advisor chat | Strategic homelab Q&A backed by full inventory context — no other ITAM tool offers this | HIGH | Claude Opus via OpenRouter; context injection from NetBox |
| Cable test workflow integration | Automated cable continuity results attached to cable records — unique to physical-lab use cases | HIGH | Treedix USB/DP/HDMI + FNIRSI FNB58; USB serial parsing |
| Live USB power measurement | FNIRSI FNB58 readings (voltage, current, watt) captured and stored against cable/PSU records | HIGH | USB serial; firmware-level protocol reverse engineering |
| Catalog quality gate | Draft → indexed → needs_research → researched → complete lifecycle prevents "garbage in, garbage out" | MEDIUM | State machine on NetBox custom field; AI drives transitions |
| SearXNG product research | AI agent uses private search instance to auto-fill missing specs, EAN/part numbers, pricing | MEDIUM | OpenRouter agent + SearXNG JSON API already available |
| Natural language inventory search | "Show me all SFP+ modules I haven't tested" — no SQL required | MEDIUM | LLM query translation → NetBox API filter; local Gemma 4 |
| NetBox as data store (not a silo) | All data lives in NetBox — reachable by existing NetBox integrations, Ansible, Terraform, etc. | MEDIUM | Constraint that becomes a feature; no vendor lock-in to HWLab |
| Fully offline operation | Standard intake, search, label print work with no internet — OpenRouter only for Tier 2/3 | MEDIUM | Local Gemma 4 + local NetBox + USB printer; already architectured |
| Universal HW-XXXXX ID scheme | Category-agnostic stable IDs printed on QR labels — works for cables, SFPs, servers equally | LOW | ID assigned at intake; encoded in QR URL |
### Anti-Features (Commonly Requested, Often Problematic)
| Feature | Why Requested | Why Problematic | Alternative |
|---------|---------------|-----------------|-------------|
| Multi-user / RBAC | "What if someone else needs access?" | Solo operator tool — RBAC adds schema, middleware, UI complexity with zero immediate payoff | NetBox has RBAC if truly needed; HWLab inherits it |
| Real-time collaboration / chat | Looks modern, like Slack for inventory | Websockets, presence, conflict resolution — massive complexity for a single-operator homelab | Lab Advisor chat is async and single-session; sufficient |
| Mobile companion app / PWA | "I want to scan in the rack" | Separate build pipeline, offline sync, camera API surface area — diverges from web-first | Responsive web UI is good enough for homelab bench work |
| SNMP / network auto-discovery | "Discover everything automatically" | SNMP is unreliable for physical inventory; discovers only what's online; misses cables, SFPs, offline gear | AI photo intake covers the full physical layer including dark equipment |
| Barcode scanner (USB HID) | Fast intake for bulk items | USB HID scanners are commodity; the value is in AI vision, not scan-gun ergonomics | Photo intake is more universal; scanner can be added later as an enhancement |
| Bulk CSV import | "I have a spreadsheet already" | Creates data quality debt immediately; bypasses quality gate and AI enrichment pipeline | Migrate via NetBox native import; use AI intake as the canonical path |
| Predictive failure / warranty alerts | "Alert me when warranty expires" | Date tracking is simple but alert infrastructure (email, push, cron) is disproportionate to homelab context | Add expiry dates as custom fields; query via Lab Advisor when needed |
| Network topology auto-update | Reflect cable changes in NetBox topology diagrams | Topology inference from cable test results is ambiguous; NetBox topology is already manually maintained | Cable test results stored as records; topology updates remain manual |
| Local LLM fine-tuning UI | "Train it on my specific hardware" | MLX fine-tuning is a separate workload; out-of-scope for an inventory tool | Prompt engineering + SearXNG research agent handles novel hardware adequately |
## Feature Dependencies
```
[NetBox API connectivity]
└──requires──> [All inventory CRUD operations]
└──requires──> [AI intake record creation]
└──requires──> [Natural language search]
└──requires──> [Lab Advisor context injection]
[AI photo intake]
└──requires──> [oMLX + Gemma 4 setup]
└──requires──> [NetBox API connectivity]
└──requires──> [NetBox custom fields schema]
└──enhances──> [Catalog quality gate] (sets initial state to "indexed")
[Three-tier AI escalation]
└──requires──> [AI photo intake] (Tier 1 base)
└──requires──> [OpenRouter API key] (Tiers 2+3)
└──requires──> [SearXNG connectivity] (Tier 2 research)
[QR label printing]
└──requires──> [HW-XXXXX ID assignment] (at intake)
└──requires──> [PRT Qutie USB driver / protocol] (hardware arrives 2026-04-13)
└──enhances──> [AI photo intake] (print label as final intake step)
[Cable test workflow]
└──requires──> [NetBox API connectivity] (attach results to cable records)
└──requires──> [Treedix / FNIRSI USB serial protocol] (hardware arrives 2026-04-13)
└──enhances──> [Catalog quality gate] (test result moves cable to "complete")
[Lab Advisor chat]
└──requires──> [NetBox API connectivity] (inventory context)
└──requires──> [OpenRouter API key] (Opus model)
└──enhances──> [Natural language search] (richer Q&A vs bare search)
[Natural language search]
└──requires──> [oMLX + Gemma 4 setup] (local query translation)
└──requires──> [NetBox API connectivity]
[Catalog quality gate]
└──requires──> [NetBox custom fields schema] (status field)
└──enhances──> [AI photo intake] (drives research escalation)
└──enhances──> [Cable test workflow] (marks completion)
```
### Dependency Notes
- **NetBox API connectivity is the foundation**: Every feature reads or writes through it. NetBox setup (custom fields, netbox-inventory plugin) must be Phase 1.
- **oMLX + Gemma 4 requires hardware verification**: 26B A4B variant needs TurboQuant testing on Mac Mini M4 16GB. If it doesn't fit, fall back to E4B. This gates all AI intake features.
- **USB hardware arrives 2026-04-13**: Label printing and cable testing are blocked until then. Protocol reverse engineering is a separate workload.
- **QR label printing enhances but doesn't block intake**: A record can be created without a label; printing is the final step.
- **Lab Advisor requires OpenRouter** — it's the only feature with a hard cloud dependency (Opus is too large for local inference).
## MVP Definition
### Launch With (v1)
Minimum viable to validate the core value proposition (photo → record → label, zero manual entry).
- [ ] NetBox connectivity + custom fields schema — all features depend on this
- [ ] AI photo intake via Gemma 4 (Tier 1) — the core differentiator, must validate
- [ ] HW-XXXXX ID assignment at intake — required for QR label content
- [ ] QR label printing via PRT Qutie — closes the intake loop physically
- [ ] Catalog quality gate (draft/indexed/complete minimum) — prevents garbage records
- [ ] Inventory dashboard with basic search — users need to find what they cataloged
- [ ] Natural language search (basic) — validates the AI search value prop
### Add After Validation (v1.x)
Add once core photo → record → label loop is validated.
- [ ] SearXNG research agent (Tier 2) — add when Tier 1 produces records with gaps
- [ ] Lab Advisor chat (Tier 3) — add after inventory has enough data to be useful for Q&A
- [ ] Cable test workflow (Treedix + FNIRSI) — add once USB protocol is understood
- [ ] Full catalog quality gate state machine (all 5 states) — add after MVP states work
### Future Consideration (v2+)
- [ ] Barcode scanner HID support — only if photo intake proves too slow for high-volume intake
- [ ] Bulk import / migration mode — only if migrating from an existing spreadsheet becomes a real need
- [ ] Advanced audit trail UI — NetBox change log is sufficient for v1
- [ ] Export / reporting views — NetBox API already provides this; UI wrapper is low priority
## Feature Prioritization Matrix
| Feature | User Value | Implementation Cost | Priority |
|---------|------------|---------------------|----------|
| NetBox API + custom fields setup | HIGH | LOW | P1 |
| AI photo intake (Gemma 4 Tier 1) | HIGH | HIGH | P1 |
| HW-XXXXX ID scheme | HIGH | LOW | P1 |
| QR label printing (PRT Qutie) | HIGH | HIGH (USB protocol) | P1 |
| Inventory dashboard + basic search | HIGH | MEDIUM | P1 |
| Natural language search | HIGH | MEDIUM | P1 |
| Catalog quality gate | MEDIUM | MEDIUM | P1 |
| SearXNG research agent (Tier 2) | MEDIUM | MEDIUM | P2 |
| Lab Advisor chat (Tier 3 / Opus) | HIGH | MEDIUM | P2 |
| Cable test workflow (Treedix) | MEDIUM | HIGH (USB protocol) | P2 |
| USB power measurement (FNIRSI) | MEDIUM | HIGH (USB protocol) | P2 |
| Audit trail UI | LOW | LOW | P3 |
| Export / reporting | LOW | LOW | P3 |
| Barcode scanner HID | LOW | LOW | P3 |
**Priority key:**
- P1: Must have for launch
- P2: Should have, add when possible
- P3: Nice to have, future consideration
## Competitor Feature Analysis
| Feature | NetBox (standalone) | Snipe-IT | Homebox | HWLab approach |
|---------|---------------------|----------|---------|----------------|
| Asset CRUD | Yes — full API + UI | Yes | Yes (home focus) | Delegates to NetBox |
| Photo intake / AI vision | No | No | Photo attachments only | AI extracts structured data from photo |
| Label printing | No native | Barcode labels | No | QR thermal label via PRT Qutie at intake |
| Natural language search | No | No | No | Local Gemma 4 query translation |
| AI chat / advisor | No | No | No | Lab Advisor via Opus + full inventory context |
| Cable test integration | No | No | No | Direct USB integration with Treedix/FNIRSI |
| Quality gate / lifecycle | No | Asset status only | No | 5-state AI-driven catalog lifecycle |
| Network documentation | Full (DCIM) | No | No | Inherits via NetBox |
| Self-hosted | Yes | Yes | Yes | Yes — all components local |
| Offline operation | Yes | Partial | Yes | Yes — Tier 1 AI + label print fully offline |
## Sources
- NetBox feature set: https://github.com/netbox-community/netbox (training knowledge, HIGH confidence)
- Snipe-IT feature set: https://github.com/grokability/snipe-it + https://blog.invgate.com/snipe-it-alternative (WebSearch + training, HIGH confidence)
- Homebox (home inventory): https://selfh.st/post/homebox/ (WebSearch, MEDIUM confidence)
- netbox-inventory plugin: https://github.com/ArnesSI/netbox-inventory (WebSearch, MEDIUM confidence)
- ITAM industry standards: training knowledge (MEDIUM confidence — verify against current tool feature lists)
- AI photo intake patterns: training knowledge, no direct competitor exists (LOW confidence for "zero-entry" claims — validate after v1)
---
*Feature research for: AI-powered self-hosted homelab hardware inventory (HWLab)*
*Researched: 2026-04-09*

280
.planning/research/PITFALLS.md Normal file
View file

@ -0,0 +1,280 @@
# Pitfalls Research
**Domain:** AI-powered homelab hardware inventory — Go + USB serial + local AI + NetBox
**Researched:** 2026-04-09
**Confidence:** MEDIUM-HIGH (domain-specific issues verified via community sources; some areas LOW confidence where official docs are thin)
---
## Critical Pitfalls
### Pitfall 1: USB Serial Port Path Churn on Device Replug
**What goes wrong:**
On macOS, `/dev/cu.usbmodem*` and `/dev/tty.usbmodem*` paths are assigned dynamically at plug time. If the Mac Mini is rebooted, USB ports are replugged in a different sequence, or a hub is involved, the same physical device gets a different `/dev/` path. Hard-coding device paths (or deriving them once at startup) means the wrong device gets written to — label printer commands going to the power meter, or goroutines blocking on a disconnected port.
**Why it happens:**
Developers test with one device plugged in, path is stable during development, then reality hits when three USB devices are present and unplugged/replugged during use.
**How to avoid:**
Enumerate devices by USB VID/PID + serial number, not by path. Use `github.com/google/gousb` for enumeration or shell out to `ioreg -p IOUSB` on macOS to resolve stable identifiers to current paths at each open. Re-resolve the path on every reconnect, not once at boot.
**Warning signs:**
- Tests pass with one device, fail silently or misbehave when all three are connected
- Label printing works but data lands in wrong device log
- Goroutine appears to be reading but produces zero bytes (hung on wrong port)
**Phase to address:**
USB device layer — the very first phase that integrates any serial peripheral. Establish the device enumeration abstraction before writing any device-specific protocol code.
---
### Pitfall 2: Goroutine Leak on USB Disconnect
**What goes wrong:**
A goroutine blocked on `serial.Read()` does not unblock when the port is closed from another goroutine or when the device is physically unplugged. The goroutine leaks indefinitely. Over a session where devices are plugged/unplugged several times, leaked goroutines accumulate. On macOS, the port's file descriptor eventually becomes invalid but the goroutine may block in a syscall until process exit.
**Why it happens:**
The standard Go serial libraries (`go.bug.st/serial`, `tarm/serial`) block in `Read()` using OS syscalls. Closing the port from a separate goroutine does not reliably interrupt the blocked read on all platforms. Developers assume `port.Close()` unblocks all readers.
**How to avoid:**
Use `go.bug.st/serial` which has explicit support for unblocking reads via `port.Close()` (tracked in their issue #13). Wrap every read loop with a `context.Context` that is cancelled before `port.Close()`. Use a `select` on a done channel so the read goroutine can exit even if the port close does not interrupt the syscall. Test with `-race` flag.
**Warning signs:**
- `runtime.NumGoroutine()` grows after each replug cycle
- `go tool pprof` goroutine profile shows multiple `read` goroutines in syscall state
- Memory creeps up over a long session
**Phase to address:**
USB device layer. Write a reconnect harness test (plug/unplug 10 times, assert goroutine count is stable) before any feature work on top of the layer.
---
### Pitfall 3: NetBox as Sole Data Store — No Offline or Degraded Mode
**What goes wrong:**
HWLab stores zero inventory data locally. When NetBox LXC 130 is unreachable (Proxmox maintenance, network hiccup, NetBox upgrade), the entire app becomes non-functional — not just degraded. Photo intake, label printing, cable testing results — all blocked on a NetBox write that can never complete. No queue, no cache, no fallback.
**Why it happens:**
The decision to use NetBox as the sole source of truth is architecturally clean and correct for the goal (no data duplication). But "no local DB" is misread as "no local state at all." The difference between inventory data (belongs in NetBox) and operation queuing (belongs locally) gets collapsed.
**How to avoid:**
Maintain a small local SQLite write-ahead queue for pending NetBox operations. Items are written to the queue first (synchronous, fast), then flushed to NetBox asynchronously with retry. The UI reflects queue state, not NetBox state, during flush. This is not a data store — it is a transactional buffer. The advisor chat history and config already live in SQLite per the design; add a `pending_operations` table to the same DB.
**Warning signs:**
- Photo intake blocks waiting for NetBox HTTP response
- Error: "NetBox unavailable" with no recovery path shown in UI
- Label is printed before NetBox record is confirmed — ID mismatch risk
**Phase to address:**
NetBox integration phase. Define the queue schema and flush logic before any intake workflow is built on top.
---
### Pitfall 4: AI Confidently Misidentifies Hardware — No Quality Gate Enforcement
**What goes wrong:**
Gemma 4 returns a plausible but wrong product identification — e.g., a PCIe riser card identified as a "USB hub," a 24-port patch panel identified as "network switch." The record is created in NetBox with wrong type, wrong manufacturer, wrong custom fields. The quality gate state machine (`draft → indexed → needs_research → researched → complete`) exists in the design but is never enforced in code: items advance automatically rather than requiring explicit confirmation for uncertain classifications.
**Why it happens:**
Multimodal LLMs have high confidence scores even on wrong answers for visually ambiguous hardware. The three-tier pipeline is designed to handle this but the escalation triggers are undefined — there is no concrete threshold for "local model is uncertain, escalate to research agent."
**How to avoid:**
The local indexer (Gemma 4) must return a structured confidence score alongside classification. Define hard thresholds: below 0.7 confidence, item is pinned at `needs_research` and flagged for manual review or automatic SearXNG escalation. Never auto-advance past `indexed` without either (a) a confidence score above threshold or (b) explicit operator confirmation. Store the raw AI response and confidence in the NetBox record's custom fields so the decision is auditable.
**Warning signs:**
- All items advance to `complete` status immediately after intake
- NetBox records show wrong `device_type` or `manufacturer` for known items you can visually verify
- SearXNG and OpenRouter tiers are never triggered in practice
**Phase to address:**
AI pipeline phase. Define confidence thresholds in config before wiring up the intake flow. Build the quality gate state machine before building the happy path.
---
### Pitfall 5: PRT Qutie Protocol Unknown — Blocking Hardware Dependency
**What goes wrong:**
The PRT Qutie uses Bluetooth-to-app communication as its primary documented interface. USB raw protocol documentation does not exist publicly. If reverse engineering reveals the device only accepts commands via its proprietary Bluetooth stack (not raw USB serial), the entire label printing architecture needs to change. This blocks any phase that delivers end-to-end intake (photo → record → label).
**Why it happens:**
Hardware ordered before protocol feasibility is confirmed. The PROJECT.md correctly notes "protocols need reverse-engineering once hardware arrives," but this risk is not explicitly sized — it could be one day of work or two weeks.
**How to avoid:**
On hardware arrival (2026-04-13), first action is protocol characterization, not feature development. Capture raw USB traffic with Wireshark + USBPCap (or macOS `usbmon` equivalent via `tcpdump`). Test if the device enumerates as a CDC-ACM or HID device. If only Bluetooth is functional, pivot to using the macOS CoreBluetooth framework via a CGo shim or a small helper process. Have a fallback plan ready: ZPL-compatible USB-C printer as alternative (Brother QL-820NWBc is well-documented).
**Warning signs:**
- `lsusb` (or macOS `system_profiler SPUSBDataType`) shows device as HID-only, not CDC-ACM
- No `/dev/cu.*` device appears when connected
- Manufacturer app communicates exclusively via Bluetooth, USB only charges
**Phase to address:**
Hardware characterization spike — must complete before committing to any label printing architecture. Do this in the first sprint after hardware arrival.
---
### Pitfall 6: 16GB Unified Memory — Gemma 4 Leaves No Headroom for the Rest of the Stack
**What goes wrong:**
On the Mac Mini M4 with 16GB unified memory, the Gemma 4 E4B model (4-bit quantized) needs approximately 5-8GB of model weights plus KV cache. oMLX, the Go backend, the React dev server, NetBox (running on a separate LXC but API calls still pass through), Proxmox overhead, macOS itself — with all running simultaneously, memory pressure triggers macOS compressed memory and swap. Inference slows catastrophically. Worse: at long context windows (intake photo + product research + NetBox context), KV cache grows and throughput drops beyond 8K tokens.
**Why it happens:**
Memory estimates are done in isolation: "Gemma 4B fits in 16GB" — true in isolation, false in production with everything else running.
**How to avoid:**
Run a memory profiling session before any feature development: load oMLX with Gemma 4 E4B, run the Go backend, open a browser tab, and watch Activity Monitor's memory pressure indicator and swap usage. If memory pressure is yellow/red, either (a) use 26B A4B with TurboQuant only for the research agent tier (not the fast indexer tier), (b) set oMLX's max concurrent requests to 1, or (c) shut down other Mac Mini workloads during intake sessions. Document the working memory budget and enforce it in oMLX config from day one.
**Warning signs:**
- macOS memory pressure bar is yellow or red during normal operation
- `vm_stat` shows high `pageouts` or `swapins` during inference
- Inference latency spikes from ~2s to 20s+ without apparent cause
- oMLX logs "out of memory" or silently returns truncated completions
**Phase to address:**
Infrastructure setup phase (before any AI pipeline work). Run the memory budget test as a gating condition before committing to model selection.
---
### Pitfall 7: NetBox Custom Fields — Write Format Differs from Read Format
**What goes wrong:**
When reading a NetBox object via REST API, `custom_fields` returns nested objects with `id`, `url`, `display`, `name`. When writing (PATCH/PUT), you must send only an array of integer IDs for object-type fields, and a flat dictionary for scalar fields. Go structs generated from the OpenAPI spec (or hand-coded) that reuse the same type for read and write will silently drop custom field updates — the PATCH succeeds with HTTP 200 but the field is not updated.
**Why it happens:**
NetBox's REST API has asymmetric read/write representations for custom fields. The official go-netbox client reflects this asymmetry but it is non-obvious. Community discussions confirm this trips up almost everyone working with custom fields programmatically.
**How to avoid:**
Write integration tests that (1) PATCH a custom field on a real NetBox object, (2) immediately GET the object back, and (3) assert the field value matches what was sent. Never assume a 200 response means the field was written. Create separate Go structs for the read and write representations of custom fields.
**Warning signs:**
- PATCH returns 200 but NetBox UI shows field still empty
- Custom fields look correct in test with scalar types but break with object-reference fields
- NetBox API returns `{"custom_fields": {"hwlab_status": null}}` after a write
**Phase to address:**
NetBox integration phase. Write the custom field round-trip test before building any intake workflow that depends on custom fields.
---
## Technical Debt Patterns
| Shortcut | Immediate Benefit | Long-term Cost | When Acceptable |
|----------|-------------------|----------------|-----------------|
| Hard-code `/dev/cu.usbmodem*` paths | Works in 5 minutes | Wrong device targeted after any replug; non-recoverable in prod | Never — use VID/PID enumeration from the start |
| Skip write queue, call NetBox synchronously in intake handler | Simpler code | Intake blocks on NetBox latency (50-200ms per call); app dead during NetBox downtime | Never for the main intake path |
| Auto-advance quality gate without confidence check | All items reach `complete` fast | NetBox fills with wrong data; impossible to clean up at scale | Never — the quality gate is the whole product value |
| Share one serial port handle across goroutines with a mutex | Avoids per-device abstraction | Deadlock if one goroutine is in a long read while another needs to write | Only acceptable for single-device prototype, never for multi-device production |
| Use `time.Sleep` polling loop to check for new USB devices | Simple device detection | Wastes CPU, misses hot-plug events, introduces latency | Never on macOS — use `IOKit` notification or enumerate on each operation |
| One monolithic AI prompt for all hardware types | Simpler prompt engineering | Low accuracy for visually ambiguous items; no structured output to parse | Only during initial prompt development, never in production |
---
## Integration Gotchas
| Integration | Common Mistake | Correct Approach |
|-------------|----------------|------------------|
| NetBox REST API | Using `PUT` when you mean `PATCH` — PUT requires ALL mandatory fields | Always use `PATCH` for partial updates; build a `PatchNetBoxObject(id, fields)` helper |
| NetBox custom fields | Passing write payload as read format (nested objects instead of ID arrays) | Maintain separate Go types for read (`CustomFieldValue`) and write (`CustomFieldPatch`) |
| oMLX / mlx-lm inference | Treating inference as a fast synchronous call (fire and forget with 5s timeout) | Use SSE streaming for long inference; set generous timeouts (60-120s); handle partial stream failures |
| FNIRSI FNB58 | Assuming energy/capacity values come directly from device | Device sends raw power+current samples at 10ms intervals; integrate on the host. Use `baryluk/fnirsi-usb-power-data-logger` as reference implementation |
| SearXNG | Sending raw AI-generated product names as queries | Sanitize queries; extract make/model tokens from AI output before querying; handle JSON `?format=json` parse errors gracefully |
| OpenRouter | No per-request cost cap | Set `max_tokens` hard limit on every OpenRouter call; log tokens consumed per pipeline run; set account-level spend limit in OpenRouter dashboard |
| NetBox NetBox-inventory plugin | Assuming plugin custom fields are available immediately after install | Custom fields must be created and assigned to object types via API or UI after plugin install; verify with a test GET before intake flow depends on them |
---
## Performance Traps
| Trap | Symptoms | Prevention | When It Breaks |
|------|----------|------------|----------------|
| Synchronous NetBox API calls on the intake hot path | Intake UI freezes for 100-500ms per item; label doesn't print until all NetBox writes succeed | Write-ahead queue in SQLite; async flush to NetBox | Day 1 with real network latency to LXC 130 |
| Unbounded KV cache during long research agent prompts | Inference latency spikes from 2s to 20s+; oMLX logs memory warnings | Set `max_tokens` on research agent tier; keep prompts focused; use `--kv-bits 4` in oMLX config | Context window >8K tokens on M4 16GB |
| Polling USB device state on a timer | CPU spike every N ms; goroutine accumulation if timer fires faster than reads complete | Event-driven reconnect with IOKit notifications (or enumerate once at request time) | With 3+ USB devices continuously polled |
| Loading full NetBox inventory for dashboard on every page load | Dashboard takes 3-5s to load; NetBox gets hammered with large paginated requests | Cache dashboard data locally (SQLite) with TTL; paginate lazily; never load all records into memory at once | Once NetBox has >200 items |
| React re-rendering entire device status panel on every SSE event | UI stutters during active cable testing with rapid updates | Use `useMemo`/`useCallback`, key by device ID, throttle SSE event processing to 10fps max | When SSE fires >5 events/second during live power testing |
---
## Security Mistakes
| Mistake | Risk | Prevention |
|---------|------|------------|
| Storing NetBox API token in plain Go config file | Token leaked via git, readable by any process | Load from environment variable or macOS Keychain; never commit token; add `config.json` to `.gitignore` |
| Passing user-supplied image data directly to Gemma without size/type validation | OOM on large images; potential prompt injection via steganographic payloads | Validate MIME type and max size (e.g., 10MB) before passing to inference; resize/normalize before encoding to base64 |
| No authentication on HWLab Go backend | Any device on the homelab network can add/modify NetBox records via HWLab | Even for solo use, add a simple bearer token or basic auth to the Go API; the backend has write access to NetBox |
| OpenRouter API key in frontend bundle | Key exposed to any browser that loads the app | OpenRouter calls must go through Go backend only; never expose key to frontend |
| SearXNG queries logged with full hardware descriptions | Sensitive inventory information in SearXNG logs | SearXNG is self-hosted so this is lower risk, but keep queries minimal — send model numbers, not full descriptions |
---
## UX Pitfalls
| Pitfall | User Impact | Better Approach |
|---------|-------------|-----------------|
| No progress feedback during AI intake (photo upload → result) | Operator thinks the app is frozen during 5-20s inference; submits photo again | Show SSE-streamed inference progress: "Indexing... Researching... Creating record..." with each tier's state |
| Quality gate status shown as enum code (`needs_research`) not human label | Operator confused about what action is needed | Display human labels: "Needs research", "Ready to print", "Complete" with action buttons per state |
| Label prints before NetBox record is confirmed | QR code points to a record that may not exist if NetBox write fails | Print label only after NetBox write is confirmed (or queue is flushed); show "printing..." state |
| Cable test results shown as raw hex/bytes | Operator can't interpret pass/fail | Parse protocol data into human-readable result: "All 8 conductors — PASS", "Pin 4 open — FAIL" |
| No way to correct a wrong AI classification without going into NetBox UI | AI errors require leaving HWLab | Provide inline edit for manufacturer/model/type on the intake confirmation screen before committing to NetBox |
---
## "Looks Done But Isn't" Checklist
- [ ] **USB device layer:** Can handle all three devices connected simultaneously AND hot-unplug/replug of any one without affecting the others — verify with integration test
- [ ] **Label printing:** QR code URL resolves to actual NetBox record (not 404) — verify that HW-XXXXX ID is written to NetBox before label is generated
- [ ] **AI intake pipeline:** Confidence threshold enforcement is active — verify that a deliberately ambiguous photo does NOT auto-advance past `indexed` status
- [ ] **NetBox custom fields:** Round-trip write+read test passes for all HWLab-specific custom fields — verify with a dedicated test script before intake is considered working
- [ ] **Three-tier escalation:** Tier 2 (SearXNG) and Tier 3 (OpenRouter) are actually triggered in practice — verify by running an item that is genuinely ambiguous and watching pipeline logs
- [ ] **Memory budget:** Mac Mini M4 stays out of memory pressure (green in Activity Monitor) with oMLX loaded and backend running — verify before declaring inference "working"
- [ ] **NetBox downtime handling:** If NetBox LXC is shut down mid-intake, the operation queues and resumes cleanly — verify with a chaos test
---
## Recovery Strategies
| Pitfall | Recovery Cost | Recovery Steps |
|---------|---------------|----------------|
| Hard-coded USB paths in production | MEDIUM | Refactor device manager to VID/PID enumeration; update all open() calls; test full replug cycle |
| NetBox filled with wrong AI classifications | HIGH | Write a NetBox API script to bulk-set affected items back to `draft` status; re-run intake on each item; no shortcut |
| Goroutine leak accumulation over long session | LOW | Restart Go backend; investigate with `pprof`; add goroutine count metric to telemetry endpoint |
| PRT Qutie only works via Bluetooth | MEDIUM | Pivot to CoreBluetooth CGo shim or external helper process; estimated 3-5 days additional work |
| 16GB memory exhausted during intake session | LOW | Restart oMLX; switch to smaller model variant; add memory monitoring to runbook |
| OpenRouter spend spike from runaway escalation | LOW-MEDIUM | Set account spend limit in OpenRouter dashboard; add per-run token counter with hard cutoff in Go pipeline code |
---
## Pitfall-to-Phase Mapping
| Pitfall | Prevention Phase | Verification |
|---------|------------------|--------------|
| USB path churn | USB device layer (first hardware phase) | Integration test: replug all 3 devices in random order, verify correct device responds |
| Goroutine leak on disconnect | USB device layer | Goroutine count stable after 10 replug cycles (test with `pprof`) |
| NetBox downtime / no offline mode | NetBox integration phase | Chaos test: kill NetBox mid-intake, verify queue persists and resumes |
| AI misidentification / quality gate bypass | AI pipeline phase | Ambiguous photo stays at `needs_research`; confident photo reaches `indexed` pending confirmation |
| PRT Qutie protocol unknown | Hardware characterization spike (day of hardware arrival) | USB traffic captured and protocol characterized before architecture is committed |
| 16GB memory exhaustion | Infrastructure setup phase | Memory pressure remains green during full stack with oMLX loaded |
| NetBox custom field write/read asymmetry | NetBox integration phase | Round-trip test: PATCH field, GET object, assert value matches |
| Three-tier escalation never triggers | AI pipeline phase | Log shows tier promotions happening on genuinely ambiguous items |
| Runaway OpenRouter spend | AI pipeline phase | `max_tokens` set on every OpenRouter call; spend limit set in dashboard; per-run cost logged |
---
## Sources
- [go.bug.st/serial — Unblock Read on Close issue #13](https://github.com/bugst/go-serial/issues/13)
- [baryluk/fnirsi-usb-power-data-logger — FNB58 protocol reverse engineering](https://github.com/baryluk/fnirsi-usb-power-data-logger)
- [NetBox issue #10134 — Invalid REST API request setting custom fields raises TypeError](https://github.com/netbox-community/netbox/issues/10134)
- [NetBox discussion #13425 — Rest API to set custom field](https://github.com/netbox-community/netbox/discussions/13425)
- [NetBox issue #14879 — Data Source custom fields not available through REST API](https://github.com/netbox-community/netbox/issues/14879)
- [oMLX README — continuous batching, SSE keep-alive, concurrent request config](https://github.com/jundot/omlx)
- [Gemma 4 Apple Silicon MLX issue — FA hang, streaming field](https://github.com/ollama/ollama/issues/15368)
- [atctwo — Reverse Engineering a Thermal Label Printer](https://atctwo.net/posts/2024/07/16/thermal-printer.html)
- [go101 — Common Concurrent Programming Mistakes](https://go101.org/article/concurrent-common-mistakes.html)
- [NetBox ArnesSI inventory plugin — asset tag / custom field sync](https://github.com/ArnesSI/netbox-inventory)
- [OpenRouter — Rate limits and guardrails documentation](https://openrouter.ai/docs/api/reference/limits)
- [Production-Grade Local LLM Inference on Apple Silicon — arxiv 2511.05502](https://arxiv.org/abs/2511.05502)
---
*Pitfalls research for: HWLab — Go + USB serial + local AI inference + NetBox*
*Researched: 2026-04-09*

171
.planning/research/STACK.md Normal file
View file

@ -0,0 +1,171 @@
# Stack Research
**Domain:** Self-hosted AI-powered hardware inventory system (Go + React)
**Researched:** 2026-04-09
**Confidence:** MEDIUM-HIGH (core stack confirmed via search; some library versions from training data)
---
## Recommended Stack
### Core Technologies
| Technology | Version | Purpose | Why Recommended |
|------------|---------|---------|-----------------|
| Go | 1.22+ | Backend API, USB/serial control, AI orchestration | Native concurrency for USB polling + HTTP serving; single binary deployment; no runtime dependency; best-in-class serial I/O on macOS |
| React | 18.x | SPA frontend | Mature ecosystem, hooks-based state management, excellent TypeScript support; right level of complexity for a single-operator tool |
| TypeScript | 5.x | Frontend type safety | Catches API contract drift early; NetBox has complex nested types that benefit from strict typing |
| Tailwind CSS | 3.x | Utility-first styling | ClickHouse-inspired design system maps well to utility classes; no CSS file sprawl; dark theme with arbitrary values is trivial |
| SQLite (via mattn/go-sqlite3) | 3.x / v1.14+ | Local-only storage (chat history, config, session state) | Zero-dependency embedded DB; perfectly suited for single-operator append-heavy chat logs; no separate process |
| oMLX | latest | Local LLM inference server on Mac Mini M4 | Native Apple Silicon MLX backend, OpenAI-compatible API endpoint, paged SSD KV caching, continuous batching; supports Gemma 4 VLM natively; drop-in for any OpenAI-SDK client |
### Go Backend Libraries
| Library | Version | Purpose | When to Use |
|---------|---------|---------|-------------|
| github.com/netbox-community/go-netbox/v4 | v4.x | NetBox REST API client | All CRUD against NetBox — officially maintained, generated from NetBox OpenAPI spec |
| go.bug.st/serial | v1.x | USB serial communication | Printer + cable tester I/O; actively maintained, cross-platform, better API than tarm/serial; supports port enumeration |
| github.com/gin-gonic/gin | v1.9+ | HTTP router and middleware | Fast, well-documented, large middleware ecosystem; handles file uploads for photo intake cleanly |
| github.com/mattn/go-sqlite3 | v1.14+ | SQLite driver (cgo) | Standard Go SQLite driver; use with database/sql for portability |
| github.com/jmoiern/sqlx | v1.3+ | database/sql extension | Named params and struct scanning reduce boilerplate for chat history queries |
| github.com/sashabaranov/go-openai | v1.x | OpenAI-compatible client | Handles both oMLX local endpoint and OpenRouter remote; single client, swap base URL per tier |
| github.com/skip2/go-qrcode | v0.0.0 | QR code generation | Generate HW-XXXXX QR codes server-side before sending to printer |
| github.com/google/uuid | v1.x | UUID generation | HW-XXXXX ID assignment; deterministic or random depending on design choice |
| github.com/spf13/viper | v1.x | Config management | YAML/env config for AI endpoints, NetBox URL/token, USB port paths |
### Frontend Libraries
| Library | Version | Purpose | When to Use |
|---------|---------|---------|-------------|
| Vite | 5.x | Build tool and dev server | Fastest HMR in class; Go backend runs separately, Vite proxies API calls in dev |
| TanStack Query (React Query) | v5 | Server state management | NetBox data is server state not client state; handles caching, refetch, stale-while-revalidate |
| TanStack Router | v1 | Type-safe routing | Better TypeScript integration than React Router v6 for complex nested routes |
| Zustand | v4 | Client state (UI state, chat) | Lightweight, no boilerplate, perfect for chat message buffer and USB device status |
| shadcn/ui | latest | Component primitives | Radix UI + Tailwind, fully owned code (not a dependency), dark mode first, ClickHouse aesthetic compatible |
| react-dropzone | v14 | Photo drag-and-drop | Standard library for file upload UX; works with multipart form to Go backend |
| react-hot-toast | v2 | Toast notifications | Inventory actions (label printed, test passed) need lightweight non-blocking feedback |
| lucide-react | latest | Icon set | Consistent, tree-shakeable, TypeScript-native, pairs well with shadcn/ui |
### Development Tools
| Tool | Purpose | Notes |
|------|---------|-------|
| Air (cosmtrek/air) | Go hot reload in development | Watch + rebuild backend on save; configure alongside Vite proxy |
| golangci-lint | Go linting | Run in CI; catches serial/concurrent code issues early |
| ESLint + typescript-eslint | Frontend linting | Strict TypeScript rules; catch NetBox API type drift |
| Prettier | Frontend formatting | Single config, no debates |
| Bruno (or httpie) | API testing | Test NetBox integration and Go endpoints locally without Postman account |
---
## Installation
```bash
# Go backend — initialize module
go mod init git.georgsen.dk/hwlab
# Core Go dependencies
go get github.com/gin-gonic/gin@latest
go get github.com/netbox-community/go-netbox/v4@latest
go get go.bug.st/serial@latest
go get github.com/mattn/go-sqlite3@latest
go get github.com/jmoiern/sqlx@latest
go get github.com/sashabaranov/go-openai@latest
go get github.com/skip2/go-qrcode@latest
go get github.com/google/uuid@latest
go get github.com/spf13/viper@latest
# Frontend — scaffold
npm create vite@latest frontend -- --template react-ts
cd frontend
# Frontend core dependencies
npm install @tanstack/react-query @tanstack/react-router zustand
npm install react-dropzone react-hot-toast lucide-react
npm install class-variance-authority clsx tailwind-merge # shadcn/ui deps
# shadcn/ui init (run in frontend dir)
npx shadcn@latest init
# Dev dependencies
npm install -D @typescript-eslint/eslint-plugin @typescript-eslint/parser eslint prettier tailwindcss autoprefixer postcss
```
---
## Alternatives Considered
| Recommended | Alternative | When to Use Alternative |
|-------------|-------------|-------------------------|
| go.bug.st/serial | tarm/serial | tarm/serial is adequate if you only need basic read/write and already have it; go.bug.st/serial is preferred for new projects due to active maintenance and port listing |
| gin-gonic/gin | chi, echo, fiber | chi if you want stdlib-compatible handlers; echo for similar feature set; fiber if you need extreme HTTP throughput (not relevant here) |
| go-openai (sashabaranov) | Custom HTTP client | Custom client only if you need streaming SSE to browser — go-openai supports streaming too, prefer it |
| TanStack Router | React Router v6 | React Router v6 is fine if team is already familiar; TanStack Router has stronger TypeScript guarantees for route params |
| TanStack Query | SWR | SWR is lighter but TanStack Query has better devtools and mutation handling for NetBox writes |
| mattn/go-sqlite3 (cgo) | modernc.org/sqlite (pure Go) | Use modernc.org/sqlite if you want zero cgo dependency (simpler cross-compilation); slightly slower but avoids gcc requirement |
| oMLX | Ollama | Ollama works but oMLX has better Apple Silicon performance via MLX backend; Gemma 4 VLM continuous batching support is oMLX-specific |
---
## What NOT to Use
| Avoid | Why | Use Instead |
|-------|-----|-------------|
| GORM | Heavy ORM abstraction for what is essentially two tables (chat logs, config); migrations and reflection overhead not worth it for SQLite local-only store | database/sql + sqlx directly |
| tarm/serial | Last commit 2018, unmaintained, no port enumeration, open issues unaddressed | go.bug.st/serial |
| Redux (React) | Massive boilerplate for a solo-operator SPA; overkill when TanStack Query handles server state | Zustand + TanStack Query |
| axios | Unnecessary HTTP client abstraction; TanStack Query works with native fetch; one fewer dependency | fetch API directly inside TanStack Query |
| Next.js | SSR/SSG overhead not needed — Go handles the API, React is a pure SPA served as static files | Vite + React SPA |
| llama.cpp Go bindings | Not optimized for Apple Silicon; MLX backend (oMLX) is 2-4x faster on M-series; cgo binding complexity | oMLX via OpenAI-compatible HTTP |
| Direct NetBox HTTP calls (no client) | NetBox v4 API has 200+ endpoints; go-netbox generates typed structs from OpenAPI spec — manual HTTP is brittle | go-netbox/v4 |
---
## Stack Patterns by Variant
**For the AI tier routing (local vs remote):**
- Use a single `go-openai` client instance per tier, configured with `BaseURL` pointing to oMLX (`http://localhost:PORT/v1`) for Tier 1, and `https://openrouter.ai/api/v1` for Tier 2/3
- Switch by returning the appropriate client from a factory based on model config
- Do NOT hardcode tier selection in business logic — make it config-driven from the start
**For USB serial (printer + testers):**
- Use `go.bug.st/serial` with a dedicated goroutine per device, communicating via channels
- Do not share a serial.Port across goroutines — open one port per device, serialize writes in the owning goroutine
- USB device paths on macOS follow `/dev/cu.usbserial-*` or `/dev/cu.usbmodem*`; use `serial.GetPortsList()` to enumerate at startup
**For photo intake (multipart upload):**
- Accept multipart/form-data in Gin; decode image in Go; send as base64 in the OpenAI vision API call to oMLX
- Store the original photo in a local temp directory only until the NetBox record is created; do not persist photos in HWLab itself
- Gemma 4 E4B fits comfortably in 16GB; Gemma 4 26B A4B needs TurboQuant KV offload — benchmark before committing to 26B
**For NetBox custom fields:**
- go-netbox v4 represents custom fields as `map[string]interface{}`; define typed wrapper structs in your own code to handle `hw_id`, `condition`, `quality_gate` etc.
- Custom fields must be created in NetBox admin before the Go code can write them — treat this as infrastructure provisioning (Phase 1 task)
---
## Version Compatibility
| Package | Compatible With | Notes |
|---------|-----------------|-------|
| go-netbox/v4 | NetBox 4.x | Generated from NetBox 4.x OpenAPI; not compatible with NetBox 3.x API responses |
| mattn/go-sqlite3 v1.14+ | Go 1.21+ | Requires cgo; needs gcc or Xcode CLT on macOS |
| shadcn/ui | React 18, Tailwind 3.x | shadcn v2+ requires Tailwind 3.x; not yet tested with Tailwind 4 alpha |
| oMLX | macOS 15+, Apple Silicon M1+ | Won't run on Intel Mac or Linux |
| go-openai v1.x | OpenAI API v1, OpenRouter, oMLX | Any OpenAI-compatible endpoint works; set `BaseURL` in `ClientConfig` |
---
## Sources
- [github.com/jundot/omlx](https://github.com/jundot/omlx) — oMLX feature set, Gemma 4 VLM support confirmed (HIGH confidence)
- [github.com/netbox-community/go-netbox](https://github.com/netbox-community/go-netbox) — Official Go client, v4 confirmed (HIGH confidence)
- [go.bug.st/serial pkg.go.dev](https://pkg.go.dev/go.bug.st/serial) — Active maintenance confirmed, preferred over tarm/serial (HIGH confidence)
- [omlx.ai](https://omlx.ai/) — macOS 15+ requirement, 16GB minimum RAM, MLX backend (HIGH confidence)
- WebSearch: tarm/serial maintenance status — last substantive commit 2018, multiple forks active (MEDIUM confidence)
- Training data: gin, TanStack Query, Zustand, Vite, shadcn/ui versions — standard 2025 React/Go stack (MEDIUM confidence, verify pinned versions before first install)
---
*Stack research for: HWLab — self-hosted AI hardware inventory (Go + React + oMLX + NetBox)*
*Researched: 2026-04-09*

212
.planning/research/SUMMARY.md Normal file
View file

@ -0,0 +1,212 @@
# Project Research Summary
**Project:** HWLab — AI-powered self-hosted homelab hardware inventory
**Domain:** USB peripheral control + local AI inference + NetBox ITAM integration
**Researched:** 2026-04-09
**Confidence:** MEDIUM-HIGH
## Executive Summary
HWLab is a single-operator homelab tool that closes the gap no existing ITAM product addresses: zero-manual-entry physical hardware cataloging via AI vision, with QR label printing and cable testing integrated at the hardware layer. The canonical pattern for this class of tool is a Go single-binary backend (HTTP API + USB serial management + AI orchestration) paired with a React SPA, with NetBox as the sole inventory data store. No local inventory database. SQLite is used only for chat history, config, and a write-ahead queue for NetBox operations. The React frontend is embedded into the Go binary for single-binary deployment — appropriate for a solo-operator homelab context.
The recommended approach centers on a three-tier AI pipeline: Tier 1 (local Gemma 4 via oMLX on Apple Silicon) handles all routine intake offline; Tier 2 (SearXNG research agent via OpenRouter) fills specification gaps; Tier 3 (Claude Opus via OpenRouter) powers the Lab Advisor strategic chat. This tiering is the core architectural differentiator — it delivers offline capability, cost control, and depth of response that no single-model approach achieves. The AI Orchestrator must own all model routing decisions; service-layer code must never reference model names directly.
The two dominant risks are hardware-dependent and must be addressed before any feature work begins. First, the PRT Qutie label printer has no public USB protocol documentation — protocol characterization must happen the day hardware arrives (2026-04-13) before committing to a label printing architecture. Second, Gemma 4 on 16GB unified memory must be profiled under full stack load before the model tier is finalized — Gemma 4 E4B (4-bit, ~5-8GB) is the safe default; Gemma 4 26B A4B needs TurboQuant testing. The USB device layer (goroutine-per-device with VID/PID enumeration) and NetBox custom field write/read asymmetry are well-documented pitfalls that must be handled from the first line of integration code.
## Key Findings
### Recommended Stack
The backend is Go 1.22+ using Gin for HTTP routing, `go.bug.st/serial` for USB serial (not `tarm/serial` — unmaintained since 2018), `go-netbox/v4` for the typed NetBox client, and `sashabaranov/go-openai` as the single OpenAI-compatible client (base URL is config-driven per tier). SQLite via `mattn/go-sqlite3` + `sqlx` handles the chat history and write-ahead queue. The frontend is React 18 + TypeScript 5 + Vite 5, with TanStack Query for server state (NetBox data), Zustand for client state (chat buffer, USB device status), and shadcn/ui + Tailwind 3 for components. oMLX on macOS 15+ (Apple Silicon M1+) is the local inference server; it is not compatible with Intel Mac or Linux.
**Core technologies:**
- Go 1.22+: backend API, USB serial control, AI orchestration — single binary deployment, native concurrency for USB polling
- React 18 + TypeScript 5 + Vite 5: SPA frontend — type safety catches NetBox API contract drift
- oMLX (latest): local Gemma 4 inference via MLX backend — OpenAI-compatible, Apple Silicon optimized, required for Tier 1
- go-netbox/v4: official typed NetBox REST client — generated from OpenAPI spec, only compatible with NetBox 4.x
- go.bug.st/serial v1.x: USB serial for printer and cable testers — actively maintained, supports port enumeration
- SQLite (mattn/go-sqlite3 v1.14+): chat history, config, write-ahead queue only — NetBox owns inventory data
- sashabaranov/go-openai v1.x: single client for all AI tiers — base URL switches between oMLX, OpenRouter
**Do not use:** GORM (over-engineered for two tables), tarm/serial (unmaintained), Redux (use Zustand + TanStack Query), Next.js (Go serves the SPA as static files), llama.cpp Go bindings (oMLX is 2-4x faster on M-series).
### Expected Features
**Must have (table stakes):**
- Asset record CRUD (delegated to NetBox via API) — every ITAM tool has this
- Unique HW-XXXXX identifiers — stable QR-encoded references, assigned at intake
- Asset search and filter — table search + AI natural language query translation
- Asset status tracking (Available / In Use / Retired / Unknown) — baseline expectation
- Category, manufacturer, model, serial number fields — NetBox native; populated by AI intake
- Location / rack / site tracking — NetBox hierarchy
- QR label printing via PRT Qutie — closes the physical intake loop
- Audit trail — NetBox change log, surfaced in record detail
**Should have (differentiators):**
- AI photo intake (zero-entry) — the core value proposition; photo to structured NetBox record via Gemma 4 vision
- Three-tier AI escalation — local Gemma 4 then SearXNG research agent then Claude Opus; offline-capable and cost-optimal
- Lab Advisor chat — strategic homelab Q&A with full inventory context; unique among ITAM tools
- Cable test workflow (Treedix + FNIRSI FNB58) — USB serial integration; test results attached to cable records
- Catalog quality gate — draft / indexed / needs_research / researched / complete; AI-driven state transitions
- Natural language inventory search — LLM query translation to NetBox API filter, running locally
- SearXNG product research (Tier 2) — auto-fills specs, EAN/part numbers, pricing for incomplete records
**Defer (v2+):**
- Barcode scanner HID support — only if photo intake proves slow for high-volume bulk intake
- Bulk CSV import — bypasses quality gate; NetBox native import is better path for migrations
- Predictive failure / warranty alerts — disproportionate infrastructure for homelab context
- Multi-user / RBAC — NetBox already provides this if genuinely needed; adds no solo-operator value
### Architecture Approach
The system is a layered modular monolith: React SPA, Go HTTP layer (thin handlers), Service layer (Inventory, Advisor, Label/Testing), Client/Adapter layer (NetBox client, AI client, SQLite), External services (NetBox LXC, oMLX, OpenRouter, SearXNG). The USB Manager runs outside the service layer, managing dedicated goroutines per device with typed command/event channels. HTTP handlers send commands to USB Manager channels and receive results via Server-Sent Events — no polling, no WebSockets. The Go binary embeds the React build via `go:embed` for single-binary deployment.
**Major components:**
1. AI Orchestrator (`internal/ai/orchestrator.go`) — owns all tier routing, escalation logic, prompt construction; services never reference model names
2. USB Manager (`internal/usb/manager.go`) — goroutine-per-device, VID/PID enumeration, command channels in / event channels out; owns PRT Qutie, 3x Treedix, FNIRSI FNB58
3. NetBox Client (`internal/netbox/client.go`) — sole integration point for all NetBox REST calls; repository pattern; separate Go types for read vs. write custom fields
4. Inventory Service (`internal/inventory/service.go`) — photo intake flow, quality gate state machine, record lifecycle
5. Advisor Service (`internal/advisor/service.go`) — chat session, NetBox context assembly, response streaming via SSE
6. SQLite store — chat history (append-only), config, write-ahead queue for pending NetBox operations
### Critical Pitfalls
1. **USB serial path churn on device replug** — macOS assigns dynamic `/dev/cu.*` paths; hard-coding them means wrong devices get targeted after any replug. Enumerate by USB VID/PID + serial number from day one; re-resolve path on every reconnect.
2. **Goroutine leak on USB disconnect** — `serial.Read()` does not reliably unblock when a port is closed from another goroutine. Wrap every read loop with `context.Context` cancellation; use `go.bug.st/serial` (has explicit unblock support); write a replug-cycle goroutine count test before any feature work.
3. **NetBox custom field write/read asymmetry** — PATCH returns HTTP 200 but the field is silently not written if you send the read format (nested object) instead of the write format (ID array). Write round-trip tests (PATCH then GET, assert value) for every custom field before building any intake flow that depends on them.
4. **AI misidentification with no quality gate enforcement** — Gemma 4 returns high-confidence wrong classifications for visually ambiguous hardware. Define hard confidence thresholds in config (e.g., below 0.7 pins at `needs_research`); never auto-advance past `indexed` without threshold clearance or explicit operator confirmation.
5. **PRT Qutie protocol unknown** — no public USB raw protocol docs exist. Hardware arrives 2026-04-13. First action on arrival is protocol characterization (Wireshark USB capture, check CDC-ACM vs. HID), not feature development. Fallback: Brother QL-820NWBc (well-documented ZPL).
6. **16GB unified memory budget** — Gemma 4 E4B uses 5-8GB; with Go backend, browser, and macOS overhead, memory pressure is a real risk. Run a full-stack memory profiling session before committing to model selection.
7. **No NetBox offline buffer** — without a write-ahead queue in SQLite, any NetBox LXC hiccup makes the entire app non-functional during intake. Add a `pending_operations` table to SQLite in the NetBox integration phase, before any intake workflow is built.
## Implications for Roadmap
Based on research, the architecture has a clear dependency DAG. The NetBox client and AI client must be stable before any services are built. USB hardware cannot be finalized until it arrives and protocols are characterized. The suggested 7-phase structure follows these hard dependencies.
### Phase 1: Foundation — Infrastructure and NetBox Integration
**Rationale:** Every single feature in the system reads or writes through NetBox. This is the hardest dependency and must be resolved first. The write-ahead queue belongs here too — retrofitting it later is a pitfall.
**Delivers:** Running Go binary with NetBox API client; all custom fields provisioned in NetBox; round-trip test suite for custom fields; SQLite schema (chat + write-ahead queue); Vite dev proxy configured.
**Addresses:** Asset CRUD, HW-XXXXX ID scheme, custom fields schema for quality gate and all HWLab-specific fields.
**Avoids:** NetBox custom field write/read asymmetry (Pitfall 7); NetBox downtime / no offline buffer (Pitfall 3).
**Research flag:** Low — NetBox REST API is well-documented; go-netbox/v4 is official.
### Phase 2: AI Pipeline — Local Inference and Photo Intake
**Rationale:** AI photo intake is the core differentiator. It must be validated before building the UI around it. oMLX memory budget must be verified before model selection is finalized.
**Delivers:** oMLX + Gemma 4 setup and profiled memory budget; AI Orchestrator with Tier 1 (local) and Tier 2 (OpenRouter) routing; photo intake endpoint (`POST /api/intake`); quality gate state machine enforcing confidence thresholds; structured output parsing from vision model.
**Uses:** go-openai client with configurable base URL; oMLX at localhost; openrouter.ai for escalation.
**Implements:** AI Orchestrator, Inventory Service (intake), quality gate state machine.
**Avoids:** AI misidentification / quality gate bypass (Pitfall 4); 16GB memory exhaustion (Pitfall 6); mixing AI tiers in service logic (Architecture Anti-Pattern 3).
**Research flag:** MEDIUM — three-tier confidence scoring calibration for vision tasks is novel; needs real hardware photos to tune thresholds.
### Phase 3: React SPA and Inventory Dashboard
**Rationale:** Frontend needs a stable backend API (Phases 1-2) to be useful. Building the UI before the backend is stable creates churn.
**Delivers:** React SPA with Vite (embedded via go:embed), inventory dashboard (TanStack Query via Go API), asset detail view with photo, quality gate status with human-readable labels and action buttons, basic text search, intake flow UI with inline AI classification correction.
**Uses:** React 18 + TypeScript 5, shadcn/ui + Tailwind 3, TanStack Query, Zustand, react-hot-toast.
**Implements:** All frontend views; SSE subscription for async intake progress updates.
**Avoids:** Quality gate status shown as enum code (UX Pitfall); no progress feedback during AI intake (UX Pitfall).
**Research flag:** Low — React + TanStack Query + shadcn/ui are well-documented, established patterns.
### Phase 4: USB Hardware Characterization and Label Printing
**Rationale:** USB hardware arrives 2026-04-13. Protocol characterization is a spike that must complete before any label printing architecture is committed to. This phase has the highest uncertainty.
**Delivers:** Protocol characterization for PRT Qutie (and decision on CDC-ACM vs. Bluetooth fallback); USB Manager with goroutine-per-device, VID/PID enumeration, channel fan-out, reconnect handling; QR label generation and print flow; SSE events for print status.
**Implements:** USB Manager, Label Service, printer sub-package.
**Avoids:** PRT Qutie protocol unknown (Pitfall 5); USB serial path churn (Pitfall 1); goroutine leak on disconnect (Pitfall 2).
**Research flag:** HIGH — PRT Qutie protocol unknown. Dedicated hardware characterization spike required on 2026-04-13. Do not plan label printing features before spike completes.
### Phase 5: Cable Test Integration
**Rationale:** Builds directly on the USB Manager foundation from Phase 4. Cannot start until Phase 4 USB Manager and Treedix serial protocols are understood.
**Delivers:** Treedix cable tester protocol implementation (3 testers); FNIRSI FNB58 power meter (continuous 10ms sample integration on host); cable test workflow UI; test results written to NetBox cable record; quality gate advance for tested cables.
**Implements:** Tester and powermeter sub-packages; Label/Testing Service.
**Avoids:** Cable test results shown as raw hex (UX Pitfall); FNIRSI raw sample integration assumption (Integration Gotcha).
**Research flag:** MEDIUM — Treedix serial protocol needs reverse engineering; FNIRSI FNB58 has a reference implementation (`baryluk/fnirsi-usb-power-data-logger`) but protocol parsing is non-trivial.
### Phase 6: Lab Advisor Chat
**Rationale:** No USB dependency; can overlap with Phase 4/5 if velocity allows. Scheduled after Phase 5 to ensure inventory has meaningful data for Q&A.
**Delivers:** Lab Advisor chat interface; AdvisorService with NetBox context assembly; Claude Opus via OpenRouter (Tier 3); streaming response via SSE; SQLite chat history (append-only); OpenRouter per-call token limit and spend safeguards.
**Implements:** Advisor Service, chat history persistence.
**Avoids:** OpenRouter runaway spend; OpenRouter key in frontend bundle.
**Research flag:** Low — SSE streaming, SQLite append-only, and OpenRouter integration are established patterns.
### Phase 7: SearXNG Research Agent and Quality Gate Automation
**Rationale:** Enhancement layer — all primitives (AI Orchestrator, NetBox client, SearXNG) exist by this point. Adds intelligence to the quality gate by automating the needs_research to researched transition.
**Delivers:** SearXNG research agent (Tier 2) with query sanitization; automated quality gate advancement; full 5-state quality gate lifecycle; natural language inventory search (Gemma 4 query translation to NetBox filter).
**Implements:** Search client (`internal/search/`); extends AI Orchestrator with research task type.
**Avoids:** SearXNG receiving unsanitized AI-generated queries; three-tier escalation never triggering in practice.
**Research flag:** Low — SearXNG JSON API is straightforward; query sanitization pattern is well-understood.
### Phase Ordering Rationale
- NetBox first because it is the dependency of every other component — no feature works without it.
- AI before UI because the intake endpoint behavior (confidence thresholds, quality gate states) defines what the UI must display.
- USB hardware characterization as a spike in Phase 4 rather than embedded in a feature phase — isolates the highest-uncertainty work and prevents it from blocking UI or AI development.
- Cable testing after label printing (Phase 5 after Phase 4) because both share the USB Manager; the Manager must be solid before adding more device types.
- Lab Advisor after core inventory is populated (Phase 6 after Phase 5) — the advisor is only valuable when it has inventory context to reason about.
- Research agent last (Phase 7) because it is pure enhancement; all tiers, the quality gate state machine, and the intake flow must exist first.
### Research Flags
Phases needing `/gsd-research-phase` during planning:
- **Phase 2 (AI Pipeline):** Confidence threshold calibration for hardware vision tasks is novel; no established benchmarks; needs real-photo experiments to tune.
- **Phase 4 (USB Hardware Characterization):** PRT Qutie protocol is completely unknown. Spike required on hardware arrival (2026-04-13) before any architecture decisions for label printing.
- **Phase 5 (Cable Test Integration):** Treedix serial protocols require reverse engineering; FNIRSI FNB58 continuous sample integration has a reference impl but is non-trivial.
Phases with standard patterns (skip research-phase):
- **Phase 1 (Foundation):** NetBox REST API and go-netbox/v4 are official and well-documented.
- **Phase 3 (React SPA):** TanStack Query + shadcn/ui + SSE subscription are standard modern React patterns.
- **Phase 6 (Lab Advisor):** OpenRouter streaming + SQLite append-only + SSE are established patterns.
- **Phase 7 (Research Agent):** SearXNG JSON API integration is straightforward.
## Confidence Assessment
| Area | Confidence | Notes |
|------|------------|-------|
| Stack | MEDIUM-HIGH | Core stack confirmed via web search and official sources. Library versions from training data — pin and verify before first install. mattn/go-sqlite3 requires cgo + gcc; modernc.org/sqlite is pure-Go fallback if cross-compilation matters. |
| Features | MEDIUM-HIGH | Table stakes features are well-established ITAM patterns (HIGH). AI photo intake zero-entry claim is novel — no competitor exists to validate against (LOW for that specific claim; validate after v1 intake loop). |
| Architecture | HIGH | Patterns are established Go idioms. NetBox repository pattern, goroutine-per-device, SSE fan-out, embedded SPA are all well-documented. Three-tier AI orchestrator follows established orchestration patterns. |
| Pitfalls | MEDIUM-HIGH | USB serial goroutine leak and path churn are confirmed community issues with documented fixes. NetBox custom field asymmetry confirmed via multiple GitHub issues. PRT Qutie protocol risk is sizing-unknown (1 day to 2 weeks). Memory budget needs empirical measurement. |
**Overall confidence:** MEDIUM-HIGH
### Gaps to Address
- **PRT Qutie USB protocol:** Complete unknown until hardware arrives 2026-04-13. Block label printing architecture decisions until characterization spike completes.
- **Gemma 4 memory budget under full stack:** Must measure empirically on the target Mac Mini M4 before committing to model tier selection.
- **Gemma 4 vision confidence scoring:** No established benchmarks for hardware photo classification. The 0.7 threshold is an informed estimate — calibrate against real photos during Phase 2.
- **Treedix cable tester serial protocol:** No public documentation. Must reverse-engineer from USB traffic capture. Estimated complexity unknown.
- **go-netbox/v4 custom field lag:** The official client may lag NetBox 4.2's custom field API. Plan to hand-roll custom field write/read wrappers regardless.
- **oMLX project longevity:** oMLX (`jundot/omlx`) is a smaller project. If it becomes unmaintained, Ollama with MLX backend is the fallback.
## Sources
### Primary (HIGH confidence)
- [github.com/jundot/omlx](https://github.com/jundot/omlx) — oMLX feature set, Gemma 4 VLM support, Apple Silicon requirements
- [github.com/netbox-community/go-netbox](https://github.com/netbox-community/go-netbox) — official Go client v4, OpenAPI-generated
- [pkg.go.dev/go.bug.st/serial](https://pkg.go.dev/go.bug.st/serial) — active maintenance, port enumeration, unblock-on-close confirmed
- [netboxlabs.com/docs — REST API overview](https://netboxlabs.com/docs/netbox/integrations/rest-api/)
- [go.bug.st/serial issue #13](https://github.com/bugst/go-serial/issues/13) — unblock read on close
- [NetBox issue #10134](https://github.com/netbox-community/netbox/issues/10134) — custom field write TypeError confirmed
- [NetBox discussion #13425](https://github.com/netbox-community/netbox/discussions/13425) — REST API custom field write format asymmetry confirmed
- [baryluk/fnirsi-usb-power-data-logger](https://github.com/baryluk/fnirsi-usb-power-data-logger) — FNB58 protocol reference implementation
### Secondary (MEDIUM confidence)
- [omlx.ai](https://omlx.ai/) — macOS 15+ requirement, 16GB minimum RAM, MLX backend
- [github.com/ArnesSI/netbox-inventory](https://github.com/ArnesSI/netbox-inventory) — netbox-inventory plugin, asset tag / custom field sync
- [atctwo.net — Reverse Engineering a Thermal Label Printer](https://atctwo.net/posts/2024/07/16/thermal-printer.html) — thermal printer protocol methodology
- [Alex Edwards: Fat Service Pattern for Go](https://www.alexedwards.net/blog/the-fat-service-pattern) — Go service architecture
- Training data: Snipe-IT, Homebox feature sets, ITAM industry standards
### Tertiary (LOW confidence — needs validation)
- AI photo intake zero-entry claim: no competitor exists to validate against; validate after v1 intake loop
- Confidence threshold of 0.7 for quality gate escalation: informed estimate, requires empirical calibration with real hardware photos
- PRT Qutie protocol nature (CDC-ACM vs. HID vs. Bluetooth-only): unknown until hardware arrival 2026-04-13
---
*Research completed: 2026-04-09*
*Ready for roadmap: yes*