---
phase: 02-ai-pipeline
verified: 2026-04-10T07:00:00Z
status: human_needed
score: 4/5 must-haves verified
overrides_applied: 0
human_verification:
  - test: "Install oMLX on Mac Mini M4 and serve Gemma 4 E4B model on port 8000. Run: HWLAB_OMLX_URL=http://localhost:8000/v1 go test -tags integration ./internal/ai/... -run TestOMLXIntegration -v"
    expected: "Test passes (non-panic IntakeResult returned, confidence in [0,1]). Activity Monitor shows peak RAM usage for E4B; document in docs/omlx-setup.md."
    why_human: "oMLX requires Mac Mini M4 hardware with Apple Silicon. Cannot install or run on dev machine. Integration test file was not created as part of deferred plan 02-04."
  - test: "POST a real hardware photo to http://localhost:8080/api/intake: curl -X POST -F 'photos=@your-photo.jpg' http://localhost:8080/api/intake | python3 -m json.tool"
    expected: "JSON response includes hw_id (HW-XXXXX format), model, manufacturer, category, specs (non-empty map), suggested_tags, confidence score, and catalog_status of 'indexed' or 'needs_research'."
    why_human: "End-to-end validation requires live oMLX inference and a running NetBox instance. Unit tests mock both dependencies."
  - test: "Swap Tier 1 model in ai_config.json to a different OpenRouter model (e.g. google/gemma-3-12b-it), restart server, send intake request, confirm response still works."
    expected: "Intake endpoint returns valid JSON response with new model. No code changes needed — only ai_config.json change."
    why_human: "Config-driven tier swapping can be verified by code inspection (BaseURL override in TierClient confirmed), but live smoke test with an actual model swap confirms the full path."
---

# Phase 2: AI Pipeline Verification Report

**Phase Goal:** Users can submit 1-3 photos and receive a structured NetBox-ready record with AI-extracted specs, suggested category/tags, and a quality gate status reflecting confidence
**Verified:** 2026-04-10T07:00:00Z
**Status:** human_needed
**Re-verification:** No — initial verification

## Goal Achievement

### Observable Truths (from ROADMAP.md success criteria)

| # | Truth | Status | Evidence |
|---|-------|--------|----------|
| 1 | oMLX serves Gemma 4 on Mac Mini M4 with measured memory budget documented | ? HUMAN NEEDED | Plan 02-04 deferred — oMLX hardware not available on dev machine. Integration test file not created. docs/omlx-setup.md does not exist. |
| 2 | POST /api/intake with 1-3 photos returns serial number, model, manufacturer, specs, category, and tags extracted by AI | ✓ VERIFIED | `internal/api/handlers/intake.go` — ServeHTTP parses multipart, calls orchestrator.Analyze, returns IntakeResponse with all fields. 6/6 unit tests pass including high/low confidence and rejection of 0 or 4+ photos. |
| 3 | Items with AI confidence below threshold are automatically set to needs_research; high-confidence items advance to indexed | ✓ VERIFIED | `internal/ai/orchestrator.go` — Analyze() maps confidence < threshold → StatusNeedsResearch, >= threshold → StatusIndexed. 5/5 orchestrator tests cover all escalation paths. TestIntakeHandlerHighConfidence (201, indexed) and TestIntakeHandlerLowConfidence (201, needs_research) confirm handler propagates status correctly. |
| 4 | Quick add mode skips review for high-confidence items and creates the NetBox record in one step | ✓ VERIFIED | `internal/api/handlers/intake.go` lines 54-77: quickAddEnabled + quickAddThresh fields. `TestIntakeHandlerQuickAdd` confirms: quick_add_enabled=true, confidence 0.95 → CreateDevice called once, 201 response. Config-driven via `cfg.AI.QuickAddEnabled` and `cfg.AI.QuickAddThreshold` in main.go. |
| 5 | Any AI tier (local oMLX, OpenRouter) can be swapped by changing a config JSON value with no code changes | ✓ VERIFIED | `internal/ai/client.go:31-32` — `NewTierClient` uses `openai.DefaultConfig(key)` + `oCfg.BaseURL = cfg.BaseURL`. `ai_config.json` has tier1 (localhost:8000) and tier2 (openrouter.ai) independently configurable. `TierConfig.BaseURL` and `TierConfig.Model` both have `mapstructure` tags wired to viper. Changing ai_config.json values requires no code changes. |

**Score:** 4/5 truths verified (1 requires human with Mac Mini M4 hardware)

### Required Artifacts

| Artifact | Expected | Status | Details |
|----------|----------|--------|---------|
| `internal/ai/types.go` | IntakeRequest, IntakeResult, TierConfig, AIConfig domain types | ✓ VERIFIED | Exists, all four types present with JSON + mapstructure tags |
| `internal/ai/client.go` | AIClient interface + TierClient production implementation | ✓ VERIFIED | AIClient interface at line 17, TierClient at line 22, NewTierClient at line 30 |
| `internal/ai/mock.go` | MockAIClient test double with fixture constructors | ✓ VERIFIED | MockAIClient, HighConfidenceResult(), LowConfidenceResult() all present |
| `internal/ai/prompts/intake.go` | BuildIntakePrompt() returning JSON-extraction prompt template | ✓ VERIFIED | File exists with BuildIntakePrompt(photoCount int) |
| `internal/ai/orchestrator.go` | Orchestrator with Analyze(ctx, IntakeRequest) → (*IntakeResult, CatalogStatus, error) | ✓ VERIFIED | NewOrchestrator and Analyze both present; all 5 tests pass |
| `internal/ai/research.go` | ResearchClient interface + NoOpResearchClient stub | ✓ VERIFIED | Both present; NoOpResearchClient returns nil, nil (Phase 7 placeholder) |
| `internal/queue/handler.go` | NetBoxOpHandler for create_device and patch_custom_fields | ✓ VERIFIED | NewNetBoxOpHandler, OpNetBoxCreateDevice, OpNetBoxPatchCustomFields constants, NetBoxOpsClient interface all present; 6 tests pass |
| `internal/api/handlers/intake.go` | POST /api/intake multipart handler | ✓ VERIFIED | IntakeHandler, NewIntakeHandler, ServeHTTP with full flow |
| `internal/api/router.go` | POST /api/intake route registered | ✓ VERIFIED | `r.Post("/intake", intakeHandler.ServeHTTP)` at line 44 |
| `cmd/hwlab/main.go` | NewNetBoxOpHandler wired as WAQ handler | ✓ VERIFIED | `queue.NewNetBoxOpHandler(nbClient)` at line 59; NoOpHandler absent |
| `internal/config/config.go` | Config struct with AI AIConfig and NetBoxDefault* fields | ✓ VERIFIED | `AI ai.AIConfig` at line 31; NetBoxDefaultDeviceTypeID/RoleID/SiteID at lines 27-29 |
| `ai_config.json` | Template config with tier1/tier2/threshold/quick_add settings | ✓ VERIFIED | File exists with all expected fields |
| `internal/ai/omlx_integration_test.go` | Integration test that skips unless HWLAB_OMLX_URL is set | ✗ MISSING | Plan 02-04 was fully deferred — file not created. Required for AI-01 validation. |
| `docs/omlx-setup.md` | oMLX installation steps, model tier selection, measured memory budget | ✗ MISSING | Deferred with plan 02-04 — docs/ directory does not exist. |

### Key Link Verification

| From | To | Via | Status | Details |
|------|----|-----|--------|---------|
| `internal/config/config.go` | `internal/ai/types.go` | `Config.AI ai.AIConfig` embeds TierConfig | ✓ WIRED | `AI ai.AIConfig \`mapstructure:"ai"\`` at line 31; AIConfig contains Tier1, Tier2 TierConfig |
| `internal/ai/client.go` | `github.com/sashabaranov/go-openai` | TierClient wraps openai.Client; BaseURL from TierConfig | ✓ WIRED | `oCfg := openai.DefaultConfig(cfg.APIKey); oCfg.BaseURL = cfg.BaseURL` in NewTierClient |
| `internal/ai/orchestrator.go` | `internal/inventory/quality_gate.go` | Returns inventory.CatalogStatus — StatusIndexed or StatusNeedsResearch | ✓ WIRED | `inventory.StatusIndexed` and `inventory.StatusNeedsResearch` used in Analyze() |
| `internal/queue/handler.go` | `internal/netbox/client.go` | NetBoxOpHandler calls CreateDevice or PatchCustomFields based on op.Type | ✓ WIRED | NetBoxOpsClient interface matches *netbox.Client methods; routing via switch op.Type |
| `internal/api/handlers/intake.go` | `internal/ai/orchestrator.go` | IntakeHandler calls orchestrator.Analyze with base64-encoded photos | ✓ WIRED | `result, status, err := h.orchestrator.Analyze(r.Context(), ai.IntakeRequest{...})` at line 146 |
| `internal/api/handlers/intake.go` | `internal/netbox/hwid.go` | AllocateNextHWID called after successful AI analysis | ✓ WIRED | `hwid, err := h.netboxClient.AllocateNextHWID(r.Context())` at line 156 |
| `internal/api/handlers/intake.go` | `internal/queue/handler.go` | WAQ.Enqueue called with OpNetBoxCreateDevice payload when NetBox unreachable | ✓ WIRED | `queue.NewPendingOp(queue.OpNetBoxCreateDevice, ...)` at line 193; TestIntakeHandlerNetBoxDown confirms 202 + WAQ enqueue |
| `internal/ai/omlx_integration_test.go` | `http://localhost:8000/v1` | TierClient with real oMLX endpoint; skips when HWLAB_OMLX_URL unset | ✗ NOT WIRED | File not created (plan 02-04 deferred) |

### Data-Flow Trace (Level 4)

| Artifact | Data Variable | Source | Produces Real Data | Status |
|----------|---------------|--------|--------------------|--------|
| `intake.go` ServeHTTP | `result *ai.IntakeResult` | `orchestrator.Analyze()` → tier1/tier2 `AnalyzePhotos()` | Yes — real HTTP call to oMLX/OpenRouter in production; MockAIClient in tests | ✓ FLOWING (mock in tests, real in prod) |
| `intake.go` ServeHTTP | `hwid string` | `netboxClient.AllocateNextHWID()` → NetBox API call | Yes — NetBox assigns sequential HW-XXXXX IDs | ✓ FLOWING |
| `orchestrator.go` Analyze | `result *ai.IntakeResult` | `tier1.AnalyzePhotos()` then optional `tier2.AnalyzePhotos()` | Yes — go-openai calls real LLM endpoint | ✓ FLOWING |

### Behavioral Spot-Checks

| Behavior | Command | Result | Status |
|----------|---------|--------|--------|
| `go build ./...` compiles clean | `go build ./... && echo BUILD OK` | BUILD OK | ✓ PASS |
| All unit tests pass (no FAIL) | `go test ./... -count=1` | 6 packages ok, 0 FAIL | ✓ PASS |
| POST /api/intake rejects 0 photos (400) | `go test ./internal/api/handlers/... -run TestIntakeHandlerRejectsZeroPhotos -v` | PASS | ✓ PASS |
| POST /api/intake rejects 4 photos (400) | `go test ./internal/api/handlers/... -run TestIntakeHandlerRejectsFourPhotos -v` | PASS | ✓ PASS |
| Orchestrator escalates tier1→tier2 on low confidence | `go test ./internal/ai/... -run TestOrchestratorLowConfidenceEscalates -v` | PASS | ✓ PASS |
| WAQ enqueues on NetBox failure (202 response) | `go test ./internal/api/handlers/... -run TestIntakeHandlerNetBoxDown -v` | PASS | ✓ PASS |
| NoOpHandler replaced in main.go | `grep NoOpHandler cmd/hwlab/main.go` | no output | ✓ PASS |
| oMLX integration test on Mac Mini | requires Mac Mini M4 hardware + oMLX installed | N/A | ? SKIP (hardware) |

### Requirements Coverage

| Requirement | Source Plan | Description | Status | Evidence |
|-------------|------------|-------------|--------|----------|
| AI-01 | 02-04 | oMLX installed on Mac Mini M4 with Gemma 4 serving OpenAI-compatible API | ? NEEDS HUMAN | Integration test file not created; oMLX hardware setup deferred to human UAT |
| AI-02 | 02-03 | User can upload 1-3 photos and AI extracts serial, model, manufacturer, specs | ✓ SATISFIED | intake.go ServeHTTP; 6 handler tests; IntakeResponse includes all fields |
| AI-03 | 02-03 | AI suggests category, tags, and location for each item | ✓ SATISFIED | IntakeResult.Category, SuggestedTags in response; SyncTags called in handler |
| AI-04 | 02-02 (stub) | AI calls SearXNG via function calling to research product specs | ✓ SATISFIED (stub) | ResearchClient interface + NoOpResearchClient in research.go. REQUIREMENTS.md traceability maps AI-04 to Phase 7 — stub satisfies Phase 2 scope. |
| AI-05 | 02-02 | Orchestrator reviews Tier 1 output for completeness and flags gaps as needs_research | ✓ SATISFIED | orchestrator.Analyze escalates low-confidence results; confidence < threshold → StatusNeedsResearch |
| AI-06 | 02-02 | Tier 2 research agent (OpenRouter) automatically enriches items flagged needs_research | ✓ SATISFIED | Orchestrator escalates to tier2 when tier1 confidence below threshold; tier2 configured as OpenRouter in ai_config.json |
| AI-07 | 02-03 | Quick add mode skips review screen for items with high AI confidence | ✓ SATISFIED | quickAddEnabled + quickAddThresh in IntakeHandler; TestIntakeHandlerQuickAdd confirms one-step NetBox create |
| AI-08 | 02-01 | All AI tiers accessed via single OpenAI-compatible client with configurable base URLs | ✓ SATISFIED | AIClient interface, TierClient wraps go-openai with BaseURL override |
| AI-09 | 02-01 | Provider routing configured via JSON file — swap any tier without code changes | ✓ SATISFIED | ai_config.json drives tier1/tier2 BaseURL + Model; mapstructure bindings confirmed |

### Anti-Patterns Found

| File | Line | Pattern | Severity | Impact |
|------|------|---------|----------|--------|
| `internal/ai/research.go` | 22-24 | `NoOpResearchClient.Search` returns `nil, nil` | ℹ️ Info | Intentional Phase 2 stub for Phase 7 SearXNG implementation. ResearchClient interface is not wired to any production path yet — no data flows through this path. Not a blocker. |

### Human Verification Required

#### 1. oMLX Memory Validation and Integration Test

**Test:** On Mac Mini M4, install oMLX and serve Gemma 4 E4B on port 8000. Then create `internal/ai/omlx_integration_test.go` (template in Plan 02-04) and run:
```
HWLAB_OMLX_URL=http://localhost:8000/v1 go test -tags integration ./internal/ai/... -run TestOMLXIntegration -v
```
While the test runs, open Activity Monitor and note the oMLX process "Real Memory" peak.
Document peak memory in `docs/omlx-setup.md`: "Gemma 4 E4B: X GB real memory on Mac Mini M4 16GB".

**Expected:** Test PASS. Peak memory for E4B expected ~8-10 GB, leaving sufficient headroom for Go backend (~200 MB) and macOS overhead.

**Why human:** Requires Apple Silicon Mac Mini M4 hardware. oMLX does not run on Intel/Linux. The integration test scaffold was part of deferred Plan 02-04.

#### 2. Live End-to-End Intake with Real Photo

**Test:** Start server (`go run cmd/hwlab/main.go`) and send a real hardware photo:
```
curl -s -X POST http://localhost:8080/api/intake \
  -F "photos=@/path/to/hardware-photo.jpg" | python3 -m json.tool
```

**Expected:** JSON response with `hw_id` (HW-XXXXX format), `model`, `manufacturer`, `category`, `specs` (non-empty), `suggested_tags`, `confidence` score, and `catalog_status` of `"indexed"` or `"needs_research"` depending on AI confidence.

**Why human:** Requires live oMLX inference on Mac Mini and a running NetBox instance. All dependencies are mocked in unit tests.

#### 3. Config-Driven Tier Swap Smoke Test

**Test:** Edit `ai_config.json` to change tier1 model, restart server, send intake request. No code changes should be needed.

**Expected:** Intake endpoint continues to respond with valid JSON. Tier1 uses the new model name from config.

**Why human:** Code inspection confirms the mechanism (BaseURL + Model from TierConfig), but live smoke test confirms the full config parse → client construction → API call path with a real endpoint.

### Gaps Summary

No blocking gaps in the code artifacts. The phase delivered all planned code for Plans 02-01, 02-02, and 02-03 with all unit tests passing. Plan 02-04 (oMLX integration validation) was explicitly deferred due to hardware unavailability and is tracked in HUMAN-UAT.md.

The two missing artifacts (`internal/ai/omlx_integration_test.go` and `docs/omlx-setup.md`) are gated on Mac Mini M4 availability and should be created as part of the human UAT process described above.

AI-04 (SearXNG function calling) is correctly stubbed — REQUIREMENTS.md maps AI-04 to Phase 7, and the `ResearchClient` interface is in place for that implementation.

---

_Verified: 2026-04-10T07:00:00Z_
_Verifier: Claude (gsd-verifier)_