homelabby/.planning/phases/02-ai-pipeline/02-03-PLAN.md

---
phase: 02-ai-pipeline
plan: "03"
type: execute
wave: 3
depends_on: [02-01, 02-02]
files_modified:
  - internal/api/handlers/intake.go
  - internal/api/handlers/intake_test.go
  - internal/api/router.go
  - cmd/hwlab/main.go
autonomous: true
requirements: [AI-02, AI-03, AI-07]

must_haves:
  truths:
    - "POST /api/intake with 1-3 JPEG/PNG files returns 200 with serial, model, manufacturer, specs, category, tags, hw_id, catalog_status"
    - "POST /api/intake with 0 or 4+ files returns 400"
    - "Quick add mode (confidence >= quick_add_threshold AND quick_add_enabled=true) creates NetBox record in one step; returns hw_id in response"
    - "When NetBox is unreachable, intake enqueues netbox.create_device op to WAQ and returns 202"
    - "WAQ real handler (NewNetBoxOpHandler) replaces NoOpHandler in main.go"
  artifacts:
    - path: "internal/api/handlers/intake.go"
      provides: "POST /api/intake multipart handler"
      exports: [IntakeHandler, NewIntakeHandler]
    - path: "internal/api/handlers/intake_test.go"
      provides: "Unit tests using MockAIClient and mock NetBox client"
    - path: "internal/api/router.go"
      provides: "POST /api/intake route registered"
      contains: "POST.*intake"
    - path: "cmd/hwlab/main.go"
      provides: "NewNetBoxOpHandler wired as WAQ handler"
      contains: "NewNetBoxOpHandler"
  key_links:
    - from: "internal/api/handlers/intake.go"
      to: "internal/ai/orchestrator.go"
      via: "IntakeHandler.ServeHTTP calls orchestrator.Analyze with base64-encoded photos"
      pattern: "orchestrator\\.Analyze"
    - from: "internal/api/handlers/intake.go"
      to: "internal/netbox/hwid.go"
      via: "AllocateNextHWID called after successful AI analysis"
      pattern: "AllocateNextHWID"
    - from: "internal/api/handlers/intake.go"
      to: "internal/queue/handler.go"
      via: "WAQ.Enqueue called with OpNetBoxCreateDevice payload when NetBox unreachable"
      pattern: "OpNetBoxCreateDevice"
---

<objective>
Implement POST /api/intake: multipart photo upload → orchestrator → HW-ID allocation → NetBox create (or WAQ enqueue on failure) → tag sync → catalog status. Wire real WAQ handler in main.go. Add quick add mode.

Purpose: This is the core end-to-end intake flow — the primary value proposition of HWLab Phase 2.
Output: Fully wired intake endpoint, updated router, updated main.go with real WAQ handler.
</objective>

<execution_context>
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/phases/02-ai-pipeline/02-CONTEXT.md
@.planning/phases/02-ai-pipeline/02-01-SUMMARY.md
@.planning/phases/02-ai-pipeline/02-02-SUMMARY.md

<interfaces>
<!-- Contracts from Plans 01 and 02 -->

From internal/ai/orchestrator.go:
```go
type Orchestrator struct{ /* ... */ }
func NewOrchestrator(tier1, tier2 AIClient, threshold float64) *Orchestrator
func (o *Orchestrator) Analyze(ctx context.Context, req IntakeRequest) (*IntakeResult, inventory.CatalogStatus, error)
```

From internal/ai/types.go:
```go
type IntakeRequest struct { PhotosBase64 []string; JobID string }
type IntakeResult struct {
    SerialNumber, Model, Manufacturer, Category string
    Specs map[string]string; SuggestedTags []string
    AINotes string; Confidence float64; ConfidenceNote string
}
type AIConfig struct {
    Tier1, Tier2        TierConfig
    ConfidenceThreshold float64
    QuickAddEnabled     bool
    QuickAddThreshold   float64
}
```

From internal/netbox/client.go:
```go
func (c *Client) CreateDevice(ctx, name, assetTag string, deviceTypeID, roleID, siteID int32) (int64, error)
func (c *Client) PatchCustomFields(ctx, deviceID int64, patch map[string]interface{}) error
func (c *Client) AllocateNextHWID(ctx) (string, error)
func (c *Client) SyncTags(ctx, tags []string) ([]netbox.TagRef, error)
```

From internal/netbox/custom_fields.go:
```go
func BuildFullCustomFieldsPatch(cf CustomFields) map[string]interface{}
```

From internal/inventory/catalog_updater.go:
```go
type CatalogUpdater struct{ /* wraps *netbox.Client */ }
func (u *CatalogUpdater) UpdateCatalogStatus(ctx, deviceID int64, current, next inventory.CatalogStatus) error
```

From internal/queue/handler.go:
```go
const OpNetBoxCreateDevice     = "netbox.create_device"
const OpNetBoxPatchCustomFields = "netbox.patch_custom_fields"
type CreateDevicePayload struct {
    Name string; AssetTag string; DeviceTypeID, RoleID, SiteID int32
}
func NewNetBoxOpHandler(client NetBoxOpsClient) OpHandler
```

From internal/queue/waq.go:
```go
func (q *WAQ) Enqueue(ctx, op PendingOp) error
func NewPendingOp(opType string, payload json.RawMessage) PendingOp
```

From internal/config/config.go:
```go
type Config struct {
    // ... existing fields ...
    AI ai.AIConfig
}
```

NetBox device defaults for new items (use these IDs for Phase 2 — they must exist in the provisioned NetBox):
- DeviceTypeID: 1 (placeholder — "Generic Device" type must be provisioned in NetBox)
- RoleID: 1 (placeholder — "Inventory Item" role must be provisioned in NetBox)  
- SiteID: 1 (placeholder — "Homelab" site provisioned in Phase 1)

Add to config.go defaults:
```go
v.SetDefault("netbox_default_device_type_id", 1)
v.SetDefault("netbox_default_role_id", 1)
v.SetDefault("netbox_default_site_id", 1)
```

These become `Config.NetBoxDefaultDeviceTypeID int32`, etc.
</interfaces>
</context>

<tasks>

<task type="auto" tdd="true">
  <name>Task 1: POST /api/intake handler with orchestrator and NetBox wiring</name>
  <files>
    internal/api/handlers/intake.go,
    internal/api/handlers/intake_test.go,
    internal/config/config.go
  </files>
  <read_first>
    - internal/api/handlers/health.go (full — understand handler pattern used in this project)
    - internal/api/handlers/health_test.go (full — understand test patterns)
    - internal/netbox/client.go (full — AllocateNextHWID, CreateDevice, PatchCustomFields, SyncTags)
    - internal/netbox/custom_fields.go (full — BuildFullCustomFieldsPatch)
    - internal/inventory/catalog_updater.go (full — UpdateCatalogStatus)
    - internal/config/config.go (full — to add new NetBox default ID fields)
    - internal/queue/waq.go (full — Enqueue, NewPendingOp)
    - internal/queue/handler.go (full — op type constants and payload structs)
  </read_first>
  <behavior>
    - Test: TestIntakeHandlerRejectsZeroPhotos — POST /api/intake with no files returns 400
    - Test: TestIntakeHandlerRejectsFourPhotos — POST /api/intake with 4 files returns 400
    - Test: TestIntakeHandlerHighConfidence — mock orchestrator returns HighConfidenceResult (0.95); mock NetBox CreateDevice succeeds; response is 201 JSON with fields: hw_id, model, manufacturer, category, catalog_status="indexed"
    - Test: TestIntakeHandlerLowConfidence — mock returns LowConfidenceResult (0.40); response is 201 with catalog_status="needs_research"
    - Test: TestIntakeHandlerQuickAdd — quick_add_enabled=true, quick_add_threshold=0.90, mock returns confidence 0.95; response is 201; CreateDevice called once (verify no review step)
    - Test: TestIntakeHandlerNetBoxDown — mock NetBox CreateDevice returns error; handler enqueues to WAQ; returns 202 with queued=true in JSON body
  </behavior>
  <action>
**Extend internal/config/config.go** — add three fields to Config struct and defaults:

```go
NetBoxDefaultDeviceTypeID int32 `mapstructure:"netbox_default_device_type_id"`
NetBoxDefaultRoleID       int32 `mapstructure:"netbox_default_role_id"`
NetBoxDefaultSiteID       int32 `mapstructure:"netbox_default_site_id"`
```

Defaults in Load():
```go
v.SetDefault("netbox_default_device_type_id", 1)
v.SetDefault("netbox_default_role_id", 1)
v.SetDefault("netbox_default_site_id", 1)
```

Bindings:
```go
_ = v.BindEnv("netbox_default_device_type_id", "HWLAB_NETBOX_DEFAULT_DEVICE_TYPE_ID")
_ = v.BindEnv("netbox_default_role_id", "HWLAB_NETBOX_DEFAULT_ROLE_ID")
_ = v.BindEnv("netbox_default_site_id", "HWLAB_NETBOX_DEFAULT_SITE_ID")
```

**internal/api/handlers/intake.go** — the intake handler:

Define interfaces for testability (handler does not import netbox.Client directly):
```go
// intakeNetBoxClient is the subset of netbox.Client the intake handler needs.
type intakeNetBoxClient interface {
    AllocateNextHWID(ctx context.Context) (string, error)
    CreateDevice(ctx context.Context, name, assetTag string, deviceTypeID, roleID, siteID int32) (int64, error)
    PatchCustomFields(ctx context.Context, deviceID int64, patch map[string]interface{}) error
    SyncTags(ctx context.Context, tags []string) ([]netbox.TagRef, error)
}

// intakeCatalogUpdater is the subset needed for catalog status.
type intakeCatalogUpdater interface {
    UpdateCatalogStatus(ctx context.Context, deviceID int64, current, next inventory.CatalogStatus) error
}

// intakeWAQ is the subset of WAQ the handler needs.
type intakeWAQ interface {
    Enqueue(ctx context.Context, op queue.PendingOp) error
}
```

IntakeHandler struct:
```go
type IntakeHandler struct {
    orchestrator    *ai.Orchestrator
    netbox          intakeNetBoxClient
    catalogUpdater  intakeCatalogUpdater
    waq             intakeWAQ  // may be nil if DragonFlyDB unavailable
    deviceTypeID    int32
    roleID          int32
    siteID          int32
    quickAddEnabled bool
    quickAddThresh  float64
}

func NewIntakeHandler(
    orch *ai.Orchestrator,
    nb intakeNetBoxClient,
    cu intakeCatalogUpdater,
    waq intakeWAQ,
    deviceTypeID, roleID, siteID int32,
    quickAddEnabled bool,
    quickAddThresh float64,
) *IntakeHandler {
    return &IntakeHandler{
        orchestrator:    orch,
        netbox:          nb,
        catalogUpdater:  cu,
        waq:             waq,
        deviceTypeID:    deviceTypeID,
        roleID:          roleID,
        siteID:          siteID,
        quickAddEnabled: quickAddEnabled,
        quickAddThresh:  quickAddThresh,
    }
}
```

IntakeResponse JSON struct:
```go
type IntakeResponse struct {
    HWID          string            `json:"hw_id"`
    Model         string            `json:"model"`
    Manufacturer  string            `json:"manufacturer"`
    Category      string            `json:"category"`
    Specs         map[string]string `json:"specs"`
    SuggestedTags []string          `json:"suggested_tags"`
    AINotes       string            `json:"ai_notes"`
    Confidence    float64           `json:"confidence"`
    CatalogStatus string            `json:"catalog_status"`
    NetBoxID      int64             `json:"netbox_id,omitempty"`
    Queued        bool              `json:"queued,omitempty"` // true if NetBox was unreachable
}
```

ServeHTTP flow:
1. `r.ParseMultipartForm(32 << 20)` — 32MB max
2. Validate files count: len(files) == 0 → 400; len(files) > 3 → 400
3. Read each file, detect MIME (http.DetectContentType on first 512 bytes), base64-encode → photosBase64 slice
4. Generate jobID: `uuid.New().String()`
5. `result, status, err := h.orchestrator.Analyze(r.Context(), ai.IntakeRequest{PhotosBase64: photosBase64, JobID: jobID})`
6. If err != nil → 500
7. `hwid, err := h.netbox.AllocateNextHWID(r.Context())`; if err → 500
8. Decide name: `result.Manufacturer + " " + result.Model` (trim spaces; if empty use hwid)
9. Quick add check: if h.quickAddEnabled && result.Confidence >= h.quickAddThresh → attempt NetBox create
10. Non-quick-add path AND quick-add path both try CreateDevice; on error → enqueue to WAQ if available → 202
11. On successful CreateDevice: PatchCustomFields with BuildFullCustomFieldsPatch, SyncTags, UpdateCatalogStatus
12. Return 201 (or 202 if queued) JSON IntakeResponse

For BuildFullCustomFieldsPatch, construct a netbox.CustomFields from IntakeResult:
```go
cf := netboxTypes.CustomFields{
    HWID:          hwid,
    CatalogStatus: string(status),
    AINotes:       result.AINotes,
}
patch := netbox.BuildFullCustomFieldsPatch(cf)
```

Note: Use encoding/json to marshal CreateDevicePayload for WAQ enqueue:
```go
payload, _ := json.Marshal(queue.CreateDevicePayload{
    Name:         deviceName,
    AssetTag:     hwid,
    DeviceTypeID: h.deviceTypeID,
    RoleID:       h.roleID,
    SiteID:       h.siteID,
})
op := queue.NewPendingOp(queue.OpNetBoxCreateDevice, payload)
h.waq.Enqueue(r.Context(), op)
```

Return 202 with `{"queued": true, "hw_id": hwid, ...}` when NetBox was unreachable.

**internal/api/handlers/intake_test.go** — six tests using mock structs:

Define mock types in intake_test.go (unexported):
- `mockOrchestrator` with `FixedResult *ai.IntakeResult`, `FixedStatus inventory.CatalogStatus` — wraps with an `Analyze` method matching the expected signature
- `mockNetBox` with configurable return values for AllocateNextHWID, CreateDevice, PatchCustomFields, SyncTags
- `mockCatalogUpdater`
- `mockWAQ` that records enqueued ops

Use `net/http/httptest` to create a recorder, call `handler.ServeHTTP(rec, req)`.

For multipart body construction in tests:
```go
var body bytes.Buffer
w := multipart.NewWriter(&body)
fw, _ := w.CreateFormFile("photos", "test.jpg")
fw.Write([]byte{0xff, 0xd8, 0xff}) // minimal JPEG header
w.Close()
req := httptest.NewRequest(http.MethodPost, "/api/intake", &body)
req.Header.Set("Content-Type", w.FormDataContentType())
```

NOTE: The handler receives `*ai.Orchestrator` but for tests you need to pass a mock orchestrator. Refactor IntakeHandler to accept an orchestratorFunc or define an `IntakeOrchestrator` interface with `Analyze(ctx, req) (*IntakeResult, CatalogStatus, error)` — use the interface instead of the concrete type. This decouples tests cleanly.

Update IntakeHandler.orchestrator field to use an interface:
```go
type intakeOrchestrator interface {
    Analyze(ctx context.Context, req ai.IntakeRequest) (*ai.IntakeResult, inventory.CatalogStatus, error)
}
```
The concrete `*ai.Orchestrator` satisfies this interface automatically.
  </action>
  <verify>
    <automated>cd /home/mikkel/homelabby && go build ./... && go test ./internal/api/... ./internal/config/... -v 2>&1 | tail -40</automated>
  </verify>
  <done>
    - All 6 TestIntakeHandler* tests pass
    - `go build ./...` clean
    - Config struct has NetBoxDefaultDeviceTypeID, NetBoxDefaultRoleID, NetBoxDefaultSiteID
    - internal/api/handlers/intake.go exists with NewIntakeHandler and ServeHTTP
  </done>
</task>

<task type="auto">
  <name>Task 2: Wire intake route and real WAQ handler in router and main.go</name>
  <files>internal/api/router.go, cmd/hwlab/main.go</files>
  <read_first>
    - internal/api/router.go (full — add POST /api/intake route)
    - cmd/hwlab/main.go (full — wire intake handler and swap NoOpHandler for NetBoxOpHandler)
    - internal/api/handlers/intake.go (skim — NewIntakeHandler signature)
    - internal/queue/handler.go (skim — NewNetBoxOpHandler signature)
    - internal/netbox/client.go (skim — *Client satisfies intakeNetBoxClient and NetBoxOpsClient interfaces)
    - internal/inventory/catalog_updater.go (skim — NewCatalogUpdater if it exists, or construct CatalogUpdater directly)
  </read_first>
  <action>
**internal/api/router.go** — add POST /api/intake:

Update NewRouter signature to accept an http.Handler for the intake endpoint:
```go
func NewRouter(staticFiles fs.FS, intakeHandler http.Handler) http.Handler {
    r := chi.NewRouter()
    // ... existing middleware ...
    r.Route("/api", func(r chi.Router) {
        r.Get("/health", handlers.Health)
        r.Post("/intake", intakeHandler.ServeHTTP)
    })
    // ... existing SPA handler ...
}
```

**cmd/hwlab/main.go** — wire everything:

1. Load config (existing)
2. Create NetBox client: `nbClient, err := netbox.NewClient(cfg.NetBoxURL, cfg.NetBoxToken)`; if err → log.Fatalf
3. Create AI tier clients:
```go
tier1 := ai.NewTierClient(cfg.AI.Tier1)
tier2 := ai.NewTierClient(cfg.AI.Tier2)
orch  := ai.NewOrchestrator(tier1, tier2, cfg.AI.ConfidenceThreshold)
```
4. Create catalog updater:
```go
catalogUpdater := &inventory.CatalogUpdater{} // or however it's constructed from Phase 1
```
   Read internal/inventory/catalog_updater.go to check its exact constructor/struct literal.

5. Create intake handler:
```go
intakeHandler := handlers.NewIntakeHandler(
    orch,
    nbClient,
    catalogUpdater,
    waq,   // may be nil — handler must handle nil waq gracefully
    cfg.NetBoxDefaultDeviceTypeID,
    cfg.NetBoxDefaultRoleID,
    cfg.NetBoxDefaultSiteID,
    cfg.AI.QuickAddEnabled,
    cfg.AI.QuickAddThreshold,
)
```

6. Swap NoOpHandler for real WAQ handler:
```go
// Replace: go waq.RunWorker(ctx, queue.NoOpHandler, ...)
// With:
nbHandler := queue.NewNetBoxOpHandler(nbClient)
go waq.RunWorker(ctx, nbHandler, cfg.WAQMaxAttempts, retryInterval)
```

7. Pass intakeHandler to router:
```go
router := api.NewRouter(staticFiles, intakeHandler)
```

Handle the case where waq is nil: IntakeHandler.waq is an interface — if waq init failed (non-fatal), pass nil. In ServeHTTP, check `if h.waq != nil` before calling Enqueue. If waq is nil and NetBox is down → return 503 (service unavailable, cannot queue).

Also handle nil waq in main.go WAQ worker section:
```go
var nbHandler queue.OpHandler
if waq != nil {
    nbHandler = queue.NewNetBoxOpHandler(nbClient)
    go waq.RunWorker(ctx, nbHandler, cfg.WAQMaxAttempts, retryInterval)
}
```

NOTE: `catalog_updater.go` from Phase 1 — read it to find the correct constructor. The struct is:
```go
type CatalogUpdater struct {
    client *netbox.Client
}
```
Construct with struct literal: `&inventory.CatalogUpdater{...}` or if it has a constructor `inventory.NewCatalogUpdater(nbClient)`. Check the actual file.

After editing, run `go build ./...` to confirm compilation. Fix any import cycle or interface mismatch errors before finishing.
  </action>
  <verify>
    <automated>cd /home/mikkel/homelabby && go build ./... 2>&1 && echo "BUILD OK" && curl -s http://localhost:8080/api/health 2>/dev/null || echo "(server not running — build check only)"</automated>
  </verify>
  <done>
    - `go build ./...` passes with zero errors
    - router.go has `r.Post("/intake", ...)` route
    - main.go uses NewNetBoxOpHandler (grep confirms; NoOpHandler no longer referenced in main.go)
    - main.go creates netbox.Client, ai.Orchestrator, and handlers.IntakeHandler
  </done>
</task>

</tasks>

<threat_model>
## Trust Boundaries

| Boundary | Description |
|----------|-------------|
| HTTP client → /api/intake | Untrusted multipart file upload from browser or curl |
| intake handler → oMLX | Base64 image data sent to local AI — image content is untrusted |
| intake handler → NetBox | Structured data written to source of truth |

## STRIDE Threat Register

| Threat ID | Category | Component | Disposition | Mitigation Plan |
|-----------|----------|-----------|-------------|-----------------|
| T-02-09 | Denial of Service | multipart upload size | mitigate | r.ParseMultipartForm(32 << 20) — 32MB hard cap on request body |
| T-02-10 | Tampering | AI-extracted device name | mitigate | device name passed directly to CreateDevice — net/http handler sanitizes via Go string (no SQL injection possible; go-netbox marshals to JSON) |
| T-02-11 | Tampering | AI-extracted tags | accept | Tags pass through normalizeTags in SyncTags (Phase 1 T-04-02 mitigation) — slug normalization strips injection characters |
| T-02-12 | Denial of Service | photo count bypass | mitigate | Explicit len check 1-3 before any processing; 0 or 4+ returns 400 immediately |
| T-02-13 | Spoofing | intake response hw_id | accept | HW-ID assigned by AllocateNextHWID, not caller-controlled; sequential allocation cannot be spoofed via this endpoint |
</threat_model>

<verification>
After plan completion:
1. `go build ./...` — zero errors
2. `go test ./internal/api/... -v` — all intake handler tests pass, health tests still pass
3. `grep "Post.*intake" internal/api/router.go` — route present
4. `grep "NewNetBoxOpHandler" cmd/hwlab/main.go` — real handler wired
5. `grep "NoOpHandler" cmd/hwlab/main.go` — NOT present (replaced)
6. `go test ./...` — zero failures (all packages)
</verification>

<success_criteria>
- POST /api/intake registered and reachable
- Handler validates 1-3 photos, returns 400 on violations
- Mock-based unit tests cover high confidence, low confidence, quick add, and NetBox-down scenarios
- WAQ real handler (NewNetBoxOpHandler) used in main.go — NoOpHandler no longer in main.go
- `go build ./...` clean with zero errors
</success_criteria>

<output>
After completion, create `.planning/phases/02-ai-pipeline/02-03-SUMMARY.md`
</output>