homelabby/.planning/phases/02-ai-pipeline/02-03-PLAN.md

phase	plan	type	wave	depends_on	files_modified	autonomous	requirements	must_haves
02-ai-pipeline	03	execute	3	02-01 02-02	internal/api/handlers/intake.go internal/api/handlers/intake_test.go internal/api/router.go cmd/hwlab/main.go	true	AI-02 AI-03 AI-07	truths artifacts key_links POST /api/intake with 1-3 JPEG/PNG files returns 200 with serial, model, manufacturer, specs, category, tags, hw_id, catalog_status POST /api/intake with 0 or 4+ files returns 400 Quick add mode (confidence >= quick_add_threshold AND quick_add_enabled=true) creates NetBox record in one step; returns hw_id in response When NetBox is unreachable, intake enqueues netbox.create_device op to WAQ and returns 202 WAQ real handler (NewNetBoxOpHandler) replaces NoOpHandler in main.go path provides exports internal/api/handlers/intake.go POST /api/intake multipart handler IntakeHandler NewIntakeHandler path provides internal/api/handlers/intake_test.go Unit tests using MockAIClient and mock NetBox client path provides contains internal/api/router.go POST /api/intake route registered POST.*intake path provides contains cmd/hwlab/main.go NewNetBoxOpHandler wired as WAQ handler NewNetBoxOpHandler from to via pattern internal/api/handlers/intake.go internal/ai/orchestrator.go IntakeHandler.ServeHTTP calls orchestrator.Analyze with base64-encoded photos orchestrator.Analyze from to via pattern internal/api/handlers/intake.go internal/netbox/hwid.go AllocateNextHWID called after successful AI analysis AllocateNextHWID from to via pattern internal/api/handlers/intake.go internal/queue/handler.go WAQ.Enqueue called with OpNetBoxCreateDevice payload when NetBox unreachable OpNetBoxCreateDevice

Implement POST /api/intake: multipart photo upload → orchestrator → HW-ID allocation → NetBox create (or WAQ enqueue on failure) → tag sync → catalog status. Wire real WAQ handler in main.go. Add quick add mode.

Purpose: This is the core end-to-end intake flow — the primary value proposition of HWLab Phase 2. Output: Fully wired intake endpoint, updated router, updated main.go with real WAQ handler.

<execution_context> @$HOME/.claude/get-shit-done/workflows/execute-plan.md @$HOME/.claude/get-shit-done/templates/summary.md </execution_context>

@.planning/phases/02-ai-pipeline/02-CONTEXT.md @.planning/phases/02-ai-pipeline/02-01-SUMMARY.md @.planning/phases/02-ai-pipeline/02-02-SUMMARY.md

From internal/ai/orchestrator.go:

type Orchestrator struct{ /* ... */ }
func NewOrchestrator(tier1, tier2 AIClient, threshold float64) *Orchestrator
func (o *Orchestrator) Analyze(ctx context.Context, req IntakeRequest) (*IntakeResult, inventory.CatalogStatus, error)

From internal/ai/types.go:

type IntakeRequest struct { PhotosBase64 []string; JobID string }
type IntakeResult struct {
    SerialNumber, Model, Manufacturer, Category string
    Specs map[string]string; SuggestedTags []string
    AINotes string; Confidence float64; ConfidenceNote string
}
type AIConfig struct {
    Tier1, Tier2        TierConfig
    ConfidenceThreshold float64
    QuickAddEnabled     bool
    QuickAddThreshold   float64
}

From internal/netbox/client.go:

func (c *Client) CreateDevice(ctx, name, assetTag string, deviceTypeID, roleID, siteID int32) (int64, error)
func (c *Client) PatchCustomFields(ctx, deviceID int64, patch map[string]interface{}) error
func (c *Client) AllocateNextHWID(ctx) (string, error)
func (c *Client) SyncTags(ctx, tags []string) ([]netbox.TagRef, error)

From internal/netbox/custom_fields.go:

func BuildFullCustomFieldsPatch(cf CustomFields) map[string]interface{}

From internal/inventory/catalog_updater.go:

type CatalogUpdater struct{ /* wraps *netbox.Client */ }
func (u *CatalogUpdater) UpdateCatalogStatus(ctx, deviceID int64, current, next inventory.CatalogStatus) error

From internal/queue/handler.go:

const OpNetBoxCreateDevice     = "netbox.create_device"
const OpNetBoxPatchCustomFields = "netbox.patch_custom_fields"
type CreateDevicePayload struct {
    Name string; AssetTag string; DeviceTypeID, RoleID, SiteID int32
}
func NewNetBoxOpHandler(client NetBoxOpsClient) OpHandler

From internal/queue/waq.go:

func (q *WAQ) Enqueue(ctx, op PendingOp) error
func NewPendingOp(opType string, payload json.RawMessage) PendingOp

From internal/config/config.go:

type Config struct {
    // ... existing fields ...
    AI ai.AIConfig
}

NetBox device defaults for new items (use these IDs for Phase 2 — they must exist in the provisioned NetBox):

• DeviceTypeID: 1 (placeholder — "Generic Device" type must be provisioned in NetBox)
• RoleID: 1 (placeholder — "Inventory Item" role must be provisioned in NetBox)
• SiteID: 1 (placeholder — "Homelab" site provisioned in Phase 1)

Add to config.go defaults:

v.SetDefault("netbox_default_device_type_id", 1)
v.SetDefault("netbox_default_role_id", 1)
v.SetDefault("netbox_default_site_id", 1)

These become Config.NetBoxDefaultDeviceTypeID int32, etc.

Task 1: POST /api/intake handler with orchestrator and NetBox wiring internal/api/handlers/intake.go, internal/api/handlers/intake_test.go, internal/config/config.go - internal/api/handlers/health.go (full — understand handler pattern used in this project) - internal/api/handlers/health_test.go (full — understand test patterns) - internal/netbox/client.go (full — AllocateNextHWID, CreateDevice, PatchCustomFields, SyncTags) - internal/netbox/custom_fields.go (full — BuildFullCustomFieldsPatch) - internal/inventory/catalog_updater.go (full — UpdateCatalogStatus) - internal/config/config.go (full — to add new NetBox default ID fields) - internal/queue/waq.go (full — Enqueue, NewPendingOp) - internal/queue/handler.go (full — op type constants and payload structs) - Test: TestIntakeHandlerRejectsZeroPhotos — POST /api/intake with no files returns 400 - Test: TestIntakeHandlerRejectsFourPhotos — POST /api/intake with 4 files returns 400 - Test: TestIntakeHandlerHighConfidence — mock orchestrator returns HighConfidenceResult (0.95); mock NetBox CreateDevice succeeds; response is 201 JSON with fields: hw_id, model, manufacturer, category, catalog_status="indexed" - Test: TestIntakeHandlerLowConfidence — mock returns LowConfidenceResult (0.40); response is 201 with catalog_status="needs_research" - Test: TestIntakeHandlerQuickAdd — quick_add_enabled=true, quick_add_threshold=0.90, mock returns confidence 0.95; response is 201; CreateDevice called once (verify no review step) - Test: TestIntakeHandlerNetBoxDown — mock NetBox CreateDevice returns error; handler enqueues to WAQ; returns 202 with queued=true in JSON body **Extend internal/config/config.go** — add three fields to Config struct and defaults:

NetBoxDefaultDeviceTypeID int32 `mapstructure:"netbox_default_device_type_id"`
NetBoxDefaultRoleID       int32 `mapstructure:"netbox_default_role_id"`
NetBoxDefaultSiteID       int32 `mapstructure:"netbox_default_site_id"`

Defaults in Load():

v.SetDefault("netbox_default_device_type_id", 1)
v.SetDefault("netbox_default_role_id", 1)
v.SetDefault("netbox_default_site_id", 1)

Bindings:

_ = v.BindEnv("netbox_default_device_type_id", "HWLAB_NETBOX_DEFAULT_DEVICE_TYPE_ID")
_ = v.BindEnv("netbox_default_role_id", "HWLAB_NETBOX_DEFAULT_ROLE_ID")
_ = v.BindEnv("netbox_default_site_id", "HWLAB_NETBOX_DEFAULT_SITE_ID")

internal/api/handlers/intake.go — the intake handler:

Define interfaces for testability (handler does not import netbox.Client directly):

// intakeNetBoxClient is the subset of netbox.Client the intake handler needs.
type intakeNetBoxClient interface {
    AllocateNextHWID(ctx context.Context) (string, error)
    CreateDevice(ctx context.Context, name, assetTag string, deviceTypeID, roleID, siteID int32) (int64, error)
    PatchCustomFields(ctx context.Context, deviceID int64, patch map[string]interface{}) error
    SyncTags(ctx context.Context, tags []string) ([]netbox.TagRef, error)
}

// intakeCatalogUpdater is the subset needed for catalog status.
type intakeCatalogUpdater interface {
    UpdateCatalogStatus(ctx context.Context, deviceID int64, current, next inventory.CatalogStatus) error
}

// intakeWAQ is the subset of WAQ the handler needs.
type intakeWAQ interface {
    Enqueue(ctx context.Context, op queue.PendingOp) error
}

IntakeHandler struct:

type IntakeHandler struct {
    orchestrator    *ai.Orchestrator
    netbox          intakeNetBoxClient
    catalogUpdater  intakeCatalogUpdater
    waq             intakeWAQ  // may be nil if DragonFlyDB unavailable
    deviceTypeID    int32
    roleID          int32
    siteID          int32
    quickAddEnabled bool
    quickAddThresh  float64
}

func NewIntakeHandler(
    orch *ai.Orchestrator,
    nb intakeNetBoxClient,
    cu intakeCatalogUpdater,
    waq intakeWAQ,
    deviceTypeID, roleID, siteID int32,
    quickAddEnabled bool,
    quickAddThresh float64,
) *IntakeHandler {
    return &IntakeHandler{
        orchestrator:    orch,
        netbox:          nb,
        catalogUpdater:  cu,
        waq:             waq,
        deviceTypeID:    deviceTypeID,
        roleID:          roleID,
        siteID:          siteID,
        quickAddEnabled: quickAddEnabled,
        quickAddThresh:  quickAddThresh,
    }
}

IntakeResponse JSON struct:

type IntakeResponse struct {
    HWID          string            `json:"hw_id"`
    Model         string            `json:"model"`
    Manufacturer  string            `json:"manufacturer"`
    Category      string            `json:"category"`
    Specs         map[string]string `json:"specs"`
    SuggestedTags []string          `json:"suggested_tags"`
    AINotes       string            `json:"ai_notes"`
    Confidence    float64           `json:"confidence"`
    CatalogStatus string            `json:"catalog_status"`
    NetBoxID      int64             `json:"netbox_id,omitempty"`
    Queued        bool              `json:"queued,omitempty"` // true if NetBox was unreachable
}

ServeHTTP flow:

1. r.ParseMultipartForm(32 << 20) — 32MB max
2. Validate files count: len(files) == 0 → 400; len(files) > 3 → 400
3. Read each file, detect MIME (http.DetectContentType on first 512 bytes), base64-encode → photosBase64 slice
4. Generate jobID: uuid.New().String()
5. result, status, err := h.orchestrator.Analyze(r.Context(), ai.IntakeRequest{PhotosBase64: photosBase64, JobID: jobID})
6. If err != nil → 500
7. hwid, err := h.netbox.AllocateNextHWID(r.Context()); if err → 500
8. Decide name: result.Manufacturer + " " + result.Model (trim spaces; if empty use hwid)
9. Quick add check: if h.quickAddEnabled && result.Confidence >= h.quickAddThresh → attempt NetBox create
10. Non-quick-add path AND quick-add path both try CreateDevice; on error → enqueue to WAQ if available → 202
11. On successful CreateDevice: PatchCustomFields with BuildFullCustomFieldsPatch, SyncTags, UpdateCatalogStatus
12. Return 201 (or 202 if queued) JSON IntakeResponse

For BuildFullCustomFieldsPatch, construct a netbox.CustomFields from IntakeResult:

cf := netboxTypes.CustomFields{
    HWID:          hwid,
    CatalogStatus: string(status),
    AINotes:       result.AINotes,
}
patch := netbox.BuildFullCustomFieldsPatch(cf)

Note: Use encoding/json to marshal CreateDevicePayload for WAQ enqueue:

payload, _ := json.Marshal(queue.CreateDevicePayload{
    Name:         deviceName,
    AssetTag:     hwid,
    DeviceTypeID: h.deviceTypeID,
    RoleID:       h.roleID,
    SiteID:       h.siteID,
})
op := queue.NewPendingOp(queue.OpNetBoxCreateDevice, payload)
h.waq.Enqueue(r.Context(), op)

Return 202 with {"queued": true, "hw_id": hwid, ...} when NetBox was unreachable.

internal/api/handlers/intake_test.go — six tests using mock structs:

Define mock types in intake_test.go (unexported):

• mockOrchestrator with FixedResult *ai.IntakeResult, FixedStatus inventory.CatalogStatus — wraps with an Analyze method matching the expected signature
• mockNetBox with configurable return values for AllocateNextHWID, CreateDevice, PatchCustomFields, SyncTags
• mockCatalogUpdater
• mockWAQ that records enqueued ops

Use net/http/httptest to create a recorder, call handler.ServeHTTP(rec, req).

For multipart body construction in tests:

var body bytes.Buffer
w := multipart.NewWriter(&body)
fw, _ := w.CreateFormFile("photos", "test.jpg")
fw.Write([]byte{0xff, 0xd8, 0xff}) // minimal JPEG header
w.Close()
req := httptest.NewRequest(http.MethodPost, "/api/intake", &body)
req.Header.Set("Content-Type", w.FormDataContentType())

NOTE: The handler receives *ai.Orchestrator but for tests you need to pass a mock orchestrator. Refactor IntakeHandler to accept an orchestratorFunc or define an IntakeOrchestrator interface with Analyze(ctx, req) (*IntakeResult, CatalogStatus, error) — use the interface instead of the concrete type. This decouples tests cleanly.

Update IntakeHandler.orchestrator field to use an interface:

type intakeOrchestrator interface {
    Analyze(ctx context.Context, req ai.IntakeRequest) (*ai.IntakeResult, inventory.CatalogStatus, error)
}

The concrete *ai.Orchestrator satisfies this interface automatically. cd /home/mikkel/homelabby && go build ./... && go test ./internal/api/... ./internal/config/... -v 2>&1 | tail -40 - All 6 TestIntakeHandler* tests pass - go build ./... clean - Config struct has NetBoxDefaultDeviceTypeID, NetBoxDefaultRoleID, NetBoxDefaultSiteID - internal/api/handlers/intake.go exists with NewIntakeHandler and ServeHTTP

Task 2: Wire intake route and real WAQ handler in router and main.go internal/api/router.go, cmd/hwlab/main.go - internal/api/router.go (full — add POST /api/intake route) - cmd/hwlab/main.go (full — wire intake handler and swap NoOpHandler for NetBoxOpHandler) - internal/api/handlers/intake.go (skim — NewIntakeHandler signature) - internal/queue/handler.go (skim — NewNetBoxOpHandler signature) - internal/netbox/client.go (skim — *Client satisfies intakeNetBoxClient and NetBoxOpsClient interfaces) - internal/inventory/catalog_updater.go (skim — NewCatalogUpdater if it exists, or construct CatalogUpdater directly) **internal/api/router.go** — add POST /api/intake:

Update NewRouter signature to accept an http.Handler for the intake endpoint:

func NewRouter(staticFiles fs.FS, intakeHandler http.Handler) http.Handler {
    r := chi.NewRouter()
    // ... existing middleware ...
    r.Route("/api", func(r chi.Router) {
        r.Get("/health", handlers.Health)
        r.Post("/intake", intakeHandler.ServeHTTP)
    })
    // ... existing SPA handler ...
}

cmd/hwlab/main.go — wire everything:

1. Load config (existing)
2. Create NetBox client: nbClient, err := netbox.NewClient(cfg.NetBoxURL, cfg.NetBoxToken); if err → log.Fatalf
3. Create AI tier clients:

tier1 := ai.NewTierClient(cfg.AI.Tier1)
tier2 := ai.NewTierClient(cfg.AI.Tier2)
orch  := ai.NewOrchestrator(tier1, tier2, cfg.AI.ConfidenceThreshold)

4. Create catalog updater:

catalogUpdater := &inventory.CatalogUpdater{} // or however it's constructed from Phase 1

Read internal/inventory/catalog_updater.go to check its exact constructor/struct literal.

5. Create intake handler:

intakeHandler := handlers.NewIntakeHandler(
    orch,
    nbClient,
    catalogUpdater,
    waq,   // may be nil — handler must handle nil waq gracefully
    cfg.NetBoxDefaultDeviceTypeID,
    cfg.NetBoxDefaultRoleID,
    cfg.NetBoxDefaultSiteID,
    cfg.AI.QuickAddEnabled,
    cfg.AI.QuickAddThreshold,
)

6. Swap NoOpHandler for real WAQ handler:

// Replace: go waq.RunWorker(ctx, queue.NoOpHandler, ...)
// With:
nbHandler := queue.NewNetBoxOpHandler(nbClient)
go waq.RunWorker(ctx, nbHandler, cfg.WAQMaxAttempts, retryInterval)

7. Pass intakeHandler to router:

router := api.NewRouter(staticFiles, intakeHandler)

Handle the case where waq is nil: IntakeHandler.waq is an interface — if waq init failed (non-fatal), pass nil. In ServeHTTP, check if h.waq != nil before calling Enqueue. If waq is nil and NetBox is down → return 503 (service unavailable, cannot queue).

Also handle nil waq in main.go WAQ worker section:

var nbHandler queue.OpHandler
if waq != nil {
    nbHandler = queue.NewNetBoxOpHandler(nbClient)
    go waq.RunWorker(ctx, nbHandler, cfg.WAQMaxAttempts, retryInterval)
}

NOTE: catalog_updater.go from Phase 1 — read it to find the correct constructor. The struct is:

type CatalogUpdater struct {
    client *netbox.Client
}

Construct with struct literal: &inventory.CatalogUpdater{...} or if it has a constructor inventory.NewCatalogUpdater(nbClient). Check the actual file.

After editing, run go build ./... to confirm compilation. Fix any import cycle or interface mismatch errors before finishing. cd /home/mikkel/homelabby && go build ./... 2>&1 && echo "BUILD OK" && curl -s http://localhost:8080/api/health 2>/dev/null || echo "(server not running — build check only)" - go build ./... passes with zero errors - router.go has r.Post("/intake", ...) route - main.go uses NewNetBoxOpHandler (grep confirms; NoOpHandler no longer referenced in main.go) - main.go creates netbox.Client, ai.Orchestrator, and handlers.IntakeHandler

<threat_model>

Trust Boundaries

Boundary	Description
HTTP client → /api/intake	Untrusted multipart file upload from browser or curl
intake handler → oMLX	Base64 image data sent to local AI — image content is untrusted
intake handler → NetBox	Structured data written to source of truth

STRIDE Threat Register

Threat ID	Category	Component	Disposition	Mitigation Plan
T-02-09	Denial of Service	multipart upload size	mitigate	r.ParseMultipartForm(32 << 20) — 32MB hard cap on request body
T-02-10	Tampering	AI-extracted device name	mitigate	device name passed directly to CreateDevice — net/http handler sanitizes via Go string (no SQL injection possible; go-netbox marshals to JSON)
T-02-11	Tampering	AI-extracted tags	accept	Tags pass through normalizeTags in SyncTags (Phase 1 T-04-02 mitigation) — slug normalization strips injection characters
T-02-12	Denial of Service	photo count bypass	mitigate	Explicit len check 1-3 before any processing; 0 or 4+ returns 400 immediately
T-02-13	Spoofing	intake response hw_id	accept	HW-ID assigned by AllocateNextHWID, not caller-controlled; sequential allocation cannot be spoofed via this endpoint
</threat_model>

After plan completion: 1. `go build ./...` — zero errors 2. `go test ./internal/api/... -v` — all intake handler tests pass, health tests still pass 3. `grep "Post.*intake" internal/api/router.go` — route present 4. `grep "NewNetBoxOpHandler" cmd/hwlab/main.go` — real handler wired 5. `grep "NoOpHandler" cmd/hwlab/main.go` — NOT present (replaced) 6. `go test ./...` — zero failures (all packages)

<success_criteria>

• POST /api/intake registered and reachable
• Handler validates 1-3 photos, returns 400 on violations
• Mock-based unit tests cover high confidence, low confidence, quick add, and NetBox-down scenarios
• WAQ real handler (NewNetBoxOpHandler) used in main.go — NoOpHandler no longer in main.go
• go build ./... clean with zero errors </success_criteria>

After completion, create `.planning/phases/02-ai-pipeline/02-03-SUMMARY.md`