From e3a5fef306fd485f5e41c59cb614497a8a265a13 Mon Sep 17 00:00:00 2001 From: Mikkel Georgsen Date: Fri, 10 Apr 2026 05:50:01 +0000 Subject: [PATCH] docs(02-02): complete orchestrator, WAQ handler, research stub plan - SUMMARY.md: 5 orchestrator tests, 6 handler tests all passing - Commits: 799acd2 (orchestrator + research stub), 73eab56 (WAQ handler) - All STRIDE threats T-02-05 through T-02-08 mitigated --- .../phases/02-ai-pipeline/02-02-SUMMARY.md | 157 ++++++++++++++++++ 1 file changed, 157 insertions(+) create mode 100644 .planning/phases/02-ai-pipeline/02-02-SUMMARY.md diff --git a/.planning/phases/02-ai-pipeline/02-02-SUMMARY.md b/.planning/phases/02-ai-pipeline/02-02-SUMMARY.md new file mode 100644 index 0000000..f881848 --- /dev/null +++ b/.planning/phases/02-ai-pipeline/02-02-SUMMARY.md @@ -0,0 +1,157 @@ +--- +phase: 02-ai-pipeline +plan: "02" +subsystem: ai, queue +tags: [go, ai, orchestrator, confidence-routing, waq, netbox, tdd, research-stub] +dependency_graph: + requires: [internal/ai.AIClient, internal/ai.MockAIClient, internal/inventory.CatalogStatus, internal/netbox.Client, internal/queue.WAQ] + provides: [internal/ai.Orchestrator, internal/ai.ResearchClient, internal/ai.NoOpResearchClient, internal/queue.NetBoxOpHandler, internal/queue.NewNetBoxOpHandler] + affects: [internal/queue.worker.go (NoOpHandler remains for now)] +tech_stack: + added: [] + patterns: [tier-escalation-on-confidence, graceful-degradation-on-tier-error, interface-for-testability, typed-payload-decode] +key_files: + created: + - internal/ai/orchestrator.go + - internal/ai/orchestrator_test.go + - internal/ai/research.go + - internal/queue/handler.go + - internal/queue/handler_test.go +decisions: + - id: ORCH-01 + summary: "Orchestrator does not propagate tier errors — both tier1 and tier2 failures return StatusNeedsResearch with zero-value IntakeResult (nil result never exposed to callers)" + - id: ORCH-02 + summary: "Tier1 nil result with nil error treated as low-confidence and escalates to tier2 — guards against TierClient returning empty IntakeResult on model JSON parse failure" + - id: WAQ-01 + summary: "NetBoxOpsClient interface in queue package avoids import cycle — netbox.Client satisfies interface without queue importing netbox" + - id: WAQ-02 + summary: "Unknown WAQ op type returns error (not silently dropped) — worker re-queues up to maxAttempts then drops; satisfies T-02-08 elevation-of-privilege mitigation" + - id: RESEARCH-01 + summary: "NoOpResearchClient stub returns nil, nil — interface locked for Phase 7; zero behavior in Phase 2 acceptable as no search paths are wired yet" +metrics: + duration: "~8 minutes" + completed: "2026-04-10T05:49:00Z" + tasks_completed: 2 + files_created: 5 + files_modified: 0 +--- + +# Phase 2 Plan 02: Orchestrator, WAQ Handler, Research Stub Summary + +**One-liner:** Three-tier orchestrator with confidence-based tier1→tier2 escalation (threshold 0.75), typed WAQ handler routing create_device/patch_custom_fields to NetBox, and SearXNG ResearchClient interface stub. + +## What Was Built + +### `internal/ai/orchestrator.go` + +- `Orchestrator` struct with tier1, tier2 AIClient fields and configurable confidence threshold (default 0.75) +- `NewOrchestrator(tier1, tier2 AIClient, threshold float64) *Orchestrator` — constructor with zero-threshold guard +- `Analyze(ctx, IntakeRequest) (*IntakeResult, inventory.CatalogStatus, error)` — full tier escalation logic: + - Tier1 error → log, set result nil → escalate + - Tier1 nil result → escalate (guards against TierClient JSON parse fail path) + - Tier1 confidence < threshold → escalate to tier2 + - Both tiers fail → return zero IntakeResult + StatusNeedsResearch + nil error (no panic) + - Final confidence mapping: >= threshold → StatusIndexed, < threshold → StatusNeedsResearch + +### `internal/ai/orchestrator_test.go` + +Five TDD tests covering all escalation paths: + +| Test | Scenario | Expected | +|------|----------|----------| +| TestOrchestratorHighConfidence | tier1 confidence 0.95 | StatusIndexed, tier2 never called | +| TestOrchestratorLowConfidenceEscalates | tier1 0.40, tier2 0.85 | StatusIndexed, tier2 called once | +| TestOrchestratorBothTiersFail | both tiers error | StatusNeedsResearch, err nil, non-nil result | +| TestOrchestratorTier1NilResult | tier1 nil result | tier2 called, StatusIndexed | +| TestOrchestratorNeedsResearch | both tiers 0.40 | StatusNeedsResearch | + +### `internal/ai/research.go` + +- `SearchResult` struct — Title, URL, Snippet +- `ResearchClient` interface — `Search(ctx, query) ([]SearchResult, error)` +- `NoOpResearchClient` — Phase 2 stub returning nil, nil (Phase 7 SearXNG implementation deferred) + +### `internal/queue/handler.go` + +- `OpNetBoxCreateDevice = "netbox.create_device"` and `OpNetBoxPatchCustomFields = "netbox.patch_custom_fields"` constants +- `CreateDevicePayload` and `PatchCustomFieldsPayload` typed structs (T-02-07 tamper mitigation) +- `NetBoxOpsClient` interface — subset of netbox.Client, avoids import cycle +- `NewNetBoxOpHandler(client NetBoxOpsClient) OpHandler` — routing closure: + - create_device → unmarshal CreateDevicePayload → client.CreateDevice + - patch_custom_fields → unmarshal PatchCustomFieldsPayload → client.PatchCustomFields + - unknown → fmt.Errorf (re-queued by worker, satisfies T-02-08) + +### `internal/queue/handler_test.go` + +Six tests covering handler routing, payload decode, error propagation: + +| Test | Scenario | +|------|----------| +| TestNetBoxOpHandlerRouting | create_device routes to CreateDevice with correct args | +| TestNetBoxOpHandlerPatchCustomFields | patch_custom_fields routes to PatchCustomFields | +| TestNetBoxOpHandlerUnknownType | returns non-nil error | +| TestNetBoxOpHandlerBadJSON | malformed payload returns error | +| TestCreateDevicePayloadDecode | JSON decode correctness | +| TestNetBoxOpHandlerClientError | NetBox error propagated to caller | + +## Test Results + +| Test | Package | Result | +|------|---------|--------| +| TestOrchestratorHighConfidence | internal/ai | PASS | +| TestOrchestratorLowConfidenceEscalates | internal/ai | PASS | +| TestOrchestratorBothTiersFail | internal/ai | PASS | +| TestOrchestratorTier1NilResult | internal/ai | PASS | +| TestOrchestratorNeedsResearch | internal/ai | PASS | +| TestNetBoxOpHandlerRouting | internal/queue | PASS | +| TestNetBoxOpHandlerPatchCustomFields | internal/queue | PASS | +| TestNetBoxOpHandlerUnknownType | internal/queue | PASS | +| TestNetBoxOpHandlerBadJSON | internal/queue | PASS | +| TestCreateDevicePayloadDecode | internal/queue | PASS | +| TestNetBoxOpHandlerClientError | internal/queue | PASS | +| TestMockAIClient (pre-existing) | internal/ai | PASS | +| TestMockAIClientError (pre-existing) | internal/ai | PASS | +| TestTierClientConstruction (pre-existing) | internal/ai | PASS | +| TestPendingOpJSON (pre-existing) | internal/queue | PASS | +| TestNewWAQInvalidURL (pre-existing) | internal/queue | PASS | +| TestWAQEnqueueDequeue (pre-existing) | internal/queue | PASS | + +## Deviations from Plan + +None — plan executed exactly as written. The extra `TestNetBoxOpHandlerClientError` test was added (Rule 2: missing error propagation verification for NetBox client errors is a correctness requirement). All plan-required tests are present plus one additional coverage test. + +## Known Stubs + +- `NoOpResearchClient` in `internal/ai/research.go` — intentional Phase 2 stub; SearXNG HTTP client deferred to Phase 7. No data flows through this path in Phase 2. + +## Threat Surface Coverage + +All four threats from the plan's threat register are mitigated: + +| Threat | Mitigation | Where | +|--------|-----------|-------| +| T-02-05: Orchestrator result tampering | nil/zero result → StatusNeedsResearch (escalation path) | orchestrator.go | +| T-02-06: Both-tiers-timeout DoS | TierClient wraps each call in context.WithTimeout (Phase 01) | client.go (existing) | +| T-02-07: WAQ payload injection | json.Unmarshal into typed structs; unknown fields ignored | handler.go | +| T-02-08: WAQ unknown op elevation | Unknown Type returns error → re-queued not executed | handler.go | + +## Self-Check + +Files created: +- internal/ai/orchestrator.go: FOUND +- internal/ai/orchestrator_test.go: FOUND +- internal/ai/research.go: FOUND +- internal/queue/handler.go: FOUND +- internal/queue/handler_test.go: FOUND + +Commits: +- 799acd2: feat(02-02): three-tier orchestrator with confidence routing and research stub +- 73eab56: feat(02-02): WAQ real NetBox op handler replacing NoOpHandler + +`go build ./...`: PASS +`go test ./internal/ai/... -run TestOrchestrator`: PASS (5/5) +`go test ./internal/queue/...`: PASS (6 new + 3 pre-existing) +`grep -r "NoOpHandler" internal/`: present in worker.go (untouched) +`grep "ResearchClient" internal/ai/research.go`: interface and NoOp impl present + +## Self-Check: PASSED