Compare commits
10 commits
16a469bfdd
...
9f59d38d49
| Author | SHA1 | Date | |
|---|---|---|---|
| 9f59d38d49 | |||
| ca0f0e351b | |||
| 712a0a39b8 | |||
| 7db093c696 | |||
| 9db7707a64 | |||
| cbe411996f | |||
| 0072aa41bd | |||
| 30cd279f49 | |||
| 34e0803661 | |||
| 987dc4b97c |
24 changed files with 2107 additions and 71 deletions
|
|
@ -18,7 +18,7 @@ Decimal phases appear between their surrounding integers in numeric order.
|
||||||
- [x] **Phase 4: USB Manager & Label Printing** - USB hardware characterization (2026-04-13), goroutine-per-device manager, QR label printing (completed 2026-04-10)
|
- [x] **Phase 4: USB Manager & Label Printing** - USB hardware characterization (2026-04-13), goroutine-per-device manager, QR label printing (completed 2026-04-10)
|
||||||
- [x] **Phase 5: Cable Test Integration** - Treedix USB/DP/HDMI testers, FNIRSI FNB58, cable test workflow UI (completed 2026-04-10)
|
- [x] **Phase 5: Cable Test Integration** - Treedix USB/DP/HDMI testers, FNIRSI FNB58, cable test workflow UI (completed 2026-04-10)
|
||||||
- [x] **Phase 6: Lab Advisor** - Claude Opus chat interface, NetBox context assembly, streaming SSE, chat history (completed 2026-04-10)
|
- [x] **Phase 6: Lab Advisor** - Claude Opus chat interface, NetBox context assembly, streaming SSE, chat history (completed 2026-04-10)
|
||||||
- [ ] **Phase 7: Research Agent & Search** - SearXNG Tier 2 research agent, natural language inventory search, quality gate automation
|
- [x] **Phase 7: Research Agent & Search** - SearXNG Tier 2 research agent, natural language inventory search, quality gate automation (completed 2026-04-10)
|
||||||
|
|
||||||
## Phase Details
|
## Phase Details
|
||||||
|
|
||||||
|
|
@ -139,7 +139,11 @@ Plans:
|
||||||
3. SearXNG queries are sanitized before dispatch — no raw AI output reaches the search engine
|
3. SearXNG queries are sanitized before dispatch — no raw AI output reaches the search engine
|
||||||
|
|
||||||
**Note:** AI-04 is the sole unmapped requirement from Phase 2 that belongs here — it requires the full orchestrator, SearXNG client, and NetBox inventory all in place. The other Phase 2 requirements cover Tier 1 intake; this requirement covers the Tier 2 research loop.
|
**Note:** AI-04 is the sole unmapped requirement from Phase 2 that belongs here — it requires the full orchestrator, SearXNG client, and NetBox inventory all in place. The other Phase 2 requirements cover Tier 1 intake; this requirement covers the Tier 2 research loop.
|
||||||
**Plans**: TBD
|
**Plans**: 2 plans
|
||||||
|
|
||||||
|
Plans:
|
||||||
|
- [x] 07-01-PLAN.md — SearXNG client, ResearchAgent worker, POST /api/research/trigger
|
||||||
|
- [x] 07-02-PLAN.md — NL search endpoint GET /api/search, dashboard search bar
|
||||||
|
|
||||||
## Progress
|
## Progress
|
||||||
|
|
||||||
|
|
@ -154,4 +158,4 @@ Phases execute in numeric order: 1 → 2 → 3 → 4 → 5 → 6 → 7
|
||||||
| 4. USB Manager & Label Printing | 5/5 | Complete | 2026-04-10 |
|
| 4. USB Manager & Label Printing | 5/5 | Complete | 2026-04-10 |
|
||||||
| 5. Cable Test Integration | 3/3 | Complete | 2026-04-10 |
|
| 5. Cable Test Integration | 3/3 | Complete | 2026-04-10 |
|
||||||
| 6. Lab Advisor | 3/3 | Complete | 2026-04-10 |
|
| 6. Lab Advisor | 3/3 | Complete | 2026-04-10 |
|
||||||
| 7. Research Agent & Search | 0/TBD | Not started | - |
|
| 7. Research Agent & Search | 2/2 | Complete | 2026-04-10 |
|
||||||
|
|
|
||||||
|
|
@ -2,15 +2,15 @@
|
||||||
gsd_state_version: 1.0
|
gsd_state_version: 1.0
|
||||||
milestone: v1.0
|
milestone: v1.0
|
||||||
milestone_name: milestone
|
milestone_name: milestone
|
||||||
status: planning
|
status: executing
|
||||||
stopped_at: Roadmap created — ready to run /gsd-plan-phase 1
|
stopped_at: Roadmap created — ready to run /gsd-plan-phase 1
|
||||||
last_updated: "2026-04-10T07:41:49.825Z"
|
last_updated: "2026-04-10T07:57:47.510Z"
|
||||||
last_activity: 2026-04-10
|
last_activity: 2026-04-10
|
||||||
progress:
|
progress:
|
||||||
total_phases: 7
|
total_phases: 7
|
||||||
completed_phases: 6
|
completed_phases: 7
|
||||||
total_plans: 25
|
total_plans: 27
|
||||||
completed_plans: 25
|
completed_plans: 27
|
||||||
percent: 100
|
percent: 100
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
@ -27,7 +27,7 @@ See: .planning/PROJECT.md (updated 2026-04-09)
|
||||||
|
|
||||||
Phase: 7 of 7 (research agent & search)
|
Phase: 7 of 7 (research agent & search)
|
||||||
Plan: Not started
|
Plan: Not started
|
||||||
Status: Ready to plan
|
Status: Ready to execute
|
||||||
Last activity: 2026-04-10
|
Last activity: 2026-04-10
|
||||||
|
|
||||||
Progress: [░░░░░░░░░░] 0%
|
Progress: [░░░░░░░░░░] 0%
|
||||||
|
|
@ -36,7 +36,7 @@ Progress: [░░░░░░░░░░] 0%
|
||||||
|
|
||||||
**Velocity:**
|
**Velocity:**
|
||||||
|
|
||||||
- Total plans completed: 25
|
- Total plans completed: 27
|
||||||
- Average duration: —
|
- Average duration: —
|
||||||
- Total execution time: 0 hours
|
- Total execution time: 0 hours
|
||||||
|
|
||||||
|
|
@ -50,6 +50,7 @@ Progress: [░░░░░░░░░░] 0%
|
||||||
| 4 | 5 | - | - |
|
| 4 | 5 | - | - |
|
||||||
| 5 | 3 | - | - |
|
| 5 | 3 | - | - |
|
||||||
| 6 | 3 | - | - |
|
| 6 | 3 | - | - |
|
||||||
|
| 7 | 2 | - | - |
|
||||||
|
|
||||||
**Recent Trend:**
|
**Recent Trend:**
|
||||||
|
|
||||||
|
|
|
||||||
319
.planning/phases/07-research-agent-search/07-01-PLAN.md
Normal file
319
.planning/phases/07-research-agent-search/07-01-PLAN.md
Normal file
|
|
@ -0,0 +1,319 @@
|
||||||
|
---
|
||||||
|
phase: 07-research-agent-search
|
||||||
|
plan: 01
|
||||||
|
type: execute
|
||||||
|
wave: 1
|
||||||
|
depends_on: []
|
||||||
|
files_modified:
|
||||||
|
- internal/config/config.go
|
||||||
|
- internal/netbox/client.go
|
||||||
|
- internal/research/searxng.go
|
||||||
|
- internal/research/agent.go
|
||||||
|
- internal/ai/research.go
|
||||||
|
- cmd/hwlab/main.go
|
||||||
|
autonomous: true
|
||||||
|
requirements:
|
||||||
|
- AI-04
|
||||||
|
|
||||||
|
must_haves:
|
||||||
|
truths:
|
||||||
|
- "A SearXNG HTTP GET to http://10.5.0.129:8080/search?q=...&format=json returns parsed results"
|
||||||
|
- "Items with catalog_status=needs_research are polled from NetBox every 10 minutes"
|
||||||
|
- "Each needs_research item is enriched by SearXNG + Tier 2 LLM and updated to catalog_status=researched in NetBox"
|
||||||
|
- "POST /api/research/trigger fires an immediate research cycle (does not wait for the 10-min ticker)"
|
||||||
|
artifacts:
|
||||||
|
- path: "internal/research/searxng.go"
|
||||||
|
provides: "SearXNGClient implementing ai.ResearchClient"
|
||||||
|
exports: ["SearXNGClient", "NewSearXNGClient"]
|
||||||
|
- path: "internal/research/agent.go"
|
||||||
|
provides: "ResearchAgent background goroutine"
|
||||||
|
exports: ["Agent", "NewAgent", "RunOnce", "Start"]
|
||||||
|
key_links:
|
||||||
|
- from: "internal/research/searxng.go"
|
||||||
|
to: "http://10.5.0.129:8080/search"
|
||||||
|
via: "net/http GET with q and format=json query params"
|
||||||
|
pattern: "http\\.Get.*search.*format=json"
|
||||||
|
- from: "internal/research/agent.go"
|
||||||
|
to: "internal/netbox/client.go"
|
||||||
|
via: "ListDevicesWithStatus(ctx, \"needs_research\")"
|
||||||
|
pattern: "ListDevicesWithStatus"
|
||||||
|
- from: "internal/research/agent.go"
|
||||||
|
to: "internal/ai/client.go"
|
||||||
|
via: "tier2.AnalyzePhotos (text-only prompt, no photos)"
|
||||||
|
pattern: "AnalyzePhotos"
|
||||||
|
---
|
||||||
|
|
||||||
|
<objective>
|
||||||
|
Build the real SearXNG research client and the ResearchAgent background worker that
|
||||||
|
closes the AI-04 research loop: items at needs_research are enriched automatically.
|
||||||
|
|
||||||
|
Purpose: Replace the Phase 2 NoOpResearchClient stub and deliver the automated
|
||||||
|
enrichment cycle that advances items from needs_research to researched in NetBox.
|
||||||
|
|
||||||
|
Output:
|
||||||
|
- internal/research/searxng.go — real HTTP client implementing ai.ResearchClient
|
||||||
|
- internal/research/agent.go — background worker with ticker + on-demand trigger
|
||||||
|
- Config additions for SearXNG URL
|
||||||
|
- main.go goroutine start + POST /api/research/trigger handler
|
||||||
|
</objective>
|
||||||
|
|
||||||
|
<execution_context>
|
||||||
|
@/home/mikkel/.claude/get-shit-done/workflows/execute-plan.md
|
||||||
|
@/home/mikkel/.claude/get-shit-done/templates/summary.md
|
||||||
|
</execution_context>
|
||||||
|
|
||||||
|
<context>
|
||||||
|
@.planning/PROJECT.md
|
||||||
|
@.planning/ROADMAP.md
|
||||||
|
@.planning/STATE.md
|
||||||
|
|
||||||
|
@internal/ai/research.go
|
||||||
|
@internal/ai/client.go
|
||||||
|
@internal/ai/orchestrator.go
|
||||||
|
@internal/netbox/client.go
|
||||||
|
@internal/netbox/custom_fields.go
|
||||||
|
@internal/netbox/types.go
|
||||||
|
@internal/inventory/catalog_updater.go
|
||||||
|
@internal/config/config.go
|
||||||
|
@cmd/hwlab/main.go
|
||||||
|
@internal/api/router.go
|
||||||
|
|
||||||
|
<interfaces>
|
||||||
|
<!-- Key contracts the executor needs. Extracted from codebase. -->
|
||||||
|
|
||||||
|
From internal/ai/research.go:
|
||||||
|
```go
|
||||||
|
type SearchResult struct {
|
||||||
|
Title string
|
||||||
|
URL string
|
||||||
|
Snippet string
|
||||||
|
}
|
||||||
|
|
||||||
|
type ResearchClient interface {
|
||||||
|
Search(ctx context.Context, query string) ([]SearchResult, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
type NoOpResearchClient struct{}
|
||||||
|
// Replace this with SearXNGClient in this plan.
|
||||||
|
```
|
||||||
|
|
||||||
|
From internal/ai/client.go:
|
||||||
|
```go
|
||||||
|
type AIClient interface {
|
||||||
|
AnalyzePhotos(ctx context.Context, req IntakeRequest) (*IntakeResult, error)
|
||||||
|
}
|
||||||
|
// IntakeRequest.PhotosBase64 may be empty — the Tier 2 model accepts text-only
|
||||||
|
// if the prompt is placed in a separate system message; use a text-only prompt
|
||||||
|
// for research enrichment (no photos).
|
||||||
|
```
|
||||||
|
|
||||||
|
From internal/netbox/client.go (method to ADD):
|
||||||
|
```go
|
||||||
|
// ListDevicesWithStatus returns devices whose catalog_status custom field equals status.
|
||||||
|
// Use status="needs_research" to find items needing enrichment.
|
||||||
|
func (c *Client) ListDevicesWithStatus(ctx context.Context, status string) ([]Device, error)
|
||||||
|
```
|
||||||
|
|
||||||
|
From internal/inventory/catalog_updater.go:
|
||||||
|
```go
|
||||||
|
func (u *CatalogUpdater) UpdateCatalogStatus(ctx context.Context, deviceID int64, current, next CatalogStatus) (CatalogStatus, error)
|
||||||
|
```
|
||||||
|
|
||||||
|
From internal/inventory (quality_gate.go constants):
|
||||||
|
```go
|
||||||
|
const StatusNeedsResearch CatalogStatus = "needs_research"
|
||||||
|
const StatusResearched CatalogStatus = "researched"
|
||||||
|
```
|
||||||
|
|
||||||
|
From internal/config/config.go (field to ADD):
|
||||||
|
```go
|
||||||
|
SearXNGURL string `mapstructure:"searxng_url"`
|
||||||
|
// default: "http://10.5.0.129:8080"
|
||||||
|
// env: HWLAB_SEARXNG_URL
|
||||||
|
```
|
||||||
|
</interfaces>
|
||||||
|
</context>
|
||||||
|
|
||||||
|
<tasks>
|
||||||
|
|
||||||
|
<task type="auto" tdd="true">
|
||||||
|
<name>Task 1: SearXNG client + netbox.ListDevicesWithStatus</name>
|
||||||
|
<files>
|
||||||
|
internal/research/searxng.go,
|
||||||
|
internal/research/searxng_test.go,
|
||||||
|
internal/netbox/client.go,
|
||||||
|
internal/config/config.go
|
||||||
|
</files>
|
||||||
|
<behavior>
|
||||||
|
- SearXNGClient.Search(ctx, "Intel NIC i350") sends GET http://10.5.0.129:8080/search?q=Intel+NIC+i350&format=json
|
||||||
|
- HTTP 200 with JSON body {"results":[{"title":"...","url":"...","content":"..."},...]} parses into []ai.SearchResult (map content->Snippet)
|
||||||
|
- HTTP non-200 returns error with status code
|
||||||
|
- Empty results array returns empty slice, no error
|
||||||
|
- Query is URL-encoded (url.QueryEscape or url.Values)
|
||||||
|
- ListDevicesWithStatus filters via custom_fields cf_catalog_status in go-netbox list call; falls back to client-side filter if API param unavailable
|
||||||
|
- ListDevicesWithStatus("needs_research") returns only devices with that catalog_status
|
||||||
|
</behavior>
|
||||||
|
<action>
|
||||||
|
Create package internal/research.
|
||||||
|
|
||||||
|
internal/research/searxng.go:
|
||||||
|
- Struct SearXNGClient with baseURL string and httpClient *http.Client (timeout 15s)
|
||||||
|
- NewSearXNGClient(baseURL string) *SearXNGClient — if baseURL empty, use "http://10.5.0.129:8080"
|
||||||
|
- Implements ai.ResearchClient interface
|
||||||
|
- Search method: build GET {baseURL}/search?q={url-encoded query}&format=json, execute, decode JSON
|
||||||
|
- SearXNG JSON response shape: {"results":[{"title":"","url":"","content":""},...]}
|
||||||
|
Map content field to SearchResult.Snippet (SearXNG uses "content" not "snippet")
|
||||||
|
- Return ([]ai.SearchResult, error). Never panic on empty results.
|
||||||
|
|
||||||
|
internal/research/searxng_test.go:
|
||||||
|
- Use httptest.NewServer to mock SearXNG responses
|
||||||
|
- Test: valid response parses correctly (2 results)
|
||||||
|
- Test: HTTP 500 returns error
|
||||||
|
- Test: empty results returns empty slice
|
||||||
|
|
||||||
|
internal/netbox/client.go — add ListDevicesWithStatus:
|
||||||
|
- List all devices (up to 200), filter client-side where CustomFields.CatalogStatus == status
|
||||||
|
- (go-netbox v4 custom field filtering via query param is schema-dependent; client-side is safer)
|
||||||
|
|
||||||
|
internal/config/config.go — add SearXNGURL:
|
||||||
|
- Field: SearXNGURL string `mapstructure:"searxng_url"`
|
||||||
|
- Default: v.SetDefault("searxng_url", "http://10.5.0.129:8080")
|
||||||
|
- Env binding: v.BindEnv("searxng_url", "HWLAB_SEARXNG_URL")
|
||||||
|
</action>
|
||||||
|
<verify>
|
||||||
|
<automated>cd /home/mikkel/homelabby && go test ./internal/research/... ./internal/config/... -v -count=1 -run TestSearXNG 2>&1 | tail -20</automated>
|
||||||
|
</verify>
|
||||||
|
<done>
|
||||||
|
SearXNGClient implements ai.ResearchClient. Tests pass with httptest mock server.
|
||||||
|
ListDevicesWithStatus added to netbox.Client. Config loads SearXNGURL with default.
|
||||||
|
</done>
|
||||||
|
</task>
|
||||||
|
|
||||||
|
<task type="auto" tdd="true">
|
||||||
|
<name>Task 2: ResearchAgent worker + main.go wiring + trigger endpoint</name>
|
||||||
|
<files>
|
||||||
|
internal/research/agent.go,
|
||||||
|
internal/research/agent_test.go,
|
||||||
|
internal/api/handlers/research.go,
|
||||||
|
internal/api/router.go,
|
||||||
|
cmd/hwlab/main.go
|
||||||
|
</files>
|
||||||
|
<behavior>
|
||||||
|
- Agent.RunOnce(ctx) polls NetBox for needs_research items, for each: builds a text-only search query from item Name, calls SearXNGClient.Search, sends results to Tier 2 LLM with a research prompt, patches NetBox custom fields (ai_notes, product_url from first result URL), transitions status to researched via CatalogUpdater
|
||||||
|
- Agent.Start(ctx, interval) runs RunOnce on ticker; logs "research agent: cycle complete, enriched N items"
|
||||||
|
- If SearXNG returns 0 results for an item, log warning and skip (do not change status)
|
||||||
|
- Tier 2 LLM research prompt: "You are enriching a hardware inventory record. Item: {name}. Search results: {formatted snippets}. Return JSON: {\"ai_notes\": \"...\", \"product_url\": \"...\"}"
|
||||||
|
- POST /api/research/trigger responds 202 Accepted and fires RunOnce in a goroutine (non-blocking)
|
||||||
|
- Query sanitization: strip characters outside [a-zA-Z0-9 .-_] before passing to SearXNG
|
||||||
|
</behavior>
|
||||||
|
<action>
|
||||||
|
internal/research/agent.go:
|
||||||
|
- Struct Agent with fields: nbClient *netbox.Client, researchClient ai.ResearchClient,
|
||||||
|
tier2 ai.AIClient, updater *inventory.CatalogUpdater
|
||||||
|
- NewAgent(nb *netbox.Client, rc ai.ResearchClient, tier2 ai.AIClient, updater *inventory.CatalogUpdater) *Agent
|
||||||
|
- sanitizeQuery(s string) string — regexp [^a-zA-Z0-9 .\-_]+ replaced with space, strings.TrimSpace
|
||||||
|
- RunOnce(ctx context.Context) (enriched int, err error):
|
||||||
|
1. ListDevicesWithStatus(ctx, "needs_research")
|
||||||
|
2. For each device:
|
||||||
|
a. query = sanitizeQuery(device.Name)
|
||||||
|
b. results = researchClient.Search(ctx, query) — skip if 0 results
|
||||||
|
c. Build text prompt with top 3 results (title + snippet)
|
||||||
|
d. tier2.AnalyzePhotos(ctx, IntakeRequest{PhotosBase64: nil, SystemPrompt: researchPrompt})
|
||||||
|
NOTE: IntakeRequest may not have SystemPrompt; build the research prompt as the
|
||||||
|
text part of the multimodal request by putting it in a single text-only message.
|
||||||
|
Check IntakeRequest fields; if no SystemPrompt, use a wrapper: set PhotosBase64 to
|
||||||
|
nil and pass the assembled prompt text in a way the TierClient accepts.
|
||||||
|
ALTERNATIVE if IntakeRequest does not support text-only: use go-openai directly
|
||||||
|
via a new ResearchTierClient method — add TextComplete(ctx, prompt) (*IntakeResult, error)
|
||||||
|
that posts a simple text ChatCompletion (no images). Prefer this approach for clarity.
|
||||||
|
e. Parse response for ai_notes and product_url
|
||||||
|
f. Patch NetBox: PatchCustomFields with ai_notes + product_url (if non-empty)
|
||||||
|
g. UpdateCatalogStatus(ctx, id, StatusNeedsResearch, StatusResearched)
|
||||||
|
h. enriched++
|
||||||
|
3. Return enriched count
|
||||||
|
- Start(ctx context.Context, interval time.Duration):
|
||||||
|
log.Printf("research agent: starting, interval=%v", interval)
|
||||||
|
RunOnce immediately, then ticker loop until ctx.Done()
|
||||||
|
|
||||||
|
For the text-only LLM call: add TextComplete to TierClient in internal/ai/client.go:
|
||||||
|
```go
|
||||||
|
func (c *TierClient) TextComplete(ctx context.Context, prompt string) (string, error)
|
||||||
|
```
|
||||||
|
This does a simple non-vision ChatCompletion with a single user message. Agent uses this.
|
||||||
|
|
||||||
|
internal/research/agent_test.go:
|
||||||
|
- Mock ResearchClient returning 2 fake SearchResults
|
||||||
|
- Mock AIClient (use existing MockAIClient pattern if available, else minimal struct)
|
||||||
|
- Mock NetBox (or use a stub struct) — test RunOnce returns enriched=1 for a fake device
|
||||||
|
- Test sanitizeQuery strips special chars
|
||||||
|
|
||||||
|
internal/api/handlers/research.go:
|
||||||
|
- ResearchHandler struct with agent *research.Agent
|
||||||
|
- NewResearchHandler(agent *research.Agent) *ResearchHandler
|
||||||
|
- TriggerResearch(w http.ResponseWriter, r *http.Request):
|
||||||
|
go func() { agent.RunOnce(context.Background()) }()
|
||||||
|
w.WriteHeader(http.StatusAccepted)
|
||||||
|
json.NewEncoder(w).Encode(map[string]string{"status": "accepted"})
|
||||||
|
|
||||||
|
internal/api/router.go:
|
||||||
|
- Add researchHandler *handlers.ResearchHandler parameter to NewRouter signature
|
||||||
|
- Add r.Post("/research/trigger", researchHandler.TriggerResearch) inside r.Route("/api", ...)
|
||||||
|
- If researchHandler is nil, register an unavailable handler (same pattern as advisorHandler)
|
||||||
|
|
||||||
|
cmd/hwlab/main.go:
|
||||||
|
- Import internal/research
|
||||||
|
- After config load: searxngClient := research.NewSearXNGClient(cfg.SearXNGURL)
|
||||||
|
- researchAgent := research.NewAgent(nbClient, searxngClient, tier2, catalogUpdater)
|
||||||
|
- go researchAgent.Start(ctx, 10*time.Minute)
|
||||||
|
- researchHandler := handlers.NewResearchHandler(researchAgent)
|
||||||
|
- Pass researchHandler to api.NewRouter(...)
|
||||||
|
</action>
|
||||||
|
<verify>
|
||||||
|
<automated>cd /home/mikkel/homelabby && go build ./... && go test ./internal/research/... -v -count=1 2>&1 | tail -30</automated>
|
||||||
|
</verify>
|
||||||
|
<done>
|
||||||
|
go build passes. Agent tests pass. POST /api/research/trigger wired in router.
|
||||||
|
Research agent goroutine starts on server launch with 10-minute interval.
|
||||||
|
</done>
|
||||||
|
</task>
|
||||||
|
|
||||||
|
</tasks>
|
||||||
|
|
||||||
|
<threat_model>
|
||||||
|
## Trust Boundaries
|
||||||
|
|
||||||
|
| Boundary | Description |
|
||||||
|
|----------|-------------|
|
||||||
|
| agent → SearXNG | AI-generated query text leaves the process and reaches the search engine |
|
||||||
|
| SearXNG → agent | External search results (HTML snippets) enter the process and are forwarded to LLM |
|
||||||
|
| trigger endpoint → agent | HTTP request from frontend triggers a research cycle |
|
||||||
|
|
||||||
|
## STRIDE Threat Register
|
||||||
|
|
||||||
|
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|
||||||
|
|-----------|----------|-----------|-------------|-----------------|
|
||||||
|
| T-07-01 | Tampering | sanitizeQuery | mitigate | Strip [^a-zA-Z0-9 .\-_]+ before dispatch; test with adversarial input in unit test |
|
||||||
|
| T-07-02 | Information Disclosure | SearXNG response snippets | accept | SearXNG is self-hosted LAN service; snippets never stored, only passed to LLM |
|
||||||
|
| T-07-03 | Denial of Service | POST /api/research/trigger | mitigate | Trigger fires goroutine but RunOnce is bounded per item; no queuing needed for MVP rate |
|
||||||
|
| T-07-04 | Spoofing | SearXNG base URL in config | accept | LAN-only service at fixed IP; no auth required by design |
|
||||||
|
</threat_model>
|
||||||
|
|
||||||
|
<verification>
|
||||||
|
1. `go build ./...` passes with no errors
|
||||||
|
2. `go test ./internal/research/...` all pass
|
||||||
|
3. SearXNG integration (manual): `curl "http://10.5.0.129:8080/search?q=Intel+i350&format=json"` returns JSON
|
||||||
|
4. Trigger endpoint: `curl -X POST http://localhost:8080/api/research/trigger` returns 202
|
||||||
|
5. Log line "research agent: starting, interval=10m0s" appears on server start
|
||||||
|
</verification>
|
||||||
|
|
||||||
|
<success_criteria>
|
||||||
|
- SearXNGClient.Search returns parsed []ai.SearchResult from live SearXNG instance
|
||||||
|
- ResearchAgent.RunOnce enriches needs_research items end-to-end: search → LLM → NetBox patch → status transition
|
||||||
|
- Research cycle runs every 10 minutes automatically and on demand via POST /api/research/trigger
|
||||||
|
- All queries sanitized before SearXNG dispatch
|
||||||
|
- go build clean, all new tests pass
|
||||||
|
</success_criteria>
|
||||||
|
|
||||||
|
<output>
|
||||||
|
After completion, create `.planning/phases/07-research-agent-search/07-01-SUMMARY.md`
|
||||||
|
</output>
|
||||||
90
.planning/phases/07-research-agent-search/07-01-SUMMARY.md
Normal file
90
.planning/phases/07-research-agent-search/07-01-SUMMARY.md
Normal file
|
|
@ -0,0 +1,90 @@
|
||||||
|
---
|
||||||
|
phase: 07-research-agent-search
|
||||||
|
plan: "01"
|
||||||
|
subsystem: research-agent
|
||||||
|
tags: [research, searxng, ai-enrichment, background-worker]
|
||||||
|
dependency_graph:
|
||||||
|
requires: [internal/ai, internal/netbox, internal/inventory, internal/config]
|
||||||
|
provides: [internal/research, internal/api/handlers/research]
|
||||||
|
affects: [cmd/hwlab/main.go, internal/api/router.go]
|
||||||
|
tech_stack:
|
||||||
|
added: [internal/research package]
|
||||||
|
patterns: [interface-injection for testability, TDD red-green, background ticker worker]
|
||||||
|
key_files:
|
||||||
|
created:
|
||||||
|
- internal/research/searxng.go
|
||||||
|
- internal/research/searxng_test.go
|
||||||
|
- internal/research/agent.go
|
||||||
|
- internal/research/agent_test.go
|
||||||
|
- internal/api/handlers/research.go
|
||||||
|
modified:
|
||||||
|
- internal/ai/client.go
|
||||||
|
- internal/netbox/client.go
|
||||||
|
- internal/config/config.go
|
||||||
|
- internal/api/router.go
|
||||||
|
- cmd/hwlab/main.go
|
||||||
|
decisions:
|
||||||
|
- "Used interface injection (NetBoxer, TextCompleter, CatalogTransitioner) in Agent instead of concrete types to enable stub-based unit tests without live NetBox"
|
||||||
|
- "Added TierClient.TextComplete as separate method rather than reusing AnalyzePhotos to keep vision and text paths distinct"
|
||||||
|
- "SanitizeQuery exported (capitalised) to allow external test package verification of T-07-01 mitigation"
|
||||||
|
- "Agent.RunOnce returns (int, error) rather than just error so callers and tests can assert enrichment count"
|
||||||
|
metrics:
|
||||||
|
duration_seconds: 256
|
||||||
|
completed_date: "2026-04-10"
|
||||||
|
tasks_completed: 2
|
||||||
|
tasks_total: 2
|
||||||
|
files_created: 5
|
||||||
|
files_modified: 5
|
||||||
|
---
|
||||||
|
|
||||||
|
# Phase 07 Plan 01: SearXNG Client + ResearchAgent Summary
|
||||||
|
|
||||||
|
**One-liner:** SearXNG HTTP client + background ResearchAgent that polls NetBox for `needs_research` items, enriches via SearXNG search + Tier 2 LLM text completion, patches NetBox custom fields, and transitions status to `researched` every 10 minutes or on-demand via `POST /api/research/trigger`.
|
||||||
|
|
||||||
|
## Tasks Completed
|
||||||
|
|
||||||
|
| Task | Name | Commit | Key Files |
|
||||||
|
|------|------|--------|-----------|
|
||||||
|
| 1 | SearXNG client + netbox.ListDevicesWithStatus | 30cd279 | internal/research/searxng.go, internal/netbox/client.go, internal/config/config.go |
|
||||||
|
| 2 | ResearchAgent worker + main.go wiring + trigger endpoint | 0072aa4 | internal/research/agent.go, internal/api/handlers/research.go, internal/api/router.go, cmd/hwlab/main.go |
|
||||||
|
|
||||||
|
## Decisions Made
|
||||||
|
|
||||||
|
1. **Interface injection for Agent dependencies** — `NetBoxer`, `TextCompleter`, and `CatalogTransitioner` interfaces in `internal/research/agent.go` allow stub injection in tests without a live NetBox or LLM. The concrete `*netbox.Client` and `*ai.TierClient` satisfy these interfaces automatically.
|
||||||
|
|
||||||
|
2. **TierClient.TextComplete as distinct method** — Rather than forcing research prompts through `AnalyzePhotos` (which builds a vision multipart message), added a clean `TextComplete(ctx, prompt) (string, error)` method on `TierClient` that posts a simple single-user-message ChatCompletion. This keeps the vision and text paths separate and makes intent clear.
|
||||||
|
|
||||||
|
3. **SanitizeQuery exported** — The T-07-01 threat mitigation (strip `[^a-zA-Z0-9 .\-_]+` before SearXNG dispatch) is tested from the `_test` package, which requires the function to be exported. Consistent with the plan's explicit mention of adversarial input testing.
|
||||||
|
|
||||||
|
4. **Product URL fallback** — If the LLM does not return a `product_url` in its JSON response, the agent falls back to the first SearXNG result URL. This ensures `product_url` is populated even when the LLM provides incomplete JSON.
|
||||||
|
|
||||||
|
## Deviations from Plan
|
||||||
|
|
||||||
|
### Auto-fixed Issues
|
||||||
|
|
||||||
|
**1. [Rule 1 - Bug] Test expectation for sanitizeQuery with `/` character**
|
||||||
|
- **Found during:** Task 2 GREEN phase
|
||||||
|
- **Issue:** Test case `"Dell<script>alert(1)</script>"` expected `"Dell script alert 1 /script "` but `/` is correctly stripped by the `[^a-zA-Z0-9 .\-_]+` regex
|
||||||
|
- **Fix:** Updated test expectation to `"Dell script alert 1 script"` (correct behavior)
|
||||||
|
- **Files modified:** internal/research/agent_test.go
|
||||||
|
- **Commit:** 0072aa4
|
||||||
|
|
||||||
|
None otherwise — plan executed as written.
|
||||||
|
|
||||||
|
## Known Stubs
|
||||||
|
|
||||||
|
None. All interfaces are fully implemented. The `NoOpResearchClient` in `internal/ai/research.go` is still present but is no longer used — it has been superseded by `research.SearXNGClient` wired in `main.go`.
|
||||||
|
|
||||||
|
## Threat Surface Scan
|
||||||
|
|
||||||
|
No new trust boundaries introduced beyond those documented in the plan's threat model. T-07-01 (query sanitization) is mitigated and tested. T-07-03 (trigger DoS) is mitigated — `RunOnce` is bounded per-item with no queuing amplification.
|
||||||
|
|
||||||
|
## Self-Check: PASSED
|
||||||
|
|
||||||
|
- internal/research/searxng.go: FOUND
|
||||||
|
- internal/research/agent.go: FOUND
|
||||||
|
- internal/api/handlers/research.go: FOUND
|
||||||
|
- Commit 30cd279: FOUND
|
||||||
|
- Commit 0072aa4: FOUND
|
||||||
|
- `go build ./...`: PASSES
|
||||||
|
- `go test ./internal/research/...`: 9/9 PASS
|
||||||
302
.planning/phases/07-research-agent-search/07-02-PLAN.md
Normal file
302
.planning/phases/07-research-agent-search/07-02-PLAN.md
Normal file
|
|
@ -0,0 +1,302 @@
|
||||||
|
---
|
||||||
|
phase: 07-research-agent-search
|
||||||
|
plan: 02
|
||||||
|
type: execute
|
||||||
|
wave: 2
|
||||||
|
depends_on:
|
||||||
|
- "07-01-PLAN.md"
|
||||||
|
files_modified:
|
||||||
|
- internal/api/handlers/search.go
|
||||||
|
- internal/api/router.go
|
||||||
|
- web/src/lib/api.ts
|
||||||
|
- web/src/pages/DashboardPage.tsx
|
||||||
|
- web/src/components/inventory/FilterBar.tsx
|
||||||
|
autonomous: true
|
||||||
|
requirements:
|
||||||
|
- AI-04
|
||||||
|
- UI-03
|
||||||
|
|
||||||
|
must_haves:
|
||||||
|
truths:
|
||||||
|
- "GET /api/search?q=show+me+free+10GbE+NICs returns matching inventory items"
|
||||||
|
- "Tier 1 (Gemma 4) translates the natural language query to NetBox filter params before the inventory lookup"
|
||||||
|
- "Dashboard has a natural language search input; submitting it calls GET /api/search and displays results using existing ItemCard/ItemRow"
|
||||||
|
- "Sanitized query — no raw NL text reaches NetBox filter params; only structured extracted values"
|
||||||
|
artifacts:
|
||||||
|
- path: "internal/api/handlers/search.go"
|
||||||
|
provides: "SearchHandler: GET /api/search?q=..."
|
||||||
|
exports: ["SearchHandler", "NewSearchHandler"]
|
||||||
|
- path: "web/src/lib/api.ts"
|
||||||
|
provides: "fetchSearch(q) function"
|
||||||
|
exports: ["fetchSearch", "SearchResponse"]
|
||||||
|
key_links:
|
||||||
|
- from: "web/src/pages/DashboardPage.tsx"
|
||||||
|
to: "/api/search"
|
||||||
|
via: "TanStack Query useQuery on nlQuery state"
|
||||||
|
pattern: "useQuery.*search"
|
||||||
|
- from: "internal/api/handlers/search.go"
|
||||||
|
to: "internal/netbox/client.go"
|
||||||
|
via: "ListDevices or ListDevicesWithStatus filtered by extracted params"
|
||||||
|
pattern: "nbClient\\.List"
|
||||||
|
---
|
||||||
|
|
||||||
|
<objective>
|
||||||
|
Natural language inventory search: a Tier 1 LLM translates the user's query to
|
||||||
|
structured NetBox filter params, fetches matching devices, and returns them.
|
||||||
|
The dashboard gains an NL search input wired to GET /api/search.
|
||||||
|
|
||||||
|
Purpose: Delivers UI-03 (natural language search) and closes the remaining AI-04
|
||||||
|
surface (research loop query path).
|
||||||
|
|
||||||
|
Output:
|
||||||
|
- internal/api/handlers/search.go — SearchHandler with NL→filter translation
|
||||||
|
- Router wired with GET /api/search
|
||||||
|
- web/src/lib/api.ts — fetchSearch function
|
||||||
|
- DashboardPage NL search bar replaces/augments the existing local text filter
|
||||||
|
</objective>
|
||||||
|
|
||||||
|
<execution_context>
|
||||||
|
@/home/mikkel/.claude/get-shit-done/workflows/execute-plan.md
|
||||||
|
@/home/mikkel/.claude/get-shit-done/templates/summary.md
|
||||||
|
</execution_context>
|
||||||
|
|
||||||
|
<context>
|
||||||
|
@.planning/PROJECT.md
|
||||||
|
@.planning/ROADMAP.md
|
||||||
|
@.planning/STATE.md
|
||||||
|
@.planning/phases/07-research-agent-search/07-01-SUMMARY.md
|
||||||
|
|
||||||
|
@internal/api/router.go
|
||||||
|
@internal/api/handlers/search.go
|
||||||
|
@internal/netbox/client.go
|
||||||
|
@internal/netbox/types.go
|
||||||
|
@internal/ai/client.go
|
||||||
|
@internal/config/config.go
|
||||||
|
@web/src/lib/api.ts
|
||||||
|
@web/src/pages/DashboardPage.tsx
|
||||||
|
@web/src/components/inventory/FilterBar.tsx
|
||||||
|
|
||||||
|
<interfaces>
|
||||||
|
<!-- Key contracts the executor needs. Extracted from codebase. -->
|
||||||
|
|
||||||
|
From internal/netbox/client.go:
|
||||||
|
```go
|
||||||
|
func (c *Client) ListDevices(ctx context.Context, limit int) ([]Device, error)
|
||||||
|
func (c *Client) ListDevicesWithStatus(ctx context.Context, status string) ([]Device, error)
|
||||||
|
// Added in Plan 01
|
||||||
|
```
|
||||||
|
|
||||||
|
From internal/netbox/types.go:
|
||||||
|
```go
|
||||||
|
type Device struct {
|
||||||
|
ID int
|
||||||
|
Name string
|
||||||
|
AssetTag string
|
||||||
|
CustomFields CustomFields
|
||||||
|
Created time.Time
|
||||||
|
LastUpdated time.Time
|
||||||
|
}
|
||||||
|
type CustomFields struct {
|
||||||
|
HWID string
|
||||||
|
CatalogStatus string
|
||||||
|
ProductURL string
|
||||||
|
FirmwareVersion string
|
||||||
|
TestDate string
|
||||||
|
TestData string
|
||||||
|
AINotes string
|
||||||
|
PhotoURLs []string
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
From internal/ai/client.go:
|
||||||
|
```go
|
||||||
|
type TierClient struct { ... }
|
||||||
|
// TextComplete added in Plan 01:
|
||||||
|
func (c *TierClient) TextComplete(ctx context.Context, prompt string) (string, error)
|
||||||
|
```
|
||||||
|
|
||||||
|
From web/src/lib/api.ts:
|
||||||
|
```typescript
|
||||||
|
export interface InventoryItem {
|
||||||
|
id: number; name: string; asset_tag: string | null; hw_id: string | null;
|
||||||
|
catalog_status: string | null; product_url: string | null;
|
||||||
|
firmware_version: string | null; test_date: string | null;
|
||||||
|
test_data: string | null; ai_notes: string | null; photo_urls: string[];
|
||||||
|
}
|
||||||
|
export const fetchInventory = (): Promise<InventoryItem[]>
|
||||||
|
// Add fetchSearch here — returns InventoryItem[] with same shape
|
||||||
|
```
|
||||||
|
|
||||||
|
From web/src/pages/DashboardPage.tsx:
|
||||||
|
```typescript
|
||||||
|
// Existing local text filter:
|
||||||
|
const [search, setSearch] = useState('')
|
||||||
|
// Filtered locally via useMemo. NL search should add a separate nlQuery state.
|
||||||
|
// When nlQuery is non-empty: show NL results (from GET /api/search) instead of local filter.
|
||||||
|
// When nlQuery is empty: use existing local search behavior unchanged.
|
||||||
|
```
|
||||||
|
</interfaces>
|
||||||
|
</context>
|
||||||
|
|
||||||
|
<tasks>
|
||||||
|
|
||||||
|
<task type="auto" tdd="true">
|
||||||
|
<name>Task 1: SearchHandler — NL query → NetBox filter → device list</name>
|
||||||
|
<files>
|
||||||
|
internal/api/handlers/search.go,
|
||||||
|
internal/api/handlers/search_test.go,
|
||||||
|
internal/api/router.go
|
||||||
|
</files>
|
||||||
|
<behavior>
|
||||||
|
- GET /api/search?q=show+me+free+10GbE+NICs → 200 JSON array of InventoryItem
|
||||||
|
- GET /api/search with empty q → 400 {"error":"q parameter required"}
|
||||||
|
- Tier 1 LLM receives: "Extract NetBox filter parameters from this inventory search query. Return JSON only: {\"catalog_status\": \"...\", \"name_contains\": \"...\", \"tag\": \"...\"}. All fields optional. Query: {user query}"
|
||||||
|
- LLM response parsed; client-side filter applied to ListDevices(ctx, 200) results
|
||||||
|
- name_contains: case-insensitive substring match on device.Name
|
||||||
|
- catalog_status: exact match on CustomFields.CatalogStatus (available → "available", etc.)
|
||||||
|
- tag: ignored for MVP (NetBox tag filtering requires separate API; log "tag filter not implemented")
|
||||||
|
- If LLM parse fails, fall back to simple substring match on device Name against raw query
|
||||||
|
- Result serialized as []map[string]interface{} matching InventoryItem TypeScript shape
|
||||||
|
</behavior>
|
||||||
|
<action>
|
||||||
|
internal/api/handlers/search.go:
|
||||||
|
- SearchHandler struct: nbClient *netbox.Client, tier1 *ai.TierClient
|
||||||
|
- NewSearchHandler(nb *netbox.Client, tier1 *ai.TierClient) *SearchHandler
|
||||||
|
- SearchDevices(w http.ResponseWriter, r *http.Request):
|
||||||
|
1. q := r.URL.Query().Get("q"); if empty → 400
|
||||||
|
2. Sanitize q: strip non-printable chars, trim to 200 chars max
|
||||||
|
3. Call tier1.TextComplete(ctx, nlFilterPrompt(q)) — 5s timeout
|
||||||
|
4. Parse JSON response into struct { CatalogStatus string `json:"catalog_status"`, NameContains string `json:"name_contains"`, Tag string `json:"tag"` }
|
||||||
|
5. If parse fails: log warning, set NameContains = q (fallback)
|
||||||
|
6. devices, _ = nbClient.ListDevices(ctx, 200)
|
||||||
|
7. Apply filters: CatalogStatus match + NameContains match (both case-insensitive)
|
||||||
|
8. Convert filtered devices to response slice using deviceToResponseMap helper
|
||||||
|
9. json.NewEncoder(w).Encode(result)
|
||||||
|
|
||||||
|
deviceToResponseMap converts netbox.Device to map[string]interface{} matching InventoryItem shape:
|
||||||
|
{ "id", "name", "asset_tag" (from AssetTag or nil), "hw_id", "catalog_status",
|
||||||
|
"product_url", "firmware_version", "test_date", "test_data", "ai_notes", "photo_urls" }
|
||||||
|
|
||||||
|
nlFilterPrompt(q string) string — returns the extraction prompt.
|
||||||
|
|
||||||
|
internal/api/handlers/search_test.go:
|
||||||
|
- Use a mock netbox client (simple struct implementing a ListDevices method via interface,
|
||||||
|
or just test the filter logic separately)
|
||||||
|
- Test: empty q returns 400
|
||||||
|
- Test: nlFilter parse failure falls back to name substring match
|
||||||
|
- Test: catalog_status filter correctly narrows results
|
||||||
|
|
||||||
|
internal/api/router.go:
|
||||||
|
- Add searchHandler *handlers.SearchHandler param to NewRouter signature
|
||||||
|
- Add r.Get("/search", searchHandler.SearchDevices) inside r.Route("/api", ...)
|
||||||
|
- nil guard: if searchHandler is nil, return 503 (same pattern as advisorHandler)
|
||||||
|
</action>
|
||||||
|
<verify>
|
||||||
|
<automated>cd /home/mikkel/homelabby && go build ./... && go test ./internal/api/handlers/... -v -count=1 -run TestSearch 2>&1 | tail -20</automated>
|
||||||
|
</verify>
|
||||||
|
<done>
|
||||||
|
GET /api/search?q=... handler compiles and handler tests pass.
|
||||||
|
Router wired. go build clean.
|
||||||
|
</done>
|
||||||
|
</task>
|
||||||
|
|
||||||
|
<task type="auto">
|
||||||
|
<name>Task 2: Frontend NL search bar + main.go wiring</name>
|
||||||
|
<files>
|
||||||
|
web/src/lib/api.ts,
|
||||||
|
web/src/pages/DashboardPage.tsx,
|
||||||
|
web/src/components/inventory/FilterBar.tsx,
|
||||||
|
cmd/hwlab/main.go
|
||||||
|
</files>
|
||||||
|
<action>
|
||||||
|
web/src/lib/api.ts — add fetchSearch:
|
||||||
|
```typescript
|
||||||
|
export const fetchSearch = (q: string): Promise<InventoryItem[]> =>
|
||||||
|
fetchJSON<InventoryItem[]>(`${BASE}/search?q=${encodeURIComponent(q)}`)
|
||||||
|
```
|
||||||
|
|
||||||
|
web/src/pages/DashboardPage.tsx changes:
|
||||||
|
1. Add state: const [nlQuery, setNlQuery] = useState('')
|
||||||
|
2. Add TanStack Query hook:
|
||||||
|
```typescript
|
||||||
|
const { data: searchResults, isLoading: searchLoading } = useQuery({
|
||||||
|
queryKey: ['search', nlQuery],
|
||||||
|
queryFn: () => fetchSearch(nlQuery),
|
||||||
|
enabled: nlQuery.trim().length > 2,
|
||||||
|
staleTime: 30_000,
|
||||||
|
})
|
||||||
|
```
|
||||||
|
3. Display logic: when nlQuery.length > 2, show searchResults (or empty state)
|
||||||
|
instead of the existing `filtered` local results.
|
||||||
|
4. When nlQuery is empty, existing local filter behavior is unchanged.
|
||||||
|
5. Pass nlQuery + setNlQuery + searchLoading to FilterBar as new props.
|
||||||
|
|
||||||
|
web/src/components/inventory/FilterBar.tsx changes:
|
||||||
|
- Add nlQuery string prop + onNlQueryChange (string) => void + nlSearchLoading boolean
|
||||||
|
- Add a second input below (or inline with) the existing search input:
|
||||||
|
- Placeholder: "Ask anything: show me free 10GbE NICs…"
|
||||||
|
- Tailwind: full-width, border-volt/40, bg-[#0a0a0a], text-white, focus:border-volt,
|
||||||
|
rounded-card, px-3 py-2 text-sm
|
||||||
|
- Right side: if nlSearchLoading show <Loader2 className="w-4 h-4 animate-spin text-volt" />
|
||||||
|
- onBlur / onChange with 400ms debounce: call onNlQueryChange
|
||||||
|
- Use a local useState for the input value; debounce via useEffect + setTimeout clearing pattern
|
||||||
|
- Keep existing search + status filter inputs intact — NL search is additive
|
||||||
|
|
||||||
|
cmd/hwlab/main.go:
|
||||||
|
- Import internal/api/handlers (already imported)
|
||||||
|
- After building tier1 TierClient: searchHandler := handlers.NewSearchHandler(nbClient, tier1)
|
||||||
|
- Pass searchHandler to api.NewRouter(...) — add as new final param
|
||||||
|
- NOTE: tier1 is already constructed as `ai.NewTierClient(cfg.AI.Tier1)` — pass it directly
|
||||||
|
but SearchHandler needs *ai.TierClient not ai.AIClient; adjust if needed (TierClient is a concrete type)
|
||||||
|
</action>
|
||||||
|
<verify>
|
||||||
|
<automated>cd /home/mikkel/homelabby && go build ./... && cd web && npm run build 2>&1 | tail -20</automated>
|
||||||
|
</verify>
|
||||||
|
<done>
|
||||||
|
go build and npm run build both pass with no errors.
|
||||||
|
Dashboard FilterBar renders NL search input. Typing a query with > 2 chars triggers
|
||||||
|
GET /api/search and displays results using existing ItemCard/ItemRow components.
|
||||||
|
Existing local search filter still works when nlQuery is empty.
|
||||||
|
</done>
|
||||||
|
</task>
|
||||||
|
|
||||||
|
</tasks>
|
||||||
|
|
||||||
|
<threat_model>
|
||||||
|
## Trust Boundaries
|
||||||
|
|
||||||
|
| Boundary | Description |
|
||||||
|
|----------|-------------|
|
||||||
|
| browser → GET /api/search | User-supplied NL query crosses HTTP boundary |
|
||||||
|
| search handler → Tier 1 LLM | Sanitized query forwarded to local oMLX |
|
||||||
|
| LLM output → NetBox filter | Structured JSON from LLM used to filter devices |
|
||||||
|
|
||||||
|
## STRIDE Threat Register
|
||||||
|
|
||||||
|
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|
||||||
|
|-----------|----------|-----------|-------------|-----------------|
|
||||||
|
| T-07-05 | Tampering | NL query → LLM prompt | mitigate | Strip non-printable chars, truncate to 200 chars before building prompt |
|
||||||
|
| T-07-06 | Tampering | LLM output → filter params | mitigate | Parse only known fields (catalog_status, name_contains, tag); ignore unknown keys; fallback on parse failure |
|
||||||
|
| T-07-07 | Denial of Service | GET /api/search fanout | accept | ListDevices(200) is bounded; Tier 1 local inference is fast; no per-user rate limiting needed for single-operator tool |
|
||||||
|
| T-07-08 | Information Disclosure | search results | accept | All results are local NetBox inventory; no cross-tenant risk in single-operator homelab |
|
||||||
|
</threat_model>
|
||||||
|
|
||||||
|
<verification>
|
||||||
|
1. `go build ./...` passes
|
||||||
|
2. `cd web && npm run build` passes
|
||||||
|
3. Manual: GET http://localhost:8080/api/search?q=show+me+available+NICs returns JSON array
|
||||||
|
4. Manual: GET http://localhost:8080/api/search (no q) returns 400
|
||||||
|
5. Dashboard: NL search input visible below existing filter bar; typing triggers spinner then results
|
||||||
|
</verification>
|
||||||
|
|
||||||
|
<success_criteria>
|
||||||
|
- GET /api/search?q=... returns filtered InventoryItem array using Tier 1 NL→filter translation
|
||||||
|
- Query sanitized (non-printable stripped, 200 char max) before LLM
|
||||||
|
- LLM parse failure falls back to name substring match (never 500)
|
||||||
|
- Dashboard NL search bar triggers live search; existing local filter unchanged when NL query empty
|
||||||
|
- go build and npm run build both clean
|
||||||
|
</success_criteria>
|
||||||
|
|
||||||
|
<output>
|
||||||
|
After completion, create `.planning/phases/07-research-agent-search/07-02-SUMMARY.md`
|
||||||
|
</output>
|
||||||
118
.planning/phases/07-research-agent-search/07-02-SUMMARY.md
Normal file
118
.planning/phases/07-research-agent-search/07-02-SUMMARY.md
Normal file
|
|
@ -0,0 +1,118 @@
|
||||||
|
---
|
||||||
|
phase: 07-research-agent-search
|
||||||
|
plan: "02"
|
||||||
|
subsystem: search
|
||||||
|
tags: [search, nlp, ai, frontend, go, react]
|
||||||
|
dependency_graph:
|
||||||
|
requires: [07-01]
|
||||||
|
provides: [search-endpoint, nl-search-ui]
|
||||||
|
affects: [DashboardPage, FilterBar, api-router]
|
||||||
|
tech_stack:
|
||||||
|
added: []
|
||||||
|
patterns:
|
||||||
|
- NL query sanitize → LLM extraction → in-memory filter (no raw text to NetBox)
|
||||||
|
- TDD: failing tests written first, handler implemented to green
|
||||||
|
- Debounced NL input (400ms) with TanStack Query (enabled guard on length > 2)
|
||||||
|
key_files:
|
||||||
|
created:
|
||||||
|
- internal/api/handlers/search.go
|
||||||
|
- internal/api/handlers/search_test.go
|
||||||
|
modified:
|
||||||
|
- internal/api/router.go
|
||||||
|
- cmd/hwlab/main.go
|
||||||
|
- web/src/lib/api.ts
|
||||||
|
- web/src/components/inventory/FilterBar.tsx
|
||||||
|
- web/src/pages/DashboardPage.tsx
|
||||||
|
decisions:
|
||||||
|
- Used SearchNetBoxClient and SearchAIClient narrow interfaces in search.go for testability without importing ai package in tests
|
||||||
|
- extractJSON helper strips markdown code fences from LLM response before JSON parse
|
||||||
|
- NL search row placed below existing filter row (additive, not replacing) to keep local filter intact
|
||||||
|
- displayLoading/displayItems derived state avoids duplicating isLoading/searchLoading logic
|
||||||
|
metrics:
|
||||||
|
duration: "~12 minutes"
|
||||||
|
completed: "2026-04-10"
|
||||||
|
tasks_completed: 2
|
||||||
|
files_changed: 7
|
||||||
|
---
|
||||||
|
|
||||||
|
# Phase 07 Plan 02: Natural Language Search Summary
|
||||||
|
|
||||||
|
Natural language inventory search via Tier 1 LLM (Gemma 4): user query sanitized, translated to structured filter params (catalog_status + name_contains), applied in-memory against ListDevices(200), returned as InventoryItem JSON; dashboard gains a debounced NL search input with volt accent styling and fallback to substring match on LLM parse failure.
|
||||||
|
|
||||||
|
## Tasks Completed
|
||||||
|
|
||||||
|
| Task | Name | Commit | Files |
|
||||||
|
|------|------|--------|-------|
|
||||||
|
| 1 | SearchHandler — NL query to NetBox filter | 9db7707 | search.go, search_test.go, router.go, main.go |
|
||||||
|
| 2 | Frontend NL search bar + api.ts wiring | 7db093c | api.ts, FilterBar.tsx, DashboardPage.tsx |
|
||||||
|
|
||||||
|
## What Was Built
|
||||||
|
|
||||||
|
### Backend (Task 1)
|
||||||
|
|
||||||
|
`internal/api/handlers/search.go` provides `SearchHandler` with `SearchDevices(w, r)`:
|
||||||
|
|
||||||
|
1. Validates `q` param — returns 400 if empty
|
||||||
|
2. Sanitizes query: strips non-printable chars, truncates to 200 runes (T-07-05)
|
||||||
|
3. Calls `tier1.TextComplete` with an extraction prompt requesting `{"catalog_status","name_contains","tag"}` JSON
|
||||||
|
4. Calls `extractJSON` to strip markdown fences before parse (LLM sometimes wraps in code blocks)
|
||||||
|
5. On parse failure: logs warning, falls back to `NameContains = rawQuery` (never 500)
|
||||||
|
6. `tag` field logged as not implemented and ignored (MVP)
|
||||||
|
7. Fetches `ListDevices(ctx, 200)`, applies in-memory filter, encodes result
|
||||||
|
|
||||||
|
Router wired: `GET /api/search` with nil-guard 503 fallback. `main.go` constructs `handlers.NewSearchHandler(nbClient, tier1)` and passes to `NewRouter`.
|
||||||
|
|
||||||
|
TDD: 4 tests written before implementation — `TestSearch_EmptyQ`, `TestSearch_LLMParseFallback`, `TestSearch_CatalogStatusFilter`, `TestSearch_NameContainsAndStatus` — all pass.
|
||||||
|
|
||||||
|
### Frontend (Task 2)
|
||||||
|
|
||||||
|
`web/src/lib/api.ts`: `fetchSearch(q)` added — calls `GET /api/search?q=<encoded>`.
|
||||||
|
|
||||||
|
`FilterBar.tsx` restructured to two rows:
|
||||||
|
- Row 1: existing local text search + status select + item count + view toggle (unchanged)
|
||||||
|
- Row 2: NL input with `Sparkles` icon (volt/60 tint), `border-volt/20` → `focus:border-volt`, `Loader2` spinner when `nlSearchLoading`
|
||||||
|
- 400ms debounce via `useEffect + setTimeout` pattern; local `nlInputValue` state, propagates via `onNlQueryChange`
|
||||||
|
|
||||||
|
`DashboardPage.tsx`:
|
||||||
|
- `nlQuery` state + `useQuery({ queryKey: ['search', nlQuery], enabled: nlQuery.trim().length > 2 })`
|
||||||
|
- `displayItems = nlQuery > 2 ? searchResults : filtered` — local filter fully preserved when NL empty
|
||||||
|
- Loading/empty state messages adapt to NL vs local mode
|
||||||
|
|
||||||
|
## Threat Mitigations Applied
|
||||||
|
|
||||||
|
| Threat | Mitigation |
|
||||||
|
|--------|-----------|
|
||||||
|
| T-07-05 | `sanitizeQuery`: non-printable chars stripped, truncated to 200 runes |
|
||||||
|
| T-07-06 | Only `catalog_status`, `name_contains`, `tag` extracted; unknown keys ignored; fallback on parse failure |
|
||||||
|
|
||||||
|
## Deviations from Plan
|
||||||
|
|
||||||
|
### Auto-added: extractJSON helper
|
||||||
|
|
||||||
|
**Found during:** Task 1 implementation
|
||||||
|
**Issue:** LLMs commonly wrap JSON responses in markdown code fences (` ```json ... ``` `), which breaks `json.Unmarshal` directly
|
||||||
|
**Fix:** Added `extractJSON(s string) string` that finds first `{` and last `}` to extract the JSON object before parsing
|
||||||
|
**Files modified:** internal/api/handlers/search.go
|
||||||
|
**Rule:** Rule 2 — missing critical functionality (robustness of LLM output parsing)
|
||||||
|
|
||||||
|
No other deviations — plan executed as specified.
|
||||||
|
|
||||||
|
## Known Stubs
|
||||||
|
|
||||||
|
None — all data paths are wired end-to-end.
|
||||||
|
|
||||||
|
## Threat Flags
|
||||||
|
|
||||||
|
None — no new network endpoints or trust boundaries beyond what the plan specified.
|
||||||
|
|
||||||
|
## Self-Check: PASSED
|
||||||
|
|
||||||
|
- internal/api/handlers/search.go: exists
|
||||||
|
- internal/api/handlers/search_test.go: exists, 4 tests pass
|
||||||
|
- internal/api/router.go: GET /api/search wired
|
||||||
|
- web/src/lib/api.ts: fetchSearch exported
|
||||||
|
- web/src/pages/DashboardPage.tsx: nlQuery state + useQuery present
|
||||||
|
- web/src/components/inventory/FilterBar.tsx: NL input with debounce present
|
||||||
|
- go build ./...: clean
|
||||||
|
- npm run build: clean (built in 3.14s)
|
||||||
|
- Commits 9db7707 and 7db093c: verified in git log
|
||||||
70
.planning/phases/07-research-agent-search/07-CONTEXT.md
Normal file
70
.planning/phases/07-research-agent-search/07-CONTEXT.md
Normal file
|
|
@ -0,0 +1,70 @@
|
||||||
|
# Phase 7: Research Agent & Search - Context
|
||||||
|
|
||||||
|
**Gathered:** 2026-04-10
|
||||||
|
**Status:** Ready for planning
|
||||||
|
**Mode:** Auto-generated (autonomous mode)
|
||||||
|
|
||||||
|
<domain>
|
||||||
|
## Phase Boundary
|
||||||
|
|
||||||
|
Items flagged needs_research are automatically enriched by a SearXNG research agent, and any inventory question can be answered via natural language search. This phase delivers the SearXNG Tier 2 client, research agent that consumes needs_research items, and natural language search endpoint.
|
||||||
|
|
||||||
|
</domain>
|
||||||
|
|
||||||
|
<decisions>
|
||||||
|
## Implementation Decisions
|
||||||
|
|
||||||
|
### SearXNG Client
|
||||||
|
- HTTP client to SearXNG JSON API at http://10.5.0.129:8080/search
|
||||||
|
- Sanitize queries before dispatch
|
||||||
|
- Return list of result objects (title, url, snippet)
|
||||||
|
|
||||||
|
### Research Agent
|
||||||
|
- Background worker that polls NetBox for catalog_status=needs_research items
|
||||||
|
- For each item: query SearXNG for product info, send results to Tier 2 LLM, extract structured data
|
||||||
|
- Update NetBox device with enriched data, set catalog_status=researched
|
||||||
|
- Run periodically (every 10 minutes) or on-demand via POST /api/research/trigger
|
||||||
|
|
||||||
|
### Natural Language Search
|
||||||
|
- Endpoint: GET /api/search?q=...
|
||||||
|
- Use Tier 1 (Gemma 4) to translate query to NetBox filter params
|
||||||
|
- Return matching devices
|
||||||
|
|
||||||
|
### Frontend
|
||||||
|
- Search bar in dashboard top
|
||||||
|
- Results page reuses dashboard cards
|
||||||
|
|
||||||
|
</decisions>
|
||||||
|
|
||||||
|
<code_context>
|
||||||
|
## Existing Code Insights
|
||||||
|
|
||||||
|
### Reusable Assets from prior phases
|
||||||
|
- internal/ai/ — AIClient, TierClient, orchestrator
|
||||||
|
- internal/ai/research.go — ResearchClient interface (NoOp from Phase 2)
|
||||||
|
- internal/netbox/client.go — ListDevices, GetDevice, PatchCustomFields
|
||||||
|
- internal/inventory/quality_gate.go + catalog_updater.go
|
||||||
|
- internal/api/router.go
|
||||||
|
- web/src/lib/api.ts
|
||||||
|
|
||||||
|
### Integration Points
|
||||||
|
- Replace NoOpResearchClient with real SearXNGResearchClient
|
||||||
|
- Add internal/research/ package
|
||||||
|
- Add internal/api/handlers/search.go
|
||||||
|
- Frontend: dashboard search bar wires GET /api/search
|
||||||
|
</code_context>
|
||||||
|
|
||||||
|
<specifics>
|
||||||
|
## Specific Ideas
|
||||||
|
|
||||||
|
- SearXNG client uses standard net/http
|
||||||
|
- Research agent runs as goroutine started from main.go
|
||||||
|
- Search endpoint translates "show me free 10GbE NICs" to filter: category=NIC, status=available, tags has 10gbe
|
||||||
|
</specifics>
|
||||||
|
|
||||||
|
<deferred>
|
||||||
|
## Deferred Ideas
|
||||||
|
|
||||||
|
- Saved searches
|
||||||
|
- Search history
|
||||||
|
</deferred>
|
||||||
30
.planning/phases/07-research-agent-search/07-HUMAN-UAT.md
Normal file
30
.planning/phases/07-research-agent-search/07-HUMAN-UAT.md
Normal file
|
|
@ -0,0 +1,30 @@
|
||||||
|
---
|
||||||
|
status: partial
|
||||||
|
phase: 07-research-agent-search
|
||||||
|
source: [07-VERIFICATION.md]
|
||||||
|
started: 2026-04-10
|
||||||
|
updated: 2026-04-10
|
||||||
|
---
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
### 1. SearXNG live query
|
||||||
|
expected: SearXNGClient.Search returns real results from http://10.5.0.129:8080
|
||||||
|
result: [pending — needs runtime test]
|
||||||
|
|
||||||
|
### 2. NL search end-to-end
|
||||||
|
expected: Type "show me all 10GbE NICs" in dashboard, see filtered results
|
||||||
|
result: [pending — needs Gemma 4 running on Mac Mini]
|
||||||
|
|
||||||
|
### 3. Research agent enrichment cycle
|
||||||
|
expected: Item with catalog_status=needs_research is enriched with SearXNG data and advances to researched after 10min
|
||||||
|
result: [pending — needs real items + Tier 2 OpenRouter key]
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
total: 3
|
||||||
|
pending: 3
|
||||||
|
|
||||||
|
## Gaps
|
||||||
|
|
||||||
|
Requires Gemma 4 running on Mac Mini for NL search and OpenRouter key for Tier 2 enrichment.
|
||||||
36
.planning/phases/07-research-agent-search/07-VERIFICATION.md
Normal file
36
.planning/phases/07-research-agent-search/07-VERIFICATION.md
Normal file
|
|
@ -0,0 +1,36 @@
|
||||||
|
---
|
||||||
|
phase: 07-research-agent-search
|
||||||
|
verified: 2026-04-10
|
||||||
|
status: human_needed
|
||||||
|
score: 3/3 (code) — live LLM + SearXNG validation pending
|
||||||
|
overrides_applied: 0
|
||||||
|
---
|
||||||
|
|
||||||
|
# Phase 7 Verification
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
Items flagged needs_research are automatically enriched by a SearXNG research agent, and any inventory question can be answered via natural language search.
|
||||||
|
|
||||||
|
## Code-Level Verification (Complete)
|
||||||
|
|
||||||
|
| # | Success Criterion | Status | Evidence |
|
||||||
|
|---|------|--------|----------|
|
||||||
|
| 1 | needs_research items auto-enriched by SearXNG → researched | ✓ | `internal/research/agent.go` RunOnce + 10min ticker |
|
||||||
|
| 2 | Natural language search returns filtered inventory | ✓ | `internal/api/handlers/search.go` Tier1 NL→filter translation |
|
||||||
|
| 3 | SearXNG queries sanitized | ✓ | `SanitizeQuery` regex `[^a-zA-Z0-9 .\-_]+` |
|
||||||
|
|
||||||
|
## All 2 Requirements Covered
|
||||||
|
AI-04 (SearXNG research) + UI-03 (NL search) — implemented and tested.
|
||||||
|
|
||||||
|
## Test Results
|
||||||
|
- `go test ./...` — all packages pass
|
||||||
|
- `cd web && npm run build` — clean
|
||||||
|
|
||||||
|
## Human Verification Required
|
||||||
|
|
||||||
|
1. Real SearXNG end-to-end test (SearXNG is live at 10.5.0.129:8080 — no auth needed)
|
||||||
|
2. Real Gemma 4 NL→filter parsing accuracy
|
||||||
|
3. Live needs_research enrichment cycle (requires real items in needs_research state + Tier 2 OpenRouter key)
|
||||||
|
|
||||||
|
## Status
|
||||||
|
`human_needed` — code complete, requires live LLM and real inventory data for end-to-end validation.
|
||||||
|
|
@ -21,6 +21,7 @@ import (
|
||||||
"git.georgsen.dk/hwlab/internal/netbox"
|
"git.georgsen.dk/hwlab/internal/netbox"
|
||||||
"git.georgsen.dk/hwlab/internal/printer"
|
"git.georgsen.dk/hwlab/internal/printer"
|
||||||
"git.georgsen.dk/hwlab/internal/queue"
|
"git.georgsen.dk/hwlab/internal/queue"
|
||||||
|
"git.georgsen.dk/hwlab/internal/research"
|
||||||
"git.georgsen.dk/hwlab/internal/store"
|
"git.georgsen.dk/hwlab/internal/store"
|
||||||
"git.georgsen.dk/hwlab/internal/usb"
|
"git.georgsen.dk/hwlab/internal/usb"
|
||||||
)
|
)
|
||||||
|
|
@ -121,6 +122,13 @@ func main() {
|
||||||
log.Printf("HWLAB_DATABASE_URL not set — advisor endpoints disabled")
|
log.Printf("HWLAB_DATABASE_URL not set — advisor endpoints disabled")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Research agent — enriches needs_research items via SearXNG + Tier 2 LLM.
|
||||||
|
searxngClient := research.NewSearXNGClient(cfg.SearXNGURL)
|
||||||
|
researchAgent := research.NewAgent(nbClient, searxngClient, tier2, catalogUpdater)
|
||||||
|
go researchAgent.Start(ctx, 10*time.Minute)
|
||||||
|
researchHandler := handlers.NewResearchHandler(researchAgent)
|
||||||
|
searchHandler := handlers.NewSearchHandler(nbClient, tier1)
|
||||||
|
|
||||||
// Wire USB Manager events to cable tester driver when a RoleCableTester device connects.
|
// Wire USB Manager events to cable tester driver when a RoleCableTester device connects.
|
||||||
// Currently a no-op stub — wires the plumbing for Phase 5 hardware integration.
|
// Currently a no-op stub — wires the plumbing for Phase 5 hardware integration.
|
||||||
go func() {
|
go func() {
|
||||||
|
|
@ -134,7 +142,7 @@ func main() {
|
||||||
}
|
}
|
||||||
}()
|
}()
|
||||||
|
|
||||||
router := api.NewRouter(staticFS, intakeHandler, inventoryHandler, labelHandler, usbEventsHandler, testHandler, advisorHandler)
|
router := api.NewRouter(staticFS, intakeHandler, inventoryHandler, labelHandler, usbEventsHandler, testHandler, advisorHandler, researchHandler, searchHandler)
|
||||||
addr := fmt.Sprintf("%s:%d", cfg.Host, cfg.Port)
|
addr := fmt.Sprintf("%s:%d", cfg.Host, cfg.Port)
|
||||||
log.Printf("HWLab starting on %s", addr)
|
log.Printf("HWLab starting on %s", addr)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -92,6 +92,28 @@ func (c *TierClient) AnalyzePhotos(ctx context.Context, req IntakeRequest) (*Int
|
||||||
return &result, nil
|
return &result, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TextComplete sends a text-only (non-vision) chat completion to the configured model.
|
||||||
|
// Used by the research agent for hardware enrichment prompts that require no images.
|
||||||
|
// Returns the raw string content of the first response choice.
|
||||||
|
func (c *TierClient) TextComplete(ctx context.Context, prompt string) (string, error) {
|
||||||
|
tctx, cancel := context.WithTimeout(ctx, c.timeout)
|
||||||
|
defer cancel()
|
||||||
|
|
||||||
|
resp, err := c.client.CreateChatCompletion(tctx, openai.ChatCompletionRequest{
|
||||||
|
Model: c.model,
|
||||||
|
Messages: []openai.ChatCompletionMessage{
|
||||||
|
{Role: openai.ChatMessageRoleUser, Content: prompt},
|
||||||
|
},
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("text complete: %w", err)
|
||||||
|
}
|
||||||
|
if len(resp.Choices) == 0 {
|
||||||
|
return "", fmt.Errorf("text complete: no choices in response")
|
||||||
|
}
|
||||||
|
return resp.Choices[0].Message.Content, nil
|
||||||
|
}
|
||||||
|
|
||||||
// buildIntakePromptWithCount is a package-internal shim to the prompts package.
|
// buildIntakePromptWithCount is a package-internal shim to the prompts package.
|
||||||
func buildIntakePromptWithCount(n int) string {
|
func buildIntakePromptWithCount(n int) string {
|
||||||
return prompts.BuildIntakePrompt(n)
|
return prompts.BuildIntakePrompt(n)
|
||||||
|
|
|
||||||
30
internal/api/handlers/research.go
Normal file
30
internal/api/handlers/research.go
Normal file
|
|
@ -0,0 +1,30 @@
|
||||||
|
package handlers
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"net/http"
|
||||||
|
|
||||||
|
"git.georgsen.dk/hwlab/internal/research"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ResearchHandler handles research-related API endpoints.
|
||||||
|
type ResearchHandler struct {
|
||||||
|
agent *research.Agent
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewResearchHandler creates a ResearchHandler backed by the given Agent.
|
||||||
|
func NewResearchHandler(agent *research.Agent) *ResearchHandler {
|
||||||
|
return &ResearchHandler{agent: agent}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TriggerResearch handles POST /api/research/trigger.
|
||||||
|
// It fires a RunOnce cycle in a background goroutine and responds 202 Accepted immediately.
|
||||||
|
func (h *ResearchHandler) TriggerResearch(w http.ResponseWriter, r *http.Request) {
|
||||||
|
go func() {
|
||||||
|
_, _ = h.agent.RunOnce(context.Background())
|
||||||
|
}()
|
||||||
|
w.Header().Set("Content-Type", "application/json")
|
||||||
|
w.WriteHeader(http.StatusAccepted)
|
||||||
|
json.NewEncoder(w).Encode(map[string]string{"status": "accepted"})
|
||||||
|
}
|
||||||
187
internal/api/handlers/search.go
Normal file
187
internal/api/handlers/search.go
Normal file
|
|
@ -0,0 +1,187 @@
|
||||||
|
package handlers
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"log"
|
||||||
|
"net/http"
|
||||||
|
"strings"
|
||||||
|
"unicode"
|
||||||
|
|
||||||
|
"git.georgsen.dk/hwlab/internal/netbox"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SearchNetBoxClient is the narrow interface the search handler needs.
|
||||||
|
type SearchNetBoxClient interface {
|
||||||
|
ListDevices(ctx context.Context, limit int) ([]netbox.Device, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// SearchAIClient is the narrow interface for NL→filter translation.
|
||||||
|
type SearchAIClient interface {
|
||||||
|
TextComplete(ctx context.Context, prompt string) (string, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// nlFilter holds the parsed output from the LLM (T-07-06: only known fields accepted).
|
||||||
|
type nlFilter struct {
|
||||||
|
CatalogStatus string `json:"catalog_status"`
|
||||||
|
NameContains string `json:"name_contains"`
|
||||||
|
Tag string `json:"tag"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// SearchHandler handles GET /api/search?q=...
|
||||||
|
type SearchHandler struct {
|
||||||
|
nbClient SearchNetBoxClient
|
||||||
|
tier1 SearchAIClient
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewSearchHandler creates a SearchHandler.
|
||||||
|
func NewSearchHandler(nb SearchNetBoxClient, tier1 SearchAIClient) *SearchHandler {
|
||||||
|
return &SearchHandler{nbClient: nb, tier1: tier1}
|
||||||
|
}
|
||||||
|
|
||||||
|
// SearchDevices handles GET /api/search?q=<natural language query>
|
||||||
|
// It translates the query to NetBox filter params via Tier 1 LLM, then fetches
|
||||||
|
// and filters devices accordingly.
|
||||||
|
func (h *SearchHandler) SearchDevices(w http.ResponseWriter, r *http.Request) {
|
||||||
|
w.Header().Set("Content-Type", "application/json")
|
||||||
|
|
||||||
|
q := r.URL.Query().Get("q")
|
||||||
|
if strings.TrimSpace(q) == "" {
|
||||||
|
w.WriteHeader(http.StatusBadRequest)
|
||||||
|
_ = json.NewEncoder(w).Encode(map[string]string{"error": "q parameter required"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Sanitize: strip non-printable chars, truncate to 200 chars (T-07-05).
|
||||||
|
q = sanitizeQuery(q)
|
||||||
|
|
||||||
|
// Translate NL query to structured filter via Tier 1 LLM.
|
||||||
|
filter := h.parseFilter(r.Context(), q)
|
||||||
|
|
||||||
|
// Fetch up to 200 devices from NetBox.
|
||||||
|
devices, err := h.nbClient.ListDevices(r.Context(), 200)
|
||||||
|
if err != nil {
|
||||||
|
w.WriteHeader(http.StatusBadGateway)
|
||||||
|
_ = json.NewEncoder(w).Encode(map[string]string{"error": "netbox unavailable: " + err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Apply filters in-memory and serialize.
|
||||||
|
results := make([]map[string]interface{}, 0)
|
||||||
|
for _, d := range devices {
|
||||||
|
if !matchesFilter(d, filter) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
results = append(results, deviceToSearchResponse(d))
|
||||||
|
}
|
||||||
|
|
||||||
|
_ = json.NewEncoder(w).Encode(results)
|
||||||
|
}
|
||||||
|
|
||||||
|
// parseFilter calls the LLM and parses its JSON output into an nlFilter.
|
||||||
|
// On any failure, falls back to a name substring match on the raw query.
|
||||||
|
func (h *SearchHandler) parseFilter(ctx context.Context, q string) nlFilter {
|
||||||
|
resp, err := h.tier1.TextComplete(ctx, nlFilterPrompt(q))
|
||||||
|
if err != nil {
|
||||||
|
log.Printf("search: LLM TextComplete error: %v — falling back to substring match", err)
|
||||||
|
return nlFilter{NameContains: q}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Extract JSON from response (LLM may wrap JSON in markdown code fences).
|
||||||
|
jsonStr := extractJSON(resp)
|
||||||
|
|
||||||
|
var f nlFilter
|
||||||
|
if err := json.Unmarshal([]byte(jsonStr), &f); err != nil {
|
||||||
|
log.Printf("search: LLM JSON parse failed: %v — falling back to substring match (raw: %.200s)", err, resp)
|
||||||
|
return nlFilter{NameContains: q}
|
||||||
|
}
|
||||||
|
|
||||||
|
if f.Tag != "" {
|
||||||
|
log.Printf("search: tag filter %q not implemented for MVP — ignoring", f.Tag)
|
||||||
|
f.Tag = ""
|
||||||
|
}
|
||||||
|
|
||||||
|
return f
|
||||||
|
}
|
||||||
|
|
||||||
|
// matchesFilter returns true if device d satisfies all active filter criteria.
|
||||||
|
func matchesFilter(d netbox.Device, f nlFilter) bool {
|
||||||
|
// catalog_status: exact match (case-insensitive) when set (T-07-06).
|
||||||
|
if f.CatalogStatus != "" {
|
||||||
|
if !strings.EqualFold(d.CustomFields.CatalogStatus, f.CatalogStatus) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// name_contains: case-insensitive substring match when set.
|
||||||
|
if f.NameContains != "" {
|
||||||
|
if !strings.Contains(strings.ToLower(d.Name), strings.ToLower(f.NameContains)) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
// deviceToSearchResponse converts a netbox.Device to the InventoryItem JSON shape.
|
||||||
|
func deviceToSearchResponse(d netbox.Device) map[string]interface{} {
|
||||||
|
cf := d.CustomFields
|
||||||
|
urls := cf.PhotoURLs
|
||||||
|
if urls == nil {
|
||||||
|
urls = []string{}
|
||||||
|
}
|
||||||
|
var assetTag interface{} = nil
|
||||||
|
if d.AssetTag != "" {
|
||||||
|
assetTag = d.AssetTag
|
||||||
|
}
|
||||||
|
return map[string]interface{}{
|
||||||
|
"id": d.ID,
|
||||||
|
"name": d.Name,
|
||||||
|
"asset_tag": assetTag,
|
||||||
|
"hw_id": nilIfEmpty(cf.HWID),
|
||||||
|
"catalog_status": nilIfEmpty(cf.CatalogStatus),
|
||||||
|
"product_url": nilIfEmpty(cf.ProductURL),
|
||||||
|
"firmware_version": nilIfEmpty(cf.FirmwareVersion),
|
||||||
|
"test_date": nilIfEmpty(cf.TestDate),
|
||||||
|
"test_data": nilIfEmpty(cf.TestData),
|
||||||
|
"ai_notes": nilIfEmpty(cf.AINotes),
|
||||||
|
"photo_urls": urls,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func nilIfEmpty(s string) interface{} {
|
||||||
|
if s == "" {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
|
||||||
|
// nlFilterPrompt builds the LLM extraction prompt for a given user query.
|
||||||
|
func nlFilterPrompt(q string) string {
|
||||||
|
return `Extract NetBox filter parameters from this inventory search query. Return JSON only: {"catalog_status": "...", "name_contains": "...", "tag": "..."}. All fields optional. Use empty string for fields that don't apply. Valid catalog_status values: draft, indexed, needs_research, researched, complete, available. Query: ` + q
|
||||||
|
}
|
||||||
|
|
||||||
|
// sanitizeQuery strips non-printable characters and truncates to 200 chars.
|
||||||
|
func sanitizeQuery(q string) string {
|
||||||
|
var sb strings.Builder
|
||||||
|
for _, r := range q {
|
||||||
|
if unicode.IsPrint(r) {
|
||||||
|
sb.WriteRune(r)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
result := sb.String()
|
||||||
|
runes := []rune(result)
|
||||||
|
if len(runes) > 200 {
|
||||||
|
result = string(runes[:200])
|
||||||
|
}
|
||||||
|
return strings.TrimSpace(result)
|
||||||
|
}
|
||||||
|
|
||||||
|
// extractJSON attempts to extract a JSON object from a string that may contain
|
||||||
|
// markdown code fences or surrounding text.
|
||||||
|
func extractJSON(s string) string {
|
||||||
|
start := strings.Index(s, "{")
|
||||||
|
end := strings.LastIndex(s, "}")
|
||||||
|
if start >= 0 && end > start {
|
||||||
|
return s[start : end+1]
|
||||||
|
}
|
||||||
|
return s
|
||||||
|
}
|
||||||
138
internal/api/handlers/search_test.go
Normal file
138
internal/api/handlers/search_test.go
Normal file
|
|
@ -0,0 +1,138 @@
|
||||||
|
package handlers_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"git.georgsen.dk/hwlab/internal/api/handlers"
|
||||||
|
"git.georgsen.dk/hwlab/internal/netbox"
|
||||||
|
)
|
||||||
|
|
||||||
|
// mockNetBoxClient satisfies the SearchNetBoxClient interface.
|
||||||
|
type mockNetBoxClient struct {
|
||||||
|
devices []netbox.Device
|
||||||
|
err error
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *mockNetBoxClient) ListDevices(_ context.Context, _ int) ([]netbox.Device, error) {
|
||||||
|
return m.devices, m.err
|
||||||
|
}
|
||||||
|
|
||||||
|
// mockAIClient returns canned JSON for TextComplete.
|
||||||
|
type mockAIClient struct {
|
||||||
|
response string
|
||||||
|
err error
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *mockAIClient) TextComplete(_ context.Context, _ string) (string, error) {
|
||||||
|
return m.response, m.err
|
||||||
|
}
|
||||||
|
|
||||||
|
func testDevices() []netbox.Device {
|
||||||
|
return []netbox.Device{
|
||||||
|
{ID: 1, Name: "10GbE NIC Intel X550", CustomFields: netbox.CustomFields{CatalogStatus: "available", HWID: "HW-0001"}},
|
||||||
|
{ID: 2, Name: "Raspberry Pi 4B", CustomFields: netbox.CustomFields{CatalogStatus: "draft", HWID: "HW-0002"}},
|
||||||
|
{ID: 3, Name: "10GbE NIC Mellanox", CustomFields: netbox.CustomFields{CatalogStatus: "complete", HWID: "HW-0003"}},
|
||||||
|
{ID: 4, Name: "USB-C Hub", CustomFields: netbox.CustomFields{CatalogStatus: "available", HWID: "HW-0004"}},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestSearch_EmptyQ verifies that missing q parameter returns 400.
|
||||||
|
func TestSearch_EmptyQ(t *testing.T) {
|
||||||
|
h := handlers.NewSearchHandler(&mockNetBoxClient{}, &mockAIClient{})
|
||||||
|
req := httptest.NewRequest(http.MethodGet, "/api/search", nil)
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
h.SearchDevices(w, req)
|
||||||
|
|
||||||
|
if w.Code != http.StatusBadRequest {
|
||||||
|
t.Fatalf("expected 400, got %d", w.Code)
|
||||||
|
}
|
||||||
|
var body map[string]string
|
||||||
|
if err := json.NewDecoder(w.Body).Decode(&body); err != nil {
|
||||||
|
t.Fatalf("decode: %v", err)
|
||||||
|
}
|
||||||
|
if !strings.Contains(body["error"], "q parameter") {
|
||||||
|
t.Errorf("expected error about q parameter, got %q", body["error"])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestSearch_LLMParseFallback verifies that when LLM returns unparseable JSON,
|
||||||
|
// the handler falls back to name substring match against the raw query.
|
||||||
|
func TestSearch_LLMParseFallback(t *testing.T) {
|
||||||
|
nb := &mockNetBoxClient{devices: testDevices()}
|
||||||
|
ai := &mockAIClient{response: "not valid json {{ broken"}
|
||||||
|
h := handlers.NewSearchHandler(nb, ai)
|
||||||
|
|
||||||
|
req := httptest.NewRequest(http.MethodGet, "/api/search?q=NIC", nil)
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
h.SearchDevices(w, req)
|
||||||
|
|
||||||
|
if w.Code != http.StatusOK {
|
||||||
|
t.Fatalf("expected 200, got %d (body: %s)", w.Code, w.Body.String())
|
||||||
|
}
|
||||||
|
var items []map[string]interface{}
|
||||||
|
if err := json.NewDecoder(w.Body).Decode(&items); err != nil {
|
||||||
|
t.Fatalf("decode: %v", err)
|
||||||
|
}
|
||||||
|
// "NIC" substring matches devices 1 and 3 (both contain "NIC")
|
||||||
|
if len(items) != 2 {
|
||||||
|
t.Errorf("expected 2 NIC results, got %d: %v", len(items), items)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestSearch_CatalogStatusFilter verifies that LLM-extracted catalog_status filters results.
|
||||||
|
func TestSearch_CatalogStatusFilter(t *testing.T) {
|
||||||
|
nb := &mockNetBoxClient{devices: testDevices()}
|
||||||
|
aiResp := `{"catalog_status": "available", "name_contains": "", "tag": ""}`
|
||||||
|
ai := &mockAIClient{response: aiResp}
|
||||||
|
h := handlers.NewSearchHandler(nb, ai)
|
||||||
|
|
||||||
|
req := httptest.NewRequest(http.MethodGet, "/api/search?q=show+me+available+items", nil)
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
h.SearchDevices(w, req)
|
||||||
|
|
||||||
|
if w.Code != http.StatusOK {
|
||||||
|
t.Fatalf("expected 200, got %d (body: %s)", w.Code, w.Body.String())
|
||||||
|
}
|
||||||
|
var items []map[string]interface{}
|
||||||
|
if err := json.NewDecoder(w.Body).Decode(&items); err != nil {
|
||||||
|
t.Fatalf("decode: %v", err)
|
||||||
|
}
|
||||||
|
// Devices 1 and 4 have catalog_status "available"
|
||||||
|
if len(items) != 2 {
|
||||||
|
t.Errorf("expected 2 available items, got %d", len(items))
|
||||||
|
}
|
||||||
|
for _, item := range items {
|
||||||
|
if item["catalog_status"] != "available" {
|
||||||
|
t.Errorf("unexpected catalog_status: %v", item["catalog_status"])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestSearch_NameContainsAndStatus verifies combined filtering works.
|
||||||
|
func TestSearch_NameContainsAndStatus(t *testing.T) {
|
||||||
|
nb := &mockNetBoxClient{devices: testDevices()}
|
||||||
|
aiResp := `{"catalog_status": "available", "name_contains": "NIC", "tag": ""}`
|
||||||
|
ai := &mockAIClient{response: aiResp}
|
||||||
|
h := handlers.NewSearchHandler(nb, ai)
|
||||||
|
|
||||||
|
req := httptest.NewRequest(http.MethodGet, "/api/search?q=available+NICs", nil)
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
h.SearchDevices(w, req)
|
||||||
|
|
||||||
|
if w.Code != http.StatusOK {
|
||||||
|
t.Fatalf("expected 200, got %d", w.Code)
|
||||||
|
}
|
||||||
|
var items []map[string]interface{}
|
||||||
|
if err := json.NewDecoder(w.Body).Decode(&items); err != nil {
|
||||||
|
t.Fatalf("decode: %v", err)
|
||||||
|
}
|
||||||
|
// Only device 1: NIC + available (device 3 is "complete")
|
||||||
|
if len(items) != 1 {
|
||||||
|
t.Errorf("expected 1 result, got %d", len(items))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -39,6 +39,8 @@ func (h spaHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||||
// usbEventsHandler handles GET /api/usb/events (SSE stream).
|
// usbEventsHandler handles GET /api/usb/events (SSE stream).
|
||||||
// testHandler handles POST /api/test/cable, GET /api/test/events, GET /api/test/recent.
|
// testHandler handles POST /api/test/cable, GET /api/test/events, GET /api/test/recent.
|
||||||
// advisorHandler handles POST /api/advisor/chat, GET /api/advisor/conversations, GET /api/advisor/conversations/{id}.
|
// advisorHandler handles POST /api/advisor/chat, GET /api/advisor/conversations, GET /api/advisor/conversations/{id}.
|
||||||
|
// researchHandler handles POST /api/research/trigger.
|
||||||
|
// searchHandler handles GET /api/search?q=...
|
||||||
func NewRouter(
|
func NewRouter(
|
||||||
staticFiles fs.FS,
|
staticFiles fs.FS,
|
||||||
intakeHandler http.Handler,
|
intakeHandler http.Handler,
|
||||||
|
|
@ -47,6 +49,8 @@ func NewRouter(
|
||||||
usbEventsHandler *handlers.USBEventsHandler,
|
usbEventsHandler *handlers.USBEventsHandler,
|
||||||
testHandler *handlers.TestHandler,
|
testHandler *handlers.TestHandler,
|
||||||
advisorHandler *advisor.AdvisorHandler,
|
advisorHandler *advisor.AdvisorHandler,
|
||||||
|
researchHandler *handlers.ResearchHandler,
|
||||||
|
searchHandler *handlers.SearchHandler,
|
||||||
) http.Handler {
|
) http.Handler {
|
||||||
r := chi.NewRouter()
|
r := chi.NewRouter()
|
||||||
r.Use(middleware.Logger)
|
r.Use(middleware.Logger)
|
||||||
|
|
@ -78,6 +82,24 @@ func NewRouter(
|
||||||
r.Get("/conversations/{id}", unavailable)
|
r.Get("/conversations/{id}", unavailable)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|
||||||
|
r.Route("/research", func(r chi.Router) {
|
||||||
|
if researchHandler != nil {
|
||||||
|
r.Post("/trigger", researchHandler.TriggerResearch)
|
||||||
|
} else {
|
||||||
|
r.Post("/trigger", func(w http.ResponseWriter, _ *http.Request) {
|
||||||
|
http.Error(w, "research unavailable", http.StatusServiceUnavailable)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
if searchHandler != nil {
|
||||||
|
r.Get("/search", searchHandler.SearchDevices)
|
||||||
|
} else {
|
||||||
|
r.Get("/search", func(w http.ResponseWriter, _ *http.Request) {
|
||||||
|
http.Error(w, "search unavailable", http.StatusServiceUnavailable)
|
||||||
|
})
|
||||||
|
}
|
||||||
})
|
})
|
||||||
|
|
||||||
// SPA fallback — serve static files; unknown paths fall back to index.html.
|
// SPA fallback — serve static files; unknown paths fall back to index.html.
|
||||||
|
|
|
||||||
|
|
@ -29,6 +29,8 @@ type Config struct {
|
||||||
NetBoxDefaultSiteID int32 `mapstructure:"netbox_default_site_id"`
|
NetBoxDefaultSiteID int32 `mapstructure:"netbox_default_site_id"`
|
||||||
|
|
||||||
AI ai.AIConfig `mapstructure:"ai"`
|
AI ai.AIConfig `mapstructure:"ai"`
|
||||||
|
|
||||||
|
SearXNGURL string `mapstructure:"searxng_url"`
|
||||||
}
|
}
|
||||||
|
|
||||||
func Load() (*Config, error) {
|
func Load() (*Config, error) {
|
||||||
|
|
@ -65,6 +67,7 @@ func Load() (*Config, error) {
|
||||||
v.SetDefault("ai.confidence_threshold", 0.75)
|
v.SetDefault("ai.confidence_threshold", 0.75)
|
||||||
v.SetDefault("ai.quick_add_enabled", false)
|
v.SetDefault("ai.quick_add_enabled", false)
|
||||||
v.SetDefault("ai.quick_add_threshold", 0.90)
|
v.SetDefault("ai.quick_add_threshold", 0.90)
|
||||||
|
v.SetDefault("searxng_url", "http://10.5.0.129:8080")
|
||||||
|
|
||||||
// Config file
|
// Config file
|
||||||
v.SetConfigName("config")
|
v.SetConfigName("config")
|
||||||
|
|
@ -103,6 +106,7 @@ func Load() (*Config, error) {
|
||||||
_ = v.BindEnv("ai.tier3.model", "HWLAB_AI_TIER3_MODEL")
|
_ = v.BindEnv("ai.tier3.model", "HWLAB_AI_TIER3_MODEL")
|
||||||
_ = v.BindEnv("ai.confidence_threshold", "HWLAB_AI_CONFIDENCE_THRESHOLD")
|
_ = v.BindEnv("ai.confidence_threshold", "HWLAB_AI_CONFIDENCE_THRESHOLD")
|
||||||
_ = v.BindEnv("ai.quick_add_enabled", "HWLAB_AI_QUICK_ADD_ENABLED")
|
_ = v.BindEnv("ai.quick_add_enabled", "HWLAB_AI_QUICK_ADD_ENABLED")
|
||||||
|
_ = v.BindEnv("searxng_url", "HWLAB_SEARXNG_URL")
|
||||||
|
|
||||||
// Read primary config file (non-fatal if missing)
|
// Read primary config file (non-fatal if missing)
|
||||||
if err := v.ReadInConfig(); err != nil {
|
if err := v.ReadInConfig(); err != nil {
|
||||||
|
|
|
||||||
|
|
@ -122,6 +122,24 @@ func (c *Client) CreateCable(ctx context.Context, label, assetTag, testDataJSON
|
||||||
return int64(result.GetId()), nil
|
return int64(result.GetId()), nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ListDevicesWithStatus returns devices whose catalog_status custom field equals status.
|
||||||
|
// Uses client-side filtering (up to 200 devices) since go-netbox v4 custom field
|
||||||
|
// query param support is schema-dependent.
|
||||||
|
func (c *Client) ListDevicesWithStatus(ctx context.Context, status string) ([]Device, error) {
|
||||||
|
res, _, err := c.api.DcimAPI.DcimDevicesList(ctx).Limit(200).Execute()
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("list devices for status %q: %w", status, err)
|
||||||
|
}
|
||||||
|
devices := make([]Device, 0)
|
||||||
|
for _, d := range res.Results {
|
||||||
|
dev := deviceFromNetBox(d)
|
||||||
|
if dev.CustomFields.CatalogStatus == status {
|
||||||
|
devices = append(devices, dev)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return devices, nil
|
||||||
|
}
|
||||||
|
|
||||||
// DeleteDevice removes a device from NetBox by its internal ID.
|
// DeleteDevice removes a device from NetBox by its internal ID.
|
||||||
// Used primarily for test cleanup after CreateDevice integration tests.
|
// Used primarily for test cleanup after CreateDevice integration tests.
|
||||||
func (c *Client) DeleteDevice(ctx context.Context, id int64) error {
|
func (c *Client) DeleteDevice(ctx context.Context, id int64) error {
|
||||||
|
|
|
||||||
185
internal/research/agent.go
Normal file
185
internal/research/agent.go
Normal file
|
|
@ -0,0 +1,185 @@
|
||||||
|
package research
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"log"
|
||||||
|
"regexp"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"git.georgsen.dk/hwlab/internal/ai"
|
||||||
|
"git.georgsen.dk/hwlab/internal/inventory"
|
||||||
|
"git.georgsen.dk/hwlab/internal/netbox"
|
||||||
|
)
|
||||||
|
|
||||||
|
// nonSafeChars matches characters that are not safe to send to SearXNG.
|
||||||
|
// Allowed: alphanumeric, space, dot, dash, underscore.
|
||||||
|
var nonSafeChars = regexp.MustCompile(`[^a-zA-Z0-9 .\-_]+`)
|
||||||
|
|
||||||
|
// SanitizeQuery strips unsafe characters from a search query string.
|
||||||
|
// Exported so it can be tested from the _test package.
|
||||||
|
func SanitizeQuery(s string) string {
|
||||||
|
sanitized := nonSafeChars.ReplaceAllString(s, " ")
|
||||||
|
return strings.TrimSpace(sanitized)
|
||||||
|
}
|
||||||
|
|
||||||
|
// NetBoxer is the subset of netbox.Client used by the Agent.
|
||||||
|
// Using an interface allows stub injection in tests.
|
||||||
|
type NetBoxer interface {
|
||||||
|
ListDevicesWithStatus(ctx context.Context, status string) ([]netbox.Device, error)
|
||||||
|
PatchCustomFields(ctx context.Context, deviceID int64, patch map[string]interface{}) error
|
||||||
|
}
|
||||||
|
|
||||||
|
// TextCompleter is the subset of ai.TierClient used by the Agent for text-only LLM calls.
|
||||||
|
type TextCompleter interface {
|
||||||
|
TextComplete(ctx context.Context, prompt string) (string, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// CatalogTransitioner is the subset of inventory.CatalogUpdater used by the Agent.
|
||||||
|
type CatalogTransitioner interface {
|
||||||
|
UpdateCatalogStatus(ctx context.Context, deviceID int64, current, next inventory.CatalogStatus) (inventory.CatalogStatus, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Agent is the background worker that enriches needs_research hardware items.
|
||||||
|
// It polls NetBox, searches SearXNG, calls a Tier 2 LLM, and transitions items to researched.
|
||||||
|
type Agent struct {
|
||||||
|
nbClient NetBoxer
|
||||||
|
researchClient ai.ResearchClient
|
||||||
|
llm TextCompleter
|
||||||
|
updater CatalogTransitioner
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewAgent creates an Agent. All arguments must be non-nil.
|
||||||
|
func NewAgent(nb NetBoxer, rc ai.ResearchClient, llm TextCompleter, updater CatalogTransitioner) *Agent {
|
||||||
|
return &Agent{
|
||||||
|
nbClient: nb,
|
||||||
|
researchClient: rc,
|
||||||
|
llm: llm,
|
||||||
|
updater: updater,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// enrichmentResponse is the expected JSON structure from the Tier 2 LLM.
|
||||||
|
type enrichmentResponse struct {
|
||||||
|
AINotes string `json:"ai_notes"`
|
||||||
|
ProductURL string `json:"product_url"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// RunOnce performs a single research cycle: finds all needs_research devices,
|
||||||
|
// enriches each via SearXNG + LLM, patches NetBox custom fields, and transitions
|
||||||
|
// the catalog status to researched. Returns the number of items enriched.
|
||||||
|
func (a *Agent) RunOnce(ctx context.Context) (int, error) {
|
||||||
|
devices, err := a.nbClient.ListDevicesWithStatus(ctx, string(inventory.StatusNeedsResearch))
|
||||||
|
if err != nil {
|
||||||
|
return 0, fmt.Errorf("research agent: list needs_research devices: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
enriched := 0
|
||||||
|
for _, dev := range devices {
|
||||||
|
query := SanitizeQuery(dev.Name)
|
||||||
|
if query == "" {
|
||||||
|
log.Printf("research agent: device %d has empty name after sanitization, skipping", dev.ID)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
results, err := a.researchClient.Search(ctx, query)
|
||||||
|
if err != nil {
|
||||||
|
log.Printf("research agent: search error for device %d (%q): %v", dev.ID, dev.Name, err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if len(results) == 0 {
|
||||||
|
log.Printf("research agent: no SearXNG results for device %d (%q), skipping", dev.ID, dev.Name)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
// Build enrichment prompt using top 3 results
|
||||||
|
top := results
|
||||||
|
if len(top) > 3 {
|
||||||
|
top = top[:3]
|
||||||
|
}
|
||||||
|
var sb strings.Builder
|
||||||
|
for i, r := range top {
|
||||||
|
sb.WriteString(fmt.Sprintf("%d. %s\n %s\n URL: %s\n", i+1, r.Title, r.Snippet, r.URL))
|
||||||
|
}
|
||||||
|
prompt := fmt.Sprintf(
|
||||||
|
"You are enriching a hardware inventory record.\nItem: %s\nSearch results:\n%s\nReturn JSON: {\"ai_notes\": \"...\", \"product_url\": \"...\"}",
|
||||||
|
dev.Name, sb.String(),
|
||||||
|
)
|
||||||
|
|
||||||
|
rawResponse, err := a.llm.TextComplete(ctx, prompt)
|
||||||
|
if err != nil {
|
||||||
|
log.Printf("research agent: LLM error for device %d (%q): %v", dev.ID, dev.Name, err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse LLM JSON response — extract ai_notes and product_url
|
||||||
|
var enrichResp enrichmentResponse
|
||||||
|
if parseErr := json.Unmarshal([]byte(rawResponse), &enrichResp); parseErr != nil {
|
||||||
|
log.Printf("research agent: LLM non-JSON for device %d: %v (raw: %.100s)", dev.ID, parseErr, rawResponse)
|
||||||
|
// Use raw response as ai_notes fallback
|
||||||
|
enrichResp.AINotes = rawResponse
|
||||||
|
}
|
||||||
|
|
||||||
|
// Patch NetBox custom fields with enrichment data
|
||||||
|
patch := map[string]interface{}{}
|
||||||
|
if enrichResp.AINotes != "" {
|
||||||
|
patch["ai_notes"] = enrichResp.AINotes
|
||||||
|
}
|
||||||
|
if enrichResp.ProductURL == "" && len(results) > 0 {
|
||||||
|
// Fall back to first SearXNG result URL if LLM didn't provide one
|
||||||
|
enrichResp.ProductURL = results[0].URL
|
||||||
|
}
|
||||||
|
if enrichResp.ProductURL != "" {
|
||||||
|
patch["product_url"] = enrichResp.ProductURL
|
||||||
|
}
|
||||||
|
if len(patch) > 0 {
|
||||||
|
if patchErr := a.nbClient.PatchCustomFields(ctx, int64(dev.ID), patch); patchErr != nil {
|
||||||
|
log.Printf("research agent: patch error for device %d: %v", dev.ID, patchErr)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Transition catalog status: needs_research -> researched
|
||||||
|
if _, transErr := a.updater.UpdateCatalogStatus(ctx, int64(dev.ID),
|
||||||
|
inventory.StatusNeedsResearch, inventory.StatusResearched); transErr != nil {
|
||||||
|
log.Printf("research agent: status transition error for device %d: %v", dev.ID, transErr)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
enriched++
|
||||||
|
}
|
||||||
|
|
||||||
|
return enriched, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Start runs the research agent on the given interval until ctx is cancelled.
|
||||||
|
// RunOnce is called immediately on start, then on each tick.
|
||||||
|
func (a *Agent) Start(ctx context.Context, interval time.Duration) {
|
||||||
|
log.Printf("research agent: starting, interval=%v", interval)
|
||||||
|
|
||||||
|
// Run immediately on startup
|
||||||
|
if n, err := a.RunOnce(ctx); err != nil {
|
||||||
|
log.Printf("research agent: initial cycle error: %v", err)
|
||||||
|
} else {
|
||||||
|
log.Printf("research agent: cycle complete, enriched %d items", n)
|
||||||
|
}
|
||||||
|
|
||||||
|
ticker := time.NewTicker(interval)
|
||||||
|
defer ticker.Stop()
|
||||||
|
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
log.Printf("research agent: shutting down")
|
||||||
|
return
|
||||||
|
case <-ticker.C:
|
||||||
|
if n, err := a.RunOnce(ctx); err != nil {
|
||||||
|
log.Printf("research agent: cycle error: %v", err)
|
||||||
|
} else {
|
||||||
|
log.Printf("research agent: cycle complete, enriched %d items", n)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
180
internal/research/agent_test.go
Normal file
180
internal/research/agent_test.go
Normal file
|
|
@ -0,0 +1,180 @@
|
||||||
|
package research_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"git.georgsen.dk/hwlab/internal/ai"
|
||||||
|
"git.georgsen.dk/hwlab/internal/inventory"
|
||||||
|
"git.georgsen.dk/hwlab/internal/netbox"
|
||||||
|
"git.georgsen.dk/hwlab/internal/research"
|
||||||
|
)
|
||||||
|
|
||||||
|
// --- Stubs ---
|
||||||
|
|
||||||
|
// stubResearchClient returns canned SearchResults.
|
||||||
|
type stubResearchClient struct {
|
||||||
|
results []ai.SearchResult
|
||||||
|
err error
|
||||||
|
calls []string
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *stubResearchClient) Search(_ context.Context, query string) ([]ai.SearchResult, error) {
|
||||||
|
s.calls = append(s.calls, query)
|
||||||
|
return s.results, s.err
|
||||||
|
}
|
||||||
|
|
||||||
|
// stubTextCompleter returns a canned LLM response text.
|
||||||
|
type stubTextCompleter struct {
|
||||||
|
response string
|
||||||
|
err error
|
||||||
|
calls []string
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *stubTextCompleter) TextComplete(_ context.Context, prompt string) (string, error) {
|
||||||
|
s.calls = append(s.calls, prompt)
|
||||||
|
return s.response, s.err
|
||||||
|
}
|
||||||
|
|
||||||
|
// stubNetBoxClient satisfies the research.NetBoxer interface used by Agent.
|
||||||
|
type stubNetBoxClient struct {
|
||||||
|
devices []netbox.Device
|
||||||
|
patches map[int64]map[string]interface{}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *stubNetBoxClient) ListDevicesWithStatus(_ context.Context, status string) ([]netbox.Device, error) {
|
||||||
|
return s.devices, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *stubNetBoxClient) PatchCustomFields(_ context.Context, deviceID int64, patch map[string]interface{}) error {
|
||||||
|
if s.patches == nil {
|
||||||
|
s.patches = make(map[int64]map[string]interface{})
|
||||||
|
}
|
||||||
|
s.patches[deviceID] = patch
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// stubCatalogUpdater records transitions.
|
||||||
|
type stubCatalogUpdater struct {
|
||||||
|
transitions []struct {
|
||||||
|
id int64
|
||||||
|
current inventory.CatalogStatus
|
||||||
|
next inventory.CatalogStatus
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *stubCatalogUpdater) UpdateCatalogStatus(_ context.Context, deviceID int64, current, next inventory.CatalogStatus) (inventory.CatalogStatus, error) {
|
||||||
|
s.transitions = append(s.transitions, struct {
|
||||||
|
id int64
|
||||||
|
current inventory.CatalogStatus
|
||||||
|
next inventory.CatalogStatus
|
||||||
|
}{deviceID, current, next})
|
||||||
|
return next, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- Tests ---
|
||||||
|
|
||||||
|
func TestSanitizeQuery(t *testing.T) {
|
||||||
|
cases := []struct {
|
||||||
|
input string
|
||||||
|
expected string
|
||||||
|
}{
|
||||||
|
{"Intel NIC i350", "Intel NIC i350"},
|
||||||
|
{"Dell<script>alert(1)</script>", "Dell script alert 1 script"},
|
||||||
|
{"HP ProLiant DL380 Gen9", "HP ProLiant DL380 Gen9"},
|
||||||
|
{" trim ", "trim"},
|
||||||
|
{"special!@#$%chars", "special chars"},
|
||||||
|
{"dots.and-dashes_ok", "dots.and-dashes_ok"},
|
||||||
|
}
|
||||||
|
for _, tc := range cases {
|
||||||
|
got := research.SanitizeQuery(tc.input)
|
||||||
|
if got != tc.expected {
|
||||||
|
t.Errorf("SanitizeQuery(%q) = %q, want %q", tc.input, got, tc.expected)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRunOnce_EnrichesDevice(t *testing.T) {
|
||||||
|
nb := &stubNetBoxClient{
|
||||||
|
devices: []netbox.Device{
|
||||||
|
{
|
||||||
|
ID: 42,
|
||||||
|
Name: "Intel i350 NIC",
|
||||||
|
CustomFields: netbox.CustomFields{
|
||||||
|
CatalogStatus: "needs_research",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
rc := &stubResearchClient{
|
||||||
|
results: []ai.SearchResult{
|
||||||
|
{Title: "Intel i350", URL: "https://ark.intel.com", Snippet: "Quad-port GbE"},
|
||||||
|
{Title: "Datasheet", URL: "https://intel.com/ds", Snippet: "Technical specs"},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
llm := &stubTextCompleter{
|
||||||
|
response: `{"ai_notes": "Intel i350 quad-port GbE adapter", "product_url": "https://ark.intel.com"}`,
|
||||||
|
}
|
||||||
|
updater := &stubCatalogUpdater{}
|
||||||
|
|
||||||
|
agent := research.NewAgent(nb, rc, llm, updater)
|
||||||
|
enriched, err := agent.RunOnce(context.Background())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("RunOnce error: %v", err)
|
||||||
|
}
|
||||||
|
if enriched != 1 {
|
||||||
|
t.Errorf("expected enriched=1, got %d", enriched)
|
||||||
|
}
|
||||||
|
if len(updater.transitions) != 1 {
|
||||||
|
t.Fatalf("expected 1 status transition, got %d", len(updater.transitions))
|
||||||
|
}
|
||||||
|
tr := updater.transitions[0]
|
||||||
|
if tr.id != 42 {
|
||||||
|
t.Errorf("expected device id=42, got %d", tr.id)
|
||||||
|
}
|
||||||
|
if tr.current != inventory.StatusNeedsResearch {
|
||||||
|
t.Errorf("unexpected current status: %s", tr.current)
|
||||||
|
}
|
||||||
|
if tr.next != inventory.StatusResearched {
|
||||||
|
t.Errorf("unexpected next status: %s", tr.next)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRunOnce_SkipsDeviceWithNoResults(t *testing.T) {
|
||||||
|
nb := &stubNetBoxClient{
|
||||||
|
devices: []netbox.Device{
|
||||||
|
{ID: 10, Name: "Mystery Device", CustomFields: netbox.CustomFields{CatalogStatus: "needs_research"}},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
rc := &stubResearchClient{results: []ai.SearchResult{}} // empty
|
||||||
|
llm := &stubTextCompleter{}
|
||||||
|
updater := &stubCatalogUpdater{}
|
||||||
|
|
||||||
|
agent := research.NewAgent(nb, rc, llm, updater)
|
||||||
|
enriched, err := agent.RunOnce(context.Background())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("RunOnce error: %v", err)
|
||||||
|
}
|
||||||
|
if enriched != 0 {
|
||||||
|
t.Errorf("expected enriched=0 (skipped), got %d", enriched)
|
||||||
|
}
|
||||||
|
if len(updater.transitions) != 0 {
|
||||||
|
t.Errorf("expected 0 transitions (device skipped), got %d", len(updater.transitions))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRunOnce_NoDevices(t *testing.T) {
|
||||||
|
nb := &stubNetBoxClient{devices: []netbox.Device{}}
|
||||||
|
rc := &stubResearchClient{}
|
||||||
|
llm := &stubTextCompleter{}
|
||||||
|
updater := &stubCatalogUpdater{}
|
||||||
|
|
||||||
|
agent := research.NewAgent(nb, rc, llm, updater)
|
||||||
|
enriched, err := agent.RunOnce(context.Background())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("unexpected error: %v", err)
|
||||||
|
}
|
||||||
|
if enriched != 0 {
|
||||||
|
t.Errorf("expected 0 enriched, got %d", enriched)
|
||||||
|
}
|
||||||
|
}
|
||||||
88
internal/research/searxng.go
Normal file
88
internal/research/searxng.go
Normal file
|
|
@ -0,0 +1,88 @@
|
||||||
|
// Package research provides the SearXNG HTTP search client and the ResearchAgent
|
||||||
|
// background worker that enriches needs_research hardware records.
|
||||||
|
package research
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
"net/url"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"encoding/json"
|
||||||
|
|
||||||
|
"git.georgsen.dk/hwlab/internal/ai"
|
||||||
|
)
|
||||||
|
|
||||||
|
const defaultSearXNGURL = "http://10.5.0.129:8080"
|
||||||
|
|
||||||
|
// searxngResponse is the parsed JSON body returned by SearXNG.
|
||||||
|
// SearXNG uses "content" for the text snippet, not "snippet".
|
||||||
|
type searxngResponse struct {
|
||||||
|
Results []searxngResult `json:"results"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type searxngResult struct {
|
||||||
|
Title string `json:"title"`
|
||||||
|
URL string `json:"url"`
|
||||||
|
Content string `json:"content"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// SearXNGClient implements ai.ResearchClient using a self-hosted SearXNG instance.
|
||||||
|
type SearXNGClient struct {
|
||||||
|
baseURL string
|
||||||
|
httpClient *http.Client
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewSearXNGClient creates a SearXNGClient. If baseURL is empty the default LAN
|
||||||
|
// address (http://10.5.0.129:8080) is used.
|
||||||
|
func NewSearXNGClient(baseURL string) *SearXNGClient {
|
||||||
|
if baseURL == "" {
|
||||||
|
baseURL = defaultSearXNGURL
|
||||||
|
}
|
||||||
|
return &SearXNGClient{
|
||||||
|
baseURL: baseURL,
|
||||||
|
httpClient: &http.Client{
|
||||||
|
Timeout: 15 * time.Second,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Search executes a GET {baseURL}/search?q={query}&format=json and returns parsed results.
|
||||||
|
// An HTTP non-2xx response is returned as an error. An empty results array is not an error.
|
||||||
|
func (c *SearXNGClient) Search(ctx context.Context, query string) ([]ai.SearchResult, error) {
|
||||||
|
params := url.Values{}
|
||||||
|
params.Set("q", query)
|
||||||
|
params.Set("format", "json")
|
||||||
|
reqURL := c.baseURL + "/search?" + params.Encode()
|
||||||
|
|
||||||
|
req, err := http.NewRequestWithContext(ctx, http.MethodGet, reqURL, nil)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("searxng: build request: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
resp, err := c.httpClient.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("searxng: http get: %w", err)
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
|
||||||
|
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
|
||||||
|
return nil, fmt.Errorf("searxng: unexpected status %d", resp.StatusCode)
|
||||||
|
}
|
||||||
|
|
||||||
|
var body searxngResponse
|
||||||
|
if err := json.NewDecoder(resp.Body).Decode(&body); err != nil {
|
||||||
|
return nil, fmt.Errorf("searxng: decode response: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
results := make([]ai.SearchResult, 0, len(body.Results))
|
||||||
|
for _, r := range body.Results {
|
||||||
|
results = append(results, ai.SearchResult{
|
||||||
|
Title: r.Title,
|
||||||
|
URL: r.URL,
|
||||||
|
Snippet: r.Content,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
return results, nil
|
||||||
|
}
|
||||||
111
internal/research/searxng_test.go
Normal file
111
internal/research/searxng_test.go
Normal file
|
|
@ -0,0 +1,111 @@
|
||||||
|
package research_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"git.georgsen.dk/hwlab/internal/research"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestSearXNGSearch_ValidResponse(t *testing.T) {
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.URL.Path != "/search" {
|
||||||
|
t.Errorf("unexpected path: %s", r.URL.Path)
|
||||||
|
}
|
||||||
|
if r.URL.Query().Get("format") != "json" {
|
||||||
|
t.Errorf("expected format=json, got %s", r.URL.Query().Get("format"))
|
||||||
|
}
|
||||||
|
if r.URL.Query().Get("q") == "" {
|
||||||
|
t.Error("expected non-empty q param")
|
||||||
|
}
|
||||||
|
resp := map[string]interface{}{
|
||||||
|
"results": []map[string]interface{}{
|
||||||
|
{"title": "Intel i350 NIC", "url": "https://ark.intel.com/i350", "content": "Quad-port Gigabit Ethernet adapter"},
|
||||||
|
{"title": "Intel i350 Datasheet", "url": "https://intel.com/datasheet", "content": "Technical specs for i350 series"},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
w.Header().Set("Content-Type", "application/json")
|
||||||
|
json.NewEncoder(w).Encode(resp)
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
client := research.NewSearXNGClient(srv.URL)
|
||||||
|
results, err := client.Search(context.Background(), "Intel NIC i350")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("unexpected error: %v", err)
|
||||||
|
}
|
||||||
|
if len(results) != 2 {
|
||||||
|
t.Fatalf("expected 2 results, got %d", len(results))
|
||||||
|
}
|
||||||
|
if results[0].Title != "Intel i350 NIC" {
|
||||||
|
t.Errorf("unexpected title: %s", results[0].Title)
|
||||||
|
}
|
||||||
|
if results[0].URL != "https://ark.intel.com/i350" {
|
||||||
|
t.Errorf("unexpected URL: %s", results[0].URL)
|
||||||
|
}
|
||||||
|
if results[0].Snippet != "Quad-port Gigabit Ethernet adapter" {
|
||||||
|
t.Errorf("unexpected snippet (content): %s", results[0].Snippet)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSearXNGSearch_HTTP500(t *testing.T) {
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
w.WriteHeader(http.StatusInternalServerError)
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
client := research.NewSearXNGClient(srv.URL)
|
||||||
|
_, err := client.Search(context.Background(), "test query")
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected error for HTTP 500, got nil")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSearXNGSearch_EmptyResults(t *testing.T) {
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
resp := map[string]interface{}{"results": []interface{}{}}
|
||||||
|
w.Header().Set("Content-Type", "application/json")
|
||||||
|
json.NewEncoder(w).Encode(resp)
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
client := research.NewSearXNGClient(srv.URL)
|
||||||
|
results, err := client.Search(context.Background(), "something obscure")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("unexpected error: %v", err)
|
||||||
|
}
|
||||||
|
if len(results) != 0 {
|
||||||
|
t.Errorf("expected 0 results, got %d", len(results))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSearXNGSearch_QueryEncoding(t *testing.T) {
|
||||||
|
var capturedQuery string
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
capturedQuery = r.URL.Query().Get("q")
|
||||||
|
resp := map[string]interface{}{"results": []interface{}{}}
|
||||||
|
w.Header().Set("Content-Type", "application/json")
|
||||||
|
json.NewEncoder(w).Encode(resp)
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
client := research.NewSearXNGClient(srv.URL)
|
||||||
|
_, err := client.Search(context.Background(), "Intel NIC i350 2.5G")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("unexpected error: %v", err)
|
||||||
|
}
|
||||||
|
if capturedQuery != "Intel NIC i350 2.5G" {
|
||||||
|
t.Errorf("unexpected decoded query: %q", capturedQuery)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNewSearXNGClient_DefaultURL(t *testing.T) {
|
||||||
|
// Empty baseURL should use the default LAN address
|
||||||
|
client := research.NewSearXNGClient("")
|
||||||
|
if client == nil {
|
||||||
|
t.Fatal("expected non-nil client")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -1,4 +1,5 @@
|
||||||
import { Search, LayoutGrid, List } from 'lucide-react'
|
import { useState, useEffect } from 'react'
|
||||||
|
import { Search, LayoutGrid, List, Loader2, Sparkles } from 'lucide-react'
|
||||||
import { Button } from '@/components/ui/button'
|
import { Button } from '@/components/ui/button'
|
||||||
import { useUIStore } from '@/store/ui'
|
import { useUIStore } from '@/store/ui'
|
||||||
|
|
||||||
|
|
@ -8,6 +9,9 @@ interface FilterBarProps {
|
||||||
statusFilter: string
|
statusFilter: string
|
||||||
onStatusChange: (v: string) => void
|
onStatusChange: (v: string) => void
|
||||||
totalCount: number
|
totalCount: number
|
||||||
|
nlQuery: string
|
||||||
|
onNlQueryChange: (v: string) => void
|
||||||
|
nlSearchLoading: boolean
|
||||||
}
|
}
|
||||||
|
|
||||||
const STATUSES = ['', 'draft', 'indexed', 'needs_research', 'researched', 'complete']
|
const STATUSES = ['', 'draft', 'indexed', 'needs_research', 'researched', 'complete']
|
||||||
|
|
@ -20,12 +24,39 @@ const STATUS_LABELS: Record<string, string> = {
|
||||||
complete: 'Complete',
|
complete: 'Complete',
|
||||||
}
|
}
|
||||||
|
|
||||||
export function FilterBar({ search, onSearchChange, statusFilter, onStatusChange, totalCount }: FilterBarProps) {
|
export function FilterBar({
|
||||||
|
search,
|
||||||
|
onSearchChange,
|
||||||
|
statusFilter,
|
||||||
|
onStatusChange,
|
||||||
|
totalCount,
|
||||||
|
nlQuery,
|
||||||
|
onNlQueryChange,
|
||||||
|
nlSearchLoading,
|
||||||
|
}: FilterBarProps) {
|
||||||
const { viewMode, setViewMode } = useUIStore()
|
const { viewMode, setViewMode } = useUIStore()
|
||||||
|
|
||||||
|
// Local state for the NL input value — debounced before calling onNlQueryChange.
|
||||||
|
const [nlInputValue, setNlInputValue] = useState(nlQuery)
|
||||||
|
|
||||||
|
// Sync if parent resets nlQuery to empty (e.g. clear action).
|
||||||
|
useEffect(() => {
|
||||||
|
if (nlQuery === '') setNlInputValue('')
|
||||||
|
}, [nlQuery])
|
||||||
|
|
||||||
|
// 400ms debounce: propagate to parent only after user stops typing.
|
||||||
|
useEffect(() => {
|
||||||
|
const timer = setTimeout(() => {
|
||||||
|
onNlQueryChange(nlInputValue)
|
||||||
|
}, 400)
|
||||||
|
return () => clearTimeout(timer)
|
||||||
|
}, [nlInputValue, onNlQueryChange])
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<div className="flex flex-wrap items-center gap-3 mb-6">
|
<div className="flex flex-col gap-3 mb-6">
|
||||||
{/* Search */}
|
{/* Row 1: text filter + status + view toggle */}
|
||||||
|
<div className="flex flex-wrap items-center gap-3">
|
||||||
|
{/* Local text search */}
|
||||||
<div className="relative flex-1 min-w-48">
|
<div className="relative flex-1 min-w-48">
|
||||||
<Search className="absolute left-3 top-1/2 -translate-y-1/2 w-4 h-4 text-[#a0a0a0]" />
|
<Search className="absolute left-3 top-1/2 -translate-y-1/2 w-4 h-4 text-[#a0a0a0]" />
|
||||||
<input
|
<input
|
||||||
|
|
@ -77,5 +108,21 @@ export function FilterBar({ search, onSearchChange, statusFilter, onStatusChange
|
||||||
</Button>
|
</Button>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
{/* Row 2: Natural language search */}
|
||||||
|
<div className="relative w-full">
|
||||||
|
<Sparkles className="absolute left-3 top-1/2 -translate-y-1/2 w-4 h-4 text-volt/60" />
|
||||||
|
<input
|
||||||
|
type="text"
|
||||||
|
placeholder="Ask anything: show me free 10GbE NICs…"
|
||||||
|
value={nlInputValue}
|
||||||
|
onChange={(e) => setNlInputValue(e.target.value)}
|
||||||
|
className="w-full pl-9 pr-9 py-2 bg-[#0a0a0a] border border-volt/20 rounded-card text-sm text-white placeholder-[#585858] focus:outline-none focus:border-volt focus:ring-1 focus:ring-volt/30 transition-colors"
|
||||||
|
/>
|
||||||
|
{nlSearchLoading && (
|
||||||
|
<Loader2 className="absolute right-3 top-1/2 -translate-y-1/2 w-4 h-4 animate-spin text-volt" />
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -28,6 +28,9 @@ async function fetchJSON<T>(url: string): Promise<T> {
|
||||||
export const fetchInventory = (): Promise<InventoryItem[]> =>
|
export const fetchInventory = (): Promise<InventoryItem[]> =>
|
||||||
fetchJSON<InventoryItem[]>(`${BASE}/inventory`)
|
fetchJSON<InventoryItem[]>(`${BASE}/inventory`)
|
||||||
|
|
||||||
|
export const fetchSearch = (q: string): Promise<InventoryItem[]> =>
|
||||||
|
fetchJSON<InventoryItem[]>(`${BASE}/search?q=${encodeURIComponent(q)}`)
|
||||||
|
|
||||||
export const fetchInventoryItem = (id: number): Promise<InventoryItem> =>
|
export const fetchInventoryItem = (id: number): Promise<InventoryItem> =>
|
||||||
fetchJSON<InventoryItem>(`${BASE}/inventory/${id}`)
|
fetchJSON<InventoryItem>(`${BASE}/inventory/${id}`)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,10 +1,12 @@
|
||||||
import { useState, useMemo } from 'react'
|
import { useState, useMemo } from 'react'
|
||||||
|
import { useQuery } from '@tanstack/react-query'
|
||||||
import { AppShell } from '@/components/layout/AppShell'
|
import { AppShell } from '@/components/layout/AppShell'
|
||||||
import { FilterBar } from '@/components/inventory/FilterBar'
|
import { FilterBar } from '@/components/inventory/FilterBar'
|
||||||
import { ItemCard } from '@/components/inventory/ItemCard'
|
import { ItemCard } from '@/components/inventory/ItemCard'
|
||||||
import { ItemRow } from '@/components/inventory/ItemRow'
|
import { ItemRow } from '@/components/inventory/ItemRow'
|
||||||
import { useInventory } from '@/hooks/useInventory'
|
import { useInventory } from '@/hooks/useInventory'
|
||||||
import { useUIStore } from '@/store/ui'
|
import { useUIStore } from '@/store/ui'
|
||||||
|
import { fetchSearch } from '@/lib/api'
|
||||||
import { Loader2, AlertCircle } from 'lucide-react'
|
import { Loader2, AlertCircle } from 'lucide-react'
|
||||||
|
|
||||||
export function DashboardPage() {
|
export function DashboardPage() {
|
||||||
|
|
@ -12,7 +14,17 @@ export function DashboardPage() {
|
||||||
const { viewMode } = useUIStore()
|
const { viewMode } = useUIStore()
|
||||||
const [search, setSearch] = useState('')
|
const [search, setSearch] = useState('')
|
||||||
const [statusFilter, setStatusFilter] = useState('')
|
const [statusFilter, setStatusFilter] = useState('')
|
||||||
|
const [nlQuery, setNlQuery] = useState('')
|
||||||
|
|
||||||
|
// NL search via GET /api/search — only fires when nlQuery has > 2 chars.
|
||||||
|
const { data: searchResults, isLoading: searchLoading } = useQuery({
|
||||||
|
queryKey: ['search', nlQuery],
|
||||||
|
queryFn: () => fetchSearch(nlQuery),
|
||||||
|
enabled: nlQuery.trim().length > 2,
|
||||||
|
staleTime: 30_000,
|
||||||
|
})
|
||||||
|
|
||||||
|
// Local filter (used when nlQuery is empty).
|
||||||
const filtered = useMemo(() => {
|
const filtered = useMemo(() => {
|
||||||
if (!items) return []
|
if (!items) return []
|
||||||
return items.filter((item) => {
|
return items.filter((item) => {
|
||||||
|
|
@ -26,6 +38,10 @@ export function DashboardPage() {
|
||||||
})
|
})
|
||||||
}, [items, search, statusFilter])
|
}, [items, search, statusFilter])
|
||||||
|
|
||||||
|
// When nlQuery is active (> 2 chars), display NL results; otherwise local filter.
|
||||||
|
const displayItems = nlQuery.trim().length > 2 ? (searchResults ?? []) : filtered
|
||||||
|
const displayLoading = nlQuery.trim().length > 2 ? searchLoading : isLoading
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<AppShell>
|
<AppShell>
|
||||||
{/* Page header */}
|
{/* Page header */}
|
||||||
|
|
@ -39,14 +55,17 @@ export function DashboardPage() {
|
||||||
onSearchChange={setSearch}
|
onSearchChange={setSearch}
|
||||||
statusFilter={statusFilter}
|
statusFilter={statusFilter}
|
||||||
onStatusChange={setStatusFilter}
|
onStatusChange={setStatusFilter}
|
||||||
totalCount={filtered.length}
|
totalCount={displayItems.length}
|
||||||
|
nlQuery={nlQuery}
|
||||||
|
onNlQueryChange={setNlQuery}
|
||||||
|
nlSearchLoading={searchLoading}
|
||||||
/>
|
/>
|
||||||
|
|
||||||
{/* Loading */}
|
{/* Loading */}
|
||||||
{isLoading && (
|
{displayLoading && (
|
||||||
<div className="flex items-center justify-center py-24 text-[#a0a0a0]">
|
<div className="flex items-center justify-center py-24 text-[#a0a0a0]">
|
||||||
<Loader2 className="w-6 h-6 animate-spin mr-2" />
|
<Loader2 className="w-6 h-6 animate-spin mr-2" />
|
||||||
Loading inventory…
|
{nlQuery.trim().length > 2 ? 'Searching…' : 'Loading inventory…'}
|
||||||
</div>
|
</div>
|
||||||
)}
|
)}
|
||||||
|
|
||||||
|
|
@ -59,28 +78,32 @@ export function DashboardPage() {
|
||||||
)}
|
)}
|
||||||
|
|
||||||
{/* Empty state */}
|
{/* Empty state */}
|
||||||
{!isLoading && !error && filtered.length === 0 && (
|
{!displayLoading && !error && displayItems.length === 0 && (
|
||||||
<div className="flex flex-col items-center justify-center py-24 text-center">
|
<div className="flex flex-col items-center justify-center py-24 text-center">
|
||||||
<p className="font-display font-black text-4xl text-volt mb-2">0</p>
|
<p className="font-display font-black text-4xl text-volt mb-2">0</p>
|
||||||
<p className="text-[#a0a0a0] text-sm">
|
<p className="text-[#a0a0a0] text-sm">
|
||||||
{items && items.length > 0 ? 'No items match your filters' : 'No items cataloged yet — add your first item'}
|
{nlQuery.trim().length > 2
|
||||||
|
? 'No items match your search'
|
||||||
|
: items && items.length > 0
|
||||||
|
? 'No items match your filters'
|
||||||
|
: 'No items cataloged yet — add your first item'}
|
||||||
</p>
|
</p>
|
||||||
</div>
|
</div>
|
||||||
)}
|
)}
|
||||||
|
|
||||||
{/* Grid view */}
|
{/* Grid view */}
|
||||||
{!isLoading && !error && filtered.length > 0 && viewMode === 'grid' && (
|
{!displayLoading && !error && displayItems.length > 0 && viewMode === 'grid' && (
|
||||||
<div className="grid grid-cols-1 sm:grid-cols-2 md:grid-cols-3 lg:grid-cols-4 xl:grid-cols-5 gap-4">
|
<div className="grid grid-cols-1 sm:grid-cols-2 md:grid-cols-3 lg:grid-cols-4 xl:grid-cols-5 gap-4">
|
||||||
{filtered.map((item) => (
|
{displayItems.map((item) => (
|
||||||
<ItemCard key={item.id} item={item} />
|
<ItemCard key={item.id} item={item} />
|
||||||
))}
|
))}
|
||||||
</div>
|
</div>
|
||||||
)}
|
)}
|
||||||
|
|
||||||
{/* List view */}
|
{/* List view */}
|
||||||
{!isLoading && !error && filtered.length > 0 && viewMode === 'list' && (
|
{!displayLoading && !error && displayItems.length > 0 && viewMode === 'list' && (
|
||||||
<div className="border border-charcoal/80 rounded-card overflow-hidden">
|
<div className="border border-charcoal/80 rounded-card overflow-hidden">
|
||||||
{filtered.map((item) => (
|
{displayItems.map((item) => (
|
||||||
<ItemRow key={item.id} item={item} />
|
<ItemRow key={item.id} item={item} />
|
||||||
))}
|
))}
|
||||||
</div>
|
</div>
|
||||||
|
|
|
||||||
Loading…
Add table
Reference in a new issue