Compare commits
No commits in common. "9f59d38d49fc3976abc8b70186081b31c7fd396c" and "16a469bfdd9ce31cb5c1630a6f11df7fba3ba5cd" have entirely different histories.
9f59d38d49
...
16a469bfdd
24 changed files with 70 additions and 2106 deletions
|
|
@ -18,7 +18,7 @@ Decimal phases appear between their surrounding integers in numeric order.
|
||||||
- [x] **Phase 4: USB Manager & Label Printing** - USB hardware characterization (2026-04-13), goroutine-per-device manager, QR label printing (completed 2026-04-10)
|
- [x] **Phase 4: USB Manager & Label Printing** - USB hardware characterization (2026-04-13), goroutine-per-device manager, QR label printing (completed 2026-04-10)
|
||||||
- [x] **Phase 5: Cable Test Integration** - Treedix USB/DP/HDMI testers, FNIRSI FNB58, cable test workflow UI (completed 2026-04-10)
|
- [x] **Phase 5: Cable Test Integration** - Treedix USB/DP/HDMI testers, FNIRSI FNB58, cable test workflow UI (completed 2026-04-10)
|
||||||
- [x] **Phase 6: Lab Advisor** - Claude Opus chat interface, NetBox context assembly, streaming SSE, chat history (completed 2026-04-10)
|
- [x] **Phase 6: Lab Advisor** - Claude Opus chat interface, NetBox context assembly, streaming SSE, chat history (completed 2026-04-10)
|
||||||
- [x] **Phase 7: Research Agent & Search** - SearXNG Tier 2 research agent, natural language inventory search, quality gate automation (completed 2026-04-10)
|
- [ ] **Phase 7: Research Agent & Search** - SearXNG Tier 2 research agent, natural language inventory search, quality gate automation
|
||||||
|
|
||||||
## Phase Details
|
## Phase Details
|
||||||
|
|
||||||
|
|
@ -139,11 +139,7 @@ Plans:
|
||||||
3. SearXNG queries are sanitized before dispatch — no raw AI output reaches the search engine
|
3. SearXNG queries are sanitized before dispatch — no raw AI output reaches the search engine
|
||||||
|
|
||||||
**Note:** AI-04 is the sole unmapped requirement from Phase 2 that belongs here — it requires the full orchestrator, SearXNG client, and NetBox inventory all in place. The other Phase 2 requirements cover Tier 1 intake; this requirement covers the Tier 2 research loop.
|
**Note:** AI-04 is the sole unmapped requirement from Phase 2 that belongs here — it requires the full orchestrator, SearXNG client, and NetBox inventory all in place. The other Phase 2 requirements cover Tier 1 intake; this requirement covers the Tier 2 research loop.
|
||||||
**Plans**: 2 plans
|
**Plans**: TBD
|
||||||
|
|
||||||
Plans:
|
|
||||||
- [x] 07-01-PLAN.md — SearXNG client, ResearchAgent worker, POST /api/research/trigger
|
|
||||||
- [x] 07-02-PLAN.md — NL search endpoint GET /api/search, dashboard search bar
|
|
||||||
|
|
||||||
## Progress
|
## Progress
|
||||||
|
|
||||||
|
|
@ -158,4 +154,4 @@ Phases execute in numeric order: 1 → 2 → 3 → 4 → 5 → 6 → 7
|
||||||
| 4. USB Manager & Label Printing | 5/5 | Complete | 2026-04-10 |
|
| 4. USB Manager & Label Printing | 5/5 | Complete | 2026-04-10 |
|
||||||
| 5. Cable Test Integration | 3/3 | Complete | 2026-04-10 |
|
| 5. Cable Test Integration | 3/3 | Complete | 2026-04-10 |
|
||||||
| 6. Lab Advisor | 3/3 | Complete | 2026-04-10 |
|
| 6. Lab Advisor | 3/3 | Complete | 2026-04-10 |
|
||||||
| 7. Research Agent & Search | 2/2 | Complete | 2026-04-10 |
|
| 7. Research Agent & Search | 0/TBD | Not started | - |
|
||||||
|
|
|
||||||
|
|
@ -2,15 +2,15 @@
|
||||||
gsd_state_version: 1.0
|
gsd_state_version: 1.0
|
||||||
milestone: v1.0
|
milestone: v1.0
|
||||||
milestone_name: milestone
|
milestone_name: milestone
|
||||||
status: executing
|
status: planning
|
||||||
stopped_at: Roadmap created — ready to run /gsd-plan-phase 1
|
stopped_at: Roadmap created — ready to run /gsd-plan-phase 1
|
||||||
last_updated: "2026-04-10T07:57:47.510Z"
|
last_updated: "2026-04-10T07:41:49.825Z"
|
||||||
last_activity: 2026-04-10
|
last_activity: 2026-04-10
|
||||||
progress:
|
progress:
|
||||||
total_phases: 7
|
total_phases: 7
|
||||||
completed_phases: 7
|
completed_phases: 6
|
||||||
total_plans: 27
|
total_plans: 25
|
||||||
completed_plans: 27
|
completed_plans: 25
|
||||||
percent: 100
|
percent: 100
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
@ -27,7 +27,7 @@ See: .planning/PROJECT.md (updated 2026-04-09)
|
||||||
|
|
||||||
Phase: 7 of 7 (research agent & search)
|
Phase: 7 of 7 (research agent & search)
|
||||||
Plan: Not started
|
Plan: Not started
|
||||||
Status: Ready to execute
|
Status: Ready to plan
|
||||||
Last activity: 2026-04-10
|
Last activity: 2026-04-10
|
||||||
|
|
||||||
Progress: [░░░░░░░░░░] 0%
|
Progress: [░░░░░░░░░░] 0%
|
||||||
|
|
@ -36,7 +36,7 @@ Progress: [░░░░░░░░░░] 0%
|
||||||
|
|
||||||
**Velocity:**
|
**Velocity:**
|
||||||
|
|
||||||
- Total plans completed: 27
|
- Total plans completed: 25
|
||||||
- Average duration: —
|
- Average duration: —
|
||||||
- Total execution time: 0 hours
|
- Total execution time: 0 hours
|
||||||
|
|
||||||
|
|
@ -50,7 +50,6 @@ Progress: [░░░░░░░░░░] 0%
|
||||||
| 4 | 5 | - | - |
|
| 4 | 5 | - | - |
|
||||||
| 5 | 3 | - | - |
|
| 5 | 3 | - | - |
|
||||||
| 6 | 3 | - | - |
|
| 6 | 3 | - | - |
|
||||||
| 7 | 2 | - | - |
|
|
||||||
|
|
||||||
**Recent Trend:**
|
**Recent Trend:**
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,319 +0,0 @@
|
||||||
---
|
|
||||||
phase: 07-research-agent-search
|
|
||||||
plan: 01
|
|
||||||
type: execute
|
|
||||||
wave: 1
|
|
||||||
depends_on: []
|
|
||||||
files_modified:
|
|
||||||
- internal/config/config.go
|
|
||||||
- internal/netbox/client.go
|
|
||||||
- internal/research/searxng.go
|
|
||||||
- internal/research/agent.go
|
|
||||||
- internal/ai/research.go
|
|
||||||
- cmd/hwlab/main.go
|
|
||||||
autonomous: true
|
|
||||||
requirements:
|
|
||||||
- AI-04
|
|
||||||
|
|
||||||
must_haves:
|
|
||||||
truths:
|
|
||||||
- "A SearXNG HTTP GET to http://10.5.0.129:8080/search?q=...&format=json returns parsed results"
|
|
||||||
- "Items with catalog_status=needs_research are polled from NetBox every 10 minutes"
|
|
||||||
- "Each needs_research item is enriched by SearXNG + Tier 2 LLM and updated to catalog_status=researched in NetBox"
|
|
||||||
- "POST /api/research/trigger fires an immediate research cycle (does not wait for the 10-min ticker)"
|
|
||||||
artifacts:
|
|
||||||
- path: "internal/research/searxng.go"
|
|
||||||
provides: "SearXNGClient implementing ai.ResearchClient"
|
|
||||||
exports: ["SearXNGClient", "NewSearXNGClient"]
|
|
||||||
- path: "internal/research/agent.go"
|
|
||||||
provides: "ResearchAgent background goroutine"
|
|
||||||
exports: ["Agent", "NewAgent", "RunOnce", "Start"]
|
|
||||||
key_links:
|
|
||||||
- from: "internal/research/searxng.go"
|
|
||||||
to: "http://10.5.0.129:8080/search"
|
|
||||||
via: "net/http GET with q and format=json query params"
|
|
||||||
pattern: "http\\.Get.*search.*format=json"
|
|
||||||
- from: "internal/research/agent.go"
|
|
||||||
to: "internal/netbox/client.go"
|
|
||||||
via: "ListDevicesWithStatus(ctx, \"needs_research\")"
|
|
||||||
pattern: "ListDevicesWithStatus"
|
|
||||||
- from: "internal/research/agent.go"
|
|
||||||
to: "internal/ai/client.go"
|
|
||||||
via: "tier2.AnalyzePhotos (text-only prompt, no photos)"
|
|
||||||
pattern: "AnalyzePhotos"
|
|
||||||
---
|
|
||||||
|
|
||||||
<objective>
|
|
||||||
Build the real SearXNG research client and the ResearchAgent background worker that
|
|
||||||
closes the AI-04 research loop: items at needs_research are enriched automatically.
|
|
||||||
|
|
||||||
Purpose: Replace the Phase 2 NoOpResearchClient stub and deliver the automated
|
|
||||||
enrichment cycle that advances items from needs_research to researched in NetBox.
|
|
||||||
|
|
||||||
Output:
|
|
||||||
- internal/research/searxng.go — real HTTP client implementing ai.ResearchClient
|
|
||||||
- internal/research/agent.go — background worker with ticker + on-demand trigger
|
|
||||||
- Config additions for SearXNG URL
|
|
||||||
- main.go goroutine start + POST /api/research/trigger handler
|
|
||||||
</objective>
|
|
||||||
|
|
||||||
<execution_context>
|
|
||||||
@/home/mikkel/.claude/get-shit-done/workflows/execute-plan.md
|
|
||||||
@/home/mikkel/.claude/get-shit-done/templates/summary.md
|
|
||||||
</execution_context>
|
|
||||||
|
|
||||||
<context>
|
|
||||||
@.planning/PROJECT.md
|
|
||||||
@.planning/ROADMAP.md
|
|
||||||
@.planning/STATE.md
|
|
||||||
|
|
||||||
@internal/ai/research.go
|
|
||||||
@internal/ai/client.go
|
|
||||||
@internal/ai/orchestrator.go
|
|
||||||
@internal/netbox/client.go
|
|
||||||
@internal/netbox/custom_fields.go
|
|
||||||
@internal/netbox/types.go
|
|
||||||
@internal/inventory/catalog_updater.go
|
|
||||||
@internal/config/config.go
|
|
||||||
@cmd/hwlab/main.go
|
|
||||||
@internal/api/router.go
|
|
||||||
|
|
||||||
<interfaces>
|
|
||||||
<!-- Key contracts the executor needs. Extracted from codebase. -->
|
|
||||||
|
|
||||||
From internal/ai/research.go:
|
|
||||||
```go
|
|
||||||
type SearchResult struct {
|
|
||||||
Title string
|
|
||||||
URL string
|
|
||||||
Snippet string
|
|
||||||
}
|
|
||||||
|
|
||||||
type ResearchClient interface {
|
|
||||||
Search(ctx context.Context, query string) ([]SearchResult, error)
|
|
||||||
}
|
|
||||||
|
|
||||||
type NoOpResearchClient struct{}
|
|
||||||
// Replace this with SearXNGClient in this plan.
|
|
||||||
```
|
|
||||||
|
|
||||||
From internal/ai/client.go:
|
|
||||||
```go
|
|
||||||
type AIClient interface {
|
|
||||||
AnalyzePhotos(ctx context.Context, req IntakeRequest) (*IntakeResult, error)
|
|
||||||
}
|
|
||||||
// IntakeRequest.PhotosBase64 may be empty — the Tier 2 model accepts text-only
|
|
||||||
// if the prompt is placed in a separate system message; use a text-only prompt
|
|
||||||
// for research enrichment (no photos).
|
|
||||||
```
|
|
||||||
|
|
||||||
From internal/netbox/client.go (method to ADD):
|
|
||||||
```go
|
|
||||||
// ListDevicesWithStatus returns devices whose catalog_status custom field equals status.
|
|
||||||
// Use status="needs_research" to find items needing enrichment.
|
|
||||||
func (c *Client) ListDevicesWithStatus(ctx context.Context, status string) ([]Device, error)
|
|
||||||
```
|
|
||||||
|
|
||||||
From internal/inventory/catalog_updater.go:
|
|
||||||
```go
|
|
||||||
func (u *CatalogUpdater) UpdateCatalogStatus(ctx context.Context, deviceID int64, current, next CatalogStatus) (CatalogStatus, error)
|
|
||||||
```
|
|
||||||
|
|
||||||
From internal/inventory (quality_gate.go constants):
|
|
||||||
```go
|
|
||||||
const StatusNeedsResearch CatalogStatus = "needs_research"
|
|
||||||
const StatusResearched CatalogStatus = "researched"
|
|
||||||
```
|
|
||||||
|
|
||||||
From internal/config/config.go (field to ADD):
|
|
||||||
```go
|
|
||||||
SearXNGURL string `mapstructure:"searxng_url"`
|
|
||||||
// default: "http://10.5.0.129:8080"
|
|
||||||
// env: HWLAB_SEARXNG_URL
|
|
||||||
```
|
|
||||||
</interfaces>
|
|
||||||
</context>
|
|
||||||
|
|
||||||
<tasks>
|
|
||||||
|
|
||||||
<task type="auto" tdd="true">
|
|
||||||
<name>Task 1: SearXNG client + netbox.ListDevicesWithStatus</name>
|
|
||||||
<files>
|
|
||||||
internal/research/searxng.go,
|
|
||||||
internal/research/searxng_test.go,
|
|
||||||
internal/netbox/client.go,
|
|
||||||
internal/config/config.go
|
|
||||||
</files>
|
|
||||||
<behavior>
|
|
||||||
- SearXNGClient.Search(ctx, "Intel NIC i350") sends GET http://10.5.0.129:8080/search?q=Intel+NIC+i350&format=json
|
|
||||||
- HTTP 200 with JSON body {"results":[{"title":"...","url":"...","content":"..."},...]} parses into []ai.SearchResult (map content->Snippet)
|
|
||||||
- HTTP non-200 returns error with status code
|
|
||||||
- Empty results array returns empty slice, no error
|
|
||||||
- Query is URL-encoded (url.QueryEscape or url.Values)
|
|
||||||
- ListDevicesWithStatus filters via custom_fields cf_catalog_status in go-netbox list call; falls back to client-side filter if API param unavailable
|
|
||||||
- ListDevicesWithStatus("needs_research") returns only devices with that catalog_status
|
|
||||||
</behavior>
|
|
||||||
<action>
|
|
||||||
Create package internal/research.
|
|
||||||
|
|
||||||
internal/research/searxng.go:
|
|
||||||
- Struct SearXNGClient with baseURL string and httpClient *http.Client (timeout 15s)
|
|
||||||
- NewSearXNGClient(baseURL string) *SearXNGClient — if baseURL empty, use "http://10.5.0.129:8080"
|
|
||||||
- Implements ai.ResearchClient interface
|
|
||||||
- Search method: build GET {baseURL}/search?q={url-encoded query}&format=json, execute, decode JSON
|
|
||||||
- SearXNG JSON response shape: {"results":[{"title":"","url":"","content":""},...]}
|
|
||||||
Map content field to SearchResult.Snippet (SearXNG uses "content" not "snippet")
|
|
||||||
- Return ([]ai.SearchResult, error). Never panic on empty results.
|
|
||||||
|
|
||||||
internal/research/searxng_test.go:
|
|
||||||
- Use httptest.NewServer to mock SearXNG responses
|
|
||||||
- Test: valid response parses correctly (2 results)
|
|
||||||
- Test: HTTP 500 returns error
|
|
||||||
- Test: empty results returns empty slice
|
|
||||||
|
|
||||||
internal/netbox/client.go — add ListDevicesWithStatus:
|
|
||||||
- List all devices (up to 200), filter client-side where CustomFields.CatalogStatus == status
|
|
||||||
- (go-netbox v4 custom field filtering via query param is schema-dependent; client-side is safer)
|
|
||||||
|
|
||||||
internal/config/config.go — add SearXNGURL:
|
|
||||||
- Field: SearXNGURL string `mapstructure:"searxng_url"`
|
|
||||||
- Default: v.SetDefault("searxng_url", "http://10.5.0.129:8080")
|
|
||||||
- Env binding: v.BindEnv("searxng_url", "HWLAB_SEARXNG_URL")
|
|
||||||
</action>
|
|
||||||
<verify>
|
|
||||||
<automated>cd /home/mikkel/homelabby && go test ./internal/research/... ./internal/config/... -v -count=1 -run TestSearXNG 2>&1 | tail -20</automated>
|
|
||||||
</verify>
|
|
||||||
<done>
|
|
||||||
SearXNGClient implements ai.ResearchClient. Tests pass with httptest mock server.
|
|
||||||
ListDevicesWithStatus added to netbox.Client. Config loads SearXNGURL with default.
|
|
||||||
</done>
|
|
||||||
</task>
|
|
||||||
|
|
||||||
<task type="auto" tdd="true">
|
|
||||||
<name>Task 2: ResearchAgent worker + main.go wiring + trigger endpoint</name>
|
|
||||||
<files>
|
|
||||||
internal/research/agent.go,
|
|
||||||
internal/research/agent_test.go,
|
|
||||||
internal/api/handlers/research.go,
|
|
||||||
internal/api/router.go,
|
|
||||||
cmd/hwlab/main.go
|
|
||||||
</files>
|
|
||||||
<behavior>
|
|
||||||
- Agent.RunOnce(ctx) polls NetBox for needs_research items, for each: builds a text-only search query from item Name, calls SearXNGClient.Search, sends results to Tier 2 LLM with a research prompt, patches NetBox custom fields (ai_notes, product_url from first result URL), transitions status to researched via CatalogUpdater
|
|
||||||
- Agent.Start(ctx, interval) runs RunOnce on ticker; logs "research agent: cycle complete, enriched N items"
|
|
||||||
- If SearXNG returns 0 results for an item, log warning and skip (do not change status)
|
|
||||||
- Tier 2 LLM research prompt: "You are enriching a hardware inventory record. Item: {name}. Search results: {formatted snippets}. Return JSON: {\"ai_notes\": \"...\", \"product_url\": \"...\"}"
|
|
||||||
- POST /api/research/trigger responds 202 Accepted and fires RunOnce in a goroutine (non-blocking)
|
|
||||||
- Query sanitization: strip characters outside [a-zA-Z0-9 .-_] before passing to SearXNG
|
|
||||||
</behavior>
|
|
||||||
<action>
|
|
||||||
internal/research/agent.go:
|
|
||||||
- Struct Agent with fields: nbClient *netbox.Client, researchClient ai.ResearchClient,
|
|
||||||
tier2 ai.AIClient, updater *inventory.CatalogUpdater
|
|
||||||
- NewAgent(nb *netbox.Client, rc ai.ResearchClient, tier2 ai.AIClient, updater *inventory.CatalogUpdater) *Agent
|
|
||||||
- sanitizeQuery(s string) string — regexp [^a-zA-Z0-9 .\-_]+ replaced with space, strings.TrimSpace
|
|
||||||
- RunOnce(ctx context.Context) (enriched int, err error):
|
|
||||||
1. ListDevicesWithStatus(ctx, "needs_research")
|
|
||||||
2. For each device:
|
|
||||||
a. query = sanitizeQuery(device.Name)
|
|
||||||
b. results = researchClient.Search(ctx, query) — skip if 0 results
|
|
||||||
c. Build text prompt with top 3 results (title + snippet)
|
|
||||||
d. tier2.AnalyzePhotos(ctx, IntakeRequest{PhotosBase64: nil, SystemPrompt: researchPrompt})
|
|
||||||
NOTE: IntakeRequest may not have SystemPrompt; build the research prompt as the
|
|
||||||
text part of the multimodal request by putting it in a single text-only message.
|
|
||||||
Check IntakeRequest fields; if no SystemPrompt, use a wrapper: set PhotosBase64 to
|
|
||||||
nil and pass the assembled prompt text in a way the TierClient accepts.
|
|
||||||
ALTERNATIVE if IntakeRequest does not support text-only: use go-openai directly
|
|
||||||
via a new ResearchTierClient method — add TextComplete(ctx, prompt) (*IntakeResult, error)
|
|
||||||
that posts a simple text ChatCompletion (no images). Prefer this approach for clarity.
|
|
||||||
e. Parse response for ai_notes and product_url
|
|
||||||
f. Patch NetBox: PatchCustomFields with ai_notes + product_url (if non-empty)
|
|
||||||
g. UpdateCatalogStatus(ctx, id, StatusNeedsResearch, StatusResearched)
|
|
||||||
h. enriched++
|
|
||||||
3. Return enriched count
|
|
||||||
- Start(ctx context.Context, interval time.Duration):
|
|
||||||
log.Printf("research agent: starting, interval=%v", interval)
|
|
||||||
RunOnce immediately, then ticker loop until ctx.Done()
|
|
||||||
|
|
||||||
For the text-only LLM call: add TextComplete to TierClient in internal/ai/client.go:
|
|
||||||
```go
|
|
||||||
func (c *TierClient) TextComplete(ctx context.Context, prompt string) (string, error)
|
|
||||||
```
|
|
||||||
This does a simple non-vision ChatCompletion with a single user message. Agent uses this.
|
|
||||||
|
|
||||||
internal/research/agent_test.go:
|
|
||||||
- Mock ResearchClient returning 2 fake SearchResults
|
|
||||||
- Mock AIClient (use existing MockAIClient pattern if available, else minimal struct)
|
|
||||||
- Mock NetBox (or use a stub struct) — test RunOnce returns enriched=1 for a fake device
|
|
||||||
- Test sanitizeQuery strips special chars
|
|
||||||
|
|
||||||
internal/api/handlers/research.go:
|
|
||||||
- ResearchHandler struct with agent *research.Agent
|
|
||||||
- NewResearchHandler(agent *research.Agent) *ResearchHandler
|
|
||||||
- TriggerResearch(w http.ResponseWriter, r *http.Request):
|
|
||||||
go func() { agent.RunOnce(context.Background()) }()
|
|
||||||
w.WriteHeader(http.StatusAccepted)
|
|
||||||
json.NewEncoder(w).Encode(map[string]string{"status": "accepted"})
|
|
||||||
|
|
||||||
internal/api/router.go:
|
|
||||||
- Add researchHandler *handlers.ResearchHandler parameter to NewRouter signature
|
|
||||||
- Add r.Post("/research/trigger", researchHandler.TriggerResearch) inside r.Route("/api", ...)
|
|
||||||
- If researchHandler is nil, register an unavailable handler (same pattern as advisorHandler)
|
|
||||||
|
|
||||||
cmd/hwlab/main.go:
|
|
||||||
- Import internal/research
|
|
||||||
- After config load: searxngClient := research.NewSearXNGClient(cfg.SearXNGURL)
|
|
||||||
- researchAgent := research.NewAgent(nbClient, searxngClient, tier2, catalogUpdater)
|
|
||||||
- go researchAgent.Start(ctx, 10*time.Minute)
|
|
||||||
- researchHandler := handlers.NewResearchHandler(researchAgent)
|
|
||||||
- Pass researchHandler to api.NewRouter(...)
|
|
||||||
</action>
|
|
||||||
<verify>
|
|
||||||
<automated>cd /home/mikkel/homelabby && go build ./... && go test ./internal/research/... -v -count=1 2>&1 | tail -30</automated>
|
|
||||||
</verify>
|
|
||||||
<done>
|
|
||||||
go build passes. Agent tests pass. POST /api/research/trigger wired in router.
|
|
||||||
Research agent goroutine starts on server launch with 10-minute interval.
|
|
||||||
</done>
|
|
||||||
</task>
|
|
||||||
|
|
||||||
</tasks>
|
|
||||||
|
|
||||||
<threat_model>
|
|
||||||
## Trust Boundaries
|
|
||||||
|
|
||||||
| Boundary | Description |
|
|
||||||
|----------|-------------|
|
|
||||||
| agent → SearXNG | AI-generated query text leaves the process and reaches the search engine |
|
|
||||||
| SearXNG → agent | External search results (HTML snippets) enter the process and are forwarded to LLM |
|
|
||||||
| trigger endpoint → agent | HTTP request from frontend triggers a research cycle |
|
|
||||||
|
|
||||||
## STRIDE Threat Register
|
|
||||||
|
|
||||||
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|
|
||||||
|-----------|----------|-----------|-------------|-----------------|
|
|
||||||
| T-07-01 | Tampering | sanitizeQuery | mitigate | Strip [^a-zA-Z0-9 .\-_]+ before dispatch; test with adversarial input in unit test |
|
|
||||||
| T-07-02 | Information Disclosure | SearXNG response snippets | accept | SearXNG is self-hosted LAN service; snippets never stored, only passed to LLM |
|
|
||||||
| T-07-03 | Denial of Service | POST /api/research/trigger | mitigate | Trigger fires goroutine but RunOnce is bounded per item; no queuing needed for MVP rate |
|
|
||||||
| T-07-04 | Spoofing | SearXNG base URL in config | accept | LAN-only service at fixed IP; no auth required by design |
|
|
||||||
</threat_model>
|
|
||||||
|
|
||||||
<verification>
|
|
||||||
1. `go build ./...` passes with no errors
|
|
||||||
2. `go test ./internal/research/...` all pass
|
|
||||||
3. SearXNG integration (manual): `curl "http://10.5.0.129:8080/search?q=Intel+i350&format=json"` returns JSON
|
|
||||||
4. Trigger endpoint: `curl -X POST http://localhost:8080/api/research/trigger` returns 202
|
|
||||||
5. Log line "research agent: starting, interval=10m0s" appears on server start
|
|
||||||
</verification>
|
|
||||||
|
|
||||||
<success_criteria>
|
|
||||||
- SearXNGClient.Search returns parsed []ai.SearchResult from live SearXNG instance
|
|
||||||
- ResearchAgent.RunOnce enriches needs_research items end-to-end: search → LLM → NetBox patch → status transition
|
|
||||||
- Research cycle runs every 10 minutes automatically and on demand via POST /api/research/trigger
|
|
||||||
- All queries sanitized before SearXNG dispatch
|
|
||||||
- go build clean, all new tests pass
|
|
||||||
</success_criteria>
|
|
||||||
|
|
||||||
<output>
|
|
||||||
After completion, create `.planning/phases/07-research-agent-search/07-01-SUMMARY.md`
|
|
||||||
</output>
|
|
||||||
|
|
@ -1,90 +0,0 @@
|
||||||
---
|
|
||||||
phase: 07-research-agent-search
|
|
||||||
plan: "01"
|
|
||||||
subsystem: research-agent
|
|
||||||
tags: [research, searxng, ai-enrichment, background-worker]
|
|
||||||
dependency_graph:
|
|
||||||
requires: [internal/ai, internal/netbox, internal/inventory, internal/config]
|
|
||||||
provides: [internal/research, internal/api/handlers/research]
|
|
||||||
affects: [cmd/hwlab/main.go, internal/api/router.go]
|
|
||||||
tech_stack:
|
|
||||||
added: [internal/research package]
|
|
||||||
patterns: [interface-injection for testability, TDD red-green, background ticker worker]
|
|
||||||
key_files:
|
|
||||||
created:
|
|
||||||
- internal/research/searxng.go
|
|
||||||
- internal/research/searxng_test.go
|
|
||||||
- internal/research/agent.go
|
|
||||||
- internal/research/agent_test.go
|
|
||||||
- internal/api/handlers/research.go
|
|
||||||
modified:
|
|
||||||
- internal/ai/client.go
|
|
||||||
- internal/netbox/client.go
|
|
||||||
- internal/config/config.go
|
|
||||||
- internal/api/router.go
|
|
||||||
- cmd/hwlab/main.go
|
|
||||||
decisions:
|
|
||||||
- "Used interface injection (NetBoxer, TextCompleter, CatalogTransitioner) in Agent instead of concrete types to enable stub-based unit tests without live NetBox"
|
|
||||||
- "Added TierClient.TextComplete as separate method rather than reusing AnalyzePhotos to keep vision and text paths distinct"
|
|
||||||
- "SanitizeQuery exported (capitalised) to allow external test package verification of T-07-01 mitigation"
|
|
||||||
- "Agent.RunOnce returns (int, error) rather than just error so callers and tests can assert enrichment count"
|
|
||||||
metrics:
|
|
||||||
duration_seconds: 256
|
|
||||||
completed_date: "2026-04-10"
|
|
||||||
tasks_completed: 2
|
|
||||||
tasks_total: 2
|
|
||||||
files_created: 5
|
|
||||||
files_modified: 5
|
|
||||||
---
|
|
||||||
|
|
||||||
# Phase 07 Plan 01: SearXNG Client + ResearchAgent Summary
|
|
||||||
|
|
||||||
**One-liner:** SearXNG HTTP client + background ResearchAgent that polls NetBox for `needs_research` items, enriches via SearXNG search + Tier 2 LLM text completion, patches NetBox custom fields, and transitions status to `researched` every 10 minutes or on-demand via `POST /api/research/trigger`.
|
|
||||||
|
|
||||||
## Tasks Completed
|
|
||||||
|
|
||||||
| Task | Name | Commit | Key Files |
|
|
||||||
|------|------|--------|-----------|
|
|
||||||
| 1 | SearXNG client + netbox.ListDevicesWithStatus | 30cd279 | internal/research/searxng.go, internal/netbox/client.go, internal/config/config.go |
|
|
||||||
| 2 | ResearchAgent worker + main.go wiring + trigger endpoint | 0072aa4 | internal/research/agent.go, internal/api/handlers/research.go, internal/api/router.go, cmd/hwlab/main.go |
|
|
||||||
|
|
||||||
## Decisions Made
|
|
||||||
|
|
||||||
1. **Interface injection for Agent dependencies** — `NetBoxer`, `TextCompleter`, and `CatalogTransitioner` interfaces in `internal/research/agent.go` allow stub injection in tests without a live NetBox or LLM. The concrete `*netbox.Client` and `*ai.TierClient` satisfy these interfaces automatically.
|
|
||||||
|
|
||||||
2. **TierClient.TextComplete as distinct method** — Rather than forcing research prompts through `AnalyzePhotos` (which builds a vision multipart message), added a clean `TextComplete(ctx, prompt) (string, error)` method on `TierClient` that posts a simple single-user-message ChatCompletion. This keeps the vision and text paths separate and makes intent clear.
|
|
||||||
|
|
||||||
3. **SanitizeQuery exported** — The T-07-01 threat mitigation (strip `[^a-zA-Z0-9 .\-_]+` before SearXNG dispatch) is tested from the `_test` package, which requires the function to be exported. Consistent with the plan's explicit mention of adversarial input testing.
|
|
||||||
|
|
||||||
4. **Product URL fallback** — If the LLM does not return a `product_url` in its JSON response, the agent falls back to the first SearXNG result URL. This ensures `product_url` is populated even when the LLM provides incomplete JSON.
|
|
||||||
|
|
||||||
## Deviations from Plan
|
|
||||||
|
|
||||||
### Auto-fixed Issues
|
|
||||||
|
|
||||||
**1. [Rule 1 - Bug] Test expectation for sanitizeQuery with `/` character**
|
|
||||||
- **Found during:** Task 2 GREEN phase
|
|
||||||
- **Issue:** Test case `"Dell<script>alert(1)</script>"` expected `"Dell script alert 1 /script "` but `/` is correctly stripped by the `[^a-zA-Z0-9 .\-_]+` regex
|
|
||||||
- **Fix:** Updated test expectation to `"Dell script alert 1 script"` (correct behavior)
|
|
||||||
- **Files modified:** internal/research/agent_test.go
|
|
||||||
- **Commit:** 0072aa4
|
|
||||||
|
|
||||||
None otherwise — plan executed as written.
|
|
||||||
|
|
||||||
## Known Stubs
|
|
||||||
|
|
||||||
None. All interfaces are fully implemented. The `NoOpResearchClient` in `internal/ai/research.go` is still present but is no longer used — it has been superseded by `research.SearXNGClient` wired in `main.go`.
|
|
||||||
|
|
||||||
## Threat Surface Scan
|
|
||||||
|
|
||||||
No new trust boundaries introduced beyond those documented in the plan's threat model. T-07-01 (query sanitization) is mitigated and tested. T-07-03 (trigger DoS) is mitigated — `RunOnce` is bounded per-item with no queuing amplification.
|
|
||||||
|
|
||||||
## Self-Check: PASSED
|
|
||||||
|
|
||||||
- internal/research/searxng.go: FOUND
|
|
||||||
- internal/research/agent.go: FOUND
|
|
||||||
- internal/api/handlers/research.go: FOUND
|
|
||||||
- Commit 30cd279: FOUND
|
|
||||||
- Commit 0072aa4: FOUND
|
|
||||||
- `go build ./...`: PASSES
|
|
||||||
- `go test ./internal/research/...`: 9/9 PASS
|
|
||||||
|
|
@ -1,302 +0,0 @@
|
||||||
---
|
|
||||||
phase: 07-research-agent-search
|
|
||||||
plan: 02
|
|
||||||
type: execute
|
|
||||||
wave: 2
|
|
||||||
depends_on:
|
|
||||||
- "07-01-PLAN.md"
|
|
||||||
files_modified:
|
|
||||||
- internal/api/handlers/search.go
|
|
||||||
- internal/api/router.go
|
|
||||||
- web/src/lib/api.ts
|
|
||||||
- web/src/pages/DashboardPage.tsx
|
|
||||||
- web/src/components/inventory/FilterBar.tsx
|
|
||||||
autonomous: true
|
|
||||||
requirements:
|
|
||||||
- AI-04
|
|
||||||
- UI-03
|
|
||||||
|
|
||||||
must_haves:
|
|
||||||
truths:
|
|
||||||
- "GET /api/search?q=show+me+free+10GbE+NICs returns matching inventory items"
|
|
||||||
- "Tier 1 (Gemma 4) translates the natural language query to NetBox filter params before the inventory lookup"
|
|
||||||
- "Dashboard has a natural language search input; submitting it calls GET /api/search and displays results using existing ItemCard/ItemRow"
|
|
||||||
- "Sanitized query — no raw NL text reaches NetBox filter params; only structured extracted values"
|
|
||||||
artifacts:
|
|
||||||
- path: "internal/api/handlers/search.go"
|
|
||||||
provides: "SearchHandler: GET /api/search?q=..."
|
|
||||||
exports: ["SearchHandler", "NewSearchHandler"]
|
|
||||||
- path: "web/src/lib/api.ts"
|
|
||||||
provides: "fetchSearch(q) function"
|
|
||||||
exports: ["fetchSearch", "SearchResponse"]
|
|
||||||
key_links:
|
|
||||||
- from: "web/src/pages/DashboardPage.tsx"
|
|
||||||
to: "/api/search"
|
|
||||||
via: "TanStack Query useQuery on nlQuery state"
|
|
||||||
pattern: "useQuery.*search"
|
|
||||||
- from: "internal/api/handlers/search.go"
|
|
||||||
to: "internal/netbox/client.go"
|
|
||||||
via: "ListDevices or ListDevicesWithStatus filtered by extracted params"
|
|
||||||
pattern: "nbClient\\.List"
|
|
||||||
---
|
|
||||||
|
|
||||||
<objective>
|
|
||||||
Natural language inventory search: a Tier 1 LLM translates the user's query to
|
|
||||||
structured NetBox filter params, fetches matching devices, and returns them.
|
|
||||||
The dashboard gains an NL search input wired to GET /api/search.
|
|
||||||
|
|
||||||
Purpose: Delivers UI-03 (natural language search) and closes the remaining AI-04
|
|
||||||
surface (research loop query path).
|
|
||||||
|
|
||||||
Output:
|
|
||||||
- internal/api/handlers/search.go — SearchHandler with NL→filter translation
|
|
||||||
- Router wired with GET /api/search
|
|
||||||
- web/src/lib/api.ts — fetchSearch function
|
|
||||||
- DashboardPage NL search bar replaces/augments the existing local text filter
|
|
||||||
</objective>
|
|
||||||
|
|
||||||
<execution_context>
|
|
||||||
@/home/mikkel/.claude/get-shit-done/workflows/execute-plan.md
|
|
||||||
@/home/mikkel/.claude/get-shit-done/templates/summary.md
|
|
||||||
</execution_context>
|
|
||||||
|
|
||||||
<context>
|
|
||||||
@.planning/PROJECT.md
|
|
||||||
@.planning/ROADMAP.md
|
|
||||||
@.planning/STATE.md
|
|
||||||
@.planning/phases/07-research-agent-search/07-01-SUMMARY.md
|
|
||||||
|
|
||||||
@internal/api/router.go
|
|
||||||
@internal/api/handlers/search.go
|
|
||||||
@internal/netbox/client.go
|
|
||||||
@internal/netbox/types.go
|
|
||||||
@internal/ai/client.go
|
|
||||||
@internal/config/config.go
|
|
||||||
@web/src/lib/api.ts
|
|
||||||
@web/src/pages/DashboardPage.tsx
|
|
||||||
@web/src/components/inventory/FilterBar.tsx
|
|
||||||
|
|
||||||
<interfaces>
|
|
||||||
<!-- Key contracts the executor needs. Extracted from codebase. -->
|
|
||||||
|
|
||||||
From internal/netbox/client.go:
|
|
||||||
```go
|
|
||||||
func (c *Client) ListDevices(ctx context.Context, limit int) ([]Device, error)
|
|
||||||
func (c *Client) ListDevicesWithStatus(ctx context.Context, status string) ([]Device, error)
|
|
||||||
// Added in Plan 01
|
|
||||||
```
|
|
||||||
|
|
||||||
From internal/netbox/types.go:
|
|
||||||
```go
|
|
||||||
type Device struct {
|
|
||||||
ID int
|
|
||||||
Name string
|
|
||||||
AssetTag string
|
|
||||||
CustomFields CustomFields
|
|
||||||
Created time.Time
|
|
||||||
LastUpdated time.Time
|
|
||||||
}
|
|
||||||
type CustomFields struct {
|
|
||||||
HWID string
|
|
||||||
CatalogStatus string
|
|
||||||
ProductURL string
|
|
||||||
FirmwareVersion string
|
|
||||||
TestDate string
|
|
||||||
TestData string
|
|
||||||
AINotes string
|
|
||||||
PhotoURLs []string
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
From internal/ai/client.go:
|
|
||||||
```go
|
|
||||||
type TierClient struct { ... }
|
|
||||||
// TextComplete added in Plan 01:
|
|
||||||
func (c *TierClient) TextComplete(ctx context.Context, prompt string) (string, error)
|
|
||||||
```
|
|
||||||
|
|
||||||
From web/src/lib/api.ts:
|
|
||||||
```typescript
|
|
||||||
export interface InventoryItem {
|
|
||||||
id: number; name: string; asset_tag: string | null; hw_id: string | null;
|
|
||||||
catalog_status: string | null; product_url: string | null;
|
|
||||||
firmware_version: string | null; test_date: string | null;
|
|
||||||
test_data: string | null; ai_notes: string | null; photo_urls: string[];
|
|
||||||
}
|
|
||||||
export const fetchInventory = (): Promise<InventoryItem[]>
|
|
||||||
// Add fetchSearch here — returns InventoryItem[] with same shape
|
|
||||||
```
|
|
||||||
|
|
||||||
From web/src/pages/DashboardPage.tsx:
|
|
||||||
```typescript
|
|
||||||
// Existing local text filter:
|
|
||||||
const [search, setSearch] = useState('')
|
|
||||||
// Filtered locally via useMemo. NL search should add a separate nlQuery state.
|
|
||||||
// When nlQuery is non-empty: show NL results (from GET /api/search) instead of local filter.
|
|
||||||
// When nlQuery is empty: use existing local search behavior unchanged.
|
|
||||||
```
|
|
||||||
</interfaces>
|
|
||||||
</context>
|
|
||||||
|
|
||||||
<tasks>
|
|
||||||
|
|
||||||
<task type="auto" tdd="true">
|
|
||||||
<name>Task 1: SearchHandler — NL query → NetBox filter → device list</name>
|
|
||||||
<files>
|
|
||||||
internal/api/handlers/search.go,
|
|
||||||
internal/api/handlers/search_test.go,
|
|
||||||
internal/api/router.go
|
|
||||||
</files>
|
|
||||||
<behavior>
|
|
||||||
- GET /api/search?q=show+me+free+10GbE+NICs → 200 JSON array of InventoryItem
|
|
||||||
- GET /api/search with empty q → 400 {"error":"q parameter required"}
|
|
||||||
- Tier 1 LLM receives: "Extract NetBox filter parameters from this inventory search query. Return JSON only: {\"catalog_status\": \"...\", \"name_contains\": \"...\", \"tag\": \"...\"}. All fields optional. Query: {user query}"
|
|
||||||
- LLM response parsed; client-side filter applied to ListDevices(ctx, 200) results
|
|
||||||
- name_contains: case-insensitive substring match on device.Name
|
|
||||||
- catalog_status: exact match on CustomFields.CatalogStatus (available → "available", etc.)
|
|
||||||
- tag: ignored for MVP (NetBox tag filtering requires separate API; log "tag filter not implemented")
|
|
||||||
- If LLM parse fails, fall back to simple substring match on device Name against raw query
|
|
||||||
- Result serialized as []map[string]interface{} matching InventoryItem TypeScript shape
|
|
||||||
</behavior>
|
|
||||||
<action>
|
|
||||||
internal/api/handlers/search.go:
|
|
||||||
- SearchHandler struct: nbClient *netbox.Client, tier1 *ai.TierClient
|
|
||||||
- NewSearchHandler(nb *netbox.Client, tier1 *ai.TierClient) *SearchHandler
|
|
||||||
- SearchDevices(w http.ResponseWriter, r *http.Request):
|
|
||||||
1. q := r.URL.Query().Get("q"); if empty → 400
|
|
||||||
2. Sanitize q: strip non-printable chars, trim to 200 chars max
|
|
||||||
3. Call tier1.TextComplete(ctx, nlFilterPrompt(q)) — 5s timeout
|
|
||||||
4. Parse JSON response into struct { CatalogStatus string `json:"catalog_status"`, NameContains string `json:"name_contains"`, Tag string `json:"tag"` }
|
|
||||||
5. If parse fails: log warning, set NameContains = q (fallback)
|
|
||||||
6. devices, _ = nbClient.ListDevices(ctx, 200)
|
|
||||||
7. Apply filters: CatalogStatus match + NameContains match (both case-insensitive)
|
|
||||||
8. Convert filtered devices to response slice using deviceToResponseMap helper
|
|
||||||
9. json.NewEncoder(w).Encode(result)
|
|
||||||
|
|
||||||
deviceToResponseMap converts netbox.Device to map[string]interface{} matching InventoryItem shape:
|
|
||||||
{ "id", "name", "asset_tag" (from AssetTag or nil), "hw_id", "catalog_status",
|
|
||||||
"product_url", "firmware_version", "test_date", "test_data", "ai_notes", "photo_urls" }
|
|
||||||
|
|
||||||
nlFilterPrompt(q string) string — returns the extraction prompt.
|
|
||||||
|
|
||||||
internal/api/handlers/search_test.go:
|
|
||||||
- Use a mock netbox client (simple struct implementing a ListDevices method via interface,
|
|
||||||
or just test the filter logic separately)
|
|
||||||
- Test: empty q returns 400
|
|
||||||
- Test: nlFilter parse failure falls back to name substring match
|
|
||||||
- Test: catalog_status filter correctly narrows results
|
|
||||||
|
|
||||||
internal/api/router.go:
|
|
||||||
- Add searchHandler *handlers.SearchHandler param to NewRouter signature
|
|
||||||
- Add r.Get("/search", searchHandler.SearchDevices) inside r.Route("/api", ...)
|
|
||||||
- nil guard: if searchHandler is nil, return 503 (same pattern as advisorHandler)
|
|
||||||
</action>
|
|
||||||
<verify>
|
|
||||||
<automated>cd /home/mikkel/homelabby && go build ./... && go test ./internal/api/handlers/... -v -count=1 -run TestSearch 2>&1 | tail -20</automated>
|
|
||||||
</verify>
|
|
||||||
<done>
|
|
||||||
GET /api/search?q=... handler compiles and handler tests pass.
|
|
||||||
Router wired. go build clean.
|
|
||||||
</done>
|
|
||||||
</task>
|
|
||||||
|
|
||||||
<task type="auto">
|
|
||||||
<name>Task 2: Frontend NL search bar + main.go wiring</name>
|
|
||||||
<files>
|
|
||||||
web/src/lib/api.ts,
|
|
||||||
web/src/pages/DashboardPage.tsx,
|
|
||||||
web/src/components/inventory/FilterBar.tsx,
|
|
||||||
cmd/hwlab/main.go
|
|
||||||
</files>
|
|
||||||
<action>
|
|
||||||
web/src/lib/api.ts — add fetchSearch:
|
|
||||||
```typescript
|
|
||||||
export const fetchSearch = (q: string): Promise<InventoryItem[]> =>
|
|
||||||
fetchJSON<InventoryItem[]>(`${BASE}/search?q=${encodeURIComponent(q)}`)
|
|
||||||
```
|
|
||||||
|
|
||||||
web/src/pages/DashboardPage.tsx changes:
|
|
||||||
1. Add state: const [nlQuery, setNlQuery] = useState('')
|
|
||||||
2. Add TanStack Query hook:
|
|
||||||
```typescript
|
|
||||||
const { data: searchResults, isLoading: searchLoading } = useQuery({
|
|
||||||
queryKey: ['search', nlQuery],
|
|
||||||
queryFn: () => fetchSearch(nlQuery),
|
|
||||||
enabled: nlQuery.trim().length > 2,
|
|
||||||
staleTime: 30_000,
|
|
||||||
})
|
|
||||||
```
|
|
||||||
3. Display logic: when nlQuery.length > 2, show searchResults (or empty state)
|
|
||||||
instead of the existing `filtered` local results.
|
|
||||||
4. When nlQuery is empty, existing local filter behavior is unchanged.
|
|
||||||
5. Pass nlQuery + setNlQuery + searchLoading to FilterBar as new props.
|
|
||||||
|
|
||||||
web/src/components/inventory/FilterBar.tsx changes:
|
|
||||||
- Add nlQuery string prop + onNlQueryChange (string) => void + nlSearchLoading boolean
|
|
||||||
- Add a second input below (or inline with) the existing search input:
|
|
||||||
- Placeholder: "Ask anything: show me free 10GbE NICs…"
|
|
||||||
- Tailwind: full-width, border-volt/40, bg-[#0a0a0a], text-white, focus:border-volt,
|
|
||||||
rounded-card, px-3 py-2 text-sm
|
|
||||||
- Right side: if nlSearchLoading show <Loader2 className="w-4 h-4 animate-spin text-volt" />
|
|
||||||
- onBlur / onChange with 400ms debounce: call onNlQueryChange
|
|
||||||
- Use a local useState for the input value; debounce via useEffect + setTimeout clearing pattern
|
|
||||||
- Keep existing search + status filter inputs intact — NL search is additive
|
|
||||||
|
|
||||||
cmd/hwlab/main.go:
|
|
||||||
- Import internal/api/handlers (already imported)
|
|
||||||
- After building tier1 TierClient: searchHandler := handlers.NewSearchHandler(nbClient, tier1)
|
|
||||||
- Pass searchHandler to api.NewRouter(...) — add as new final param
|
|
||||||
- NOTE: tier1 is already constructed as `ai.NewTierClient(cfg.AI.Tier1)` — pass it directly
|
|
||||||
but SearchHandler needs *ai.TierClient not ai.AIClient; adjust if needed (TierClient is a concrete type)
|
|
||||||
</action>
|
|
||||||
<verify>
|
|
||||||
<automated>cd /home/mikkel/homelabby && go build ./... && cd web && npm run build 2>&1 | tail -20</automated>
|
|
||||||
</verify>
|
|
||||||
<done>
|
|
||||||
go build and npm run build both pass with no errors.
|
|
||||||
Dashboard FilterBar renders NL search input. Typing a query with > 2 chars triggers
|
|
||||||
GET /api/search and displays results using existing ItemCard/ItemRow components.
|
|
||||||
Existing local search filter still works when nlQuery is empty.
|
|
||||||
</done>
|
|
||||||
</task>
|
|
||||||
|
|
||||||
</tasks>
|
|
||||||
|
|
||||||
<threat_model>
|
|
||||||
## Trust Boundaries
|
|
||||||
|
|
||||||
| Boundary | Description |
|
|
||||||
|----------|-------------|
|
|
||||||
| browser → GET /api/search | User-supplied NL query crosses HTTP boundary |
|
|
||||||
| search handler → Tier 1 LLM | Sanitized query forwarded to local oMLX |
|
|
||||||
| LLM output → NetBox filter | Structured JSON from LLM used to filter devices |
|
|
||||||
|
|
||||||
## STRIDE Threat Register
|
|
||||||
|
|
||||||
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|
|
||||||
|-----------|----------|-----------|-------------|-----------------|
|
|
||||||
| T-07-05 | Tampering | NL query → LLM prompt | mitigate | Strip non-printable chars, truncate to 200 chars before building prompt |
|
|
||||||
| T-07-06 | Tampering | LLM output → filter params | mitigate | Parse only known fields (catalog_status, name_contains, tag); ignore unknown keys; fallback on parse failure |
|
|
||||||
| T-07-07 | Denial of Service | GET /api/search fanout | accept | ListDevices(200) is bounded; Tier 1 local inference is fast; no per-user rate limiting needed for single-operator tool |
|
|
||||||
| T-07-08 | Information Disclosure | search results | accept | All results are local NetBox inventory; no cross-tenant risk in single-operator homelab |
|
|
||||||
</threat_model>
|
|
||||||
|
|
||||||
<verification>
|
|
||||||
1. `go build ./...` passes
|
|
||||||
2. `cd web && npm run build` passes
|
|
||||||
3. Manual: GET http://localhost:8080/api/search?q=show+me+available+NICs returns JSON array
|
|
||||||
4. Manual: GET http://localhost:8080/api/search (no q) returns 400
|
|
||||||
5. Dashboard: NL search input visible below existing filter bar; typing triggers spinner then results
|
|
||||||
</verification>
|
|
||||||
|
|
||||||
<success_criteria>
|
|
||||||
- GET /api/search?q=... returns filtered InventoryItem array using Tier 1 NL→filter translation
|
|
||||||
- Query sanitized (non-printable stripped, 200 char max) before LLM
|
|
||||||
- LLM parse failure falls back to name substring match (never 500)
|
|
||||||
- Dashboard NL search bar triggers live search; existing local filter unchanged when NL query empty
|
|
||||||
- go build and npm run build both clean
|
|
||||||
</success_criteria>
|
|
||||||
|
|
||||||
<output>
|
|
||||||
After completion, create `.planning/phases/07-research-agent-search/07-02-SUMMARY.md`
|
|
||||||
</output>
|
|
||||||
|
|
@ -1,118 +0,0 @@
|
||||||
---
|
|
||||||
phase: 07-research-agent-search
|
|
||||||
plan: "02"
|
|
||||||
subsystem: search
|
|
||||||
tags: [search, nlp, ai, frontend, go, react]
|
|
||||||
dependency_graph:
|
|
||||||
requires: [07-01]
|
|
||||||
provides: [search-endpoint, nl-search-ui]
|
|
||||||
affects: [DashboardPage, FilterBar, api-router]
|
|
||||||
tech_stack:
|
|
||||||
added: []
|
|
||||||
patterns:
|
|
||||||
- NL query sanitize → LLM extraction → in-memory filter (no raw text to NetBox)
|
|
||||||
- TDD: failing tests written first, handler implemented to green
|
|
||||||
- Debounced NL input (400ms) with TanStack Query (enabled guard on length > 2)
|
|
||||||
key_files:
|
|
||||||
created:
|
|
||||||
- internal/api/handlers/search.go
|
|
||||||
- internal/api/handlers/search_test.go
|
|
||||||
modified:
|
|
||||||
- internal/api/router.go
|
|
||||||
- cmd/hwlab/main.go
|
|
||||||
- web/src/lib/api.ts
|
|
||||||
- web/src/components/inventory/FilterBar.tsx
|
|
||||||
- web/src/pages/DashboardPage.tsx
|
|
||||||
decisions:
|
|
||||||
- Used SearchNetBoxClient and SearchAIClient narrow interfaces in search.go for testability without importing ai package in tests
|
|
||||||
- extractJSON helper strips markdown code fences from LLM response before JSON parse
|
|
||||||
- NL search row placed below existing filter row (additive, not replacing) to keep local filter intact
|
|
||||||
- displayLoading/displayItems derived state avoids duplicating isLoading/searchLoading logic
|
|
||||||
metrics:
|
|
||||||
duration: "~12 minutes"
|
|
||||||
completed: "2026-04-10"
|
|
||||||
tasks_completed: 2
|
|
||||||
files_changed: 7
|
|
||||||
---
|
|
||||||
|
|
||||||
# Phase 07 Plan 02: Natural Language Search Summary
|
|
||||||
|
|
||||||
Natural language inventory search via Tier 1 LLM (Gemma 4): user query sanitized, translated to structured filter params (catalog_status + name_contains), applied in-memory against ListDevices(200), returned as InventoryItem JSON; dashboard gains a debounced NL search input with volt accent styling and fallback to substring match on LLM parse failure.
|
|
||||||
|
|
||||||
## Tasks Completed
|
|
||||||
|
|
||||||
| Task | Name | Commit | Files |
|
|
||||||
|------|------|--------|-------|
|
|
||||||
| 1 | SearchHandler — NL query to NetBox filter | 9db7707 | search.go, search_test.go, router.go, main.go |
|
|
||||||
| 2 | Frontend NL search bar + api.ts wiring | 7db093c | api.ts, FilterBar.tsx, DashboardPage.tsx |
|
|
||||||
|
|
||||||
## What Was Built
|
|
||||||
|
|
||||||
### Backend (Task 1)
|
|
||||||
|
|
||||||
`internal/api/handlers/search.go` provides `SearchHandler` with `SearchDevices(w, r)`:
|
|
||||||
|
|
||||||
1. Validates `q` param — returns 400 if empty
|
|
||||||
2. Sanitizes query: strips non-printable chars, truncates to 200 runes (T-07-05)
|
|
||||||
3. Calls `tier1.TextComplete` with an extraction prompt requesting `{"catalog_status","name_contains","tag"}` JSON
|
|
||||||
4. Calls `extractJSON` to strip markdown fences before parse (LLM sometimes wraps in code blocks)
|
|
||||||
5. On parse failure: logs warning, falls back to `NameContains = rawQuery` (never 500)
|
|
||||||
6. `tag` field logged as not implemented and ignored (MVP)
|
|
||||||
7. Fetches `ListDevices(ctx, 200)`, applies in-memory filter, encodes result
|
|
||||||
|
|
||||||
Router wired: `GET /api/search` with nil-guard 503 fallback. `main.go` constructs `handlers.NewSearchHandler(nbClient, tier1)` and passes to `NewRouter`.
|
|
||||||
|
|
||||||
TDD: 4 tests written before implementation — `TestSearch_EmptyQ`, `TestSearch_LLMParseFallback`, `TestSearch_CatalogStatusFilter`, `TestSearch_NameContainsAndStatus` — all pass.
|
|
||||||
|
|
||||||
### Frontend (Task 2)
|
|
||||||
|
|
||||||
`web/src/lib/api.ts`: `fetchSearch(q)` added — calls `GET /api/search?q=<encoded>`.
|
|
||||||
|
|
||||||
`FilterBar.tsx` restructured to two rows:
|
|
||||||
- Row 1: existing local text search + status select + item count + view toggle (unchanged)
|
|
||||||
- Row 2: NL input with `Sparkles` icon (volt/60 tint), `border-volt/20` → `focus:border-volt`, `Loader2` spinner when `nlSearchLoading`
|
|
||||||
- 400ms debounce via `useEffect + setTimeout` pattern; local `nlInputValue` state, propagates via `onNlQueryChange`
|
|
||||||
|
|
||||||
`DashboardPage.tsx`:
|
|
||||||
- `nlQuery` state + `useQuery({ queryKey: ['search', nlQuery], enabled: nlQuery.trim().length > 2 })`
|
|
||||||
- `displayItems = nlQuery > 2 ? searchResults : filtered` — local filter fully preserved when NL empty
|
|
||||||
- Loading/empty state messages adapt to NL vs local mode
|
|
||||||
|
|
||||||
## Threat Mitigations Applied
|
|
||||||
|
|
||||||
| Threat | Mitigation |
|
|
||||||
|--------|-----------|
|
|
||||||
| T-07-05 | `sanitizeQuery`: non-printable chars stripped, truncated to 200 runes |
|
|
||||||
| T-07-06 | Only `catalog_status`, `name_contains`, `tag` extracted; unknown keys ignored; fallback on parse failure |
|
|
||||||
|
|
||||||
## Deviations from Plan
|
|
||||||
|
|
||||||
### Auto-added: extractJSON helper
|
|
||||||
|
|
||||||
**Found during:** Task 1 implementation
|
|
||||||
**Issue:** LLMs commonly wrap JSON responses in markdown code fences (` ```json ... ``` `), which breaks `json.Unmarshal` directly
|
|
||||||
**Fix:** Added `extractJSON(s string) string` that finds first `{` and last `}` to extract the JSON object before parsing
|
|
||||||
**Files modified:** internal/api/handlers/search.go
|
|
||||||
**Rule:** Rule 2 — missing critical functionality (robustness of LLM output parsing)
|
|
||||||
|
|
||||||
No other deviations — plan executed as specified.
|
|
||||||
|
|
||||||
## Known Stubs
|
|
||||||
|
|
||||||
None — all data paths are wired end-to-end.
|
|
||||||
|
|
||||||
## Threat Flags
|
|
||||||
|
|
||||||
None — no new network endpoints or trust boundaries beyond what the plan specified.
|
|
||||||
|
|
||||||
## Self-Check: PASSED
|
|
||||||
|
|
||||||
- internal/api/handlers/search.go: exists
|
|
||||||
- internal/api/handlers/search_test.go: exists, 4 tests pass
|
|
||||||
- internal/api/router.go: GET /api/search wired
|
|
||||||
- web/src/lib/api.ts: fetchSearch exported
|
|
||||||
- web/src/pages/DashboardPage.tsx: nlQuery state + useQuery present
|
|
||||||
- web/src/components/inventory/FilterBar.tsx: NL input with debounce present
|
|
||||||
- go build ./...: clean
|
|
||||||
- npm run build: clean (built in 3.14s)
|
|
||||||
- Commits 9db7707 and 7db093c: verified in git log
|
|
||||||
|
|
@ -1,70 +0,0 @@
|
||||||
# Phase 7: Research Agent & Search - Context
|
|
||||||
|
|
||||||
**Gathered:** 2026-04-10
|
|
||||||
**Status:** Ready for planning
|
|
||||||
**Mode:** Auto-generated (autonomous mode)
|
|
||||||
|
|
||||||
<domain>
|
|
||||||
## Phase Boundary
|
|
||||||
|
|
||||||
Items flagged needs_research are automatically enriched by a SearXNG research agent, and any inventory question can be answered via natural language search. This phase delivers the SearXNG Tier 2 client, research agent that consumes needs_research items, and natural language search endpoint.
|
|
||||||
|
|
||||||
</domain>
|
|
||||||
|
|
||||||
<decisions>
|
|
||||||
## Implementation Decisions
|
|
||||||
|
|
||||||
### SearXNG Client
|
|
||||||
- HTTP client to SearXNG JSON API at http://10.5.0.129:8080/search
|
|
||||||
- Sanitize queries before dispatch
|
|
||||||
- Return list of result objects (title, url, snippet)
|
|
||||||
|
|
||||||
### Research Agent
|
|
||||||
- Background worker that polls NetBox for catalog_status=needs_research items
|
|
||||||
- For each item: query SearXNG for product info, send results to Tier 2 LLM, extract structured data
|
|
||||||
- Update NetBox device with enriched data, set catalog_status=researched
|
|
||||||
- Run periodically (every 10 minutes) or on-demand via POST /api/research/trigger
|
|
||||||
|
|
||||||
### Natural Language Search
|
|
||||||
- Endpoint: GET /api/search?q=...
|
|
||||||
- Use Tier 1 (Gemma 4) to translate query to NetBox filter params
|
|
||||||
- Return matching devices
|
|
||||||
|
|
||||||
### Frontend
|
|
||||||
- Search bar in dashboard top
|
|
||||||
- Results page reuses dashboard cards
|
|
||||||
|
|
||||||
</decisions>
|
|
||||||
|
|
||||||
<code_context>
|
|
||||||
## Existing Code Insights
|
|
||||||
|
|
||||||
### Reusable Assets from prior phases
|
|
||||||
- internal/ai/ — AIClient, TierClient, orchestrator
|
|
||||||
- internal/ai/research.go — ResearchClient interface (NoOp from Phase 2)
|
|
||||||
- internal/netbox/client.go — ListDevices, GetDevice, PatchCustomFields
|
|
||||||
- internal/inventory/quality_gate.go + catalog_updater.go
|
|
||||||
- internal/api/router.go
|
|
||||||
- web/src/lib/api.ts
|
|
||||||
|
|
||||||
### Integration Points
|
|
||||||
- Replace NoOpResearchClient with real SearXNGResearchClient
|
|
||||||
- Add internal/research/ package
|
|
||||||
- Add internal/api/handlers/search.go
|
|
||||||
- Frontend: dashboard search bar wires GET /api/search
|
|
||||||
</code_context>
|
|
||||||
|
|
||||||
<specifics>
|
|
||||||
## Specific Ideas
|
|
||||||
|
|
||||||
- SearXNG client uses standard net/http
|
|
||||||
- Research agent runs as goroutine started from main.go
|
|
||||||
- Search endpoint translates "show me free 10GbE NICs" to filter: category=NIC, status=available, tags has 10gbe
|
|
||||||
</specifics>
|
|
||||||
|
|
||||||
<deferred>
|
|
||||||
## Deferred Ideas
|
|
||||||
|
|
||||||
- Saved searches
|
|
||||||
- Search history
|
|
||||||
</deferred>
|
|
||||||
|
|
@ -1,30 +0,0 @@
|
||||||
---
|
|
||||||
status: partial
|
|
||||||
phase: 07-research-agent-search
|
|
||||||
source: [07-VERIFICATION.md]
|
|
||||||
started: 2026-04-10
|
|
||||||
updated: 2026-04-10
|
|
||||||
---
|
|
||||||
|
|
||||||
## Tests
|
|
||||||
|
|
||||||
### 1. SearXNG live query
|
|
||||||
expected: SearXNGClient.Search returns real results from http://10.5.0.129:8080
|
|
||||||
result: [pending — needs runtime test]
|
|
||||||
|
|
||||||
### 2. NL search end-to-end
|
|
||||||
expected: Type "show me all 10GbE NICs" in dashboard, see filtered results
|
|
||||||
result: [pending — needs Gemma 4 running on Mac Mini]
|
|
||||||
|
|
||||||
### 3. Research agent enrichment cycle
|
|
||||||
expected: Item with catalog_status=needs_research is enriched with SearXNG data and advances to researched after 10min
|
|
||||||
result: [pending — needs real items + Tier 2 OpenRouter key]
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
total: 3
|
|
||||||
pending: 3
|
|
||||||
|
|
||||||
## Gaps
|
|
||||||
|
|
||||||
Requires Gemma 4 running on Mac Mini for NL search and OpenRouter key for Tier 2 enrichment.
|
|
||||||
|
|
@ -1,36 +0,0 @@
|
||||||
---
|
|
||||||
phase: 07-research-agent-search
|
|
||||||
verified: 2026-04-10
|
|
||||||
status: human_needed
|
|
||||||
score: 3/3 (code) — live LLM + SearXNG validation pending
|
|
||||||
overrides_applied: 0
|
|
||||||
---
|
|
||||||
|
|
||||||
# Phase 7 Verification
|
|
||||||
|
|
||||||
## Goal
|
|
||||||
Items flagged needs_research are automatically enriched by a SearXNG research agent, and any inventory question can be answered via natural language search.
|
|
||||||
|
|
||||||
## Code-Level Verification (Complete)
|
|
||||||
|
|
||||||
| # | Success Criterion | Status | Evidence |
|
|
||||||
|---|------|--------|----------|
|
|
||||||
| 1 | needs_research items auto-enriched by SearXNG → researched | ✓ | `internal/research/agent.go` RunOnce + 10min ticker |
|
|
||||||
| 2 | Natural language search returns filtered inventory | ✓ | `internal/api/handlers/search.go` Tier1 NL→filter translation |
|
|
||||||
| 3 | SearXNG queries sanitized | ✓ | `SanitizeQuery` regex `[^a-zA-Z0-9 .\-_]+` |
|
|
||||||
|
|
||||||
## All 2 Requirements Covered
|
|
||||||
AI-04 (SearXNG research) + UI-03 (NL search) — implemented and tested.
|
|
||||||
|
|
||||||
## Test Results
|
|
||||||
- `go test ./...` — all packages pass
|
|
||||||
- `cd web && npm run build` — clean
|
|
||||||
|
|
||||||
## Human Verification Required
|
|
||||||
|
|
||||||
1. Real SearXNG end-to-end test (SearXNG is live at 10.5.0.129:8080 — no auth needed)
|
|
||||||
2. Real Gemma 4 NL→filter parsing accuracy
|
|
||||||
3. Live needs_research enrichment cycle (requires real items in needs_research state + Tier 2 OpenRouter key)
|
|
||||||
|
|
||||||
## Status
|
|
||||||
`human_needed` — code complete, requires live LLM and real inventory data for end-to-end validation.
|
|
||||||
|
|
@ -21,7 +21,6 @@ import (
|
||||||
"git.georgsen.dk/hwlab/internal/netbox"
|
"git.georgsen.dk/hwlab/internal/netbox"
|
||||||
"git.georgsen.dk/hwlab/internal/printer"
|
"git.georgsen.dk/hwlab/internal/printer"
|
||||||
"git.georgsen.dk/hwlab/internal/queue"
|
"git.georgsen.dk/hwlab/internal/queue"
|
||||||
"git.georgsen.dk/hwlab/internal/research"
|
|
||||||
"git.georgsen.dk/hwlab/internal/store"
|
"git.georgsen.dk/hwlab/internal/store"
|
||||||
"git.georgsen.dk/hwlab/internal/usb"
|
"git.georgsen.dk/hwlab/internal/usb"
|
||||||
)
|
)
|
||||||
|
|
@ -122,13 +121,6 @@ func main() {
|
||||||
log.Printf("HWLAB_DATABASE_URL not set — advisor endpoints disabled")
|
log.Printf("HWLAB_DATABASE_URL not set — advisor endpoints disabled")
|
||||||
}
|
}
|
||||||
|
|
||||||
// Research agent — enriches needs_research items via SearXNG + Tier 2 LLM.
|
|
||||||
searxngClient := research.NewSearXNGClient(cfg.SearXNGURL)
|
|
||||||
researchAgent := research.NewAgent(nbClient, searxngClient, tier2, catalogUpdater)
|
|
||||||
go researchAgent.Start(ctx, 10*time.Minute)
|
|
||||||
researchHandler := handlers.NewResearchHandler(researchAgent)
|
|
||||||
searchHandler := handlers.NewSearchHandler(nbClient, tier1)
|
|
||||||
|
|
||||||
// Wire USB Manager events to cable tester driver when a RoleCableTester device connects.
|
// Wire USB Manager events to cable tester driver when a RoleCableTester device connects.
|
||||||
// Currently a no-op stub — wires the plumbing for Phase 5 hardware integration.
|
// Currently a no-op stub — wires the plumbing for Phase 5 hardware integration.
|
||||||
go func() {
|
go func() {
|
||||||
|
|
@ -142,7 +134,7 @@ func main() {
|
||||||
}
|
}
|
||||||
}()
|
}()
|
||||||
|
|
||||||
router := api.NewRouter(staticFS, intakeHandler, inventoryHandler, labelHandler, usbEventsHandler, testHandler, advisorHandler, researchHandler, searchHandler)
|
router := api.NewRouter(staticFS, intakeHandler, inventoryHandler, labelHandler, usbEventsHandler, testHandler, advisorHandler)
|
||||||
addr := fmt.Sprintf("%s:%d", cfg.Host, cfg.Port)
|
addr := fmt.Sprintf("%s:%d", cfg.Host, cfg.Port)
|
||||||
log.Printf("HWLab starting on %s", addr)
|
log.Printf("HWLab starting on %s", addr)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -92,28 +92,6 @@ func (c *TierClient) AnalyzePhotos(ctx context.Context, req IntakeRequest) (*Int
|
||||||
return &result, nil
|
return &result, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// TextComplete sends a text-only (non-vision) chat completion to the configured model.
|
|
||||||
// Used by the research agent for hardware enrichment prompts that require no images.
|
|
||||||
// Returns the raw string content of the first response choice.
|
|
||||||
func (c *TierClient) TextComplete(ctx context.Context, prompt string) (string, error) {
|
|
||||||
tctx, cancel := context.WithTimeout(ctx, c.timeout)
|
|
||||||
defer cancel()
|
|
||||||
|
|
||||||
resp, err := c.client.CreateChatCompletion(tctx, openai.ChatCompletionRequest{
|
|
||||||
Model: c.model,
|
|
||||||
Messages: []openai.ChatCompletionMessage{
|
|
||||||
{Role: openai.ChatMessageRoleUser, Content: prompt},
|
|
||||||
},
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
return "", fmt.Errorf("text complete: %w", err)
|
|
||||||
}
|
|
||||||
if len(resp.Choices) == 0 {
|
|
||||||
return "", fmt.Errorf("text complete: no choices in response")
|
|
||||||
}
|
|
||||||
return resp.Choices[0].Message.Content, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// buildIntakePromptWithCount is a package-internal shim to the prompts package.
|
// buildIntakePromptWithCount is a package-internal shim to the prompts package.
|
||||||
func buildIntakePromptWithCount(n int) string {
|
func buildIntakePromptWithCount(n int) string {
|
||||||
return prompts.BuildIntakePrompt(n)
|
return prompts.BuildIntakePrompt(n)
|
||||||
|
|
|
||||||
|
|
@ -1,30 +0,0 @@
|
||||||
package handlers
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"encoding/json"
|
|
||||||
"net/http"
|
|
||||||
|
|
||||||
"git.georgsen.dk/hwlab/internal/research"
|
|
||||||
)
|
|
||||||
|
|
||||||
// ResearchHandler handles research-related API endpoints.
|
|
||||||
type ResearchHandler struct {
|
|
||||||
agent *research.Agent
|
|
||||||
}
|
|
||||||
|
|
||||||
// NewResearchHandler creates a ResearchHandler backed by the given Agent.
|
|
||||||
func NewResearchHandler(agent *research.Agent) *ResearchHandler {
|
|
||||||
return &ResearchHandler{agent: agent}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TriggerResearch handles POST /api/research/trigger.
|
|
||||||
// It fires a RunOnce cycle in a background goroutine and responds 202 Accepted immediately.
|
|
||||||
func (h *ResearchHandler) TriggerResearch(w http.ResponseWriter, r *http.Request) {
|
|
||||||
go func() {
|
|
||||||
_, _ = h.agent.RunOnce(context.Background())
|
|
||||||
}()
|
|
||||||
w.Header().Set("Content-Type", "application/json")
|
|
||||||
w.WriteHeader(http.StatusAccepted)
|
|
||||||
json.NewEncoder(w).Encode(map[string]string{"status": "accepted"})
|
|
||||||
}
|
|
||||||
|
|
@ -1,187 +0,0 @@
|
||||||
package handlers
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"encoding/json"
|
|
||||||
"log"
|
|
||||||
"net/http"
|
|
||||||
"strings"
|
|
||||||
"unicode"
|
|
||||||
|
|
||||||
"git.georgsen.dk/hwlab/internal/netbox"
|
|
||||||
)
|
|
||||||
|
|
||||||
// SearchNetBoxClient is the narrow interface the search handler needs.
|
|
||||||
type SearchNetBoxClient interface {
|
|
||||||
ListDevices(ctx context.Context, limit int) ([]netbox.Device, error)
|
|
||||||
}
|
|
||||||
|
|
||||||
// SearchAIClient is the narrow interface for NL→filter translation.
|
|
||||||
type SearchAIClient interface {
|
|
||||||
TextComplete(ctx context.Context, prompt string) (string, error)
|
|
||||||
}
|
|
||||||
|
|
||||||
// nlFilter holds the parsed output from the LLM (T-07-06: only known fields accepted).
|
|
||||||
type nlFilter struct {
|
|
||||||
CatalogStatus string `json:"catalog_status"`
|
|
||||||
NameContains string `json:"name_contains"`
|
|
||||||
Tag string `json:"tag"`
|
|
||||||
}
|
|
||||||
|
|
||||||
// SearchHandler handles GET /api/search?q=...
|
|
||||||
type SearchHandler struct {
|
|
||||||
nbClient SearchNetBoxClient
|
|
||||||
tier1 SearchAIClient
|
|
||||||
}
|
|
||||||
|
|
||||||
// NewSearchHandler creates a SearchHandler.
|
|
||||||
func NewSearchHandler(nb SearchNetBoxClient, tier1 SearchAIClient) *SearchHandler {
|
|
||||||
return &SearchHandler{nbClient: nb, tier1: tier1}
|
|
||||||
}
|
|
||||||
|
|
||||||
// SearchDevices handles GET /api/search?q=<natural language query>
|
|
||||||
// It translates the query to NetBox filter params via Tier 1 LLM, then fetches
|
|
||||||
// and filters devices accordingly.
|
|
||||||
func (h *SearchHandler) SearchDevices(w http.ResponseWriter, r *http.Request) {
|
|
||||||
w.Header().Set("Content-Type", "application/json")
|
|
||||||
|
|
||||||
q := r.URL.Query().Get("q")
|
|
||||||
if strings.TrimSpace(q) == "" {
|
|
||||||
w.WriteHeader(http.StatusBadRequest)
|
|
||||||
_ = json.NewEncoder(w).Encode(map[string]string{"error": "q parameter required"})
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
// Sanitize: strip non-printable chars, truncate to 200 chars (T-07-05).
|
|
||||||
q = sanitizeQuery(q)
|
|
||||||
|
|
||||||
// Translate NL query to structured filter via Tier 1 LLM.
|
|
||||||
filter := h.parseFilter(r.Context(), q)
|
|
||||||
|
|
||||||
// Fetch up to 200 devices from NetBox.
|
|
||||||
devices, err := h.nbClient.ListDevices(r.Context(), 200)
|
|
||||||
if err != nil {
|
|
||||||
w.WriteHeader(http.StatusBadGateway)
|
|
||||||
_ = json.NewEncoder(w).Encode(map[string]string{"error": "netbox unavailable: " + err.Error()})
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
// Apply filters in-memory and serialize.
|
|
||||||
results := make([]map[string]interface{}, 0)
|
|
||||||
for _, d := range devices {
|
|
||||||
if !matchesFilter(d, filter) {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
results = append(results, deviceToSearchResponse(d))
|
|
||||||
}
|
|
||||||
|
|
||||||
_ = json.NewEncoder(w).Encode(results)
|
|
||||||
}
|
|
||||||
|
|
||||||
// parseFilter calls the LLM and parses its JSON output into an nlFilter.
|
|
||||||
// On any failure, falls back to a name substring match on the raw query.
|
|
||||||
func (h *SearchHandler) parseFilter(ctx context.Context, q string) nlFilter {
|
|
||||||
resp, err := h.tier1.TextComplete(ctx, nlFilterPrompt(q))
|
|
||||||
if err != nil {
|
|
||||||
log.Printf("search: LLM TextComplete error: %v — falling back to substring match", err)
|
|
||||||
return nlFilter{NameContains: q}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Extract JSON from response (LLM may wrap JSON in markdown code fences).
|
|
||||||
jsonStr := extractJSON(resp)
|
|
||||||
|
|
||||||
var f nlFilter
|
|
||||||
if err := json.Unmarshal([]byte(jsonStr), &f); err != nil {
|
|
||||||
log.Printf("search: LLM JSON parse failed: %v — falling back to substring match (raw: %.200s)", err, resp)
|
|
||||||
return nlFilter{NameContains: q}
|
|
||||||
}
|
|
||||||
|
|
||||||
if f.Tag != "" {
|
|
||||||
log.Printf("search: tag filter %q not implemented for MVP — ignoring", f.Tag)
|
|
||||||
f.Tag = ""
|
|
||||||
}
|
|
||||||
|
|
||||||
return f
|
|
||||||
}
|
|
||||||
|
|
||||||
// matchesFilter returns true if device d satisfies all active filter criteria.
|
|
||||||
func matchesFilter(d netbox.Device, f nlFilter) bool {
|
|
||||||
// catalog_status: exact match (case-insensitive) when set (T-07-06).
|
|
||||||
if f.CatalogStatus != "" {
|
|
||||||
if !strings.EqualFold(d.CustomFields.CatalogStatus, f.CatalogStatus) {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// name_contains: case-insensitive substring match when set.
|
|
||||||
if f.NameContains != "" {
|
|
||||||
if !strings.Contains(strings.ToLower(d.Name), strings.ToLower(f.NameContains)) {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return true
|
|
||||||
}
|
|
||||||
|
|
||||||
// deviceToSearchResponse converts a netbox.Device to the InventoryItem JSON shape.
|
|
||||||
func deviceToSearchResponse(d netbox.Device) map[string]interface{} {
|
|
||||||
cf := d.CustomFields
|
|
||||||
urls := cf.PhotoURLs
|
|
||||||
if urls == nil {
|
|
||||||
urls = []string{}
|
|
||||||
}
|
|
||||||
var assetTag interface{} = nil
|
|
||||||
if d.AssetTag != "" {
|
|
||||||
assetTag = d.AssetTag
|
|
||||||
}
|
|
||||||
return map[string]interface{}{
|
|
||||||
"id": d.ID,
|
|
||||||
"name": d.Name,
|
|
||||||
"asset_tag": assetTag,
|
|
||||||
"hw_id": nilIfEmpty(cf.HWID),
|
|
||||||
"catalog_status": nilIfEmpty(cf.CatalogStatus),
|
|
||||||
"product_url": nilIfEmpty(cf.ProductURL),
|
|
||||||
"firmware_version": nilIfEmpty(cf.FirmwareVersion),
|
|
||||||
"test_date": nilIfEmpty(cf.TestDate),
|
|
||||||
"test_data": nilIfEmpty(cf.TestData),
|
|
||||||
"ai_notes": nilIfEmpty(cf.AINotes),
|
|
||||||
"photo_urls": urls,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func nilIfEmpty(s string) interface{} {
|
|
||||||
if s == "" {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
return s
|
|
||||||
}
|
|
||||||
|
|
||||||
// nlFilterPrompt builds the LLM extraction prompt for a given user query.
|
|
||||||
func nlFilterPrompt(q string) string {
|
|
||||||
return `Extract NetBox filter parameters from this inventory search query. Return JSON only: {"catalog_status": "...", "name_contains": "...", "tag": "..."}. All fields optional. Use empty string for fields that don't apply. Valid catalog_status values: draft, indexed, needs_research, researched, complete, available. Query: ` + q
|
|
||||||
}
|
|
||||||
|
|
||||||
// sanitizeQuery strips non-printable characters and truncates to 200 chars.
|
|
||||||
func sanitizeQuery(q string) string {
|
|
||||||
var sb strings.Builder
|
|
||||||
for _, r := range q {
|
|
||||||
if unicode.IsPrint(r) {
|
|
||||||
sb.WriteRune(r)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
result := sb.String()
|
|
||||||
runes := []rune(result)
|
|
||||||
if len(runes) > 200 {
|
|
||||||
result = string(runes[:200])
|
|
||||||
}
|
|
||||||
return strings.TrimSpace(result)
|
|
||||||
}
|
|
||||||
|
|
||||||
// extractJSON attempts to extract a JSON object from a string that may contain
|
|
||||||
// markdown code fences or surrounding text.
|
|
||||||
func extractJSON(s string) string {
|
|
||||||
start := strings.Index(s, "{")
|
|
||||||
end := strings.LastIndex(s, "}")
|
|
||||||
if start >= 0 && end > start {
|
|
||||||
return s[start : end+1]
|
|
||||||
}
|
|
||||||
return s
|
|
||||||
}
|
|
||||||
|
|
@ -1,138 +0,0 @@
|
||||||
package handlers_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"encoding/json"
|
|
||||||
"net/http"
|
|
||||||
"net/http/httptest"
|
|
||||||
"strings"
|
|
||||||
"testing"
|
|
||||||
|
|
||||||
"git.georgsen.dk/hwlab/internal/api/handlers"
|
|
||||||
"git.georgsen.dk/hwlab/internal/netbox"
|
|
||||||
)
|
|
||||||
|
|
||||||
// mockNetBoxClient satisfies the SearchNetBoxClient interface.
|
|
||||||
type mockNetBoxClient struct {
|
|
||||||
devices []netbox.Device
|
|
||||||
err error
|
|
||||||
}
|
|
||||||
|
|
||||||
func (m *mockNetBoxClient) ListDevices(_ context.Context, _ int) ([]netbox.Device, error) {
|
|
||||||
return m.devices, m.err
|
|
||||||
}
|
|
||||||
|
|
||||||
// mockAIClient returns canned JSON for TextComplete.
|
|
||||||
type mockAIClient struct {
|
|
||||||
response string
|
|
||||||
err error
|
|
||||||
}
|
|
||||||
|
|
||||||
func (m *mockAIClient) TextComplete(_ context.Context, _ string) (string, error) {
|
|
||||||
return m.response, m.err
|
|
||||||
}
|
|
||||||
|
|
||||||
func testDevices() []netbox.Device {
|
|
||||||
return []netbox.Device{
|
|
||||||
{ID: 1, Name: "10GbE NIC Intel X550", CustomFields: netbox.CustomFields{CatalogStatus: "available", HWID: "HW-0001"}},
|
|
||||||
{ID: 2, Name: "Raspberry Pi 4B", CustomFields: netbox.CustomFields{CatalogStatus: "draft", HWID: "HW-0002"}},
|
|
||||||
{ID: 3, Name: "10GbE NIC Mellanox", CustomFields: netbox.CustomFields{CatalogStatus: "complete", HWID: "HW-0003"}},
|
|
||||||
{ID: 4, Name: "USB-C Hub", CustomFields: netbox.CustomFields{CatalogStatus: "available", HWID: "HW-0004"}},
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestSearch_EmptyQ verifies that missing q parameter returns 400.
|
|
||||||
func TestSearch_EmptyQ(t *testing.T) {
|
|
||||||
h := handlers.NewSearchHandler(&mockNetBoxClient{}, &mockAIClient{})
|
|
||||||
req := httptest.NewRequest(http.MethodGet, "/api/search", nil)
|
|
||||||
w := httptest.NewRecorder()
|
|
||||||
h.SearchDevices(w, req)
|
|
||||||
|
|
||||||
if w.Code != http.StatusBadRequest {
|
|
||||||
t.Fatalf("expected 400, got %d", w.Code)
|
|
||||||
}
|
|
||||||
var body map[string]string
|
|
||||||
if err := json.NewDecoder(w.Body).Decode(&body); err != nil {
|
|
||||||
t.Fatalf("decode: %v", err)
|
|
||||||
}
|
|
||||||
if !strings.Contains(body["error"], "q parameter") {
|
|
||||||
t.Errorf("expected error about q parameter, got %q", body["error"])
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestSearch_LLMParseFallback verifies that when LLM returns unparseable JSON,
|
|
||||||
// the handler falls back to name substring match against the raw query.
|
|
||||||
func TestSearch_LLMParseFallback(t *testing.T) {
|
|
||||||
nb := &mockNetBoxClient{devices: testDevices()}
|
|
||||||
ai := &mockAIClient{response: "not valid json {{ broken"}
|
|
||||||
h := handlers.NewSearchHandler(nb, ai)
|
|
||||||
|
|
||||||
req := httptest.NewRequest(http.MethodGet, "/api/search?q=NIC", nil)
|
|
||||||
w := httptest.NewRecorder()
|
|
||||||
h.SearchDevices(w, req)
|
|
||||||
|
|
||||||
if w.Code != http.StatusOK {
|
|
||||||
t.Fatalf("expected 200, got %d (body: %s)", w.Code, w.Body.String())
|
|
||||||
}
|
|
||||||
var items []map[string]interface{}
|
|
||||||
if err := json.NewDecoder(w.Body).Decode(&items); err != nil {
|
|
||||||
t.Fatalf("decode: %v", err)
|
|
||||||
}
|
|
||||||
// "NIC" substring matches devices 1 and 3 (both contain "NIC")
|
|
||||||
if len(items) != 2 {
|
|
||||||
t.Errorf("expected 2 NIC results, got %d: %v", len(items), items)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestSearch_CatalogStatusFilter verifies that LLM-extracted catalog_status filters results.
|
|
||||||
func TestSearch_CatalogStatusFilter(t *testing.T) {
|
|
||||||
nb := &mockNetBoxClient{devices: testDevices()}
|
|
||||||
aiResp := `{"catalog_status": "available", "name_contains": "", "tag": ""}`
|
|
||||||
ai := &mockAIClient{response: aiResp}
|
|
||||||
h := handlers.NewSearchHandler(nb, ai)
|
|
||||||
|
|
||||||
req := httptest.NewRequest(http.MethodGet, "/api/search?q=show+me+available+items", nil)
|
|
||||||
w := httptest.NewRecorder()
|
|
||||||
h.SearchDevices(w, req)
|
|
||||||
|
|
||||||
if w.Code != http.StatusOK {
|
|
||||||
t.Fatalf("expected 200, got %d (body: %s)", w.Code, w.Body.String())
|
|
||||||
}
|
|
||||||
var items []map[string]interface{}
|
|
||||||
if err := json.NewDecoder(w.Body).Decode(&items); err != nil {
|
|
||||||
t.Fatalf("decode: %v", err)
|
|
||||||
}
|
|
||||||
// Devices 1 and 4 have catalog_status "available"
|
|
||||||
if len(items) != 2 {
|
|
||||||
t.Errorf("expected 2 available items, got %d", len(items))
|
|
||||||
}
|
|
||||||
for _, item := range items {
|
|
||||||
if item["catalog_status"] != "available" {
|
|
||||||
t.Errorf("unexpected catalog_status: %v", item["catalog_status"])
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestSearch_NameContainsAndStatus verifies combined filtering works.
|
|
||||||
func TestSearch_NameContainsAndStatus(t *testing.T) {
|
|
||||||
nb := &mockNetBoxClient{devices: testDevices()}
|
|
||||||
aiResp := `{"catalog_status": "available", "name_contains": "NIC", "tag": ""}`
|
|
||||||
ai := &mockAIClient{response: aiResp}
|
|
||||||
h := handlers.NewSearchHandler(nb, ai)
|
|
||||||
|
|
||||||
req := httptest.NewRequest(http.MethodGet, "/api/search?q=available+NICs", nil)
|
|
||||||
w := httptest.NewRecorder()
|
|
||||||
h.SearchDevices(w, req)
|
|
||||||
|
|
||||||
if w.Code != http.StatusOK {
|
|
||||||
t.Fatalf("expected 200, got %d", w.Code)
|
|
||||||
}
|
|
||||||
var items []map[string]interface{}
|
|
||||||
if err := json.NewDecoder(w.Body).Decode(&items); err != nil {
|
|
||||||
t.Fatalf("decode: %v", err)
|
|
||||||
}
|
|
||||||
// Only device 1: NIC + available (device 3 is "complete")
|
|
||||||
if len(items) != 1 {
|
|
||||||
t.Errorf("expected 1 result, got %d", len(items))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
@ -39,8 +39,6 @@ func (h spaHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||||
// usbEventsHandler handles GET /api/usb/events (SSE stream).
|
// usbEventsHandler handles GET /api/usb/events (SSE stream).
|
||||||
// testHandler handles POST /api/test/cable, GET /api/test/events, GET /api/test/recent.
|
// testHandler handles POST /api/test/cable, GET /api/test/events, GET /api/test/recent.
|
||||||
// advisorHandler handles POST /api/advisor/chat, GET /api/advisor/conversations, GET /api/advisor/conversations/{id}.
|
// advisorHandler handles POST /api/advisor/chat, GET /api/advisor/conversations, GET /api/advisor/conversations/{id}.
|
||||||
// researchHandler handles POST /api/research/trigger.
|
|
||||||
// searchHandler handles GET /api/search?q=...
|
|
||||||
func NewRouter(
|
func NewRouter(
|
||||||
staticFiles fs.FS,
|
staticFiles fs.FS,
|
||||||
intakeHandler http.Handler,
|
intakeHandler http.Handler,
|
||||||
|
|
@ -49,8 +47,6 @@ func NewRouter(
|
||||||
usbEventsHandler *handlers.USBEventsHandler,
|
usbEventsHandler *handlers.USBEventsHandler,
|
||||||
testHandler *handlers.TestHandler,
|
testHandler *handlers.TestHandler,
|
||||||
advisorHandler *advisor.AdvisorHandler,
|
advisorHandler *advisor.AdvisorHandler,
|
||||||
researchHandler *handlers.ResearchHandler,
|
|
||||||
searchHandler *handlers.SearchHandler,
|
|
||||||
) http.Handler {
|
) http.Handler {
|
||||||
r := chi.NewRouter()
|
r := chi.NewRouter()
|
||||||
r.Use(middleware.Logger)
|
r.Use(middleware.Logger)
|
||||||
|
|
@ -82,24 +78,6 @@ func NewRouter(
|
||||||
r.Get("/conversations/{id}", unavailable)
|
r.Get("/conversations/{id}", unavailable)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|
||||||
r.Route("/research", func(r chi.Router) {
|
|
||||||
if researchHandler != nil {
|
|
||||||
r.Post("/trigger", researchHandler.TriggerResearch)
|
|
||||||
} else {
|
|
||||||
r.Post("/trigger", func(w http.ResponseWriter, _ *http.Request) {
|
|
||||||
http.Error(w, "research unavailable", http.StatusServiceUnavailable)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
})
|
|
||||||
|
|
||||||
if searchHandler != nil {
|
|
||||||
r.Get("/search", searchHandler.SearchDevices)
|
|
||||||
} else {
|
|
||||||
r.Get("/search", func(w http.ResponseWriter, _ *http.Request) {
|
|
||||||
http.Error(w, "search unavailable", http.StatusServiceUnavailable)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
})
|
})
|
||||||
|
|
||||||
// SPA fallback — serve static files; unknown paths fall back to index.html.
|
// SPA fallback — serve static files; unknown paths fall back to index.html.
|
||||||
|
|
|
||||||
|
|
@ -29,8 +29,6 @@ type Config struct {
|
||||||
NetBoxDefaultSiteID int32 `mapstructure:"netbox_default_site_id"`
|
NetBoxDefaultSiteID int32 `mapstructure:"netbox_default_site_id"`
|
||||||
|
|
||||||
AI ai.AIConfig `mapstructure:"ai"`
|
AI ai.AIConfig `mapstructure:"ai"`
|
||||||
|
|
||||||
SearXNGURL string `mapstructure:"searxng_url"`
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func Load() (*Config, error) {
|
func Load() (*Config, error) {
|
||||||
|
|
@ -67,7 +65,6 @@ func Load() (*Config, error) {
|
||||||
v.SetDefault("ai.confidence_threshold", 0.75)
|
v.SetDefault("ai.confidence_threshold", 0.75)
|
||||||
v.SetDefault("ai.quick_add_enabled", false)
|
v.SetDefault("ai.quick_add_enabled", false)
|
||||||
v.SetDefault("ai.quick_add_threshold", 0.90)
|
v.SetDefault("ai.quick_add_threshold", 0.90)
|
||||||
v.SetDefault("searxng_url", "http://10.5.0.129:8080")
|
|
||||||
|
|
||||||
// Config file
|
// Config file
|
||||||
v.SetConfigName("config")
|
v.SetConfigName("config")
|
||||||
|
|
@ -106,7 +103,6 @@ func Load() (*Config, error) {
|
||||||
_ = v.BindEnv("ai.tier3.model", "HWLAB_AI_TIER3_MODEL")
|
_ = v.BindEnv("ai.tier3.model", "HWLAB_AI_TIER3_MODEL")
|
||||||
_ = v.BindEnv("ai.confidence_threshold", "HWLAB_AI_CONFIDENCE_THRESHOLD")
|
_ = v.BindEnv("ai.confidence_threshold", "HWLAB_AI_CONFIDENCE_THRESHOLD")
|
||||||
_ = v.BindEnv("ai.quick_add_enabled", "HWLAB_AI_QUICK_ADD_ENABLED")
|
_ = v.BindEnv("ai.quick_add_enabled", "HWLAB_AI_QUICK_ADD_ENABLED")
|
||||||
_ = v.BindEnv("searxng_url", "HWLAB_SEARXNG_URL")
|
|
||||||
|
|
||||||
// Read primary config file (non-fatal if missing)
|
// Read primary config file (non-fatal if missing)
|
||||||
if err := v.ReadInConfig(); err != nil {
|
if err := v.ReadInConfig(); err != nil {
|
||||||
|
|
|
||||||
|
|
@ -122,24 +122,6 @@ func (c *Client) CreateCable(ctx context.Context, label, assetTag, testDataJSON
|
||||||
return int64(result.GetId()), nil
|
return int64(result.GetId()), nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// ListDevicesWithStatus returns devices whose catalog_status custom field equals status.
|
|
||||||
// Uses client-side filtering (up to 200 devices) since go-netbox v4 custom field
|
|
||||||
// query param support is schema-dependent.
|
|
||||||
func (c *Client) ListDevicesWithStatus(ctx context.Context, status string) ([]Device, error) {
|
|
||||||
res, _, err := c.api.DcimAPI.DcimDevicesList(ctx).Limit(200).Execute()
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("list devices for status %q: %w", status, err)
|
|
||||||
}
|
|
||||||
devices := make([]Device, 0)
|
|
||||||
for _, d := range res.Results {
|
|
||||||
dev := deviceFromNetBox(d)
|
|
||||||
if dev.CustomFields.CatalogStatus == status {
|
|
||||||
devices = append(devices, dev)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return devices, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// DeleteDevice removes a device from NetBox by its internal ID.
|
// DeleteDevice removes a device from NetBox by its internal ID.
|
||||||
// Used primarily for test cleanup after CreateDevice integration tests.
|
// Used primarily for test cleanup after CreateDevice integration tests.
|
||||||
func (c *Client) DeleteDevice(ctx context.Context, id int64) error {
|
func (c *Client) DeleteDevice(ctx context.Context, id int64) error {
|
||||||
|
|
|
||||||
|
|
@ -1,185 +0,0 @@
|
||||||
package research
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"log"
|
|
||||||
"regexp"
|
|
||||||
"strings"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"git.georgsen.dk/hwlab/internal/ai"
|
|
||||||
"git.georgsen.dk/hwlab/internal/inventory"
|
|
||||||
"git.georgsen.dk/hwlab/internal/netbox"
|
|
||||||
)
|
|
||||||
|
|
||||||
// nonSafeChars matches characters that are not safe to send to SearXNG.
|
|
||||||
// Allowed: alphanumeric, space, dot, dash, underscore.
|
|
||||||
var nonSafeChars = regexp.MustCompile(`[^a-zA-Z0-9 .\-_]+`)
|
|
||||||
|
|
||||||
// SanitizeQuery strips unsafe characters from a search query string.
|
|
||||||
// Exported so it can be tested from the _test package.
|
|
||||||
func SanitizeQuery(s string) string {
|
|
||||||
sanitized := nonSafeChars.ReplaceAllString(s, " ")
|
|
||||||
return strings.TrimSpace(sanitized)
|
|
||||||
}
|
|
||||||
|
|
||||||
// NetBoxer is the subset of netbox.Client used by the Agent.
|
|
||||||
// Using an interface allows stub injection in tests.
|
|
||||||
type NetBoxer interface {
|
|
||||||
ListDevicesWithStatus(ctx context.Context, status string) ([]netbox.Device, error)
|
|
||||||
PatchCustomFields(ctx context.Context, deviceID int64, patch map[string]interface{}) error
|
|
||||||
}
|
|
||||||
|
|
||||||
// TextCompleter is the subset of ai.TierClient used by the Agent for text-only LLM calls.
|
|
||||||
type TextCompleter interface {
|
|
||||||
TextComplete(ctx context.Context, prompt string) (string, error)
|
|
||||||
}
|
|
||||||
|
|
||||||
// CatalogTransitioner is the subset of inventory.CatalogUpdater used by the Agent.
|
|
||||||
type CatalogTransitioner interface {
|
|
||||||
UpdateCatalogStatus(ctx context.Context, deviceID int64, current, next inventory.CatalogStatus) (inventory.CatalogStatus, error)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Agent is the background worker that enriches needs_research hardware items.
|
|
||||||
// It polls NetBox, searches SearXNG, calls a Tier 2 LLM, and transitions items to researched.
|
|
||||||
type Agent struct {
|
|
||||||
nbClient NetBoxer
|
|
||||||
researchClient ai.ResearchClient
|
|
||||||
llm TextCompleter
|
|
||||||
updater CatalogTransitioner
|
|
||||||
}
|
|
||||||
|
|
||||||
// NewAgent creates an Agent. All arguments must be non-nil.
|
|
||||||
func NewAgent(nb NetBoxer, rc ai.ResearchClient, llm TextCompleter, updater CatalogTransitioner) *Agent {
|
|
||||||
return &Agent{
|
|
||||||
nbClient: nb,
|
|
||||||
researchClient: rc,
|
|
||||||
llm: llm,
|
|
||||||
updater: updater,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// enrichmentResponse is the expected JSON structure from the Tier 2 LLM.
|
|
||||||
type enrichmentResponse struct {
|
|
||||||
AINotes string `json:"ai_notes"`
|
|
||||||
ProductURL string `json:"product_url"`
|
|
||||||
}
|
|
||||||
|
|
||||||
// RunOnce performs a single research cycle: finds all needs_research devices,
|
|
||||||
// enriches each via SearXNG + LLM, patches NetBox custom fields, and transitions
|
|
||||||
// the catalog status to researched. Returns the number of items enriched.
|
|
||||||
func (a *Agent) RunOnce(ctx context.Context) (int, error) {
|
|
||||||
devices, err := a.nbClient.ListDevicesWithStatus(ctx, string(inventory.StatusNeedsResearch))
|
|
||||||
if err != nil {
|
|
||||||
return 0, fmt.Errorf("research agent: list needs_research devices: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
enriched := 0
|
|
||||||
for _, dev := range devices {
|
|
||||||
query := SanitizeQuery(dev.Name)
|
|
||||||
if query == "" {
|
|
||||||
log.Printf("research agent: device %d has empty name after sanitization, skipping", dev.ID)
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
|
|
||||||
results, err := a.researchClient.Search(ctx, query)
|
|
||||||
if err != nil {
|
|
||||||
log.Printf("research agent: search error for device %d (%q): %v", dev.ID, dev.Name, err)
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if len(results) == 0 {
|
|
||||||
log.Printf("research agent: no SearXNG results for device %d (%q), skipping", dev.ID, dev.Name)
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
|
|
||||||
// Build enrichment prompt using top 3 results
|
|
||||||
top := results
|
|
||||||
if len(top) > 3 {
|
|
||||||
top = top[:3]
|
|
||||||
}
|
|
||||||
var sb strings.Builder
|
|
||||||
for i, r := range top {
|
|
||||||
sb.WriteString(fmt.Sprintf("%d. %s\n %s\n URL: %s\n", i+1, r.Title, r.Snippet, r.URL))
|
|
||||||
}
|
|
||||||
prompt := fmt.Sprintf(
|
|
||||||
"You are enriching a hardware inventory record.\nItem: %s\nSearch results:\n%s\nReturn JSON: {\"ai_notes\": \"...\", \"product_url\": \"...\"}",
|
|
||||||
dev.Name, sb.String(),
|
|
||||||
)
|
|
||||||
|
|
||||||
rawResponse, err := a.llm.TextComplete(ctx, prompt)
|
|
||||||
if err != nil {
|
|
||||||
log.Printf("research agent: LLM error for device %d (%q): %v", dev.ID, dev.Name, err)
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
|
|
||||||
// Parse LLM JSON response — extract ai_notes and product_url
|
|
||||||
var enrichResp enrichmentResponse
|
|
||||||
if parseErr := json.Unmarshal([]byte(rawResponse), &enrichResp); parseErr != nil {
|
|
||||||
log.Printf("research agent: LLM non-JSON for device %d: %v (raw: %.100s)", dev.ID, parseErr, rawResponse)
|
|
||||||
// Use raw response as ai_notes fallback
|
|
||||||
enrichResp.AINotes = rawResponse
|
|
||||||
}
|
|
||||||
|
|
||||||
// Patch NetBox custom fields with enrichment data
|
|
||||||
patch := map[string]interface{}{}
|
|
||||||
if enrichResp.AINotes != "" {
|
|
||||||
patch["ai_notes"] = enrichResp.AINotes
|
|
||||||
}
|
|
||||||
if enrichResp.ProductURL == "" && len(results) > 0 {
|
|
||||||
// Fall back to first SearXNG result URL if LLM didn't provide one
|
|
||||||
enrichResp.ProductURL = results[0].URL
|
|
||||||
}
|
|
||||||
if enrichResp.ProductURL != "" {
|
|
||||||
patch["product_url"] = enrichResp.ProductURL
|
|
||||||
}
|
|
||||||
if len(patch) > 0 {
|
|
||||||
if patchErr := a.nbClient.PatchCustomFields(ctx, int64(dev.ID), patch); patchErr != nil {
|
|
||||||
log.Printf("research agent: patch error for device %d: %v", dev.ID, patchErr)
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Transition catalog status: needs_research -> researched
|
|
||||||
if _, transErr := a.updater.UpdateCatalogStatus(ctx, int64(dev.ID),
|
|
||||||
inventory.StatusNeedsResearch, inventory.StatusResearched); transErr != nil {
|
|
||||||
log.Printf("research agent: status transition error for device %d: %v", dev.ID, transErr)
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
|
|
||||||
enriched++
|
|
||||||
}
|
|
||||||
|
|
||||||
return enriched, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// Start runs the research agent on the given interval until ctx is cancelled.
|
|
||||||
// RunOnce is called immediately on start, then on each tick.
|
|
||||||
func (a *Agent) Start(ctx context.Context, interval time.Duration) {
|
|
||||||
log.Printf("research agent: starting, interval=%v", interval)
|
|
||||||
|
|
||||||
// Run immediately on startup
|
|
||||||
if n, err := a.RunOnce(ctx); err != nil {
|
|
||||||
log.Printf("research agent: initial cycle error: %v", err)
|
|
||||||
} else {
|
|
||||||
log.Printf("research agent: cycle complete, enriched %d items", n)
|
|
||||||
}
|
|
||||||
|
|
||||||
ticker := time.NewTicker(interval)
|
|
||||||
defer ticker.Stop()
|
|
||||||
|
|
||||||
for {
|
|
||||||
select {
|
|
||||||
case <-ctx.Done():
|
|
||||||
log.Printf("research agent: shutting down")
|
|
||||||
return
|
|
||||||
case <-ticker.C:
|
|
||||||
if n, err := a.RunOnce(ctx); err != nil {
|
|
||||||
log.Printf("research agent: cycle error: %v", err)
|
|
||||||
} else {
|
|
||||||
log.Printf("research agent: cycle complete, enriched %d items", n)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
@ -1,180 +0,0 @@
|
||||||
package research_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"testing"
|
|
||||||
|
|
||||||
"git.georgsen.dk/hwlab/internal/ai"
|
|
||||||
"git.georgsen.dk/hwlab/internal/inventory"
|
|
||||||
"git.georgsen.dk/hwlab/internal/netbox"
|
|
||||||
"git.georgsen.dk/hwlab/internal/research"
|
|
||||||
)
|
|
||||||
|
|
||||||
// --- Stubs ---
|
|
||||||
|
|
||||||
// stubResearchClient returns canned SearchResults.
|
|
||||||
type stubResearchClient struct {
|
|
||||||
results []ai.SearchResult
|
|
||||||
err error
|
|
||||||
calls []string
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *stubResearchClient) Search(_ context.Context, query string) ([]ai.SearchResult, error) {
|
|
||||||
s.calls = append(s.calls, query)
|
|
||||||
return s.results, s.err
|
|
||||||
}
|
|
||||||
|
|
||||||
// stubTextCompleter returns a canned LLM response text.
|
|
||||||
type stubTextCompleter struct {
|
|
||||||
response string
|
|
||||||
err error
|
|
||||||
calls []string
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *stubTextCompleter) TextComplete(_ context.Context, prompt string) (string, error) {
|
|
||||||
s.calls = append(s.calls, prompt)
|
|
||||||
return s.response, s.err
|
|
||||||
}
|
|
||||||
|
|
||||||
// stubNetBoxClient satisfies the research.NetBoxer interface used by Agent.
|
|
||||||
type stubNetBoxClient struct {
|
|
||||||
devices []netbox.Device
|
|
||||||
patches map[int64]map[string]interface{}
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *stubNetBoxClient) ListDevicesWithStatus(_ context.Context, status string) ([]netbox.Device, error) {
|
|
||||||
return s.devices, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *stubNetBoxClient) PatchCustomFields(_ context.Context, deviceID int64, patch map[string]interface{}) error {
|
|
||||||
if s.patches == nil {
|
|
||||||
s.patches = make(map[int64]map[string]interface{})
|
|
||||||
}
|
|
||||||
s.patches[deviceID] = patch
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// stubCatalogUpdater records transitions.
|
|
||||||
type stubCatalogUpdater struct {
|
|
||||||
transitions []struct {
|
|
||||||
id int64
|
|
||||||
current inventory.CatalogStatus
|
|
||||||
next inventory.CatalogStatus
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *stubCatalogUpdater) UpdateCatalogStatus(_ context.Context, deviceID int64, current, next inventory.CatalogStatus) (inventory.CatalogStatus, error) {
|
|
||||||
s.transitions = append(s.transitions, struct {
|
|
||||||
id int64
|
|
||||||
current inventory.CatalogStatus
|
|
||||||
next inventory.CatalogStatus
|
|
||||||
}{deviceID, current, next})
|
|
||||||
return next, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// --- Tests ---
|
|
||||||
|
|
||||||
func TestSanitizeQuery(t *testing.T) {
|
|
||||||
cases := []struct {
|
|
||||||
input string
|
|
||||||
expected string
|
|
||||||
}{
|
|
||||||
{"Intel NIC i350", "Intel NIC i350"},
|
|
||||||
{"Dell<script>alert(1)</script>", "Dell script alert 1 script"},
|
|
||||||
{"HP ProLiant DL380 Gen9", "HP ProLiant DL380 Gen9"},
|
|
||||||
{" trim ", "trim"},
|
|
||||||
{"special!@#$%chars", "special chars"},
|
|
||||||
{"dots.and-dashes_ok", "dots.and-dashes_ok"},
|
|
||||||
}
|
|
||||||
for _, tc := range cases {
|
|
||||||
got := research.SanitizeQuery(tc.input)
|
|
||||||
if got != tc.expected {
|
|
||||||
t.Errorf("SanitizeQuery(%q) = %q, want %q", tc.input, got, tc.expected)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestRunOnce_EnrichesDevice(t *testing.T) {
|
|
||||||
nb := &stubNetBoxClient{
|
|
||||||
devices: []netbox.Device{
|
|
||||||
{
|
|
||||||
ID: 42,
|
|
||||||
Name: "Intel i350 NIC",
|
|
||||||
CustomFields: netbox.CustomFields{
|
|
||||||
CatalogStatus: "needs_research",
|
|
||||||
},
|
|
||||||
},
|
|
||||||
},
|
|
||||||
}
|
|
||||||
rc := &stubResearchClient{
|
|
||||||
results: []ai.SearchResult{
|
|
||||||
{Title: "Intel i350", URL: "https://ark.intel.com", Snippet: "Quad-port GbE"},
|
|
||||||
{Title: "Datasheet", URL: "https://intel.com/ds", Snippet: "Technical specs"},
|
|
||||||
},
|
|
||||||
}
|
|
||||||
llm := &stubTextCompleter{
|
|
||||||
response: `{"ai_notes": "Intel i350 quad-port GbE adapter", "product_url": "https://ark.intel.com"}`,
|
|
||||||
}
|
|
||||||
updater := &stubCatalogUpdater{}
|
|
||||||
|
|
||||||
agent := research.NewAgent(nb, rc, llm, updater)
|
|
||||||
enriched, err := agent.RunOnce(context.Background())
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("RunOnce error: %v", err)
|
|
||||||
}
|
|
||||||
if enriched != 1 {
|
|
||||||
t.Errorf("expected enriched=1, got %d", enriched)
|
|
||||||
}
|
|
||||||
if len(updater.transitions) != 1 {
|
|
||||||
t.Fatalf("expected 1 status transition, got %d", len(updater.transitions))
|
|
||||||
}
|
|
||||||
tr := updater.transitions[0]
|
|
||||||
if tr.id != 42 {
|
|
||||||
t.Errorf("expected device id=42, got %d", tr.id)
|
|
||||||
}
|
|
||||||
if tr.current != inventory.StatusNeedsResearch {
|
|
||||||
t.Errorf("unexpected current status: %s", tr.current)
|
|
||||||
}
|
|
||||||
if tr.next != inventory.StatusResearched {
|
|
||||||
t.Errorf("unexpected next status: %s", tr.next)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestRunOnce_SkipsDeviceWithNoResults(t *testing.T) {
|
|
||||||
nb := &stubNetBoxClient{
|
|
||||||
devices: []netbox.Device{
|
|
||||||
{ID: 10, Name: "Mystery Device", CustomFields: netbox.CustomFields{CatalogStatus: "needs_research"}},
|
|
||||||
},
|
|
||||||
}
|
|
||||||
rc := &stubResearchClient{results: []ai.SearchResult{}} // empty
|
|
||||||
llm := &stubTextCompleter{}
|
|
||||||
updater := &stubCatalogUpdater{}
|
|
||||||
|
|
||||||
agent := research.NewAgent(nb, rc, llm, updater)
|
|
||||||
enriched, err := agent.RunOnce(context.Background())
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("RunOnce error: %v", err)
|
|
||||||
}
|
|
||||||
if enriched != 0 {
|
|
||||||
t.Errorf("expected enriched=0 (skipped), got %d", enriched)
|
|
||||||
}
|
|
||||||
if len(updater.transitions) != 0 {
|
|
||||||
t.Errorf("expected 0 transitions (device skipped), got %d", len(updater.transitions))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestRunOnce_NoDevices(t *testing.T) {
|
|
||||||
nb := &stubNetBoxClient{devices: []netbox.Device{}}
|
|
||||||
rc := &stubResearchClient{}
|
|
||||||
llm := &stubTextCompleter{}
|
|
||||||
updater := &stubCatalogUpdater{}
|
|
||||||
|
|
||||||
agent := research.NewAgent(nb, rc, llm, updater)
|
|
||||||
enriched, err := agent.RunOnce(context.Background())
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("unexpected error: %v", err)
|
|
||||||
}
|
|
||||||
if enriched != 0 {
|
|
||||||
t.Errorf("expected 0 enriched, got %d", enriched)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
@ -1,88 +0,0 @@
|
||||||
// Package research provides the SearXNG HTTP search client and the ResearchAgent
|
|
||||||
// background worker that enriches needs_research hardware records.
|
|
||||||
package research
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"fmt"
|
|
||||||
"net/http"
|
|
||||||
"net/url"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"encoding/json"
|
|
||||||
|
|
||||||
"git.georgsen.dk/hwlab/internal/ai"
|
|
||||||
)
|
|
||||||
|
|
||||||
const defaultSearXNGURL = "http://10.5.0.129:8080"
|
|
||||||
|
|
||||||
// searxngResponse is the parsed JSON body returned by SearXNG.
|
|
||||||
// SearXNG uses "content" for the text snippet, not "snippet".
|
|
||||||
type searxngResponse struct {
|
|
||||||
Results []searxngResult `json:"results"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type searxngResult struct {
|
|
||||||
Title string `json:"title"`
|
|
||||||
URL string `json:"url"`
|
|
||||||
Content string `json:"content"`
|
|
||||||
}
|
|
||||||
|
|
||||||
// SearXNGClient implements ai.ResearchClient using a self-hosted SearXNG instance.
|
|
||||||
type SearXNGClient struct {
|
|
||||||
baseURL string
|
|
||||||
httpClient *http.Client
|
|
||||||
}
|
|
||||||
|
|
||||||
// NewSearXNGClient creates a SearXNGClient. If baseURL is empty the default LAN
|
|
||||||
// address (http://10.5.0.129:8080) is used.
|
|
||||||
func NewSearXNGClient(baseURL string) *SearXNGClient {
|
|
||||||
if baseURL == "" {
|
|
||||||
baseURL = defaultSearXNGURL
|
|
||||||
}
|
|
||||||
return &SearXNGClient{
|
|
||||||
baseURL: baseURL,
|
|
||||||
httpClient: &http.Client{
|
|
||||||
Timeout: 15 * time.Second,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Search executes a GET {baseURL}/search?q={query}&format=json and returns parsed results.
|
|
||||||
// An HTTP non-2xx response is returned as an error. An empty results array is not an error.
|
|
||||||
func (c *SearXNGClient) Search(ctx context.Context, query string) ([]ai.SearchResult, error) {
|
|
||||||
params := url.Values{}
|
|
||||||
params.Set("q", query)
|
|
||||||
params.Set("format", "json")
|
|
||||||
reqURL := c.baseURL + "/search?" + params.Encode()
|
|
||||||
|
|
||||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, reqURL, nil)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("searxng: build request: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
resp, err := c.httpClient.Do(req)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("searxng: http get: %w", err)
|
|
||||||
}
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
|
|
||||||
return nil, fmt.Errorf("searxng: unexpected status %d", resp.StatusCode)
|
|
||||||
}
|
|
||||||
|
|
||||||
var body searxngResponse
|
|
||||||
if err := json.NewDecoder(resp.Body).Decode(&body); err != nil {
|
|
||||||
return nil, fmt.Errorf("searxng: decode response: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
results := make([]ai.SearchResult, 0, len(body.Results))
|
|
||||||
for _, r := range body.Results {
|
|
||||||
results = append(results, ai.SearchResult{
|
|
||||||
Title: r.Title,
|
|
||||||
URL: r.URL,
|
|
||||||
Snippet: r.Content,
|
|
||||||
})
|
|
||||||
}
|
|
||||||
return results, nil
|
|
||||||
}
|
|
||||||
|
|
@ -1,111 +0,0 @@
|
||||||
package research_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"encoding/json"
|
|
||||||
"net/http"
|
|
||||||
"net/http/httptest"
|
|
||||||
"testing"
|
|
||||||
|
|
||||||
"git.georgsen.dk/hwlab/internal/research"
|
|
||||||
)
|
|
||||||
|
|
||||||
func TestSearXNGSearch_ValidResponse(t *testing.T) {
|
|
||||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
||||||
if r.URL.Path != "/search" {
|
|
||||||
t.Errorf("unexpected path: %s", r.URL.Path)
|
|
||||||
}
|
|
||||||
if r.URL.Query().Get("format") != "json" {
|
|
||||||
t.Errorf("expected format=json, got %s", r.URL.Query().Get("format"))
|
|
||||||
}
|
|
||||||
if r.URL.Query().Get("q") == "" {
|
|
||||||
t.Error("expected non-empty q param")
|
|
||||||
}
|
|
||||||
resp := map[string]interface{}{
|
|
||||||
"results": []map[string]interface{}{
|
|
||||||
{"title": "Intel i350 NIC", "url": "https://ark.intel.com/i350", "content": "Quad-port Gigabit Ethernet adapter"},
|
|
||||||
{"title": "Intel i350 Datasheet", "url": "https://intel.com/datasheet", "content": "Technical specs for i350 series"},
|
|
||||||
},
|
|
||||||
}
|
|
||||||
w.Header().Set("Content-Type", "application/json")
|
|
||||||
json.NewEncoder(w).Encode(resp)
|
|
||||||
}))
|
|
||||||
defer srv.Close()
|
|
||||||
|
|
||||||
client := research.NewSearXNGClient(srv.URL)
|
|
||||||
results, err := client.Search(context.Background(), "Intel NIC i350")
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("unexpected error: %v", err)
|
|
||||||
}
|
|
||||||
if len(results) != 2 {
|
|
||||||
t.Fatalf("expected 2 results, got %d", len(results))
|
|
||||||
}
|
|
||||||
if results[0].Title != "Intel i350 NIC" {
|
|
||||||
t.Errorf("unexpected title: %s", results[0].Title)
|
|
||||||
}
|
|
||||||
if results[0].URL != "https://ark.intel.com/i350" {
|
|
||||||
t.Errorf("unexpected URL: %s", results[0].URL)
|
|
||||||
}
|
|
||||||
if results[0].Snippet != "Quad-port Gigabit Ethernet adapter" {
|
|
||||||
t.Errorf("unexpected snippet (content): %s", results[0].Snippet)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestSearXNGSearch_HTTP500(t *testing.T) {
|
|
||||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
||||||
w.WriteHeader(http.StatusInternalServerError)
|
|
||||||
}))
|
|
||||||
defer srv.Close()
|
|
||||||
|
|
||||||
client := research.NewSearXNGClient(srv.URL)
|
|
||||||
_, err := client.Search(context.Background(), "test query")
|
|
||||||
if err == nil {
|
|
||||||
t.Fatal("expected error for HTTP 500, got nil")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestSearXNGSearch_EmptyResults(t *testing.T) {
|
|
||||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
||||||
resp := map[string]interface{}{"results": []interface{}{}}
|
|
||||||
w.Header().Set("Content-Type", "application/json")
|
|
||||||
json.NewEncoder(w).Encode(resp)
|
|
||||||
}))
|
|
||||||
defer srv.Close()
|
|
||||||
|
|
||||||
client := research.NewSearXNGClient(srv.URL)
|
|
||||||
results, err := client.Search(context.Background(), "something obscure")
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("unexpected error: %v", err)
|
|
||||||
}
|
|
||||||
if len(results) != 0 {
|
|
||||||
t.Errorf("expected 0 results, got %d", len(results))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestSearXNGSearch_QueryEncoding(t *testing.T) {
|
|
||||||
var capturedQuery string
|
|
||||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
||||||
capturedQuery = r.URL.Query().Get("q")
|
|
||||||
resp := map[string]interface{}{"results": []interface{}{}}
|
|
||||||
w.Header().Set("Content-Type", "application/json")
|
|
||||||
json.NewEncoder(w).Encode(resp)
|
|
||||||
}))
|
|
||||||
defer srv.Close()
|
|
||||||
|
|
||||||
client := research.NewSearXNGClient(srv.URL)
|
|
||||||
_, err := client.Search(context.Background(), "Intel NIC i350 2.5G")
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("unexpected error: %v", err)
|
|
||||||
}
|
|
||||||
if capturedQuery != "Intel NIC i350 2.5G" {
|
|
||||||
t.Errorf("unexpected decoded query: %q", capturedQuery)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestNewSearXNGClient_DefaultURL(t *testing.T) {
|
|
||||||
// Empty baseURL should use the default LAN address
|
|
||||||
client := research.NewSearXNGClient("")
|
|
||||||
if client == nil {
|
|
||||||
t.Fatal("expected non-nil client")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
@ -1,5 +1,4 @@
|
||||||
import { useState, useEffect } from 'react'
|
import { Search, LayoutGrid, List } from 'lucide-react'
|
||||||
import { Search, LayoutGrid, List, Loader2, Sparkles } from 'lucide-react'
|
|
||||||
import { Button } from '@/components/ui/button'
|
import { Button } from '@/components/ui/button'
|
||||||
import { useUIStore } from '@/store/ui'
|
import { useUIStore } from '@/store/ui'
|
||||||
|
|
||||||
|
|
@ -9,9 +8,6 @@ interface FilterBarProps {
|
||||||
statusFilter: string
|
statusFilter: string
|
||||||
onStatusChange: (v: string) => void
|
onStatusChange: (v: string) => void
|
||||||
totalCount: number
|
totalCount: number
|
||||||
nlQuery: string
|
|
||||||
onNlQueryChange: (v: string) => void
|
|
||||||
nlSearchLoading: boolean
|
|
||||||
}
|
}
|
||||||
|
|
||||||
const STATUSES = ['', 'draft', 'indexed', 'needs_research', 'researched', 'complete']
|
const STATUSES = ['', 'draft', 'indexed', 'needs_research', 'researched', 'complete']
|
||||||
|
|
@ -24,104 +20,61 @@ const STATUS_LABELS: Record<string, string> = {
|
||||||
complete: 'Complete',
|
complete: 'Complete',
|
||||||
}
|
}
|
||||||
|
|
||||||
export function FilterBar({
|
export function FilterBar({ search, onSearchChange, statusFilter, onStatusChange, totalCount }: FilterBarProps) {
|
||||||
search,
|
|
||||||
onSearchChange,
|
|
||||||
statusFilter,
|
|
||||||
onStatusChange,
|
|
||||||
totalCount,
|
|
||||||
nlQuery,
|
|
||||||
onNlQueryChange,
|
|
||||||
nlSearchLoading,
|
|
||||||
}: FilterBarProps) {
|
|
||||||
const { viewMode, setViewMode } = useUIStore()
|
const { viewMode, setViewMode } = useUIStore()
|
||||||
|
|
||||||
// Local state for the NL input value — debounced before calling onNlQueryChange.
|
|
||||||
const [nlInputValue, setNlInputValue] = useState(nlQuery)
|
|
||||||
|
|
||||||
// Sync if parent resets nlQuery to empty (e.g. clear action).
|
|
||||||
useEffect(() => {
|
|
||||||
if (nlQuery === '') setNlInputValue('')
|
|
||||||
}, [nlQuery])
|
|
||||||
|
|
||||||
// 400ms debounce: propagate to parent only after user stops typing.
|
|
||||||
useEffect(() => {
|
|
||||||
const timer = setTimeout(() => {
|
|
||||||
onNlQueryChange(nlInputValue)
|
|
||||||
}, 400)
|
|
||||||
return () => clearTimeout(timer)
|
|
||||||
}, [nlInputValue, onNlQueryChange])
|
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<div className="flex flex-col gap-3 mb-6">
|
<div className="flex flex-wrap items-center gap-3 mb-6">
|
||||||
{/* Row 1: text filter + status + view toggle */}
|
{/* Search */}
|
||||||
<div className="flex flex-wrap items-center gap-3">
|
<div className="relative flex-1 min-w-48">
|
||||||
{/* Local text search */}
|
<Search className="absolute left-3 top-1/2 -translate-y-1/2 w-4 h-4 text-[#a0a0a0]" />
|
||||||
<div className="relative flex-1 min-w-48">
|
|
||||||
<Search className="absolute left-3 top-1/2 -translate-y-1/2 w-4 h-4 text-[#a0a0a0]" />
|
|
||||||
<input
|
|
||||||
type="text"
|
|
||||||
placeholder="Search items…"
|
|
||||||
value={search}
|
|
||||||
onChange={(e) => onSearchChange(e.target.value)}
|
|
||||||
className="w-full pl-9 pr-3 py-2 bg-near-black border border-charcoal/80 rounded-sharp text-sm text-white placeholder-[#585858] focus:outline-none focus:border-volt/60 focus:ring-1 focus:ring-volt/30"
|
|
||||||
/>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
{/* Status filter */}
|
|
||||||
<select
|
|
||||||
value={statusFilter}
|
|
||||||
onChange={(e) => onStatusChange(e.target.value)}
|
|
||||||
className="px-3 py-2 bg-near-black border border-charcoal/80 rounded-sharp text-sm text-white focus:outline-none focus:border-volt/60"
|
|
||||||
>
|
|
||||||
{STATUSES.map((s) => (
|
|
||||||
<option key={s} value={s} className="bg-near-black">
|
|
||||||
{STATUS_LABELS[s]}
|
|
||||||
</option>
|
|
||||||
))}
|
|
||||||
</select>
|
|
||||||
|
|
||||||
{/* Item count */}
|
|
||||||
<span className="text-xs text-[#a0a0a0] label-upper mr-auto">
|
|
||||||
{totalCount} items
|
|
||||||
</span>
|
|
||||||
|
|
||||||
{/* View toggle */}
|
|
||||||
<div className="flex rounded-sharp border border-charcoal/80 overflow-hidden">
|
|
||||||
<Button
|
|
||||||
variant={viewMode === 'grid' ? 'default' : 'secondary'}
|
|
||||||
size="icon"
|
|
||||||
className="rounded-none h-9 w-9"
|
|
||||||
onClick={() => setViewMode('grid')}
|
|
||||||
title="Grid view"
|
|
||||||
>
|
|
||||||
<LayoutGrid className="w-4 h-4" />
|
|
||||||
</Button>
|
|
||||||
<Button
|
|
||||||
variant={viewMode === 'list' ? 'default' : 'secondary'}
|
|
||||||
size="icon"
|
|
||||||
className="rounded-none h-9 w-9 border-l border-charcoal/80"
|
|
||||||
onClick={() => setViewMode('list')}
|
|
||||||
title="List view"
|
|
||||||
>
|
|
||||||
<List className="w-4 h-4" />
|
|
||||||
</Button>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
{/* Row 2: Natural language search */}
|
|
||||||
<div className="relative w-full">
|
|
||||||
<Sparkles className="absolute left-3 top-1/2 -translate-y-1/2 w-4 h-4 text-volt/60" />
|
|
||||||
<input
|
<input
|
||||||
type="text"
|
type="text"
|
||||||
placeholder="Ask anything: show me free 10GbE NICs…"
|
placeholder="Search items…"
|
||||||
value={nlInputValue}
|
value={search}
|
||||||
onChange={(e) => setNlInputValue(e.target.value)}
|
onChange={(e) => onSearchChange(e.target.value)}
|
||||||
className="w-full pl-9 pr-9 py-2 bg-[#0a0a0a] border border-volt/20 rounded-card text-sm text-white placeholder-[#585858] focus:outline-none focus:border-volt focus:ring-1 focus:ring-volt/30 transition-colors"
|
className="w-full pl-9 pr-3 py-2 bg-near-black border border-charcoal/80 rounded-sharp text-sm text-white placeholder-[#585858] focus:outline-none focus:border-volt/60 focus:ring-1 focus:ring-volt/30"
|
||||||
/>
|
/>
|
||||||
{nlSearchLoading && (
|
</div>
|
||||||
<Loader2 className="absolute right-3 top-1/2 -translate-y-1/2 w-4 h-4 animate-spin text-volt" />
|
|
||||||
)}
|
{/* Status filter */}
|
||||||
|
<select
|
||||||
|
value={statusFilter}
|
||||||
|
onChange={(e) => onStatusChange(e.target.value)}
|
||||||
|
className="px-3 py-2 bg-near-black border border-charcoal/80 rounded-sharp text-sm text-white focus:outline-none focus:border-volt/60"
|
||||||
|
>
|
||||||
|
{STATUSES.map((s) => (
|
||||||
|
<option key={s} value={s} className="bg-near-black">
|
||||||
|
{STATUS_LABELS[s]}
|
||||||
|
</option>
|
||||||
|
))}
|
||||||
|
</select>
|
||||||
|
|
||||||
|
{/* Item count */}
|
||||||
|
<span className="text-xs text-[#a0a0a0] label-upper mr-auto">
|
||||||
|
{totalCount} items
|
||||||
|
</span>
|
||||||
|
|
||||||
|
{/* View toggle */}
|
||||||
|
<div className="flex rounded-sharp border border-charcoal/80 overflow-hidden">
|
||||||
|
<Button
|
||||||
|
variant={viewMode === 'grid' ? 'default' : 'secondary'}
|
||||||
|
size="icon"
|
||||||
|
className="rounded-none h-9 w-9"
|
||||||
|
onClick={() => setViewMode('grid')}
|
||||||
|
title="Grid view"
|
||||||
|
>
|
||||||
|
<LayoutGrid className="w-4 h-4" />
|
||||||
|
</Button>
|
||||||
|
<Button
|
||||||
|
variant={viewMode === 'list' ? 'default' : 'secondary'}
|
||||||
|
size="icon"
|
||||||
|
className="rounded-none h-9 w-9 border-l border-charcoal/80"
|
||||||
|
onClick={() => setViewMode('list')}
|
||||||
|
title="List view"
|
||||||
|
>
|
||||||
|
<List className="w-4 h-4" />
|
||||||
|
</Button>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
)
|
)
|
||||||
|
|
|
||||||
|
|
@ -28,9 +28,6 @@ async function fetchJSON<T>(url: string): Promise<T> {
|
||||||
export const fetchInventory = (): Promise<InventoryItem[]> =>
|
export const fetchInventory = (): Promise<InventoryItem[]> =>
|
||||||
fetchJSON<InventoryItem[]>(`${BASE}/inventory`)
|
fetchJSON<InventoryItem[]>(`${BASE}/inventory`)
|
||||||
|
|
||||||
export const fetchSearch = (q: string): Promise<InventoryItem[]> =>
|
|
||||||
fetchJSON<InventoryItem[]>(`${BASE}/search?q=${encodeURIComponent(q)}`)
|
|
||||||
|
|
||||||
export const fetchInventoryItem = (id: number): Promise<InventoryItem> =>
|
export const fetchInventoryItem = (id: number): Promise<InventoryItem> =>
|
||||||
fetchJSON<InventoryItem>(`${BASE}/inventory/${id}`)
|
fetchJSON<InventoryItem>(`${BASE}/inventory/${id}`)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,12 +1,10 @@
|
||||||
import { useState, useMemo } from 'react'
|
import { useState, useMemo } from 'react'
|
||||||
import { useQuery } from '@tanstack/react-query'
|
|
||||||
import { AppShell } from '@/components/layout/AppShell'
|
import { AppShell } from '@/components/layout/AppShell'
|
||||||
import { FilterBar } from '@/components/inventory/FilterBar'
|
import { FilterBar } from '@/components/inventory/FilterBar'
|
||||||
import { ItemCard } from '@/components/inventory/ItemCard'
|
import { ItemCard } from '@/components/inventory/ItemCard'
|
||||||
import { ItemRow } from '@/components/inventory/ItemRow'
|
import { ItemRow } from '@/components/inventory/ItemRow'
|
||||||
import { useInventory } from '@/hooks/useInventory'
|
import { useInventory } from '@/hooks/useInventory'
|
||||||
import { useUIStore } from '@/store/ui'
|
import { useUIStore } from '@/store/ui'
|
||||||
import { fetchSearch } from '@/lib/api'
|
|
||||||
import { Loader2, AlertCircle } from 'lucide-react'
|
import { Loader2, AlertCircle } from 'lucide-react'
|
||||||
|
|
||||||
export function DashboardPage() {
|
export function DashboardPage() {
|
||||||
|
|
@ -14,17 +12,7 @@ export function DashboardPage() {
|
||||||
const { viewMode } = useUIStore()
|
const { viewMode } = useUIStore()
|
||||||
const [search, setSearch] = useState('')
|
const [search, setSearch] = useState('')
|
||||||
const [statusFilter, setStatusFilter] = useState('')
|
const [statusFilter, setStatusFilter] = useState('')
|
||||||
const [nlQuery, setNlQuery] = useState('')
|
|
||||||
|
|
||||||
// NL search via GET /api/search — only fires when nlQuery has > 2 chars.
|
|
||||||
const { data: searchResults, isLoading: searchLoading } = useQuery({
|
|
||||||
queryKey: ['search', nlQuery],
|
|
||||||
queryFn: () => fetchSearch(nlQuery),
|
|
||||||
enabled: nlQuery.trim().length > 2,
|
|
||||||
staleTime: 30_000,
|
|
||||||
})
|
|
||||||
|
|
||||||
// Local filter (used when nlQuery is empty).
|
|
||||||
const filtered = useMemo(() => {
|
const filtered = useMemo(() => {
|
||||||
if (!items) return []
|
if (!items) return []
|
||||||
return items.filter((item) => {
|
return items.filter((item) => {
|
||||||
|
|
@ -38,10 +26,6 @@ export function DashboardPage() {
|
||||||
})
|
})
|
||||||
}, [items, search, statusFilter])
|
}, [items, search, statusFilter])
|
||||||
|
|
||||||
// When nlQuery is active (> 2 chars), display NL results; otherwise local filter.
|
|
||||||
const displayItems = nlQuery.trim().length > 2 ? (searchResults ?? []) : filtered
|
|
||||||
const displayLoading = nlQuery.trim().length > 2 ? searchLoading : isLoading
|
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<AppShell>
|
<AppShell>
|
||||||
{/* Page header */}
|
{/* Page header */}
|
||||||
|
|
@ -55,17 +39,14 @@ export function DashboardPage() {
|
||||||
onSearchChange={setSearch}
|
onSearchChange={setSearch}
|
||||||
statusFilter={statusFilter}
|
statusFilter={statusFilter}
|
||||||
onStatusChange={setStatusFilter}
|
onStatusChange={setStatusFilter}
|
||||||
totalCount={displayItems.length}
|
totalCount={filtered.length}
|
||||||
nlQuery={nlQuery}
|
|
||||||
onNlQueryChange={setNlQuery}
|
|
||||||
nlSearchLoading={searchLoading}
|
|
||||||
/>
|
/>
|
||||||
|
|
||||||
{/* Loading */}
|
{/* Loading */}
|
||||||
{displayLoading && (
|
{isLoading && (
|
||||||
<div className="flex items-center justify-center py-24 text-[#a0a0a0]">
|
<div className="flex items-center justify-center py-24 text-[#a0a0a0]">
|
||||||
<Loader2 className="w-6 h-6 animate-spin mr-2" />
|
<Loader2 className="w-6 h-6 animate-spin mr-2" />
|
||||||
{nlQuery.trim().length > 2 ? 'Searching…' : 'Loading inventory…'}
|
Loading inventory…
|
||||||
</div>
|
</div>
|
||||||
)}
|
)}
|
||||||
|
|
||||||
|
|
@ -78,32 +59,28 @@ export function DashboardPage() {
|
||||||
)}
|
)}
|
||||||
|
|
||||||
{/* Empty state */}
|
{/* Empty state */}
|
||||||
{!displayLoading && !error && displayItems.length === 0 && (
|
{!isLoading && !error && filtered.length === 0 && (
|
||||||
<div className="flex flex-col items-center justify-center py-24 text-center">
|
<div className="flex flex-col items-center justify-center py-24 text-center">
|
||||||
<p className="font-display font-black text-4xl text-volt mb-2">0</p>
|
<p className="font-display font-black text-4xl text-volt mb-2">0</p>
|
||||||
<p className="text-[#a0a0a0] text-sm">
|
<p className="text-[#a0a0a0] text-sm">
|
||||||
{nlQuery.trim().length > 2
|
{items && items.length > 0 ? 'No items match your filters' : 'No items cataloged yet — add your first item'}
|
||||||
? 'No items match your search'
|
|
||||||
: items && items.length > 0
|
|
||||||
? 'No items match your filters'
|
|
||||||
: 'No items cataloged yet — add your first item'}
|
|
||||||
</p>
|
</p>
|
||||||
</div>
|
</div>
|
||||||
)}
|
)}
|
||||||
|
|
||||||
{/* Grid view */}
|
{/* Grid view */}
|
||||||
{!displayLoading && !error && displayItems.length > 0 && viewMode === 'grid' && (
|
{!isLoading && !error && filtered.length > 0 && viewMode === 'grid' && (
|
||||||
<div className="grid grid-cols-1 sm:grid-cols-2 md:grid-cols-3 lg:grid-cols-4 xl:grid-cols-5 gap-4">
|
<div className="grid grid-cols-1 sm:grid-cols-2 md:grid-cols-3 lg:grid-cols-4 xl:grid-cols-5 gap-4">
|
||||||
{displayItems.map((item) => (
|
{filtered.map((item) => (
|
||||||
<ItemCard key={item.id} item={item} />
|
<ItemCard key={item.id} item={item} />
|
||||||
))}
|
))}
|
||||||
</div>
|
</div>
|
||||||
)}
|
)}
|
||||||
|
|
||||||
{/* List view */}
|
{/* List view */}
|
||||||
{!displayLoading && !error && displayItems.length > 0 && viewMode === 'list' && (
|
{!isLoading && !error && filtered.length > 0 && viewMode === 'list' && (
|
||||||
<div className="border border-charcoal/80 rounded-card overflow-hidden">
|
<div className="border border-charcoal/80 rounded-card overflow-hidden">
|
||||||
{displayItems.map((item) => (
|
{filtered.map((item) => (
|
||||||
<ItemRow key={item.id} item={item} />
|
<ItemRow key={item.id} item={item} />
|
||||||
))}
|
))}
|
||||||
</div>
|
</div>
|
||||||
|
|
|
||||||
Loading…
Add table
Reference in a new issue