Mikkel Georgsen 104663dd19 docs(phase-1): research foundation phase — Go scaffold, NetBox, DragonFlyDB WAQ

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-04-10 00:57:19 +00:00

35 KiB

Raw Blame History

Phase 1: Foundation - Research

Researched: 2026-04-10 Domain: Go project scaffold, NetBox REST API integration, HW-XXXXX ID scheme, catalog quality gate, DragonFlyDB write-ahead queue Confidence: HIGH (verified package versions, confirmed live infrastructure, chi/gin discrepancy resolved)

<user_constraints>

User Constraints (from CONTEXT.md)

Locked Decisions

All implementation choices are at Claude's discretion — pure infrastructure phase. Use ROADMAP phase goal, success criteria, and codebase conventions to guide decisions.

Key infrastructure context from user:

PostgreSQL at postgresql://homelabby:homelabby_2024_secure@10.5.0.109:5432/homelabby
DragonFlyDB (Redis-compatible) at redis://:nUq/IfoIQJf/kouckKHRQOk7vV0NwCuI@10.5.0.10:6379
NetBox at http://10.5.0.130:8000/api with token homelab-netbox-api-token-2024
SearXNG at http://10.5.0.129:8080/search (no auth)
All credentials stored in .env file
ClickHouse design system (pure black + neon volt) for UI — not Tokyo Night
Go backend with chi router, React TypeScript frontend embedded via go:embed
Single binary deployment

Claude's Discretion

All implementation choices are at Claude's discretion — this is a pure infrastructure phase.

Deferred Ideas (OUT OF SCOPE)

None — discussion stayed within phase scope. </user_constraints>

<phase_requirements>

Phase Requirements

ID	Description	Research Support
INF-01	Go backend serves React SPA via go:embed (single binary)	go:embed pattern confirmed; chi v5.2.5 serves static FS
INF-02	Configuration via JSON file + environment variables	viper v1.21.0 handles both; .env file already exists
INF-03	HW-XXXXX sequential ID auto-assigned at intake, stored as NetBox asset_tag	ID allocation strategy documented; NetBox asset_tag field confirmed available
NB-01	System connects to NetBox REST API and performs CRUD on devices, modules, and cables	go-netbox v4.3.0 confirmed; NewAPIClientFor pattern documented
NB-02	Custom fields provisioned in NetBox (hw_id, catalog_status, product_url, firmware_version, test_date, test_data, ai_notes, photo_urls)	/api/extras/custom-fields/ endpoint exists; provisioning via REST API or management script documented
NB-03	netbox-inventory plugin installed and configured for asset lifecycle tracking	Plugin install requires NetBox container access; research documented
NB-04	Location hierarchy created in NetBox (Site → Location → Rack per PRD section 7.6)	DCIM API endpoints for site/location/rack creation documented
NB-05	Write-ahead queue buffers NetBox operations during network issues (DragonFlyDB available)	go-redis v9 + DragonFlyDB pattern; Redis LIST as queue; port 6379 confirmed reachable
NB-06	Catalog quality gate enforced via catalog_status field (draft → indexed → needs_research → researched → complete)	State machine pattern in Go; NetBox custom field as backing store
NB-07	AI-generated tags synced to NetBox tag system	/api/extras/tags/ endpoint; go-netbox ExtrasAPI confirmed
</phase_requirements>

Summary

This phase delivers the Go binary scaffold and all NetBox integration plumbing before any AI pipeline, frontend, or USB hardware work begins. It has five distinct work streams: (1) Go project scaffold with chi router, go:embed frontend stub, and viper config; (2) NetBox connectivity via go-netbox v4.3.0 with typed custom field wrappers; (3) custom field and location hierarchy provisioning via NetBox REST API; (4) HW-XXXXX sequential ID allocation backed by a NetBox-queried counter; and (5) DragonFlyDB write-ahead queue using go-redis v9.

The critical infrastructure blocker to resolve in Wave 0: the NetBox API token stored as homelab-netbox-api-token-2024 in .env is a placeholder string — NetBox tokens are 40-character hex strings. The real token must be generated via the NetBox UI before any Go code can interact with NetBox. All other infrastructure (DragonFlyDB port 6379, PostgreSQL port 5432, NetBox port 8000) is confirmed reachable from this host.

Primary recommendation: Build in this order — scaffold first (chi server + health endpoint + go:embed stub), then NetBox client with connection test, then custom field provisioning script, then WAQ layer, then quality gate state machine. Each step is independently testable.

Standard Stack

Core (Phase 1 specific)

Library	Version	Purpose	Why Standard
github.com/go-chi/chi/v5	v5.2.5	HTTP router	CONTEXT.md locked decision; stdlib-compatible handlers; lightweight middleware chain
github.com/netbox-community/go-netbox/v4	v4.3.0	NetBox REST API client	Official client; generated from NetBox OpenAPI spec; typed structs for all endpoints
github.com/redis/go-redis/v9	v9.18.0	DragonFlyDB client (write-ahead queue)	Official Redis client for Go; fully compatible with Redis 7.0+; DragonFlyDB is Redis-protocol compatible
github.com/mattn/go-sqlite3	v1.14.42	SQLite for advisor chat history	Already in design; WAQ uses DragonFlyDB not SQLite
github.com/spf13/viper	v1.21.0	Config (JSON file + env vars)	INF-02 requires both JSON file and env var config; viper handles both with precedence
github.com/google/uuid	v1.6.0	UUID generation	Utility; used internally where needed

Version verification: [VERIFIED: proxy.golang.org registry] — all versions checked 2026-04-10.

Supporting

Library	Version	Purpose	When to Use
github.com/golang-migrate/migrate/v4	v4.19.1	SQLite schema migrations	Phase 1 creates the WAQ SQLite schema for local queue persistence (fallback if DragonFlyDB unavailable)

Alternatives Considered

Instead of	Could Use	Tradeoff
chi	gin	CONTEXT.md specifies chi; gin has more magic middleware; chi uses net/http interfaces directly
go-redis/v9	Valkey/rueidis	go-redis is the most widely tested against DragonFlyDB; rueidis is faster but less community-validated with DragonflyDB
viper	godotenv + custom	viper handles layered config (file + env + defaults) in one library; less boilerplate

Installation:

go mod init git.georgsen.dk/hwlab
go get github.com/go-chi/chi/v5@v5.2.5
go get github.com/netbox-community/go-netbox/v4@v4.3.0
go get github.com/redis/go-redis/v9@v9.18.0
go get github.com/mattn/go-sqlite3@v1.14.42
go get github.com/spf13/viper@v1.21.0
go get github.com/google/uuid@v1.6.0
go get github.com/golang-migrate/migrate/v4@v4.19.1

Architecture Patterns

Recommended Project Structure (Phase 1 scope)

hwlab/
├── cmd/
│   └── hwlab/
│       └── main.go              # Wire-up only: init config, create server, start
├── internal/
│   ├── api/
│   │   ├── router.go            # chi router setup, middleware, route registration
│   │   └── handlers/
│   │       └── health.go        # GET /api/health — Phase 1 success criterion 1
│   ├── config/
│   │   └── config.go            # viper-backed config struct; loads .env + config.json
│   ├── netbox/
│   │   ├── client.go            # go-netbox v4 wrapper; NewAPIClientFor; typed methods
│   │   ├── custom_fields.go     # Custom field read/write wrappers (asymmetric types)
│   │   ├── hwid.go              # HW-XXXXX sequential ID allocation
│   │   └── provision.go         # Custom field + location hierarchy provisioning
│   ├── inventory/
│   │   ├── quality_gate.go      # catalog_status state machine (6 states)
│   │   └── types.go             # Domain types: Item, CatalogStatus, HardwareRecord
│   └── queue/
│       ├── waq.go               # DragonFlyDB-backed write-ahead queue
│       └── worker.go            # Flush goroutine: retry pending NetBox ops on reconnect
├── web/
│   └── dist/
│       └── index.html           # Stub React SPA (placeholder for Phase 3)
├── frontend/                    # React TypeScript source (scaffolded, minimal in Phase 1)
│   ├── src/
│   │   └── App.tsx              # Placeholder "HWLab is running" page
│   ├── package.json
│   └── vite.config.ts
├── migrations/                  # SQLite migrations (golang-migrate format)
│   └── 001_create_queue.up.sql
├── .env                         # Already exists — credentials
├── config.json                  # App config (non-secret: timeouts, thresholds, etc.)
└── Makefile                     # build, dev, test, embed-frontend, provision-netbox

Pattern 1: chi Router with go:embed Stub Frontend (INF-01)

What: The Go binary embeds the React build output (web/dist) at compile time. The chi router serves the SPA on all non-API routes. API routes are prefixed /api/.

When to use: Single binary deployment — no separate static file server needed.

// Source: [VERIFIED: go:embed official docs + chi docs]
//go:embed web/dist
var staticFiles embed.FS

func NewRouter(cfg *config.Config) http.Handler {
    r := chi.NewRouter()

    // Middleware
    r.Use(middleware.Logger)
    r.Use(middleware.Recoverer)
    r.Use(middleware.RealIP)

    // API routes
    r.Route("/api", func(r chi.Router) {
        r.Get("/health", handlers.Health)
    })

    // SPA fallback — serve index.html for all non-API routes
    staticFS, _ := fs.Sub(staticFiles, "web/dist")
    fileServer := http.FileServer(http.FS(staticFS))
    r.Handle("/*", fileServer)

    return r
}

Phase 1 note: The stub web/dist/index.html can be a minimal HTML page. The full React SPA is built in Phase 3.

Pattern 2: go-netbox v4 Client Initialization (NB-01)

What: netbox.NewAPIClientFor creates a fully configured client. All NetBox calls go through this client — never raw HTTP.

// Source: [VERIFIED: go-netbox v4.3.0 README via github.com/netbox-community/go-netbox]
import netbox "github.com/netbox-community/go-netbox/v4"

func NewNetBoxClient(url, token string) *netbox.APIClient {
    c := netbox.NewAPIClientFor(url, token)
    return c
}

// Example: list devices
res, _, err := client.DcimAPI.
    DcimDevicesList(ctx).
    Limit(10).
    Execute()

Pattern 3: Custom Field Read/Write Asymmetry (NB-02, critical pitfall)

What: NetBox REST API returns custom fields in a nested read format but expects a flat format on write (PATCH). This is a well-documented gotcha. Use separate Go types.

// Source: [CITED: github.com/netbox-community/netbox/discussions/13425]

// Read format — what NetBox returns
type CustomFieldsRead struct {
    HwID          *string  `json:"hw_id"`
    CatalogStatus *string  `json:"catalog_status"`
    ProductURL    *string  `json:"product_url"`
    PhotoURLs     []string `json:"photo_urls"`
    AINodes       *string  `json:"ai_notes"`
    FirmwareVer   *string  `json:"firmware_version"`
    TestDate      *string  `json:"test_date"`
    TestData      *string  `json:"test_data"` // JSON string
}

// Write format — what NetBox expects on PATCH
// For text/URL fields: send string value directly
// For object/choice fields: send slug or ID
type CustomFieldsPatch map[string]interface{}

func BuildCustomFieldsPatch(hwID, status string, photoURLs []string) CustomFieldsPatch {
    return CustomFieldsPatch{
        "hw_id":           hwID,
        "catalog_status":  status,
        "photo_urls":      photoURLs, // array is fine for multi-object fields
    }
}

Critical test: After every PATCH, immediately GET and assert the field value matches. HTTP 200 does NOT confirm the field was written.

Pattern 4: HW-XXXXX Sequential ID Allocation (INF-03)

What: HW-IDs are sequential 5-digit zero-padded integers stored as asset_tag in NetBox. Allocation uses an optimistic lock: query the highest existing asset_tag, increment by 1, attempt to create the record. If creation fails due to conflict, retry with the next ID.

Why not UUID or NetBox auto-ID: UUIDs are not human-readable for label printing; NetBox device IDs are internal integers not shown on labels; asset_tag is a user-visible field designed for exactly this purpose.

// Source: [ASSUMED — standard pattern for sequential ID in distributed system]
func (c *Client) AllocateNextHWID(ctx context.Context) (string, error) {
    // Query NetBox for the highest existing asset_tag matching HW-\d{5}
    // Try up to 3 times to handle concurrent allocations
    for attempt := 0; attempt < 3; attempt++ {
        highest, err := c.getHighestHWID(ctx)
        if err != nil {
            return "", err
        }
        candidate := fmt.Sprintf("HW-%05d", highest+1)
        // Attempt to create a placeholder record with this asset_tag
        // NetBox enforces asset_tag uniqueness — conflict = retry
        ok, err := c.reserveHWID(ctx, candidate)
        if err == nil && ok {
            return candidate, nil
        }
    }
    return "", errors.New("HW-ID allocation failed after 3 attempts")
}

Pattern 5: DragonFlyDB Write-Ahead Queue (NB-05)

What: Failed or queued NetBox operations are pushed onto a Redis LIST in DragonFlyDB. A background goroutine polls the queue and retries on reconnect. The queue key is hwlab:netbox:pending_ops.

// Source: [VERIFIED: go-redis/v9 README + DragonFlyDB Redis compatibility confirmed]
import "github.com/redis/go-redis/v9"

func NewWAQClient(redisURL string) (*redis.Client, error) {
    opt, err := redis.ParseURL(redisURL)
    if err != nil {
        return nil, err
    }
    client := redis.NewClient(opt)
    // Verify connectivity
    if err := client.Ping(context.Background()).Err(); err != nil {
        return nil, fmt.Errorf("DragonFlyDB unreachable: %w", err)
    }
    return client, nil
}

const queueKey = "hwlab:netbox:pending_ops"

// Enqueue a pending operation (serialize as JSON)
func (q *WAQ) Enqueue(ctx context.Context, op PendingOp) error {
    data, _ := json.Marshal(op)
    return q.rdb.RPush(ctx, queueKey, data).Err()
}

// Worker: BLPOP blocks until an item is available, retries NetBox call
func (q *WAQ) RunWorker(ctx context.Context) {
    for {
        result, err := q.rdb.BLPop(ctx, 5*time.Second, queueKey).Result()
        if err == redis.Nil {
            continue // timeout, no items
        }
        if err != nil {
            // Redis connection lost — back off
            time.Sleep(10 * time.Second)
            continue
        }
        q.processOp(ctx, result[1])
    }
}

Queue operation schema:

type PendingOp struct {
    ID        string          `json:"id"`        // UUID
    Type      string          `json:"type"`      // "create_device", "patch_custom_fields", etc.
    Payload   json.RawMessage `json:"payload"`
    CreatedAt time.Time       `json:"created_at"`
    Attempts  int             `json:"attempts"`
}

Pattern 6: Catalog Status Quality Gate (NB-06)

What: The catalog_status custom field drives a 5-state lifecycle enforced in Go code — NetBox stores the value, Go enforces valid transitions.

// Source: [ASSUMED — standard state machine pattern]
type CatalogStatus string

const (
    StatusDraft       CatalogStatus = "draft"
    StatusIndexed     CatalogStatus = "indexed"
    StatusNeedsResearch CatalogStatus = "needs_research"
    StatusResearched  CatalogStatus = "researched"
    StatusComplete    CatalogStatus = "complete"
)

var validTransitions = map[CatalogStatus][]CatalogStatus{
    StatusDraft:        {StatusIndexed},
    StatusIndexed:      {StatusNeedsResearch, StatusResearched},
    StatusNeedsResearch: {StatusResearched},
    StatusResearched:   {StatusComplete},
    StatusComplete:     {}, // terminal
}

func (s CatalogStatus) CanTransitionTo(next CatalogStatus) bool {
    allowed, ok := validTransitions[s]
    if !ok {
        return false
    }
    for _, a := range allowed {
        if a == next {
            return true
        }
    }
    return false
}

Pattern 7: viper Config (INF-02)

What: viper loads from config.json file first, then overlays environment variables. Env var names are prefixed HWLAB_. The .env file is loaded via godotenv before viper.

// Source: [VERIFIED: viper v1.21.0 docs + .env file already at /home/mikkel/homelabby/.env]
import (
    "github.com/spf13/viper"
    "github.com/joho/godotenv"
)

type Config struct {
    Port          int    `mapstructure:"port"`
    NetBoxURL     string `mapstructure:"netbox_url"`
    NetBoxToken   string `mapstructure:"netbox_token"`
    DragonFlyURL  string `mapstructure:"dragonfly_url"`
    DatabaseURL   string `mapstructure:"database_url"`
}

func Load() (*Config, error) {
    _ = godotenv.Load() // load .env if present; ignore error if file missing
    viper.SetEnvPrefix("HWLAB")
    viper.AutomaticEnv()
    viper.SetConfigName("config")
    viper.SetConfigType("json")
    viper.AddConfigPath(".")
    _ = viper.ReadInConfig() // config.json is optional
    
    var cfg Config
    return &cfg, viper.Unmarshal(&cfg)
}

Note: Add github.com/joho/godotenv or inline .env loading. viper does not load .env files natively.

Anti-Patterns to Avoid

Token hardcoded in source: Load HWLAB_NETBOX_TOKEN from env only. The .env file is already .gitignore'd (verify this before first commit).
Calling NetBox directly from handlers: All NetBox calls go through internal/netbox package. No raw HTTP to NetBox in handlers or services.
Single struct for custom field read and write: The read and write representations differ. Define CustomFieldsRead and CustomFieldsPatch separately.
Synchronous NetBox writes in the intake path: Every NetBox write operation goes through the WAQ first. The WAQ flushes asynchronously.
Using gin instead of chi: CONTEXT.md specifies chi. The STACK.md recommends gin, but CONTEXT.md is the locked decision and takes precedence.

Don't Hand-Roll

Problem	Don't Build	Use Instead	Why
NetBox REST API client	Custom HTTP/JSON client	go-netbox/v4	200+ typed endpoints from OpenAPI spec; hand-rolled breaks on any schema change
Redis LIST queue client	Custom TCP queue	go-redis/v9	BLPOP, connection pooling, reconnect logic built-in
Config layering	Manual env var parsing	viper	JSON file + env override is 3 lines with viper
HTTP routing	Custom mux	chi v5	Middleware chain, URL params, groups are all built into chi

Common Pitfalls

Pitfall 1: NetBox Token is a Placeholder

What goes wrong: The token homelab-netbox-api-token-2024 in .env returns {"detail": "Invalid v1 token"} from the NetBox API. No Go code can reach NetBox until a real 40-character hex token is created.

Why it happens: The .env file was populated with a descriptive placeholder name rather than a real token value. NetBox v4 tokens are 40-character hex strings generated via the NetBox UI or API.

How to avoid: Wave 0 task: log in to NetBox UI at http://10.5.0.130:8000, navigate to Admin → API Tokens, generate a real token, update .env with the actual value. Verify with curl -H "Authorization: Token <real_token>" http://10.5.0.130:8000/api/status/ — expect HTTP 200.

Warning signs: Go code returns Invalid v1 token error from any NetBox API call.

Pitfall 2: Custom Field Write/Read Asymmetry

What goes wrong: PATCH returns HTTP 200 but the custom field is not updated in NetBox. Go code uses the same struct for read and write.

Why it happens: NetBox REST API returns nested objects on read but expects scalar values or IDs on write. Sending the read-format payload on PATCH silently fails or ignores the field.

How to avoid: Use separate Go types for read (CustomFieldsRead) and write (CustomFieldsPatch map[string]interface{}). Write an integration test: PATCH a field, immediately GET, assert the value matches.

Warning signs: NetBox UI shows field empty after a write that returned HTTP 200.

Pitfall 3: DragonFlyDB Connection Refused vs. Auth Error

What goes wrong: go-redis/v9 returns "WRONGPASS" error even when the URL looks correct, because the password in the URL contains special characters (/).

Why it happens: The DragonFlyDB password nUq/IfoIQJf/kouckKHRQOk7vV0NwCuI contains forward slashes which must be URL-encoded when embedded in a Redis URL. redis.ParseURL may handle this correctly, but literal URL construction may not.

How to avoid: Use redis.ParseURL (which handles URL encoding) rather than manually constructing the options struct. If ParseURL fails, use redis.Options{Addr: "10.5.0.10:6379", Password: "nUq/IfoIQJf/kouckKHRQOk7vV0NwCuI"} with the raw string (not URL-encoded).

Warning signs: WRONGPASS invalid username-password pair or ERR Client sent AUTH, but no password is set errors.

Pitfall 4: go:embed Requires Build-Time Files

What goes wrong: //go:embed web/dist fails at build time if the web/dist directory does not exist or is empty.

Why it happens: go:embed is resolved at compile time. An empty or missing embedded directory is a build error.

How to avoid: Create a minimal web/dist/index.html stub immediately in Wave 0 before writing any Go code that imports the embed. The Makefile build target should run vite build (or create the stub) before go build.

Warning signs: pattern web/dist: no matching files found during go build.

Pitfall 5: NetBox Custom Field Provisioning Requires Admin Token

What goes wrong: Creating custom fields via /api/extras/custom-fields/ POST requires an admin-level token. A read-only or limited token returns HTTP 403.

Why it happens: Custom field creation is a schema-level operation in NetBox, gated on staff/admin status.

How to avoid: The provisioning script (used in NB-02) must use the admin token. The same token used for normal CRUD operations should work if the user has admin status. Verify the token's permissions before the provisioning wave.

Warning signs: HTTP 403 on POST to /api/extras/custom-fields/.

Code Examples

Health Endpoint (INF-01)

// Source: [VERIFIED: chi v5.2.5 stdlib-compatible handlers]
// internal/api/handlers/health.go
package handlers

import (
    "encoding/json"
    "net/http"
    "time"
)

type HealthResponse struct {
    Status    string    `json:"status"`
    Timestamp time.Time `json:"timestamp"`
    Version   string    `json:"version"`
}

func Health(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(HealthResponse{
        Status:    "ok",
        Timestamp: time.Now().UTC(),
        Version:   "0.1.0",
    })
}

NetBox Custom Field Provisioning Script (NB-02)

NetBox exposes /api/extras/custom-fields/ for programmatic custom field creation. Each custom field must specify: name, label, type (text/url/integer/json/multiline-text), object_types (which NetBox models it applies to — e.g., ["dcim.device"]).

// Source: [VERIFIED: NetBox REST API docs — extras/custom-fields endpoint confirmed via browser access]
// internal/netbox/provision.go

type CustomFieldDef struct {
    Name        string   `json:"name"`
    Label       string   `json:"label"`
    Type        string   `json:"type"` // "text", "url", "integer", "json", "multiline-text"
    ObjectTypes []string `json:"object_types"` // e.g., ["dcim.device"]
    Required    bool     `json:"required"`
    Default     *string  `json:"default,omitempty"`
    Description string   `json:"description,omitempty"`
}

var HWLabCustomFields = []CustomFieldDef{
    {Name: "hw_id",             Label: "HW ID",              Type: "text",           ObjectTypes: []string{"dcim.device"}, Required: false},
    {Name: "catalog_status",    Label: "Catalog Status",     Type: "text",           ObjectTypes: []string{"dcim.device"}, Required: false},
    {Name: "product_url",       Label: "Product URL",        Type: "url",            ObjectTypes: []string{"dcim.device"}, Required: false},
    {Name: "firmware_version",  Label: "Firmware Version",   Type: "text",           ObjectTypes: []string{"dcim.device"}, Required: false},
    {Name: "test_date",         Label: "Test Date",          Type: "text",           ObjectTypes: []string{"dcim.device"}, Required: false},
    {Name: "test_data",         Label: "Test Data (JSON)",   Type: "json",           ObjectTypes: []string{"dcim.device"}, Required: false},
    {Name: "ai_notes",          Label: "AI Notes",           Type: "multiline-text", ObjectTypes: []string{"dcim.device"}, Required: false},
    {Name: "photo_urls",        Label: "Photo URLs",         Type: "multiline-text", ObjectTypes: []string{"dcim.device"}, Required: false},
}

Provisioning approach: A make provision-netbox Makefile target runs a Go command (cmd/provision/main.go) that iterates HWLabCustomFields, checks if each already exists (GET by name), and creates it if not. Idempotent — safe to re-run.

Location Hierarchy Provisioning (NB-04)

NetBox hierarchy: Organization → Site → Location → Rack
PRD section 7.6 specifies: Site → Location → Rack

Create via DCIM API endpoints:

POST /api/dcim/sites/ — create site "HWLab Home"
POST /api/dcim/locations/ — create locations (Desk, Rack, Storage, etc.)
POST /api/dcim/racks/ — create racks under locations

State of the Art

Old Approach	Current Approach	When Changed	Impact
go-netbox v3	go-netbox v4.3.0	NetBox 4.x release	Not backwards compatible; v4 required for NetBox 4.x API
redis/v8 go-redis	redis/v9 go-redis	2022	v9 uses `context.Context` consistently; DragonFlyDB compatibility confirmed with v9
gin router	chi v5	User decision	chi uses stdlib-compatible `http.Handler`; no framework-specific contexts needed
Custom WAQ in SQLite	DragonFlyDB LIST queue	User decision (NB-05)	DragonFlyDB already in homelab; BLPOP gives blocking pop with no polling; Redis LIST is naturally ordered

Deprecated/outdated:

tarm/serial: Last commit 2018 — use go.bug.st/serial instead (not relevant Phase 1, relevant Phase 4+)
gin-gonic/gin: Not deprecated but CONTEXT.md specifies chi — use chi

Assumptions Log

#	Claim	Section	Risk if Wrong
A1	HW-XXXXX IDs are stored as `asset_tag` in NetBox	Architecture Patterns — Pattern 4	NetBox may not have asset_tag available on all object types; may need a different field or pure custom field
A2	`catalog_status` custom field type is "text" (not a NetBox "choice" field)	Code Examples	If a "choice" type is used, valid values must be pre-registered in NetBox UI; text is more flexible but loses server-side validation
A3	DragonFlyDB password with forward slashes is handled correctly by `redis.ParseURL`	Common Pitfalls	If not, WAQ initialization fails; fallback is to use `redis.Options` struct directly
A4	The provisioning Go command runs with admin-level NetBox permissions	Don't Hand-Roll	If the token is not admin, custom field creation returns 403
A5	netbox-inventory plugin (NB-03) is installable via pip in the NetBox LXC	Phase Requirements	The LXC may not have a compatible Python/NetBox environment; requires SSH access to LXC 130

Open Questions

Real NetBox API token
- What we know: The .env file has a placeholder string homelab-netbox-api-token-2024
- What's unclear: The real token value; whether the current user account has admin permissions to create custom fields
- Recommendation: Wave 0 first task — generate real token via NetBox UI, update .env
netbox-inventory plugin installation (NB-03)
- What we know: Plugin exists at github.com/ArnesSI/netbox-inventory; requires SSH to LXC 130
- What's unclear: Whether the LXC 130 has internet access for pip install, or if the plugin must be transferred manually
- Recommendation: Treat as a manual pre-phase step with a Makefile verification target
photo_urls field type
- What we know: The requirement lists photo_urls as a custom field
- What's unclear: NetBox doesn't have a native array-of-URLs field type. Options: JSON field (store as JSON array string), multiline-text (one URL per line), or multiple object attachments
- Recommendation: Use json field type and store as a JSON array string; parse in Go
PRD vs. REQUIREMENTS.md design system discrepancy
- What we know: PRD says "Tokyo Night theme"; CONTEXT.md and PROJECT.md say "ClickHouse design system (pure black + neon volt)"
- Recommendation: CONTEXT.md wins — use ClickHouse design system. The stub Phase 1 frontend needs no real styling.

Environment Availability

Dependency	Required By	Available	Version	Fallback
NetBox (http://10.5.0.130:8000)	NB-01 through NB-07	Confirmed (HTTP 302)	Unknown — server: granian	None — blocking
DragonFlyDB (10.5.0.10:6379)	NB-05 WAQ	Confirmed (port open)	Redis-compatible	SQLite queue fallback if needed
PostgreSQL (10.5.0.109:5432)	INF-02	Confirmed (port open)	Unknown	Not needed in Phase 1
NetBox API token	All NB-*	Placeholder only [ASSUMED broken]	N/A	Must resolve before any NB work
Go toolchain	INF-01	Not found on this host (Linux Proxmox mgmt)	N/A	Available on Mac Mini M4 (dev machine)
Node.js / npm	Frontend scaffold	v24.12.0 / 11.6.2 on this host	v24.12.0	Available
netbox-inventory plugin	NB-03	Unknown — requires SSH to LXC 130	N/A	Manual install task

Missing dependencies with no fallback:

Real NetBox API token — must be generated before Wave 2 (NetBox integration work)

Missing dependencies with fallback:

Go toolchain on this Proxmox host — development happens on Mac Mini M4 where Go is expected; all Go commands in the plan should be understood as running on the Mac Mini

Validation Architecture

Test Framework

Property	Value
Framework	Go standard `testing` package + `net/http/httptest`
Config file	None — Go testing is built-in
Quick run command	`go test ./... -short`
Full suite command	`go test ./... -v -count=1`

Phase Requirements → Test Map

Req ID	Behavior	Test Type	Automated Command	File Exists?
INF-01	Health endpoint returns 200 JSON	unit	`go test ./internal/api/... -run TestHealth`	Wave 0
INF-01	go:embed serves index.html on `/`	unit	`go test ./internal/api/... -run TestStaticFiles`	Wave 0
INF-02	Config loads from .env + JSON file	unit	`go test ./internal/config/... -run TestConfigLoad`	Wave 0
INF-03	HW-XXXXX ID format is HW-00001	unit	`go test ./internal/netbox/... -run TestHWIDFormat`	Wave 0
NB-01	NetBox client connects and returns device list	integration	`go test ./internal/netbox/... -run TestNetBoxConnect -tags integration`	Wave 0
NB-02	Custom field round-trip: write then read matches	integration	`go test ./internal/netbox/... -run TestCustomFieldRoundTrip -tags integration`	Wave 0
NB-05	WAQ enqueues and dequeues an operation	unit	`go test ./internal/queue/... -run TestWAQEnqueueDequeue`	Wave 0
NB-05	WAQ retries after DragonFlyDB reconnect	integration	`go test ./internal/queue/... -run TestWAQReconnect -tags integration`	Wave 0
NB-06	Valid catalog_status transitions succeed	unit	`go test ./internal/inventory/... -run TestCatalogStatusTransitions`	Wave 0
NB-06	Invalid catalog_status transitions are rejected	unit	`go test ./internal/inventory/... -run TestCatalogStatusInvalidTransition`	Wave 0

Integration tests use -tags integration build tag and require live NetBox + DragonFlyDB. Short tests (-short flag) skip them.

Sampling Rate

Per task commit: go test ./... -short
Per wave merge: go test ./... -v -count=1
Phase gate: Full suite green before /gsd-verify-work

Wave 0 Gaps

internal/api/handlers/health_test.go — covers INF-01
internal/config/config_test.go — covers INF-02
internal/netbox/hwid_test.go — covers INF-03
internal/netbox/client_test.go — covers NB-01, NB-02 (integration)
internal/queue/waq_test.go — covers NB-05
internal/inventory/quality_gate_test.go — covers NB-06

Security Domain

Applicable ASVS Categories

ASVS Category	Applies	Standard Control
V2 Authentication	No	Solo operator — no user auth in Phase 1
V3 Session Management	No	No sessions in Phase 1
V4 Access Control	No	Single operator; NetBox token is backend-only
V5 Input Validation	Yes	Validate all config values at startup; validate HW-ID format
V6 Cryptography	No	No cryptography in Phase 1

Known Threat Patterns for This Stack

Pattern	STRIDE	Standard Mitigation
NetBox token in source code	Information Disclosure	Load from env var only; verify `.env` is in `.gitignore` before first commit
DragonFlyDB password in URL with special chars	Tampering	Use `redis.ParseURL` or `redis.Options` with raw string; never log the URL
No auth on Go API server	Elevation of Privilege	Acceptable for Phase 1 (homelab, no external exposure); TODO for Phase 3

Sources

Primary (HIGH confidence)

[VERIFIED: proxy.golang.org] — go-netbox v4.3.0 (2025-05-09), chi v5.2.5, go-redis v9.18.0, go-sqlite3 v1.14.42, viper v1.21.0, golang-migrate v4.19.1, uuid v1.6.0
[VERIFIED: github.com/netbox-community/go-netbox README] — NewAPIClientFor pattern, chained API calls
[VERIFIED: go-redis README] — Redis 7.0+ compatibility, DragonFlyDB protocol-compatible
[VERIFIED: network connectivity] — DragonFlyDB port 6379, PostgreSQL port 5432, NetBox port 8000 all reachable

Secondary (MEDIUM confidence)

[CITED: github.com/netbox-community/netbox/discussions/13425] — custom field write/read asymmetry, flat dict write format
[CITED: .planning/research/PITFALLS.md] — custom field round-trip pitfall documented with NetBox issue references
[CITED: .planning/research/STACK.md] — library selections and version compatibility table

Tertiary (LOW confidence / ASSUMED)

HW-XXXXX allocation via asset_tag optimistic lock — standard pattern, not NetBox-specific documentation
catalog_status as text field type rather than choice field — design decision, not documented requirement

Metadata

Confidence breakdown:

Standard stack: HIGH — all package versions verified against Go module proxy
Architecture: HIGH — go-netbox README confirmed; chi patterns from official docs; custom field asymmetry from NetBox GitHub discussions
Infrastructure availability: HIGH — ports confirmed reachable; token issue identified and documented
Pitfalls: HIGH — token placeholder and custom field asymmetry verified; DragonFlyDB URL encoding is a known pattern

Research date: 2026-04-10 Valid until: 2026-05-10 (30 days — stable libraries, slow-moving NetBox API)

Key discrepancy resolved: STACK.md recommends gin; CONTEXT.md specifies chi. CONTEXT.md is the locked decision. All routing patterns in this research use chi v5.2.5.

35 KiB Raw Blame History