homelabby/.planning/phases/01-foundation/01-03-PLAN.md

phase	plan	type	wave	depends_on	files_modified	autonomous	requirements	must_haves
01-foundation	03	execute	2	01-02-PLAN.md	internal/netbox/provision.go internal/netbox/provision_test.go scripts/provision-netbox.go	true	NB-02 NB-03 NB-04	truths artifacts key_links All 8 HWLab custom fields exist in NetBox after provisioning runs GET /api/extras/custom-fields/?name=hw_id returns the hw_id field definition GET /api/extras/custom-fields/?name=catalog_status returns the catalog_status field Location hierarchy exists: at least one Site, one Location, one Rack in NetBox Provisioning script is idempotent — running it twice does not create duplicates path provides exports internal/netbox/provision.go Provision() function: creates custom fields + location hierarchy if not present Provision ProvisionCustomFields ProvisionLocationHierarchy path provides scripts/provision-netbox.go Standalone CLI: `go run scripts/provision-netbox.go` — provisions NetBox from .env from to via pattern scripts/provision-netbox.go internal/netbox/provision.go direct function call Provision(client) Provision from to via pattern internal/netbox/provision.go http://10.5.0.130:8000/api REST POST /api/extras/custom-fields/, /api/dcim/sites/, /api/dcim/locations/, /api/dcim/racks/ ExtrasAPI|DcimAPI

Provision the NetBox instance with all HWLab custom fields and location hierarchy. This is an idempotent provisioning operation: if a custom field already exists, skip it; if the location hierarchy exists, skip it.

Purpose: Custom fields must exist in NetBox before any Go code can write them. NB-02 requires 8 specific fields; NB-03 requires the netbox-inventory plugin installed; NB-04 requires the Site→Location→Rack hierarchy. Output: A Provision() function and a standalone script that a human can run against the live NetBox instance to bootstrap it.

<execution_context> @$HOME/.claude/get-shit-done/workflows/execute-plan.md @$HOME/.claude/get-shit-done/templates/summary.md </execution_context>

@.planning/PROJECT.md @.planning/ROADMAP.md @.env @.planning/phases/01-foundation/01-RESEARCH.md @.planning/phases/01-foundation/01-02-SUMMARY.md From internal/netbox/client.go: ```go type Client struct { ... } func NewClient(url, token string) (*Client, error) // Client.api is *nb.APIClient (go-netbox v4) // Access raw API via: client.api.ExtrasAPI, client.api.DcimAPI ```

NetBox custom field POST payload (REST API):

{
  "name": "hw_id",
  "label": "HW ID",
  "type": "text",
  "object_types": ["dcim.device"],
  "required": false,
  "description": "HWLab sequential identifier (HW-XXXXX)"
}

NetBox location hierarchy (DCIM API):

• Site: POST /api/dcim/sites/ {name, slug}
• Location: POST /api/dcim/locations/ {name, slug, site: {id}}
• Rack: POST /api/dcim/racks/ {name, site: {id}, location: {id}, u_height: 42}

Check for existing custom field by name: GET /api/extras/custom-fields/?name=hw_id

• If count > 0, field exists — skip creation

Task 1: Custom field provisioning (NB-02) internal/netbox/provision.go, internal/netbox/provision_test.go

<read_first> - /home/mikkel/homelabby/internal/netbox/client.go (Client struct, api field access pattern) - /home/mikkel/homelabby/.planning/phases/01-foundation/01-RESEARCH.md (phase requirements table, lines 38-48) - /home/mikkel/homelabby/.env (HWLAB_NETBOX_TOKEN — check if real 40-char token) </read_first>

- Test 1: customFieldSpec("hw_id") returns a spec with Name="hw_id", Type="text", ObjectTypes=["dcim.device"] - Test 2: customFieldSpec("catalog_status") returns Type="text" (stored as free text, not NetBox choice field, to avoid NetBox admin dependency) - Test 3: customFieldSpec("photo_urls") returns a spec with Type="text" and description mentioning comma-separated - Test 4 (INTEGRATION — skip if no real token): ProvisionCustomFields() creates all 8 fields and returns no error - Test 5 (INTEGRATION — skip if no real token): ProvisionCustomFields() called twice is idempotent (no error, no duplicate fields) All 8 custom fields to provision (NB-02): - hw_id: text, dcim.device — "HWLab sequential identifier (HW-XXXXX)" - catalog_status: text, dcim.device — "Lifecycle status: draft|indexed|needs_research|researched|complete" - product_url: url, dcim.device — "Manufacturer product page URL" - firmware_version: text, dcim.device — "Current firmware/software version" - test_date: date, dcim.device — "Date of last cable/hardware test" - test_data: text, dcim.device — "Structured JSON test results from cable testers" - ai_notes: text, dcim.device — "AI-generated notes from intake analysis" - photo_urls: text, dcim.device — "Comma-separated photo URLs captured during intake"

NOTE on photo_urls: Use type "text" (not multi-object) to avoid NetBox v4 multi-value custom field complexity. Store as comma-separated string. The Go layer will split/join as needed.

NOTE on catalog_status: Use type "text" not NetBox "selection" field — the Go quality gate owns the valid values; we don't want to maintain a parallel list in NetBox admin UI.

Create `internal/netbox/provision.go`:
```go
package netbox

import (
    "context"
    "fmt"
    "log"
)

// CustomFieldSpec defines a NetBox custom field to provision.
type CustomFieldSpec struct {
    Name        string
    Label       string
    Type        string // "text", "url", "date", "integer", "boolean"
    ObjectTypes []string
    Description string
    Required    bool
}

// hwlabCustomFields is the canonical list of all HWLab custom fields.
// These MUST be provisioned in NetBox before any item can be created.
var hwlabCustomFields = []CustomFieldSpec{
    {Name: "hw_id", Label: "HW ID", Type: "text", ObjectTypes: []string{"dcim.device"}, Description: "HWLab sequential identifier (HW-XXXXX)"},
    {Name: "catalog_status", Label: "Catalog Status", Type: "text", ObjectTypes: []string{"dcim.device"}, Description: "Lifecycle: draft|indexed|needs_research|researched|complete"},
    {Name: "product_url", Label: "Product URL", Type: "url", ObjectTypes: []string{"dcim.device"}, Description: "Manufacturer product page URL"},
    {Name: "firmware_version", Label: "Firmware Version", Type: "text", ObjectTypes: []string{"dcim.device"}, Description: "Current firmware/software version"},
    {Name: "test_date", Label: "Test Date", Type: "date", ObjectTypes: []string{"dcim.device"}, Description: "Date of last cable/hardware test (ISO 8601)"},
    {Name: "test_data", Label: "Test Data", Type: "text", ObjectTypes: []string{"dcim.device"}, Description: "Structured JSON test results from cable testers"},
    {Name: "ai_notes", Label: "AI Notes", Type: "text", ObjectTypes: []string{"dcim.device"}, Description: "AI-generated notes from intake photo analysis"},
    {Name: "photo_urls", Label: "Photo URLs", Type: "text", ObjectTypes: []string{"dcim.device"}, Description: "Comma-separated photo URLs captured during intake"},
}

// customFieldSpec returns the spec for a named custom field (for testing).
func customFieldSpec(name string) *CustomFieldSpec {
    for i := range hwlabCustomFields {
        if hwlabCustomFields[i].Name == name {
            return &hwlabCustomFields[i]
        }
    }
    return nil
}

// ProvisionCustomFields ensures all HWLab custom fields exist in NetBox.
// Idempotent: fields that already exist are skipped.
// Returns the count of fields created (0 if all existed).
func (c *Client) ProvisionCustomFields(ctx context.Context) (int, error) {
    created := 0
    for _, spec := range hwlabCustomFields {
        exists, err := c.customFieldExists(ctx, spec.Name)
        if err != nil {
            return created, fmt.Errorf("check field %s: %w", spec.Name, err)
        }
        if exists {
            log.Printf("custom field %q already exists — skipping", spec.Name)
            continue
        }
        if err := c.createCustomField(ctx, spec); err != nil {
            return created, fmt.Errorf("create field %s: %w", spec.Name, err)
        }
        log.Printf("created custom field %q", spec.Name)
        created++
    }
    return created, nil
}

// customFieldExists checks if a custom field with the given name already exists.
func (c *Client) customFieldExists(ctx context.Context, name string) (bool, error) {
    res, _, err := c.api.ExtrasAPI.ExtrasCustomFieldsList(ctx).Name([]string{name}).Execute()
    if err != nil {
        return false, err
    }
    return res.GetCount() > 0, nil
}

// createCustomField creates a single custom field in NetBox via the Extras API.
// Uses the go-netbox v4 generated WritableCustomFieldRequest type.
func (c *Client) createCustomField(ctx context.Context, spec CustomFieldSpec) error {
    // go-netbox v4: use CustomFieldTypeValue for the Type field
    // Available types: "text", "longtext", "integer", "decimal", "boolean",
    //                  "date", "datetime", "url", "json", "select", "multiselect",
    //                  "object", "multiobject"
    // NOTE: Executor must check the actual enum values in go-netbox v4 generated code:
    //   grep -r "CustomFieldTypeValue" $(go env GOPATH)/pkg/mod/github.com/netbox-community/go-netbox/ | head -10

    nb_pkg := "github.com/netbox-community/go-netbox/v4"
    _ = nb_pkg

    // Pseudocode — executor must use real go-netbox v4 WritableCustomFieldRequest:
    // req := nb.WritableCustomFieldRequest{
    //     Name:        spec.Name,
    //     Label:       nb.PtrString(spec.Label),
    //     Type:        nb.CustomFieldTypeValue(spec.Type),
    //     ObjectTypes: spec.ObjectTypes,
    //     Description: nb.PtrString(spec.Description),
    // }
    // _, _, err := c.api.ExtrasAPI.ExtrasCustomFieldsCreate(ctx).
    //     WritableCustomFieldRequest(req).Execute()
    // return err

    return fmt.Errorf("createCustomField: implement using go-netbox v4 WritableCustomFieldRequest — check generated types at $(go env GOPATH)/pkg/mod/github.com/netbox-community/go-netbox/v4@*/model_writable_custom_field_request.go")
}

// Provision runs all provisioning steps: custom fields + location hierarchy.
func (c *Client) Provision(ctx context.Context) error {
    n, err := c.ProvisionCustomFields(ctx)
    if err != nil {
        return fmt.Errorf("provision custom fields: %w", err)
    }
    log.Printf("custom fields: %d created", n)

    if err := c.ProvisionLocationHierarchy(ctx); err != nil {
        return fmt.Errorf("provision locations: %w", err)
    }
    return nil
}
```

IMPORTANT: The `createCustomField` function contains a stub that will not compile. The executor MUST:
1. Run: `ls $(go env GOPATH)/pkg/mod/github.com/netbox-community/go-netbox/v4@v4.3.0/` to find generated files
2. Run: `grep -l "WritableCustomFieldRequest" $(go env GOPATH)/pkg/mod/github.com/netbox-community/go-netbox/v4@v4.3.0/*.go`
3. Read the relevant generated file to get the exact field names and enum types
4. Replace the pseudocode with real go-netbox v4 API calls

Create `internal/netbox/provision_test.go`:
```go
package netbox

import (
    "testing"
)

func TestCustomFieldSpec(t *testing.T) {
    spec := customFieldSpec("hw_id")
    if spec == nil {
        t.Fatal("hw_id spec not found")
    }
    if spec.Type != "text" {
        t.Errorf("hw_id type: want text, got %s", spec.Type)
    }
    for _, ot := range spec.ObjectTypes {
        if ot == "dcim.device" {
            return
        }
    }
    t.Error("hw_id ObjectTypes must include dcim.device")
}

func TestCustomFieldSpecCatalogStatus(t *testing.T) {
    spec := customFieldSpec("catalog_status")
    if spec == nil {
        t.Fatal("catalog_status spec not found")
    }
    if spec.Type != "text" {
        t.Errorf("catalog_status type: want text (not selection), got %s", spec.Type)
    }
}

func TestCustomFieldSpecPhotoURLs(t *testing.T) {
    spec := customFieldSpec("photo_urls")
    if spec == nil {
        t.Fatal("photo_urls spec not found")
    }
    if spec.Description == "" {
        t.Error("photo_urls must have a description")
    }
}

func TestAllEightFieldsDefined(t *testing.T) {
    expected := []string{"hw_id", "catalog_status", "product_url", "firmware_version",
        "test_date", "test_data", "ai_notes", "photo_urls"}
    for _, name := range expected {
        if customFieldSpec(name) == nil {
            t.Errorf("missing custom field spec: %s", name)
        }
    }
    if len(hwlabCustomFields) != 8 {
        t.Errorf("want 8 custom fields, got %d", len(hwlabCustomFields))
    }
}
```

cd /home/mikkel/homelabby && go test ./internal/netbox/... -v -run "TestCustomFieldSpec|TestAllEight"

<acceptance_criteria> - go test ./internal/netbox/... -run "TestCustomFieldSpec|TestAllEight" passes all 4 unit tests - grep -c "Name:" internal/netbox/provision.go returns 8 or more (one per custom field) - grep "photo_urls" internal/netbox/provision.go appears in hwlabCustomFields slice - grep "ai_notes" internal/netbox/provision.go appears in hwlabCustomFields slice - createCustomField is implemented with real go-netbox v4 API calls (not stub returning error string) - go build ./internal/netbox/... exits 0 </acceptance_criteria>

All 8 custom field specs defined and tested. ProvisionCustomFields implemented with idempotent check-before-create. createCustomField uses real go-netbox v4 API (not stub).

Task 2: Location hierarchy provisioning + NB-03 plugin check + provision CLI (NB-03, NB-04) internal/netbox/provision.go, scripts/provision-netbox.go

<read_first> - /home/mikkel/homelabby/internal/netbox/provision.go (ProvisionCustomFields added in Task 1) - /home/mikkel/homelabby/.planning/phases/01-foundation/01-RESEARCH.md (phase requirements NB-03, NB-04 lines 43-48) - /home/mikkel/homelabby/.env (HWLAB_NETBOX_URL, HWLAB_NETBOX_TOKEN) </read_first>

Location hierarchy to create (per ROADMAP "Site → Location → Rack per PRD section 7.6"): - Site: name="Homelab", slug="homelab" - Location: name="Lab Bench", slug="lab-bench", site=homelab - Rack: name="Primary Rack", site=homelab, location=lab-bench, u_height=42

1. Add `ProvisionLocationHierarchy` to `internal/netbox/provision.go`:
   ```go
   // ProvisionLocationHierarchy creates the Site → Location → Rack hierarchy.
   // Idempotent: each level is checked before creation.
   func (c *Client) ProvisionLocationHierarchy(ctx context.Context) error {
       // Step 1: Create or find Site "Homelab"
       siteID, err := c.ensureSite(ctx, "Homelab", "homelab")
       if err != nil {
           return fmt.Errorf("ensure site: %w", err)
       }

       // Step 2: Create or find Location "Lab Bench" under the site
       locationID, err := c.ensureLocation(ctx, "Lab Bench", "lab-bench", siteID)
       if err != nil {
           return fmt.Errorf("ensure location: %w", err)
       }

       // Step 3: Create or find Rack "Primary Rack" under the site + location
       if err := c.ensureRack(ctx, "Primary Rack", siteID, locationID); err != nil {
           return fmt.Errorf("ensure rack: %w", err)
       }

       return nil
   }

   func (c *Client) ensureSite(ctx context.Context, name, slug string) (int32, error) {
       // Check if site exists by slug
       res, _, err := c.api.DcimAPI.DcimSitesList(ctx).Slug([]string{slug}).Execute()
       if err != nil {
           return 0, err
       }
       if res.GetCount() > 0 {
           log.Printf("site %q already exists — skipping", name)
           return res.Results[0].GetId(), nil
       }
       // Create site
       // NOTE: Executor must use go-netbox v4 WritableSiteRequest type
       // Pattern: c.api.DcimAPI.DcimSitesCreate(ctx).WritableSiteRequest(req).Execute()
       // Required fields: Name (string), Slug (string)
       // Status: use nb.SiteStatusValue("active") or equivalent
       return 0, fmt.Errorf("ensureSite: implement using go-netbox v4 WritableSiteRequest")
   }

   func (c *Client) ensureLocation(ctx context.Context, name, slug string, siteID int32) (int32, error) {
       res, _, err := c.api.DcimAPI.DcimLocationsList(ctx).Slug([]string{slug}).Execute()
       if err != nil {
           return 0, err
       }
       if res.GetCount() > 0 {
           log.Printf("location %q already exists — skipping", name)
           return res.Results[0].GetId(), nil
       }
       // Create location under site
       // NOTE: use go-netbox v4 WritableLocationRequest
       // Required: Name, Slug, Site (NestedSiteRequest with ID)
       return 0, fmt.Errorf("ensureLocation: implement using go-netbox v4 WritableLocationRequest")
   }

   func (c *Client) ensureRack(ctx context.Context, name string, siteID, locationID int32) error {
       res, _, err := c.api.DcimAPI.DcimRacksList(ctx).Name([]string{name}).Execute()
       if err != nil {
           return err
       }
       if res.GetCount() > 0 {
           log.Printf("rack %q already exists — skipping", name)
           return nil
       }
       // Create rack under site + location
       // NOTE: use go-netbox v4 WritableRackRequest
       // Required: Name, Site (NestedSiteRequest), Location (NestedLocationRequest), UHeight (int32)
       // UHeight = 42
       return fmt.Errorf("ensureRack: implement using go-netbox v4 WritableRackRequest")
   }
   ```

   EXECUTOR MUST replace all stub returns with real go-netbox v4 API calls. Locate the generated types:
   ```
   ls $(go env GOPATH)/pkg/mod/github.com/netbox-community/go-netbox/v4@v4.3.0/model_writable_site_request.go
   ls $(go env GOPATH)/pkg/mod/github.com/netbox-community/go-netbox/v4@v4.3.0/model_writable_location_request.go
   ls $(go env GOPATH)/pkg/mod/github.com/netbox-community/go-netbox/v4@v4.3.0/model_writable_rack_request.go
   ```

2. Add NB-03 plugin check note in provision.go as a comment (NB-03 requires SSH to LXC 130 — this is a manual verification step documented in VALIDATION.md):
   ```go
   // CheckNetBoxInventoryPlugin verifies the netbox-inventory plugin is installed.
   // This check uses the NetBox plugins API endpoint.
   // NB-03: netbox-inventory plugin must be installed on LXC 130.
   // Manual verification: SSH to LXC 130, run: pip show netbox-inventory
   // API check: GET /api/plugins/ lists installed plugin API endpoints.
   func (c *Client) CheckNetBoxInventoryPlugin(ctx context.Context) (bool, error) {
       // The netbox-inventory plugin registers under /api/plugins/inventory/
       // We can check by hitting that endpoint and seeing if we get a 200 vs 404.
       resp, err := c.api.GetConfig().HTTPClient.Get(c.url[:len(c.url)-4] + "/api/plugins/")
       if err != nil {
           return false, err
       }
       defer resp.Body.Close()
       return resp.StatusCode == 200, nil
   }
   ```

3. Create `scripts/provision-netbox.go` — standalone CLI script:
   ```go
   //go:build ignore

   // Run with: go run scripts/provision-netbox.go
   // Provisions NetBox with all HWLab custom fields and location hierarchy.
   // Reads HWLAB_NETBOX_URL and HWLAB_NETBOX_TOKEN from environment (.env auto-loaded).

   package main

   import (
       "context"
       "log"
       "os"

       "github.com/joho/godotenv"

       "git.georgsen.dk/hwlab/internal/netbox"
   )

   func main() {
       // Load .env
       if err := godotenv.Load(); err != nil {
           log.Printf("no .env file: %v", err)
       }

       url := os.Getenv("HWLAB_NETBOX_URL")
       token := os.Getenv("HWLAB_NETBOX_TOKEN")

       if url == "" || token == "" {
           log.Fatal("HWLAB_NETBOX_URL and HWLAB_NETBOX_TOKEN must be set")
       }

       client, err := netbox.NewClient(url, token)
       if err != nil {
           log.Fatalf("netbox client: %v", err)
       }

       ctx := context.Background()

       log.Println("Provisioning NetBox...")
       if err := client.Provision(ctx); err != nil {
           log.Fatalf("provision: %v", err)
       }

       // Check netbox-inventory plugin (NB-03)
       ok, err := client.CheckNetBoxInventoryPlugin(ctx)
       if err != nil {
           log.Printf("plugin check error: %v", err)
       } else if ok {
           log.Println("netbox-inventory plugin: INSTALLED")
       } else {
           log.Println("WARNING: netbox-inventory plugin may not be installed")
           log.Println("  Manual check: SSH to LXC 130, run: pip show netbox-inventory")
           log.Println("  Install if missing: pip install netbox-inventory")
       }

       log.Println("Provisioning complete.")
   }
   ```

cd /home/mikkel/homelabby && go build ./internal/netbox/... && go vet ./internal/netbox/...

<acceptance_criteria> - go build ./internal/netbox/... exits 0 (all stubs replaced with real go-netbox v4 API calls) - go vet ./internal/netbox/... exits 0 (no vet errors) - grep "ProvisionLocationHierarchy" internal/netbox/provision.go returns the exported function - grep "ensureSite\|ensureLocation\|ensureRack" internal/netbox/provision.go returns 3 matches (not stub errors) - grep "CheckNetBoxInventoryPlugin" internal/netbox/provision.go returns the function - File scripts/provision-netbox.go exists - grep "go:build ignore" scripts/provision-netbox.go returns a match (build tag prevents accidental inclusion) - If real token available: go run scripts/provision-netbox.go exits 0 and logs "Provisioning complete." - If real token available: curl -H "Authorization: Token REAL_TOKEN" http://10.5.0.130:8000/api/extras/custom-fields/?name=hw_id returns count > 0 </acceptance_criteria>

Location hierarchy provisioning implemented. Provision CLI script created. All functions use real go-netbox v4 API (no stubs). go build and go vet clean.

<threat_model>

Trust Boundaries

Boundary	Description
scripts/provision-netbox.go → NetBox API	Write access to NetBox via authenticated REST API
Provisioning logic → NetBox admin state	Creates objects in NetBox; idempotency prevents data corruption

STRIDE Threat Register

Threat ID	Category	Component	Disposition	Mitigation Plan
T-03-01	Tampering	ProvisionCustomFields idempotency	mitigate	Check-before-create pattern ensures no duplicates; NetBox also enforces unique names on custom fields
T-03-02	Information Disclosure	scripts/provision-netbox.go logs	accept	Logs go to stdout only; no sensitive data logged beyond field names
T-03-03	Denial of Service	Provisioning script run in loop	accept	Script has //go:build ignore tag, requires explicit go run; single operator context
T-03-04	Elevation of Privilege	Provisioning requires NetBox admin rights	accept	NetBox token grants admin-level access by design; this is the operator's own homelab
</threat_model>

After both tasks complete: - `go test ./internal/netbox/... -v` all tests pass (unit) or skip (integration without real token) - `go build ./...` exits 0 - `go vet ./...` exits 0 - If real token: `go run scripts/provision-netbox.go` provisions cleanly - If real token: `curl -s -H "Authorization: Token $HWLAB_NETBOX_TOKEN" "http://10.5.0.130:8000/api/extras/custom-fields/" | python3 -c "import sys,json; d=json.load(sys.stdin); print(d['count'])"` shows 8 or more custom fields

<success_criteria>

1. All 8 custom field specs defined in hwlabCustomFields slice with correct names, types, object_types
2. ProvisionCustomFields is idempotent (check-before-create)
3. ProvisionLocationHierarchy creates Site "Homelab" → Location "Lab Bench" → Rack "Primary Rack"
4. Standalone script runs with go run scripts/provision-netbox.go against live NetBox
5. go build and go vet clean on all packages </success_criteria>

After completion, create `.planning/phases/01-foundation/01-03-SUMMARY.md` with: - Whether provisioning ran against live NetBox (yes/no) - If yes: which custom fields were created vs. already existed - If yes: whether netbox-inventory plugin was detected - Any go-netbox v4 API quirks encountered (WritableSiteRequest fields, slug handling, etc.) - Files created/modified