docs(35): research npx buildthis CLI phase domain

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Nexus Dev 2026-04-03 22:49:06 +00:00
parent f489f5f4d1
commit e0e60be6b6

View file

@ -0,0 +1,453 @@
# Phase 35: npx buildthis CLI - Research
**Researched:** 2026-04-01
**Domain:** Node.js CLI packaging, npx entrypoints, hardware detection, interactive terminal UX
**Confidence:** HIGH
---
<user_constraints>
## User Constraints (from CONTEXT.md)
### Locked Decisions
None — all implementation choices are at Claude's discretion.
### Claude's Discretion
All implementation choices are at Claude's discretion.
### Deferred Ideas (OUT OF SCOPE)
None.
</user_constraints>
<phase_requirements>
## Phase Requirements
| ID | Description | Research Support |
|----|-------------|------------------|
| CLI-01 | User can run `npx buildthis` to bootstrap Nexus from scratch | `buildthis` package with `bin.buildthis` pointing to CLI entry; detects running instance vs fresh install path |
| CLI-02 | CLI bootstrapper detects hardware and walks through same provider tiering as web onboarding | Hardware detection via inline `os` module (no `systeminformation` needed for Apple Silicon; GPU probe optional) + provider tiering display mirrored from web onboarding |
</phase_requirements>
---
## Summary
Phase 35 introduces a standalone `npx buildthis` entrypoint — a developer-facing bootstrapper that a user can run on a fresh machine without having cloned the repo. When invoked it follows two distinct paths: if Nexus is already running on the default port (3100) it opens the browser immediately; if not, it walks the user through hardware-aware provider selection (matching the web onboarding Phase 30-32 logic) and then calls the existing `paperclipai onboard --run` flow.
The `buildthis` npm package is a new, minimal package under `packages/buildthis/` (or as a second `bin` entry on the existing `paperclipai` package — see Architecture Patterns). It bundles its own tiny entry with esbuild, following the same pattern as `cli/`. Hardware detection can be done entirely with the Node.js built-in `os` module for Apple Silicon (no `systeminformation` needed at the CLI layer) and a 3-second `si.graphics()` probe for GPU — or, if Nexus is running, by calling `GET /system/providers` directly. The second approach is simpler and avoids adding `systeminformation` to the `buildthis` package.
The hardest design question is where `buildthis` lives: as a new `packages/buildthis/` workspace, or as a second `bin` on the existing `cli/package.json`. The new package approach is cleaner for users (`npx buildthis` vs `npx paperclipai buildthis`) and avoids leaking all of `paperclipai`'s heavy dependencies into a lightweight bootstrapper download.
**Primary recommendation:** Create `packages/buildthis/` as a new, minimal workspace package with `bin: { buildthis: "./dist/index.js" }`, a single `src/index.ts` entry, and a thin esbuild build. Hardware detection uses inline `os` + optional `systeminformation` (already in monorepo) for the local probe path.
---
## Standard Stack
### Core
| Library | Version | Purpose | Why Standard |
|---------|---------|---------|--------------|
| `@clack/prompts` | `^0.10.0` | Interactive terminal prompts (spinners, select, text, confirm) | Already used across all `cli/` commands; consistent UX |
| `commander` | `^13.1.0` | CLI argument parsing | Already used in `cli/`; mature, zero-dep |
| `picocolors` | `^1.1.1` | Terminal colouring | Already used in `cli/`; smallest colour lib |
| `open` | `^11.0.0` | Open URLs in the default browser | Already used in `@paperclipai/server` for `PAPERCLIP_OPEN_ON_LISTEN`; same version |
| `systeminformation` | `5` | GPU detection (VRAM probe) | Already used in `@paperclipai/server`; pinned to v5 per project decision |
### Supporting
| Library | Version | Purpose | When to Use |
|---------|---------|---------|-------------|
| `esbuild` | workspace devDep | Bundle to single `dist/index.js` | Build step, same pattern as `cli/esbuild.config.mjs` |
| `tsx` | `^4.19.2` | Dev mode entrypoint (`pnpm dev`) | Dev only, same pattern as `cli/` |
### Alternatives Considered
| Instead of | Could Use | Tradeoff |
|------------|-----------|----------|
| New `packages/buildthis/` package | Second `bin` on `cli/package.json` | `cli/` is 1.8 MB unpacked with many deps; `buildthis` should be a lightweight download — separate package wins |
| Inline `os`-only hardware detection | Call running server's `/system/providers` | If server is not yet running, we need local detection; use local probe first, server probe as fallback |
| `systeminformation` for GPU | Skip GPU detection in `buildthis` | GPU detection is part of CLI-02; include it but behind the same 3-second `Promise.race` timeout pattern |
**Installation (new package):**
```bash
pnpm add @clack/prompts commander picocolors open systeminformation --filter buildthis
```
**Version verification (confirmed 2026-04-01):**
- `@clack/prompts`: 1.2.0 (latest)
- `commander`: 14.0.3 (latest; `^13.1.0` in cli — update to `^14.0.0` or pin to `^13.1.0` for consistency)
- `picocolors`: 1.1.1 (latest)
- `open`: 11.0.0 (latest)
- `systeminformation`: 5.31.5 (latest v5)
---
## Architecture Patterns
### Recommended Project Structure
```
packages/buildthis/
├── package.json # name: "buildthis", bin: { buildthis: "./dist/index.js" }
├── tsconfig.json # extends ../../tsconfig.base.json
├── esbuild.config.mjs # mirror of cli/esbuild.config.mjs, single entry
├── src/
│ ├── index.ts # CLI entry: program setup + default command = bootstrapCommand
│ ├── bootstrap.ts # main logic: detect-running → open, or guide-install → open
│ ├── hardware.ts # inline hardware detection (copied from server/src/services/hardware.ts)
│ └── banner.ts # printBuildthisBanner (mirrors cli/src/utils/banner.ts)
└── dist/ # gitignored, built artifact
```
### Pattern 1: npx entrypoint — `bin` field
**What:** npm `bin` field maps a command name to a file. When `npx buildthis` is run, npm downloads the package and executes `./dist/index.js` directly.
**When to use:** Any time you want `npx <name>` to work without global install.
**Example:**
```json
// packages/buildthis/package.json
{
"name": "buildthis",
"bin": { "buildthis": "./dist/index.js" },
"files": ["dist"],
"publishConfig": { "access": "public" }
}
```
The bundled `dist/index.js` **must** have a `#!/usr/bin/env node` shebang — the esbuild `banner.js` option injects this automatically (see `cli/esbuild.config.mjs`).
### Pattern 2: Detect running instance before prompting
**What:** Probe `http://127.0.0.1:<port>/api/health` first. Port is read from config if config exists, else try the default 3100.
**When to use:** `buildthis`'s primary happy path — "already running, just open browser."
**Example:**
```typescript
// Source: onboard.ts bootstrapNexusAgents() health-check pattern
async function probeRunningInstance(port: number): Promise<boolean> {
try {
const res = await fetch(`http://127.0.0.1:${port}/api/health`, { signal: AbortSignal.timeout(2000) });
return res.ok;
} catch {
return false;
}
}
```
Health URL: `http://127.0.0.1:<port>/api/health` — confirmed from `server/src/app.ts` (mounted at `/api/health`).
### Pattern 3: Hardware detection — inline os + systeminformation
**What:** Same logic as `server/src/services/hardware.ts`. Apple Silicon path uses only `os` module (no `systeminformation` call). GPU path uses `si.graphics()` with a 3-second `Promise.race` timeout.
**When to use:** CLI-02 requirement — detect tier before presenting provider options.
**Example:**
```typescript
// Source: server/src/services/hardware.ts (copy-adapted for buildthis)
import os from "node:os";
import si from "systeminformation";
export type HardwareTier = "gpu" | "apple_silicon" | "cpu_only";
export async function detectHardware(): Promise<{ tier: HardwareTier; totalGb: number }> {
const totalGb = Math.round(os.totalmem() / (1024 ** 3) * 10) / 10;
const cpuModel = os.cpus()[0]?.model ?? null;
if (process.platform === "darwin" && cpuModel?.startsWith("Apple")) {
return { tier: "apple_silicon", totalGb };
}
try {
const result = await Promise.race([
si.graphics(),
new Promise<never>((_, reject) => setTimeout(() => reject(new Error("timeout")), 3000)),
]);
const vramGb = (result.controllers[0]?.vram ?? 0) / 1024;
if (vramGb >= 4) return { tier: "gpu", totalGb };
} catch { /* fallthrough to cpu_only */ }
return { tier: "cpu_only", totalGb };
}
```
### Pattern 4: Provider tiering display (CLI-02)
**What:** After hardware detection, present the same tier-appropriate options as the web wizard:
- GPU / Apple Silicon: recommend local Ollama + show top model from catalog
- Any tier: offer Puter (zero-config), Google OAuth, API key
- CPU-only: up-front note that cloud AI is recommended
**When to use:** Second path (no running instance, fresh install).
**CLI display pattern (clack/prompts):**
```typescript
import * as p from "@clack/prompts";
p.log.info(`Hardware: ${tier} | RAM: ${totalGb} GB`);
const provider = await p.select({
message: "Choose a starting provider",
options: [
{ value: "puter", label: "Puter -- free, zero-config", hint: "No API key needed" },
{ value: "google", label: "Google -- Gemini free tier", hint: "Sign in with Google" },
{ value: "apikey", label: "API key -- subscription provider", hint: "OpenAI, Anthropic, Groq" },
...(tier !== "cpu_only" ? [{ value: "local", label: "Local AI (Ollama)", hint: "Private, offline" }] : []),
{ value: "skip", label: "Skip for now" },
],
});
```
### Pattern 5: Delegating to `paperclipai onboard --run`
**What:** After provider guidance, `buildthis` hands off to the existing `paperclipai onboard` or `paperclipai run` flow. This avoids duplicating the full install logic.
**Constraint:** `buildthis` cannot `import` from `@paperclipai/*` workspace packages — it is a standalone public package. It must invoke `paperclipai` as a subprocess via `child_process.spawn` (if already installed) or guide the user to install it first.
**Key decision:** `buildthis` is a *bootstrapper*, not a full CLI. Its job ends when it either opens the browser or tells the user the next command to run. It does NOT embed the full install logic.
### Pattern 6: Browser open
**What:** Use `open` package to launch the browser.
**Example:**
```typescript
import open from "open";
await open(`http://127.0.0.1:${port}`);
```
Server already uses `open` via dynamic import; `buildthis` can use it as a static dependency.
### Anti-Patterns to Avoid
- **Bundling `@paperclipai/server` into `buildthis`:** Server is 1.8 MB+ and has native deps (embedded postgres). `buildthis` must stay small — it's what `npx` downloads on a fresh machine.
- **Skipping hardware detection on first path:** Even when Nexus is already running, CLI-02 says hardware detection must be part of the flow. If server is running, call `GET /system/providers` instead of local probe — avoids adding `systeminformation` at all.
- **Using `PAPERCLIP_OPEN_ON_LISTEN` env pattern:** That pattern is for the server process. `buildthis` opens the browser itself after confirming the server is listening.
- **Blocking forever on GPU probe:** Must use the 3-second `Promise.race` timeout, same as `server/src/services/hardware.ts`.
---
## Don't Hand-Roll
| Problem | Don't Build | Use Instead | Why |
|---------|-------------|-------------|-----|
| Terminal prompts | Custom readline | `@clack/prompts` | Already used project-wide; handles TTY edge cases, cancellation, piped input |
| Terminal colour | ANSI escape codes | `picocolors` | Tree-shakeable, no deps, already project standard |
| Browser open | Platform-specific shell commands | `open@11` | Handles macOS/Linux/Windows correctly; already in server |
| GPU VRAM detection | Custom WMI/system calls | `systeminformation@5` | Handles cross-platform GPU probing; project already uses v5 |
| CLI arg parsing | Manual `process.argv` | `commander` | Already project standard; handles flags, help, errors |
**Key insight:** `buildthis` can be very thin — 200-300 lines of TypeScript. Every heavy problem is already solved by existing project libraries.
---
## Common Pitfalls
### Pitfall 1: `buildthis` name already taken on npm
**What goes wrong:** `npm publish` fails with E403 because `buildthis` is registered.
**Why it happens:** `buildthis` (404 on npm as of 2026-04-01 — confirmed) is not registered, but this can change.
**How to avoid:** `npm view buildthis` returned 404 — name is available. Reserve it early by publishing the package.
**Warning signs:** `npm error 403` during publish.
### Pitfall 2: `#!/usr/bin/env node` shebang missing from dist
**What goes wrong:** `npx buildthis` runs but produces "Permission denied" or "SyntaxError: Unexpected token #".
**Why it happens:** esbuild doesn't add the shebang automatically unless configured.
**How to avoid:** In `esbuild.config.mjs`, set `banner: { js: "#!/usr/bin/env node" }` — same as `cli/esbuild.config.mjs`. Also run `chmod +x dist/index.js` in build script.
**Warning signs:** `permission denied` from npx; file doesn't start with `#!/usr/bin/env node`.
### Pitfall 3: `AbortSignal.timeout` not available on old Node.js
**What goes wrong:** `AbortSignal.timeout(2000)` throws `TypeError: AbortSignal.timeout is not a function` on Node 16.
**Why it happens:** `AbortSignal.timeout` was added in Node 17.3.
**How to avoid:** Target Node 20 in esbuild config (`target: "node20"`). Project already uses `node20` in `cli/esbuild.config.mjs`. Add `"engines": { "node": ">=20" }` to `package.json`.
**Warning signs:** Errors on Node 16/18.
### Pitfall 4: `systeminformation` native bindings in esbuild bundle
**What goes wrong:** `systeminformation` includes platform-specific binaries that esbuild can't bundle; build succeeds but crashes at runtime.
**Why it happens:** Some `systeminformation` functions use optional native bindings.
**How to avoid:** Mark `systeminformation` as `external` in esbuild config (same as how `cli/esbuild.config.mjs` handles all npm deps). It will be a regular `node_modules` dependency, not bundled.
**Warning signs:** `Cannot find module` errors for `.node` files at runtime.
### Pitfall 5: Health URL path confusion
**What goes wrong:** Probing the wrong URL — `http://127.0.0.1:3100/health` instead of `/api/health`.
**Why it happens:** The server mounts health at `/api/health` (inside the `api` router at `app.ts:237`) not `/health`.
**How to avoid:** Use `http://127.0.0.1:${port}/api/health` — confirmed from `server/src/app.ts` line 237 + 138.
**Warning signs:** 404 on health probe even when server is running.
### Pitfall 6: Non-TTY environments (CI, piped input)
**What goes wrong:** `@clack/prompts` hangs or crashes when stdin is not a TTY.
**Why it happens:** Interactive prompts require a TTY.
**How to avoid:** Check `process.stdin.isTTY && process.stdout.isTTY` before entering interactive mode (same guard used in `onboard.ts:395`). In non-TTY, print instructions and exit 0.
**Warning signs:** `buildthis` hanging in CI.
---
## Code Examples
### Detect running instance
```typescript
// Source: adapted from cli/src/commands/onboard.ts bootstrapNexusAgents()
async function probeRunningInstance(port: number): Promise<boolean> {
try {
const res = await fetch(`http://127.0.0.1:${port}/api/health`, {
signal: AbortSignal.timeout(2000),
});
return res.ok;
} catch {
return false;
}
}
```
### Read port from existing config (if present)
```typescript
// Source: pattern from cli/src/config/store.ts + cli/src/config/schema.ts
import { configExists, readConfig } from "./config/store.js";
function resolveNexusPort(): number {
const DEFAULT_PORT = 3100;
try {
if (configExists()) {
const config = readConfig();
return config?.server?.port ?? DEFAULT_PORT;
}
} catch { /* config parse error — use default */ }
return DEFAULT_PORT;
}
```
Note: `buildthis` cannot import from workspace packages directly. It must either replicate the config-reading logic inline or accept that it only checks port 3100. The simplest approach: probe 3100 only. Config path resolution is an advanced feature for a v2.
### Hardware detection (server GET /system/providers fallback)
```typescript
// If server is running, use its hardware endpoint instead of local probe
async function fetchHardwareFromServer(port: number): Promise<HardwareInfo | null> {
try {
const res = await fetch(`http://127.0.0.1:${port}/system/providers`, {
signal: AbortSignal.timeout(3000),
});
if (res.ok) return res.json() as Promise<HardwareInfo>;
} catch { /* server not running */ }
return null;
}
```
### esbuild config for buildthis
```javascript
// Source: modelled on cli/esbuild.config.mjs
export default {
entryPoints: ["src/index.ts"],
bundle: true,
platform: "node",
target: "node20",
format: "esm",
outfile: "dist/index.js",
banner: { js: "#!/usr/bin/env node" },
external: [
"systeminformation", // native bindings — must stay external
"open", // uses dynamic import internally
],
treeShaking: true,
sourcemap: true,
};
```
---
## Runtime State Inventory
> Skipped — this is a greenfield package creation phase. No rename, refactor, or migration involved.
---
## Environment Availability
| Dependency | Required By | Available | Version | Fallback |
|------------|------------|-----------|---------|----------|
| Node.js 20+ | Build + runtime | ✓ | Linux 6.17.4 / node present | — |
| pnpm | Workspace management | ✓ | workspace in use | — |
| npm registry | `npx buildthis` resolution | ✓ (confirmed `paperclipai` 2026.325.0 live) | — | publish step is manual |
| `systeminformation` v5 | GPU detection | ✓ (in server/node_modules) | 5.31.5 | Apple Silicon + cpu_only paths work without it |
| `open` v11 | Browser open | ✓ (in server/node_modules) | 11.0.0 | Print URL to console as fallback |
| `@clack/prompts` | Interactive prompts | ✓ (in cli/node_modules) | 1.2.0 | — |
**Missing dependencies with no fallback:** None.
**Missing dependencies with fallback:** None — all required packages already exist in the monorepo.
---
## Validation Architecture
### Test Framework
| Property | Value |
|----------|-------|
| Framework | vitest 2.x (workspace project) |
| Config file | `packages/buildthis/vitest.config.ts` (Wave 0 gap — doesn't exist yet) |
| Quick run command | `npx vitest run packages/buildthis/src/__tests__/` |
| Full suite command | `npx vitest run` |
### Phase Requirements → Test Map
| Req ID | Behavior | Test Type | Automated Command | File Exists? |
|--------|----------|-----------|-------------------|-------------|
| CLI-01 | `probeRunningInstance` returns true when server responds 200 | unit | `npx vitest run packages/buildthis/src/__tests__/bootstrap.test.ts` | ❌ Wave 0 |
| CLI-01 | `probeRunningInstance` returns false when port closed / timeout | unit | `npx vitest run packages/buildthis/src/__tests__/bootstrap.test.ts` | ❌ Wave 0 |
| CLI-02 | `detectHardware` returns `apple_silicon` on darwin + Apple CPU | unit | `npx vitest run packages/buildthis/src/__tests__/hardware.test.ts` | ❌ Wave 0 |
| CLI-02 | `detectHardware` returns `cpu_only` when GPU probe times out | unit | `npx vitest run packages/buildthis/src/__tests__/hardware.test.ts` | ❌ Wave 0 |
| CLI-02 | Provider options include local AI only for non-cpu_only tiers | unit | `npx vitest run packages/buildthis/src/__tests__/bootstrap.test.ts` | ❌ Wave 0 |
### Sampling Rate
- **Per task commit:** `npx vitest run packages/buildthis/src/__tests__/`
- **Per wave merge:** `npx vitest run`
- **Phase gate:** Full suite green before `/gsd:verify-work`
### Wave 0 Gaps
- [ ] `packages/buildthis/vitest.config.ts` — vitest project config
- [ ] `packages/buildthis/src/__tests__/bootstrap.test.ts` — covers CLI-01 (probe, open, non-TTY guard)
- [ ] `packages/buildthis/src/__tests__/hardware.test.ts` — covers CLI-02 (tier detection by platform/CPU/GPU)
---
## State of the Art
| Old Approach | Current Approach | When Changed | Impact |
|--------------|------------------|--------------|--------|
| Single binary with all logic | Thin bootstrapper + delegates to heavy CLI | Phase 35 | `buildthis` stays small (~200KB bundle) |
| Hardware detection only in server | Hardware detection available in CLI layer | Phase 35 | No server needed for pre-install guidance |
**Deprecated/outdated:**
- None relevant to this phase.
---
## Open Questions
1. **Should `buildthis` invoke `paperclipai` as a subprocess, or just print instructions?**
- What we know: `buildthis` cannot bundle `@paperclipai/*` workspace packages; subprocess invoke requires `paperclipai` to be installed globally or via docker.
- What's unclear: Is `paperclipai` expected to be installed alongside `buildthis`, or is `buildthis` truly a fresh-machine bootstrapper?
- Recommendation: Print clear next-step instructions (`npx paperclipai@latest onboard` or `npm install -g paperclipai && paperclipai run`) rather than spawning a subprocess. This is simpler and more reliable.
2. **Should the `buildthis` package version track the `paperclipai` version?**
- What we know: `paperclipai` uses date-based versioning (2026.MDD.P). `buildthis` is a different package.
- What's unclear: Whether both should be published together in the same release script.
- Recommendation: Start with independent versioning (0.1.0); add to the release script in a follow-up. The release script in `scripts/release.sh` only handles `cli/`.
3. **Where does `buildthis` live in the workspace — `packages/buildthis/` or `cli-bootstrap/`?**
- What we know: `pnpm-workspace.yaml` includes `packages/*`.
- What's unclear: Naming convention preference.
- Recommendation: `packages/buildthis/` — consistent with workspace layout; `packages/*` is already in workspace glob.
---
## Sources
### Primary (HIGH confidence)
- `cli/src/index.ts` — Commander registration, `onboard` and `run` command wiring
- `cli/src/commands/onboard.ts` — Full interactive onboard flow, health-check pattern, `bootstrapNexusAgents`
- `cli/src/commands/run.ts``runCommand` implementation, server start, `PAPERCLIP_OPEN_ON_LISTEN`
- `cli/esbuild.config.mjs` — esbuild bundle strategy, shebang pattern, external deps
- `cli/package.json` — Exact dependency versions, build/publish setup
- `server/src/services/hardware.ts` — Hardware detection logic (Apple Silicon path, GPU probe, 3-second timeout)
- `server/src/app.ts` — Health route mounted at `/api/health` (line 237 + 138 confirmation)
- `server/src/index.ts``PAPERCLIP_OPEN_ON_LISTEN` env var + `open` package usage
- `pnpm-workspace.yaml` — Workspace package globs
- `scripts/build-npm.sh` — CLI build pipeline steps
### Secondary (MEDIUM confidence)
- npm registry check: `buildthis` package name — confirmed 404 (available) as of 2026-04-01
- npm registry check: `paperclipai` — confirmed published at 2026.325.0
### Tertiary (LOW confidence)
- None.
---
## Metadata
**Confidence breakdown:**
- Standard stack: HIGH — all libraries already in use in project; versions confirmed via `npm view`
- Architecture: HIGH — based on direct codebase inspection; patterns copied from existing `cli/` implementation
- Pitfalls: HIGH — derived from existing code patterns and confirmed npm registry state
**Research date:** 2026-04-01
**Valid until:** 2026-05-01 (stable domain; npm package availability could change sooner)