nexus/.planning/phases/45-content-as-skills/45-RESEARCH.md

# Phase 45: Content as Skills - Research

**Researched:** 2026-04-04
**Domain:** Nexus skill registry — local SKILL.md authoring, source seeding, group membership, generalist agent preloading
**Confidence:** HIGH

## Summary

Phase 45 bridges the content generation work of Phases 41-44 with the skill registry system already in place. Every content type (diagram, icon, theme, wallpaper, social, convert, pdf-document, brand-kit, presentation) must become an installable Markdown skill, the built-in "Creative" group must contain all of them, and the Generalist agent must ship pre-loaded with that group.

The skill registry infrastructure is complete. The registry stores skills in a libSQL database under `~/.paperclip/instances/default/skills/registry.db`. Skills are fetched from remote GitHub sources (`anthropic-marketplace` and `github-tree` types) or from native agent runtimes (Hermes). There is currently no local-filesystem source type. Phase 45 must add one, seed it on server startup, and wire the Creative group to the new skills.

The generalist agent preloading mechanism already exists: `index.ts` seeds `pendingSkillGroups: ["Creative"]` in agent metadata at creation time, and a fire-and-forget startup reconciler assigns the group (using `assignGroup`) to every agent that has that metadata flag. Phase 45 needs the Creative group to have members before the reconciler runs — which means the skill SKILL.md files and their registry entries must be in place before any agent is created.

**Primary recommendation:** Add a `"local-nexus-content"` source type to the skill registry fetcher. On server startup, register nine content SKILL.md files from `server/src/skills/content/` into the registry DB and add all of them to the `builtin/creative` group.

---

<user_constraints>
## User Constraints (from CONTEXT.md)

### Locked Decisions
All implementation choices are at Claude's discretion — discuss phase was skipped per user setting.

### Claude's Discretion
All implementation choices are at Claude's discretion. Use ROADMAP phase goal, success criteria, and codebase conventions to guide decisions.

### Deferred Ideas (OUT OF SCOPE)
None.
</user_constraints>

---

<phase_requirements>
## Phase Requirements

| ID | Description | Research Support |
|----|-------------|------------------|
| SKILL-01 | Each content type is implemented as an installable Nexus skill | Nine SKILL.md files authored; registered in skill registry DB via local-nexus-content source type; installable through existing `svc.install()` path |
| SKILL-02 | Generalist agent is pre-loaded with a "Creative" skill group | `builtin/creative` group already exists in DB seed; group members populated on startup; `pendingSkillGroups` reconciler already assigns the group to Generalist agents |
| SKILL-03 | Users can add or remove content type skills through the Skill Aggregator | Content skills appear in the existing SkillBrowser UI once registered; install/uninstall routes already work for any registered skill |
</phase_requirements>

---

## Standard Stack

### Core

| Library | Version | Purpose | Why Standard |
|---------|---------|---------|--------------|
| libSQL / drizzle-orm | Already installed | Skill registry persistence | All skill data lives in this SQLite DB — no new dep needed |
| node:fs/promises | Node built-in | Read SKILL.md files from disk | Used by existing skill cache logic |
| vitest | Already installed | Unit tests | Project standard in `server/` |

### Supporting

No new packages required. The full content generation stack (jobType dispatch, renderers, SSE) is already in place from Phases 40-44.

**Installation:** None required.

---

## Architecture Patterns

### Recommended Project Structure

```
server/src/
├── skills/
│   └── content/
│       ├── diagram.SKILL.md
│       ├── icon-set.SKILL.md
│       ├── theme-palette.SKILL.md
│       ├── wallpaper.SKILL.md
│       ├── social-post.SKILL.md
│       ├── convert.SKILL.md
│       ├── pdf-document.SKILL.md
│       ├── brand-kit.SKILL.md
│       └── presentation.SKILL.md
├── services/
│   └── skill-registry-fetcher.ts   ← add local-nexus-content source type
│   └── skill-registry-db.ts        ← seed Creative group members on init
└── index.ts                         ← call seedContentSkills on startup
```

### Pattern 1: Local-Filesystem Source Type

**What:** A new `"local-nexus-content"` entry in `SkillSourceConfig` union and a `fetchLocalNexusContent()` function that reads SKILL.md files from `server/src/skills/content/` and upserts them into the registry DB.

**When to use:** Server startup — called once after `getSkillRegistryDb()` initialises.

**Example:**

```typescript
// In skill-registry-fetcher.ts — extend the union type
export type SkillSourceConfig =
  | { id: string; type: "anthropic-marketplace"; owner: string; repo: string; ref: string; label: string }
  | { id: string; type: "github-tree"; owner: string; repo: string; ref: string; label: string }
  | { id: string; type: "local-nexus-content"; dir: string; label: string };

// New handler called by fetchAllSources when type === "local-nexus-content"
async function fetchLocalNexusContent(
  source: Extract<SkillSourceConfig, { type: "local-nexus-content" }>,
  db: SkillRegistryDb,
): Promise<number> {
  const entries = await readdir(source.dir, { withFileTypes: true });
  let fetched = 0;
  for (const entry of entries) {
    if (!entry.name.endsWith(".SKILL.md")) continue;
    const slug = entry.name.replace(/\.SKILL\.md$/, "");
    const skillId = `${source.id}/${slug}`;
    const skillMdContent = await readFile(path.join(source.dir, entry.name), "utf-8");
    const { name, description } = parseSkillFrontmatter(skillMdContent);
    // Use a content hash as the version SHA (deterministic, no network needed)
    const sha = crypto.createHash("sha1").update(skillMdContent).digest("hex");
    await upsertSkill(db, { skillId, sourceId: source.id, name: name ?? slug, description, sourceUrl: "" });
    await cacheSkillVersion(db, { skillId, sha, skillMdContent, skillMdUrl: "" });
    await upsertCommunityRatingsStub(db, skillId, source.id);
    fetched++;
  }
  return fetched;
}
```

**BUILT_IN_SOURCES addition:**

```typescript
export const BUILT_IN_SOURCES: SkillSourceConfig[] = [
  // … existing remote sources …
  {
    id: "nexus-content",
    type: "local-nexus-content",
    dir: path.join(path.dirname(fileURLToPath(import.meta.url)), "..", "skills", "content"),
    label: "Nexus Content Tools",
  },
];
```

### Pattern 2: Creative Group Membership Seeding

**What:** After the local skills are registered, add them as members of the `builtin/creative` group. This should happen inside `getSkillRegistryDb()` (or a startup helper called right after) so the group has members before the `pendingSkillGroups` reconciler runs.

**When to use:** Server startup, idempotent (uses `INSERT OR IGNORE`).

```typescript
// In skill-registry-db.ts — extend seedBuiltinGroups or add seedCreativeGroupMembers
const NEXUS_CONTENT_SKILL_IDS = [
  "nexus-content/diagram",
  "nexus-content/icon-set",
  "nexus-content/theme-palette",
  "nexus-content/wallpaper",
  "nexus-content/social-post",
  "nexus-content/convert",
  "nexus-content/pdf-document",
  "nexus-content/brand-kit",
  "nexus-content/presentation",
] as const;

async function seedCreativeGroupMembers(client: LibSQLClient): Promise<void> {
  const now = Date.now();
  for (const skillId of NEXUS_CONTENT_SKILL_IDS) {
    await client.execute({
      sql: `INSERT OR IGNORE INTO skill_group_members (group_id, skill_id, added_at) VALUES (?, ?, ?)`,
      args: ["builtin/creative", skillId, now],
    });
  }
}
```

### Pattern 3: SKILL.md Authoring Convention

Each content skill SKILL.md must follow the same frontmatter convention as existing skills:

```markdown
---
name: diagram
description: >
  Generate diagrams from a natural language description. Produces Mermaid
  syntax rendered to SVG/PNG. Supports architecture, flowchart, ERD,
  sequence, and mind-map types. Use when a user asks to visualise a
  system, process, or data structure as a diagram.
---

# Diagram Generation

Generates diagrams from a text description via the Nexus content API.

## Usage

Submit a `POST /api/content-jobs` with:
- `jobType: "diagram"`
- `input.description` — natural language description of the diagram
- `input.type` (optional) — `"flowchart"`, `"architecture"`, `"erd"`, `"sequence"`, or `"mindmap"`

The job returns a 202 with a `jobId`. Poll `GET /api/content-jobs/:jobId` or subscribe
to SSE `content_job.done` events to retrieve the resulting asset URL.

## Output

Returns an SVG file (also available as PNG via the export button in the UI).
```

### Startup Wiring

In `server/src/index.ts`, the existing startup block already calls `getSkillRegistryDb()` as fire-and-forget. Extend that block to also call `fetchAllSources()` with `BUILT_IN_SOURCES` immediately after DB init so content skills are seeded on every cold start:

```typescript
// [nexus] Initialize skill registry and seed content skills
void (async () => {
  try {
    const { getSkillRegistryDb } = await import("./services/skill-registry-db.js");
    await getSkillRegistryDb();          // creates tables + builtin groups
    const { skillRegistryService } = await import("./services/skill-registry.js");
    await skillRegistryService().fetchAll();  // seeds local-nexus-content skills
    logger.info("skill registry database initialized and content skills seeded");
  } catch (err) {
    logger.error({ err }, "skill registry init failed");
  }
})();
```

### Anti-Patterns to Avoid

- **Seeding Creative group members before the skills table rows exist:** `skill_group_members.skill_id` has no FK constraint, so INSERT succeeds even if the skill row is absent — but `resolveEffectiveSkills` then returns orphaned IDs. Always seed skill rows first, group members second.
- **Calling `fetchAll()` before `getSkillRegistryDb()`:** DB tables must exist before the local fetcher tries to upsert.
- **Using mutable remote SHA for local file versioning:** Local files don't have a git SHA. Use a SHA-1 of the file content instead — deterministic and cheap.
- **Blocking HTTP on `fetchAll()` at startup:** Keep as fire-and-forget; the external github-tree fetches can be slow. Content skills (local path) are synchronous but the remote sources may timeout.

---

## Don't Hand-Roll

| Problem | Don't Build | Use Instead | Why |
|---------|-------------|-------------|-----|
| Skill install/uninstall | Custom file copy logic | `skillRegistryService().install()` / `uninstall()` | Already handles version tracking, cache dirs, agentSkills table |
| Group assignment | Direct INSERT into agentSkillGroups | `skillGroupService().assignGroup()` | Handles effective skill resolution, per-skill install, dedup |
| Skill Browser display | New UI component | Existing SkillBrowser page at `/skills` | Already renders all registered skills with install/remove actions |
| Group membership seeding | New service | `INSERT OR IGNORE INTO skill_group_members` on DB init | Simpler than adding a new route; idempotent |

---

## Common Pitfalls

### Pitfall 1: Creative group populated but skills not yet in registry
**What goes wrong:** `pendingSkillGroups` reconciler runs at startup and calls `assignGroup("builtin/creative", agentId, skillsDir)`. `resolveEffectiveSkills` returns the 9 skill IDs, but `svc.install()` fails with "Skill not found" because local skills haven't been fetched yet.
**Why it happens:** The skill-registry init and the `pendingSkillGroups` reconciler are both fire-and-forget; their order is non-deterministic.
**How to avoid:** Seed creative group members only after `fetchAll()` completes (not at DB init time). Alternatively, sequence the two startup blocks: skill init → fetchAll → then let the reconciler run.
**Warning signs:** `skipped` array non-empty in `assignGroup` result; Generalist agent has group assigned but no SKILL.md files on disk.

### Pitfall 2: Local source dir resolution broken in production
**What goes wrong:** `path.join(path.dirname(fileURLToPath(import.meta.url)), "..", "skills", "content")` resolves to `server/src/skills/content` in dev but points to a non-existent path in a compiled/packaged build.
**Why it happens:** `import.meta.url` refers to the compiled output location.
**How to avoid:** In `tsconfig.json`, include `server/src/skills/content/*.SKILL.md` in the assets copied to dist, OR read the source dir via `process.cwd()` with a known repo-relative path. Check that the resolved dir exists before iterating; log a clear error if absent.

### Pitfall 3: SKILL.md frontmatter parse returns undefined name
**What goes wrong:** `parseSkillFrontmatter` fails silently on a multi-line YAML description block (`description: >\n  line1\n  line2`). The `name` field parses correctly but `description` is truncated to `>` literal.
**Why it happens:** `parseSkillFrontmatter` uses a single-line regex (`^description:\s*(.+)$`) — it does not handle YAML block scalars.
**How to avoid:** For the `name` field (used for registry display), keep it on a single line. For `description` use a single-line string or expand the frontmatter parser to handle folded scalars. Alternatively, accept the `>` literal as description — it won't break anything.

### Pitfall 4: Agent SKILL.md dir wrong for Generalist agent
**What goes wrong:** `assignGroup` copies skills to `agentSkillsDir` but the Generalist agent's adapter is `claude_local` — the skills dir is resolved by `resolveAdapterSkillConfig("claude_local").skillDir`. If the agent's workspace does not exist yet (new onboard), `cp` fails.
**Why it happens:** The workspace dir is created lazily when the agent first runs.
**How to avoid:** In `assignGroup`, `mkdir(agentSkillsDir, { recursive: true })` before attempting the copy. Inspect the existing code path — if this `mkdir` is already there, no action needed.

### Pitfall 5: Duplicate registration on restarts
**What goes wrong:** Every server restart calls `fetchAll()`, which re-registers all 9 local skills. Without idempotency guards, this creates duplicate `skill_versions` and `skill_files` rows.
**Why it happens:** The content hash SHA is deterministic, so `versionExists()` returns true and `cacheSkillVersion` is skipped. But `upsertSkill` uses `onConflictDoUpdate` — the name/description are updated but no duplicate is created.
**How to avoid:** The existing `versionExists()` check handles this. Confirm the content-hash versionId format matches: `${skillId}@${sha}`.

---

## Code Examples

### Registering skills from disk (local-nexus-content handler)

```typescript
// Source: server/src/services/skill-registry-fetcher.ts (new function)
import { readdir, readFile } from "node:fs/promises";
import crypto from "node:crypto";

async function fetchLocalNexusContent(
  source: Extract<SkillSourceConfig, { type: "local-nexus-content" }>,
  db: SkillRegistryDb,
): Promise<number> {
  let entries: Awaited<ReturnType<typeof readdir>>;
  try {
    entries = await readdir(source.dir, { withFileTypes: true });
  } catch {
    // Directory doesn't exist (e.g. missing after a build step) — log and skip
    return 0;
  }
  let fetched = 0;
  for (const entry of entries) {
    if (!entry.isFile() || !entry.name.endsWith(".SKILL.md")) continue;
    const slug = entry.name.replace(/\.SKILL\.md$/, "");
    const skillId = `${source.id}/${slug}`;
    const content = await readFile(path.join(source.dir, entry.name), "utf-8");
    const sha = crypto.createHash("sha1").update(content).digest("hex");
    const { name, description } = parseSkillFrontmatter(content);
    await upsertSkill(db, {
      skillId, sourceId: source.id,
      name: name ?? slug, description,
      sourceUrl: "",
    });
    await cacheSkillVersion(db, { skillId, sha, skillMdContent: content, skillMdUrl: "" });
    await upsertCommunityRatingsStub(db, skillId, source.id);
    fetched++;
  }
  return fetched;
}
```

### Seeding Creative group membership after fetchAll

```typescript
// In skill-registry-db.ts — called after seedBuiltinGroups
async function seedCreativeGroupMembers(client: LibSQLClient): Promise<void> {
  const now = Date.now();
  const SKILLS = [
    "nexus-content/diagram", "nexus-content/icon-set", "nexus-content/theme-palette",
    "nexus-content/wallpaper", "nexus-content/social-post", "nexus-content/convert",
    "nexus-content/pdf-document", "nexus-content/brand-kit", "nexus-content/presentation",
  ];
  for (const skillId of SKILLS) {
    await client.execute({
      sql: `INSERT OR IGNORE INTO skill_group_members (group_id, skill_id, added_at) VALUES (?, ?, ?)`,
      args: ["builtin/creative", skillId, now],
    });
  }
}
```

### Ensuring Creative is assigned to a new Generalist agent

```typescript
// The existing path in index.ts (already present, no change needed):
await agentSvc.create(company.id, {
  name: "Generalist",
  role: "general",
  adapterType: "claude_local",
  adapterConfig: {},
  runtimeConfig: {},
  metadata: { pendingSkillGroups: ["Creative"], backfilled: true },
});
// The reconciler at server startup calls svc.assignGroup("builtin/creative", agentId, skillsDir)
// which installs all Creative group members to the agent's .claude/skills/ directory.
```

---

## State of the Art

| Old Approach | Current Approach | When Changed | Impact |
|--------------|------------------|--------------|--------|
| Skills only from remote GitHub repos | Skills can come from local filesystem too (new local-nexus-content type) | Phase 45 | Nexus-specific content skills don't need a public GitHub repo |
| Creative group is empty (seed only creates the group row) | Creative group has 9 content skill members after server init | Phase 45 | Generalist agents automatically get all content tools on first run |

---

## Open Questions

1. **Should `cacheSkillVersion` write files to a tmpdir for local skills?**
   - What we know: `cacheSkillVersion` writes `SKILL.md` to `~/.paperclip/instances/default/skills/cache/nexus-content/<slug>/<sha>/SKILL.md`, then `svc.install()` copies from that cache dir to the agent's skills dir.
   - What's unclear: For local skills, caching to disk before installing is redundant — the source file is already on disk.
   - Recommendation: Keep the cache write for consistency (install always reads from cache); the file is tiny. If performance matters, skip caching and have install copy the source file directly.

2. **Where exactly is the `server/src/skills/content/` dir resolved in compiled builds?**
   - What we know: The server compiles TypeScript but does not currently bundle assets — source files are referenced at runtime via `import.meta.url`.
   - What's unclear: Whether `.SKILL.md` files are included in the dist output.
   - Recommendation: Add `server/src/skills/content/` to the list of directories copied during build (check the server's build script), or use `process.env.SKILL_CONTENT_DIR` as an override for production.

3. **Should the Skill Aggregator UI show a "Creative" filter tab?**
   - What we know: `SkillBrowser.tsx` filters by adapter compatibility; the "Groups" tab shows group membership via `skillGroupsApi`. Content skills installed via the registry appear in the installed tab.
   - What's unclear: Whether the browse experience needs a dedicated "Creative" section or if the existing group-based filtering is sufficient.
   - Recommendation: No UI changes needed for SKILL-03. The existing SkillBrowser already supports install/remove for any registered skill. Adding a dedicated Creative tab is a future enhancement.

---

## Environment Availability

Step 2.6: SKIPPED (no new external dependencies — local filesystem only)

---

## Validation Architecture

### Test Framework

| Property | Value |
|----------|-------|
| Framework | Vitest (node environment) |
| Config file | `server/vitest.config.ts` |
| Quick run command | `cd /opt/nexus/server && npx vitest run --reporter=verbose src/__tests__/skill-registry-content-skills.test.ts` |
| Full suite command | `cd /opt/nexus/server && npx vitest run --reporter=verbose` |

### Phase Requirements → Test Map

| Req ID | Behavior | Test Type | Automated Command | File Exists? |
|--------|----------|-----------|-------------------|-------------|
| SKILL-01 | 9 SKILL.md files are seeded into registry DB via local-nexus-content source | unit | `npx vitest run src/__tests__/skill-registry-content-skills.test.ts -t "seeds content skills"` | ❌ Wave 0 |
| SKILL-01 | Each skill is installable to an agent skills dir | unit | `npx vitest run src/__tests__/skill-registry-content-skills.test.ts -t "installs content skill"` | ❌ Wave 0 |
| SKILL-02 | Creative group has 9 members after DB init | unit | `npx vitest run src/__tests__/skill-registry-content-skills.test.ts -t "Creative group members"` | ❌ Wave 0 |
| SKILL-03 | Content skills appear in registry list | unit | `npx vitest run src/__tests__/skill-registry-content-skills.test.ts -t "list includes content skills"` | ❌ Wave 0 |

### Sampling Rate

- **Per task commit:** `cd /opt/nexus/server && npx vitest run src/__tests__/skill-registry-content-skills.test.ts`
- **Per wave merge:** `cd /opt/nexus/server && npx vitest run`
- **Phase gate:** Full suite green before `/gsd:verify-work`

### Wave 0 Gaps

- [ ] `server/src/__tests__/skill-registry-content-skills.test.ts` — covers SKILL-01, SKILL-02, SKILL-03
- [ ] `server/src/skills/content/*.SKILL.md` — 9 skill files (authored, not test infra)

---

## Sources

### Primary (HIGH confidence)

- Direct source code inspection — `server/src/services/skill-registry-fetcher.ts`, `skill-registry-db.ts`, `skill-registry-groups.ts`, `skill-registry.ts`
- Direct source code inspection — `server/src/index.ts` lines 256-279 (ensureGeneralistAgents), 628-658 (pendingSkillGroups reconciler)
- Direct source code inspection — `server/src/adapters/registry.ts`, `server/src/routes/skill-registry.ts`, `server/src/routes/skill-registry-groups.ts`

### Secondary (MEDIUM confidence)

- Existing test patterns from `server/src/__tests__/skill-registry-install.test.ts` and `hermes-dual-source.test.ts` — confirms vitest + real temp dirs as test pattern

---

## Metadata

**Confidence breakdown:**
- Standard stack: HIGH — all libraries already installed; no new deps
- Architecture: HIGH — local source type follows identical pattern to existing github-tree handler
- Pitfalls: HIGH — race conditions and dir resolution issues verified against actual startup code

**Research date:** 2026-04-04
**Valid until:** 2026-05-04 (stable codebase)