nexus/.planning/phases/33-persistent-memory/33-01-PLAN.md

11 KiB

phase plan type wave depends_on files_modified autonomous requirements must_haves
33-persistent-memory 01 execute 1
server/src/services/assistant-memory.ts
server/src/services/memory-sanitizer.ts
server/src/routes/assistant-memory.ts
server/src/app.ts
server/src/__tests__/33-assistant-memory.test.ts
server/src/__tests__/33-memory-sanitization.test.ts
true
ASST-01
ASST-02
truths artifacts key_links
Memory facts persist to disk and survive server restart
API key patterns (sk-*, ghp_*, AIza*) pasted into chat are scrubbed to [REDACTED] before storage
JWT-shaped values are scrubbed before storage
Key=value patterns with sensitive keys (api_key, token, secret, password, bearer) are scrubbed
Memory is scoped per companyId (separate files)
Memory capped at 50 facts (FIFO eviction)
path provides exports
server/src/services/assistant-memory.ts File-backed memory service with get/append/clear
assistantMemoryService
path provides exports
server/src/services/memory-sanitizer.ts Credential scrubbing for plain-text memory facts
sanitizeMemoryFact
path provides exports
server/src/routes/assistant-memory.ts GET/PATCH/DELETE memory endpoints
assistantMemoryRoutes
path provides
server/src/__tests__/33-assistant-memory.test.ts Unit tests for memory service CRUD
path provides
server/src/__tests__/33-memory-sanitization.test.ts Unit tests for credential scrubbing
from to via pattern
server/src/services/assistant-memory.ts data/assistant-memory/<companyId>.json fs.readFileSync/writeFileSync readFileSync.*assistant-memory
from to via pattern
server/src/services/assistant-memory.ts server/src/services/memory-sanitizer.ts sanitizeMemoryFact import sanitizeMemoryFact
from to via pattern
server/src/routes/assistant-memory.ts server/src/services/assistant-memory.ts service import assistantMemoryService
Create the file-backed assistant memory service with write-time credential sanitization and REST endpoints.

Purpose: ASST-01 requires persistent memory across sessions; ASST-02 requires sanitization at write time so credentials never reach disk. Output: Memory service, sanitizer, routes, and comprehensive unit tests.

<execution_context> @$HOME/.claude/get-shit-done/workflows/execute-plan.md @$HOME/.claude/get-shit-done/templates/summary.md </execution_context>

@.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/phases/33-persistent-memory/33-RESEARCH.md @server/src/services/nexus-settings.ts @server/src/redaction.ts @server/src/home-paths.ts

From server/src/services/nexus-settings.ts:

import fs from "node:fs";
import path from "node:path";
import { z } from "zod";
import { resolvePaperclipInstanceRoot } from "../home-paths.js";

// Pattern: file-backed JSON service
// resolveXxxPath() -> fs.readFileSync -> zod safeParse -> fallback default
// set() -> mkdirSync recursive -> writeFileSync
export function nexusSettingsService() {
  async function get(): Promise<NexusSettings> { ... }
  async function set(patch: Partial<NexusSettings>): Promise<NexusSettings> { ... }
  return { get, set };
}

From server/src/redaction.ts:

const SECRET_PAYLOAD_KEY_RE =
  /(api[-_]?key|access[-_]?token|auth(?:_?token)?|authorization|bearer|secret|passwd|password|credential|jwt|private[-_]?key|cookie|connectionstring)/i;
const JWT_VALUE_RE = /^[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+(?:\.[A-Za-z0-9_-]+)?$/;

From server/src/routes/nexus-settings.ts (route mounting pattern):

// Routes are mounted in server/src/app.ts via:
// app.use("/api", someRoutes(db));
Task 1: Create memory sanitizer and assistant memory service server/src/services/memory-sanitizer.ts, server/src/services/assistant-memory.ts, server/src/__tests__/33-memory-sanitization.test.ts, server/src/__tests__/33-assistant-memory.test.ts - sanitizeMemoryFact("My API key is sk-abc123def456ghijklmnopqrst") returns "My API key is [REDACTED]" - sanitizeMemoryFact("ghp_aBcDeFgHiJkLmNoPqRsTuVwXyZaBcDeFgHiJk") returns "[REDACTED]" - sanitizeMemoryFact("AIzaSyA1234567890abcdefghijklmnopqrstuv") returns "[REDACTED]" - sanitizeMemoryFact("token=abc123longvalue") returns "[REDACTED]" - sanitizeMemoryFact("api_key: sk-something") returns "[REDACTED]" - sanitizeMemoryFact("password = hunter2") returns "[REDACTED]" - sanitizeMemoryFact("eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0.dozjgNryP4J3jVmNHl0w5N_XgL0n3I9PlFUP0THsR8U") returns "[REDACTED]" - sanitizeMemoryFact("I prefer TypeScript over JavaScript") returns unchanged - sanitizeMemoryFact("Use port 3000 for the server") returns unchanged - assistantMemoryService.get(companyId) returns { facts: [], updatedAt: null } when no file exists - assistantMemoryService.append(companyId, "I prefer TypeScript") writes fact to file and returns updated memory - assistantMemoryService.append() with credential text stores sanitized version - assistantMemoryService.clear(companyId) removes all facts - After appending 51 facts, only 50 remain (FIFO eviction of oldest) - Two different companyIds get separate memory files Create `server/src/services/memory-sanitizer.ts`: - Export `sanitizeMemoryFact(raw: string): string` - Define `CREDENTIAL_INLINE_RE` matching: `sk-` (OpenAI), `ghp_` (GitHub PAT), `AIza` (Google API), and generic long alphanumeric tokens with dots (JWTs) - Pattern: `/\b(sk-[A-Za-z0-9]{20,}|ghp_[A-Za-z0-9]{36}|AIza[0-9A-Za-z_-]{35}|[A-Za-z0-9_-]{20,}\.[A-Za-z0-9_-]{6,}\.[A-Za-z0-9_-]{20,})/g` - Define `SENSITIVE_KEY_VALUE_RE` matching key=value or key: value patterns where key is api_key, token, secret, password, bearer, auth, etc. - Pattern: `/(?:api[_-]?key|token|secret|password|bearer|auth)\s*[:=]\s*\S+/gi` - Apply both regexes, replacing matches with `[REDACTED]`
Create `server/src/services/assistant-memory.ts`:
- Export `assistantMemoryService()` returning `{ get, append, clear }`
- Schema: `z.object({ facts: z.array(z.string()).default([]), updatedAt: z.string().nullable().default(null) })`
- `resolveMemoryPath(companyId)`: `path.resolve(resolvePaperclipInstanceRoot(), "data", "assistant-memory", \`${companyId}.json\`)`
- `get(companyId)`: read file, safeParse, return default `{ facts: [], updatedAt: null }` on any error
- `append(companyId, rawFact)`: call `sanitizeMemoryFact(rawFact)`, skip if result is empty or only `[REDACTED]`, read current, push to facts array, cap at 50 (shift oldest), set updatedAt to ISO string, mkdirSync recursive, writeFileSync
- `clear(companyId)`: write `{ facts: [], updatedAt: null }` to file (or delete file)

Create test files using vitest. Use `os.tmpdir()` + random dir for test isolation (mock `resolvePaperclipInstanceRoot` to return temp dir). Test all behaviors listed above.
server/src/services/nexus-settings.ts, server/src/redaction.ts, server/src/home-paths.ts pnpm --filter @paperclipai/server vitest run --reporter=verbose src/__tests__/33-memory-sanitization.test.ts src/__tests__/33-assistant-memory.test.ts - grep -q "sanitizeMemoryFact" server/src/services/memory-sanitizer.ts - grep -q "CREDENTIAL_INLINE_RE" server/src/services/memory-sanitizer.ts - grep -q "SENSITIVE_KEY_VALUE_RE" server/src/services/memory-sanitizer.ts - grep -q "assistantMemoryService" server/src/services/assistant-memory.ts - grep -q "resolveMemoryPath" server/src/services/assistant-memory.ts - grep -q "sanitizeMemoryFact" server/src/services/assistant-memory.ts - grep -q "50" server/src/services/assistant-memory.ts - grep -q "REDACTED" server/src/__tests__/33-memory-sanitization.test.ts - grep -q "sk-" server/src/__tests__/33-memory-sanitization.test.ts - grep -q "ghp_" server/src/__tests__/33-memory-sanitization.test.ts Memory service reads/writes per-company JSON files. Sanitizer scrubs API keys, JWTs, and key=value credential patterns. 50-fact FIFO cap enforced. All tests pass. Task 2: Create memory REST routes and wire to app server/src/routes/assistant-memory.ts, server/src/app.ts Create `server/src/routes/assistant-memory.ts`: - Export `assistantMemoryRoutes(): Router` - `GET /api/assistant-memory/:companyId` — calls `assertBoard(req)`, `assertCompanyAccess(req, companyId)`, returns `assistantMemoryService().get(companyId)` - `PATCH /api/assistant-memory/:companyId` — accepts `{ fact: string }` body, calls `assistantMemoryService().append(companyId, fact)`, returns updated memory - `DELETE /api/assistant-memory/:companyId` — calls `assistantMemoryService().clear(companyId)`, returns 204 - Import `assertBoard`, `assertCompanyAccess` from `./authz.js` - Import `assistantMemoryService` from `../services/assistant-memory.js`
Wire in `server/src/app.ts`:
- Import `assistantMemoryRoutes` from `./routes/assistant-memory.js`
- Mount: `app.use("/api", assistantMemoryRoutes())`
- Place after existing route mounts (follow existing pattern order)
server/src/app.ts, server/src/routes/nexus-settings.ts, server/src/routes/authz.ts pnpm --filter @paperclipai/server tsc --noEmit - grep -q "assistantMemoryRoutes" server/src/routes/assistant-memory.ts - grep -q "assertBoard" server/src/routes/assistant-memory.ts - grep -q "assertCompanyAccess" server/src/routes/assistant-memory.ts - grep -q "assistantMemoryRoutes" server/src/app.ts - grep -q 'GET.*assistant-memory\|get.*assistant-memory' server/src/routes/assistant-memory.ts - grep -q 'PATCH\|patch\|post' server/src/routes/assistant-memory.ts - grep -q 'DELETE\|delete' server/src/routes/assistant-memory.ts Memory CRUD routes mounted at /api/assistant-memory/:companyId. GET returns facts, PATCH appends a sanitized fact, DELETE clears memory. Auth enforced. TypeScript compiles clean. pnpm --filter @paperclipai/server vitest run --reporter=verbose src/__tests__/33-*.test.ts pnpm --filter @paperclipai/server tsc --noEmit

<success_criteria>

  • assistantMemoryService reads/writes per-company JSON at data/assistant-memory/.json
  • sanitizeMemoryFact scrubs sk-, ghp_, AIza*, JWTs, and key=value credential patterns
  • Memory capped at 50 facts with FIFO eviction
  • REST endpoints mounted and auth-gated
  • All unit tests pass </success_criteria>
After completion, create `.planning/phases/33-persistent-memory/33-01-SUMMARY.md`