18 KiB
18 KiB
| phase | plan | type | wave | depends_on | files_modified | autonomous | requirements | must_haves | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 40-job-infrastructure | 01 | execute | 1 |
|
true |
|
|
Purpose: Phase 40 is the hard dependency for phases 41-45. This plan establishes the data model and async execution engine so Plan 02 can wire HTTP routes on top. Output: Schema file, two migrations, updated constants, contentJobStore service, contentJobRunner service.
<execution_context> @$HOME/.claude/get-shit-done/workflows/execute-plan.md @$HOME/.claude/get-shit-done/templates/summary.md </execution_context>
@.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/phases/40-job-infrastructure/40-RESEARCH.mdFrom packages/db/src/schema/assets.ts:
export const assets = pgTable("assets", {
id: uuid("id").primaryKey().defaultRandom(),
companyId: uuid("company_id").notNull().references(() => companies.id),
provider: text("provider").notNull(),
objectKey: text("object_key").notNull(),
contentType: text("content_type").notNull(),
byteSize: integer("byte_size").notNull(),
sha256: text("sha256").notNull(),
originalFilename: text("original_filename"),
createdByAgentId: uuid("created_by_agent_id").references(() => agents.id),
createdByUserId: text("created_by_user_id"),
createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
updatedAt: timestamp("updated_at", { withTimezone: true }).notNull().defaultNow(),
}, (table) => ({ ... }));
From packages/shared/src/constants.ts:
export const LIVE_EVENT_TYPES = [
"heartbeat.run.queued",
"heartbeat.run.status",
"heartbeat.run.event",
"heartbeat.run.log",
"agent.status",
"activity.logged",
"plugin.ui.updated",
"plugin.worker.crashed",
"plugin.worker.restarted",
] as const;
export type LiveEventType = (typeof LIVE_EVENT_TYPES)[number];
From server/src/attachment-types.ts:
export const MAX_ATTACHMENT_BYTES =
Number(process.env.PAPERCLIP_ATTACHMENT_MAX_BYTES) || 10 * 1024 * 1024;
From server/src/services/live-events.ts:
export function publishLiveEvent(input: {
companyId: string;
type: LiveEventType;
payload?: Record<string, unknown>;
}): LiveEvent;
export function subscribeCompanyLiveEvents(companyId: string, listener: (event: LiveEvent) => void): () => void;
From server/src/services/assets.ts:
export function assetService(db: Db) {
return {
create: (companyId: string, data: Omit<typeof assets.$inferInsert, "companyId">) => ...,
getById: (id: string) => ...,
};
}
From server/src/services/index.ts — barrel export pattern:
export { assetService } from "./assets.js";
export { publishLiveEvent, subscribeCompanyLiveEvents } from "./live-events.js";
From packages/db/src/schema/index.ts — barrel export pattern:
export { assets } from "./assets.js";
2. Modify `packages/db/src/schema/assets.ts`:
- Add column after `createdByUserId`: `sourceTaskId: text("source_task_id"),` (nullable, no FK, no default)
- No other changes
3. Modify `packages/db/src/schema/index.ts`:
- Add line: `export { contentJobs, CONTENT_JOB_STATUSES } from "./content_jobs.js";`
- Place it after the `assets` export line
4. Modify `packages/shared/src/constants.ts`:
- Add four entries to LIVE_EVENT_TYPES array before the `] as const;` closing:
`"content_job.queued",`
`"content_job.running",`
`"content_job.done",`
`"content_job.failed",`
5. Modify `server/src/attachment-types.ts`:
- Add after the existing `MAX_ATTACHMENT_BYTES` export:
```
export const MAX_GENERATED_ASSET_BYTES =
Number(process.env.PAPERCLIP_GENERATED_ASSET_MAX_BYTES) || 500 * 1024 * 1024;
```
6. Generate migrations:
- Run `cd /opt/nexus && pnpm db:generate`
- Verify two new migration files appear (0056 for content_jobs, 0057 for assets source_task_id — or combined)
- Run `pnpm db:migrate` to apply
Anti-patterns to avoid:
- Do NOT add a FK constraint for sourceTaskId — task IDs are string identifiers, not UUID FKs
- Do NOT add resultAssetId as a FK to assets — it's populated after asset creation, circular FK causes issues
export function contentJobStore(db: Db) {
return {
create: (companyId: string, data: { jobType: string; input: Record<string, unknown>; sourceTaskId: string | null }) =>
db.insert(contentJobs).values({ companyId, ...data }).returning().then((r) => r[0]),
getById: (id: string) =>
db.select().from(contentJobs).where(eq(contentJobs.id, id)).then((r) => r[0] ?? null),
listByCompany: (companyId: string) =>
db.select().from(contentJobs)
.where(eq(contentJobs.companyId, companyId))
.orderBy(desc(contentJobs.createdAt)),
transition: (id: string, patch: Partial<typeof contentJobs.$inferInsert>) =>
db.update(contentJobs).set({ ...patch, updatedAt: new Date() }).where(eq(contentJobs.id, id)),
};
}
```
2. Create `server/src/services/content-job-runner.ts`:
- Import publishLiveEvent from "./live-events.js"
- Import contentJobStore from "./content-job-store.js"
- Import assetService from "./assets.js"
- Import MAX_GENERATED_ASSET_BYTES from "../attachment-types.js"
- Import type { Db } from "@paperclipai/db"
- Import type { StorageService } from "../storage/types.js"
- Import type { contentJobs } from "@paperclipai/db"
Define type `ContentJob = typeof contentJobs.$inferSelect`
Export `renderContent` stub function:
```typescript
export async function renderContent(
_jobType: string,
_input: Record<string, unknown>,
): Promise<{ filename: string; contentType: string; buffer: Buffer }> {
// Stub — phases 41-45 will add real renderers keyed by jobType
return {
filename: "placeholder.txt",
contentType: "text/plain",
buffer: Buffer.from("placeholder output"),
};
}
```
Export `contentJobRunner` object with a single `dispatch` method:
```typescript
export const contentJobRunner = {
dispatch(db: Db, storage: StorageService, job: ContentJob): void {
void runJob(db, storage, job);
},
};
```
Implement `async function runJob(db, storage, job)`:
- Create store via `contentJobStore(db)`
- Transition to "running" with startedAt: new Date(), then publishLiveEvent type "content_job.running" payload { jobId: job.id }
- Try block:
- Call renderContent(job.jobType, job.input as Record<string, unknown>)
- Check result.buffer.byteLength <= MAX_GENERATED_ASSET_BYTES, throw if exceeded
- Call storage.putFile({ companyId: job.companyId, namespace: "generated", originalFilename: result.filename, contentType: result.contentType, body: result.buffer })
- Call assetService(db).create(job.companyId, { ...stored, sourceTaskId: job.sourceTaskId, createdByAgentId: null, createdByUserId: null })
- Transition to "done" with resultAssetId: asset.id, finishedAt: new Date()
- publishLiveEvent type "content_job.done" payload { jobId: job.id, assetId: asset.id }
- Catch block:
- Extract errorMessage from err
- Transition to "failed" with errorMessage, finishedAt: new Date()
- publishLiveEvent type "content_job.failed" payload { jobId: job.id, errorMessage }
3. Modify `server/src/services/index.ts`:
- Add: `export { contentJobStore } from "./content-job-store.js";`
- Add: `export { contentJobRunner } from "./content-job-runner.js";`
Anti-patterns to avoid:
- Do NOT await dispatch in the caller — it must be fire-and-forget (`void`)
- Do NOT publish live event before DB write completes — always `await store.transition(...)` first
- Do NOT import multer anywhere in runner code — runner writes directly via storage.putFile
<success_criteria>
- content_jobs schema has all lifecycle columns with correct types
- assets.sourceTaskId column exists (nullable text)
- LIVE_EVENT_TYPES includes 4 content_job.* entries
- MAX_GENERATED_ASSET_BYTES = 500MB default
- contentJobStore CRUD works against DB
- contentJobRunner dispatches async, transitions lifecycle, stores asset, publishes events
- TypeScript compiles clean across all affected packages </success_criteria>