PayCargo · AI-Sandbox · Process

6.cAsync / queue / batch backend pattern

SDK surface for vibe-scripter ergonomic of background work: SQS-backed queues, worker handlers, scheduled triggers, and Step Functions wrappers for long-running tasks. Specced before vibe app 2 (Sales monthly reports CSV ingest) demands it; built when it does.
Block: 6.c (Console doctrine extension) · First consumer: Sales monthly reports CSV ingest (Phase B) · State: Specced; built in Phase B
Specced

This doctrine doc captures the SDK surface for async work. Engineering team writes the actual code when vibe app 2 hits a need (CSV that takes 5+ minutes to process; long-running agent task; scheduled report generation).

1. Why async / queue / batch is its own SDK package

Lambda's 15-minute execution cap is the most common shape that breaks "Lambda + Function URL handles everything." When a vibe scripter writes code that processes a 100MB CSV, summarizes a long thread, or runs a multi-step agent — the work might not fit in a single Lambda invocation. Without a platform pattern, the vibe scripter writes their own SQS plumbing, their own DLQ handling, their own retry semantics. That's exactly the engineering-abstraction territory §19 specifies the engineering team to own.

The SDK provides three primitives that compose:

2. SDK surface — vibe-scripter ergonomic

Enqueue a job

// In any vibe app's route handler — vibe scripter writes:
import { queue } from 'vibecode-sdk';

await queue.enqueue('process-csv', {
  csvKey: 'uploads/2026-05-sales.csv',
  ownerId: ctx.user.id,
  reportId: report.id,
});
// Returns; the worker handles the rest.

Define a worker

// cartridges/sales/monthly-reports/workers/process-csv.js
import { worker } from 'vibecode-sdk';

export default worker('process-csv', async (payload, ctx) => {
  const { csvKey, ownerId, reportId } = payload;

  // Stream the CSV (vibecode/sdk/v1/data/documentStore handles this)
  const stream = await ctx.store.documents.stream(csvKey);

  // Process row-by-row, emit audit events
  for await (const row of stream) {
    await ctx.audit.emit({ op: 'csv.row.processed', payload: { reportId, rowIndex: row.idx } });
    // ... business logic
  }

  await ctx.store.documents.put(`reports/${reportId}.json`, summary);
});
// Worker returns; SDK acks the SQS message. Failure throws → SDK NACKs → retry.

Scheduled job

// cartridges/engineering/obs/scheduled/daily-narrative.js
import { scheduled } from 'vibecode-sdk';

export default scheduled('daily-narrative', {
  cron: 'cron(0 13 * * ? *)',  // 09:00 ET / 13:00 UTC
  handler: async (ctx) => {
    const narrative = await ctx.agent.invokeAgentSync({
      model: 'claude-haiku-4-5',
      messages: [{ role: 'user', content: 'Summarize yesterday...' }],
      attribution: { app: 'obs', purpose: 'daily_narrative', cell: 'eng-low-risk' },
    });
    await ctx.cache.put('narrative', narrative);
  },
});

Step Function workflow (for >15 min)

import { workflow } from 'vibecode-sdk';

export default workflow('large-csv-import', {
  steps: [
    { name: 'split-csv', handler: async (payload, ctx) => { ... return chunks; } },
    { name: 'process-each-chunk', map: true, handler: async (chunk, ctx) => { ... } },
    { name: 'merge-and-publish', handler: async (results, ctx) => { ... } },
  ],
});

3. Contracts that carry over

ContractApplies to async/queue/batch as
governance_cellSQS queue + DDB job-state table encrypted with the app's governance cell CMK; workflow state encrypted similarly
lockbox_only_piiWorker payloads scanned by the PII detector on receive; raw PII in worker payload is a contract violation
audit_emit_requiredEvery queue.enqueue() and every worker invocation emits a structured audit event with the same shape as HTTP route audit
cost_attributeSQS + Lambda + Step Function costs tagged with app + cell + purpose; visible in cross-app cost dashboards

4. Substrate primitives

Per-app substrate (provisioned by the cartridge's Terraform via the app-template):

Cross-app substrate (shared):

5. Failure modes — what each layer catches

FailureCaught byOperator sees
Worker code throws on first attemptSQS retry semantics2 more retries with backoff; audit event records "retried"
Worker code throws on all 3 retriesDLQMessage lands in DLQ; CloudWatch alarm fires; obs app's audit tab surfaces it
Worker exceeds Lambda 15minLambda timeoutAudit event with status: 'timeout'; message retries; if persistent, lands in DLQ
Worker writes PII to non-lockbox tablePII detector + DB boundary tag checkSDK throws; message NACKed; DLQ; alarm
Worker calls off-allowlist external hostSDK's SoR helpers (no helper for the host); fallback to NFWSDK refuses; if bypassed, NFW drops; audit shows attempted off-allowlist call
Scheduled job runs concurrently due to overlapping invocationsSDK's single-instance lock (DDB conditional write on a lock row)Second invocation no-ops with audit "skipped-locked"

6. When this gets built

Phase B, alongside vibe app 2 (Sales monthly reports). The CSV ingest is the first consumer; engineering team builds the SDK packages against this spec; vibe app 2 ships consuming them. By Phase C, vibe app 3's likely needs (scheduled reports, agent workflows) inherit the same SDK with zero additional engineering.

7. Doctrine

Step 6.c doctrine: "Background work is a substrate primitive, not an app concern."

Every vibe app that needs >15 minutes, scheduled execution, or multi-step orchestration consumes the same SDK helpers. The substrate provides the SQS queues, DLQ wiring, Step Functions state machines, and EventBridge rules. The vibe scripter writes the worker handler body. The split is consistent with §19: engineering owns the substrate plumbing; vibe scripters write business logic.