The Forge model is supposed to make cartridges thin: declare your app, inherit substrate (KMS, network, guardrails, schedules, obs, SDK), and ship business logic. We have two cartridges that prove the model for browser-fronted vibe apps (chatbot, obs). We do not have a cartridge that proves the model for event-driven background work. Adding one closes that gap.
We could close the gap by absorbing all of Ammar's paycargo-ai work into cartridges/engineering/code-agents/ directly. The trade is large: 9 Terraform modules to rewrite against the substrate, 11 Docker agents to reroute through AgentCore Gateway with our IAM, ECR + CodeBuild + deploy pipelines to migrate. Multi-day effort with real risk of regression on something Ammar already has running.
The smaller, safer move: pick the simplest agent (golden-spec, which reads an issue and writes a spec), rebuild it on the Forge in roughly one focused day, and demo the full pipeline. If it works, we know the doctrine is real for event-driven cartridges. If we then decide to absorb the rest of paycargo-ai, we have a working template. If we decide to keep paycargo-ai standalone, we lost a day but proved the model.
paycargo-ai uses Bedrock AgentCore Runtime (Docker containers, AgentCore Memory, AgentCore Gateway). This cartridge uses plain Bedrock InvokeModel from Lambda in a two-turn pattern (Section 4). They are different primitives with different tradeoffs.
The two-turn pattern (planner picks the relevant files, Lambda fetches them, writer produces the output with those files in context) is the right answer for read-once-then-write agents. The model does not iteratively discover files during generation; it picks the set up front. golden-spec, code-review, qa, and ai-intake all fit this shape. The output for golden-spec is not a generic paragraph; it is a structured spec whose "Files to touch" section names specific paths and per-file change recommendations grounded in the actual code the writer turn saw.
AgentCore is the right answer for read-modify-iterate agents: ones that must discover new files mid-generation, run tools (e.g. tests) and read the output, or revisit earlier decisions after seeing more context. implement (writes code patches across many files, may need to retract and rewrite) and react-migration (iterates over component files) both fit AgentCore. When we land the first of those, we add a substrate/modules/platform/agentcore module and the relevant cartridges opt in. The two-turn-on-Lambda cartridges keep their pattern; both cohabit.
The clean line: golden-spec OUTPUTS a path-to-code plan as a structured spec a human reviews. implement EXECUTES a path-to-code plan by writing actual file edits. Different jobs, different primitives.
Every story you label = one row in the cost-by-user table. Every spec generation has a dollar amount you can show to anyone. Quarterly reviews start writing themselves.
sdlc shows up as a fourth row in the obs Cartridges tab alongside chatbot, obs-app, and substrate. MTD spend, invocation count, budget, status, retention, and access all populate. Real-time cost attribution by token-log query, no waiting for Cost Explorer activation.
Diff: two lines in cartridges/administrative/obs/lambda/cartridges.js. Add 'sdlc' to APP_OWNERS. Add 'sdlc': '/aws/lambda/ai-sandbox-sdlc-golden-spec' to COST_LOG_GROUPS. The merge logic already handles cartridges with no Bedrock activity by showing $0 until the first invocation.
The operator clicks the sdlc row in the Cartridges modal and sees, in addition to the standard Budget / Access / Retention / Notes sections, a new Trigger config section showing one row per agent. Each row is editable; model and prompt travel together because the right prompt depends on the model (Opus reasons over more context, Haiku rewards tighter instructions).
| Field | Control | Notes |
|---|---|---|
| Label | text input | The GitHub issue label that fires this agent. Day one: golden-spec. |
| Model | select dropdown | Choices populated from the pricing map: Claude Opus 4.6 / Claude Sonnet 4.5 / Claude Haiku 4.5. Default for golden-spec: Opus 4.6. |
| Prompt template | textarea | Multi-line markdown. Substitutions like {{issue.title}}, {{issue.body}}, {{comments}}, {{repo}} are resolved at invocation time. Canonical default stored at cartridges/engineering/code-agents/lambda/prompts/<agent>.md; operator edits override per-environment, persisted in DDB. |
| Max output tokens | number | Per-agent cap. Default 4,000. |
| Status | toggle | Enabled / paused. Paused agents log a warning when their label fires but skip the Bedrock call. |
Plus, on the same section:
PayCargoDevOps/ai-sandbox only. Expandable to multiple PayCargoDevOps repos.
State lives in DDB under agent_session_id="sdlc-agent-config", one item per agent. The Lambda reads at cold start and caches for 15 minutes (same pattern the schedules manifest uses, Step.6.k). Operator edits land within 15 minutes without redeploying anything.
The cartridge is a first-class member of the platform.
cartridges/engineering/code-agents/. Owned by the engineering vibe-scripter cohort per Step 6.d. Not under global/ because these are engineering's productivity tools (finance won't use golden-spec; HR won't use code-review), even though they read PayCargo-wide repos.cartridges/engineering/code-agents/cartridge.tf declares app_id, owner, data classification, retention, supported labels, log group ARN, trigger endpoint, source repo URL.module "sdlc_cartridge" {} block in substrate/environments/ai-sandbox/main.tf alongside chatbot_cartridge and observability_app.App = "sdlc"./cartridges/engineering/code-agents/ @engineering-vibe-scripters (today fronted by @mmbtang until the team exists as a GitHub team).global/.The handler is a two-turn pipe: a planner Bedrock call decides which repo files matter for this specific issue, the Lambda fetches those files via GitHub REST, then a writer Bedrock call generates the spec with that context loaded. Repo-aware generation without an AgentCore tool-use loop. Bedrock itself never holds the GitHub PAT; both calls receive only assembled text.
The cost optimization is real. A 200-entry file tree is roughly 4 KB of tokens; reading 10 actual files is often 40-80 KB. Doing both reads in one turn would burn the input-token budget on files the model wasn't going to reference. Letting the model pick first, with only the index in front of it, keeps the writer turn focused and the total bill close to single-turn cost (~2x, not 10x).
The PAT lives in Secrets Manager, fetched by the Lambda on cold start and cached in module scope. Bedrock receives only assembled text. There is no path for the model to call GitHub directly (that's the AgentCore tool-use pattern, not this one). CloudTrail shows SecretsManager:GetSecretValue per cold start; CloudWatch shows the exact GitHub API calls the Lambda made between turns. Full audit trail.
cartridges/engineering/code-agents/
├── README.md — purpose, supported labels, how to add agents
├── cartridge.tf — platform manifest (app_id, owner, retention)
├── auth.tf — IAM role + permission boundary
├── backend.tf — Lambda + API GW + Secrets Manager refs
├── outputs.tf
├── variables.tf
├── versions.tf
└── lambda/
├── handler.js — main webhook handler (~250 LOC zero-npm)
├── github-client.js — minimal GitHub REST wrapper (~80 LOC)
├── verify-signature.js — HMAC-SHA256 webhook verify (~30 LOC)
└── prompts/
└── golden-spec.md — copied verbatim from paycargo-ai/agents/golden-spec/
Plus three small edits outside the cartridge:
substrate/environments/ai-sandbox/main.tf: new module "sdlc_cartridge" block.cartridges/administrative/obs/lambda/cartridges.js: APP_OWNERS, COST_LOG_GROUPS, plus Opus 4.6 row in BEDROCK_PRICING_USD_PER_M_TOKENS..github/CODEOWNERS: route the new folder to the owning team.
Opus 4.6 is the strongest Claude model in the family. The trade is cost: roughly 5x Sonnet 4.5 per token, 15x Haiku 4.5. golden-spec writes architecture-level specs, so the extra reasoning is worth it; if quality is good enough on Sonnet 4.5 for some agents (e.g. code-review on small diffs), we run those cheaper. The per-cartridge model is configurable from the obs modal, so the decision can land per-agent.
| Component | Quantity | Cost |
|---|---|---|
| Turn 1 (planner) input tokens | ~6,000 @ $15.00 / M (issue + tree) | $0.090 |
| Turn 1 (planner) output tokens | ~300 @ $75.00 / M (JSON file list) | $0.023 |
| Turn 2 (writer) input tokens | ~25,000 @ $15.00 / M (issue + 5-10 file contents) | $0.375 |
| Turn 2 (writer) output tokens | ~4,000 @ $75.00 / M (the spec) | $0.300 |
| Lambda (8-25s wall clock for both turns) | ~1 GB-s avg | ~$0.001 |
| API Gateway request | 1 | ~$0.000001 |
| GitHub API calls (~12: 1 tree, 1 comments, 5-10 file gets, 1-2 writes) | ~12 requests | $0 (within free PAT limit) |
| S3 / DDB / CloudWatch incidental | tiny | ~$0.001 |
| One spec generation (two-turn) | ~$0.79 |
The planner turn is cheap (small input + tiny JSON output). The writer turn carries most of the spend because it pulls in actual repo file contents. Per-token breakdown is logged twice (once per turn) so the obs Cartridges tab can show the operator exactly where the bill landed; if planner cost ever creeps up, the dial is "show the model fewer tree entries up front."
| Resource | Monthly |
|---|---|
| Lambda idle | $0 |
| API Gateway HTTP API idle | $0 |
| Secrets Manager (GitHub PAT + webhook secret) | $0.80 |
| CloudWatch log group (90d, CMK) | ~$0.10 |
| IAM, KMS (reuses substrate keys) | $0 |
| Fixed baseline | ~$1 |
The 2-turn-with-Opus row above is the baseline. The substrate already lets us pick from five strategies via the model router (Step 6.m). The operator changes route names in the obs Trigger config section; no code change.
| Strategy | Scout backend | Writer backend | Per spec | At 50 specs/day |
|---|---|---|---|---|
| A. Naive single-vendor (Opus everywhere) | Opus 4.6 | Opus 4.6 | $0.87 | $1,305/mo |
| B. Cloud-only tiered (Haiku scout, Opus writer) | Haiku 4.5 | Opus 4.6 | $0.87 | $1,305/mo |
| C. Cloud-only with aggressive selectivity | Haiku 4.5 × multi-turn | Opus 4.6 (narrow input) | $0.48 | $720/mo |
| D. Mac Mini scout + Opus writer (Phase 1.5) | Local (free) | Opus 4.6 (narrow input) | $0.48 | $720/mo |
| E. Mac Mini scout + Sonnet writer (Phase 1.5 default) | Local (free) | Sonnet 4.5 | $0.10 | $150/mo |
Strategy E is roughly 88% cheaper than naive single-vendor. The point is not which row to pick; the point is that the substrate lets the operator pick D for executive-facing artifacts and E for routine work on the same agent, by editing a config. The economics flip from "cost of running agents" to "cost of running specifically the writer turn." Once the writer goes local in Phase 2 (Step 6.m trajectory), even that goes to electricity.
| Tier | Specs/day | Variable | Plus fixed | Monthly total |
|---|---|---|---|---|
| Demo / showing it works | 1-3 | ~$3-$10 | $1 | $4-$11 |
| Real pilot (small eng team) | 5-10 | ~$15-$30 | $1 | $16-$31 |
| Full team (50 devs) | 50-150 | ~$150-$450 | $1 | $151-$451 |
Opus 4.6 is expensive enough that ungated team-wide adoption could surprise. The substrate gives us two dials. First, the cartridge budget + warning recipients in the obs modal (set $100 with 80% threshold; SES alerts the cartridge owner on breach). Second, the cartridge access roles: restrict to admins initially so only operators can label issues, expand after we know real usage. Both already exist as substrate primitives, no new code required.
| From | What the cartridge gets free |
|---|---|
| Bootstrap (Step 1) | OIDC for GitHub Actions deploy, permission boundary on every IAM role. |
| Identity (Step 2) | Data-governance CMKs. sdlc picks eng-low-risk for the Lambda env vars + log group, no new keys. |
| Network (Step 3) | Egress allowlist (Salesforce / Monday / GitHub / GitBook). The sdlc Lambda's only external call is GitHub, which is already on the allowlist. |
| Observability (Step 4) | CloudTrail data events, GuardDuty, CloudWatch log group encryption per tier. |
| obs vibe app (Step 4.5) | Shared Cognito pool (no new pool). Cartridges tab automatically surfaces sdlc as a row once APP_OWNERS and COST_LOG_GROUPS are updated. |
| Guardrails (Step 6.f area) | Platform default guardrail applied on every InvokeModel. Same KMS Decrypt fix from Step.6.j doctrine update #4 applies (caller IAM grants kms:Decrypt on the guardrail CMK). |
| Console (Step 6.g/h) | Currently nothing visible; sdlc is webhook-driven, no UI. When the Trigger config UI lands, the obs Cartridges modal hosts it (no new SDK modules needed). |
| API Gateway + shared secret (Step 6.j doctrine #1) | Not used here; webhook traffic comes from GitHub IPs directly to API Gateway, secured by HMAC signature. Different threat model than browser-to-Lambda, different auth. |
| Schedules (Step 6.k) | Not used at first. If we add a "weekly digest" or "stale-spec sweep" later, it slots into the platform/schedules module. |
sdlc adds roughly 250 LOC of Lambda code plus ~150 LOC of Terraform. Everything else (KMS, network, guardrails, log encryption, cost attribution, budget alerts, access control config, retention declaration) comes from substrate primitives that already exist. Same lesson as chatbot: the cartridge is thin because the platform is thick.
The chatbot and obs cartridges use X-Origin-Secret in a CloudFront origin custom_header as the API Gateway auth (Step 6.j doctrine #1). That pattern works because CloudFront is the only legitimate caller; anyone hitting API Gateway directly is unauthorized.
Webhook cartridges are different. The legitimate caller is GitHub, hitting the API Gateway URL directly from GitHub's outbound IPs. There is no CloudFront in front. The right auth is the X-Hub-Signature-256 HMAC header that GitHub computes against a shared secret over the raw request body. The Lambda verifies; mismatched signatures get 401.
Pin as cartridge doctrine: pick the auth pattern by traffic shape. Browser-fronted via CloudFront → X-Origin-Secret. Webhook-fronted direct from a third party → provider-specific HMAC. Don't try to mix them.
Single-turn generation is sufficient when the trigger payload contains everything the model needs (chatbot is this: user message in, model answer out). It is insufficient the moment the model needs context that lives in a repo (code, conventions, prior art). For golden-spec on the ai-sandbox repo, a single-turn call sees only the issue body and produces generic AI advice. The actual codebase is invisible.
The temptation here is to reach for AgentCore tool-use, which lets the model iteratively call back into a github_read_file tool Lambda mid-generation. That works, but it brings the AgentCore Runtime stack (Docker images, Memory, Gateway, per-second compute) for what is fundamentally a read-once decision: "given this issue, which files do I need?"
Pin as cartridge doctrine: when the trigger payload alone is insufficient, use the two-turn planner-fetcher-writer pattern, not AgentCore. The Lambda makes two Bedrock calls:
The pattern's properties:
*-complete log line per turn (turn:"planner" / turn:"writer"). The obs Cartridges tab attributes them separately so the operator can see which turn is driving cost.
When the two-turn pattern is wrong: when the model needs to iteratively read, modify, and re-read based on what it discovers (paycargo-ai's implement agent works this way). That's the AgentCore case. The two-turn pattern is for read-once-then-write agents (golden-spec, code-review, qa, ai-intake). The AgentCore pattern is for read-modify-iterate agents (implement, react-migration). Both deserve to live on the Forge; they just live on different substrate modules.
AgentCore and InvokeModel are both Bedrock. They are not the same primitive. AgentCore (Runtime + Memory + Gateway) is the right answer when the agent needs persistent memory across turns, long-running tool orchestration, or 5+ minute tasks. InvokeModel from Lambda is the right answer for single-turn generation under 15 minutes.
paycargo-ai's implement agent reads many files, writes many files, and runs for minutes. It belongs on AgentCore. The golden-spec agent reads one issue and writes one spec; it belongs on InvokeModel. Both deserve to live as cartridges. The substrate's job is to wire each pattern once (KMS, IAM, log group, cost log line) and let each cartridge pick what fits.
When we add the first AgentCore-based cartridge, the new substrate addition is one terraform/modules/platform/agentcore module that handles Runtime, Memory, Gateway plumbing. The Lambda+InvokeModel cartridges keep their existing pattern. Both cohabit cleanly.
The traditional separation (engineers ship code with hardcoded model IDs; operators tune knobs) does not survive contact with reality on LLM-based agents. A prompt written for Opus reasons differently than a prompt written for Haiku, even at the same task. An operator who downgrades an agent from Opus to Haiku without retuning the prompt gets noticeably worse output. An operator who edits the prompt without considering the model picks vocabulary the model does not respond well to.
Pin as substrate doctrine: every agent stored in DDB is a (model, prompt, max_output_tokens) tuple. The obs Trigger config UI surfaces all three together in one row. Edits to any field are saved atomically. The agent's default tuple ships in cartridges/engineering/code-agents/lambda/prompts/<agent>.md as a frontmatter+body markdown file:
--- model: us.anthropic.claude-opus-4-6-<datestamp>-v1:0 max_output_tokens: 4000 --- You are golden-spec, an architect-level assistant. Given a GitHub issue, write an implementation specification. Sections: Context, Approach, Files to touch, Tests, Risks. ...
On apply, the cartridge backfills the DDB row only if one does not exist (it never overwrites operator edits). The Lambda reads the active tuple at invocation time.
Sonnet 4.5 ($3 in / $15 out per million tokens) and Haiku 4.5 ($1 / $5) make per-invocation cost a rounding error. Opus 4.6 (~$15 / $75) does not. One golden-spec run is roughly half a dollar. A team labeling 50 issues a day burns $675 a month from one cartridge.
The substrate gives us three independent dials and they should all be configured at cartridge launch:
admins only; expand after observed usage. For sdlc the gate is "who can apply the label," enforced by GitHub repo permissions; the obs row records the GitHub login for cost attribution but does not gate.
Pin as cartridge doctrine: when a cartridge ships with Opus as its default model, the cartridge's cartridge.tf manifest must set initial_budget_usd and initial_access_roles = ["admins"] so the obs modal starts gated. The model dial is operator-discoverable in the Trigger config section; no manifest gate needed because the operator can freely move down the price ladder.
Cartridges must not call Bedrock directly. They call the model router (Step 6.m) with a logical route name: scout-local, scout-cloud, writer-opus, writer-sonnet. The router decides which physical backend serves that role (Bedrock, Mac Mini via Tailscale, future on-prem cluster, future partner endpoint). The decision lives in an SSM-published JSON manifest the operator edits in the obs UI.
Why the router is the right separation: every Bedrock-invoking cartridge today (chatbot, obs narrative, code-agents) has a hard-coded vendor model ID in its TF or its prompt manifest. When a cheaper or better model lands, every cartridge has to change. That is not the right place for the decision. The cartridge should declare the role it needs ("scout for navigation," "writer for output") and the substrate should pick the backend. Same separation we already enforce for KMS keys (cartridges declare data class, substrate picks key), for guardrails (cartridges declare risk tier, substrate picks guardrail), and for network egress (cartridges declare egress need, substrate picks allowlist entry).
Why this is a moat, not just an optimization: Bedrock pricing is rising as Anthropic invests in better models. Open-source quality is rising while local inference cost falls toward electricity. The router captures every step of the gap widening. Competitors who wrote new BedrockRuntimeClient() directly in every Lambda inherit Anthropic's pricing power as a permanent dependency; migrating their stack later is a multi-quarter refactor. For us it is a JSON edit. Asymmetric switching cost compounds without us doing anything beyond the initial substrate work. Full strategic rationale lives in Memo — PayCargo's AI infrastructure moat in the Memos rail.
Pin as substrate doctrine: no cartridge calls a vendor model ID directly. All Bedrock-invoking Lambdas route through substrate/modules/platform/model-router/. Every route declares a fallback chain. Operators tune the route table; engineers ship the abstraction.
Phase 1 ships golden-spec on cloud-only routes (Haiku scout, Opus or Sonnet writer). Phase 1.5 swaps the scout turns to a Mac Mini running Qwen 2.5 Coder via the model router substrate (Step 6.m). The cartridge code does not change. The agent's per-turn config flips one field: route: "scout-cloud" → route: "scout-local".
Network setup checklist (the only non-trivial part):
10.42.0.10:11434. Nothing else on the local network is reachable.substrate/modules/platform/local-inference-mesh/. ~30 lines of TF.secrets/local-inference-token) that the Mac Mini's reverse proxy enforces on every request. Belt and suspenders on top of the Tailscale ACL.scout-local registered in the SSM manifest (Step 6.m, Section 2).What changes for golden-spec specifically: per-spec cost drops from $0.48 (Strategy C in the table above) to ~$0.48 (Strategy D, no change) because the writer turn still dominates. The interesting savings come from the Sonnet writer variant (Strategy E): with free scouts the writer can be Sonnet 4.5 without losing the quality benefits of multi-pass narrowing. Per-spec cost drops to ~$0.10. At 50 specs/day, that is $150/mo instead of $1,305/mo for naive single-vendor.
The fallback chain means Phase 1.5 does not raise availability risk. When the Mac Mini is unreachable, scout-local falls back to scout-cloud at $0.005-$0.03 per turn. The agent still runs. The obs Cartridges tab shows the fallback rate so the operator knows when the local backend is sick.
agents/shared/knowledge/*.md (platform.md, payments.md, integrations.md, cpp.md) is rich context that improves agent output. The first version of this cartridge uses a slim version (just the system prompt + issue). When we want better quality, we copy the knowledge files into cartridges/engineering/code-agents/lambda/prompts/knowledge/ and prepend them to the system message.