VERIFICATION FOR SOFTWARE CHANGES
Verification factory for software changes, now in pilot
FetchSandbox is piloting a verification factory for teams shipping software changes. Business rules come from your incidents and requirements. Checks run in a controlled sandbox. Evidence comes from your application.
Bring a risky change. We will scope what verification looks like for your service.
Start with a change your team is worried about. We’ll agree on what to test.
Failure surfaces in support
A synthetic Jira ticket flags a duplicate charge. Sanitized worker telemetry is correlated alongside it.
JIRA-BILL-142EVT-002 · worker_exit_before_ackPAY-001 + PAY-002 → INV-1042The incident starts as a complaint. Sanitized telemetry from the async worker is correlated with the ticket. The team gets context, not just a description.
The animation shows a synthetic reference application, not a real customer integration. The full team verification workflow is a pilot design in development. A free integration twin engine is available with supported setup.
Read the full walkthrough
- Failure surfaces in support
The incident starts as a complaint. Sanitized telemetry from the async worker is correlated with the ticket. The team gets context, not just a description.
- Incident becomes proposed rule
Someone translates the incident into a precise expectation. If a payment event arrives twice, charge exactly once. The rule is proposed. A human reviews and versions it before it governs anything.
- Approved rule anchors sandbox
Once the rule is approved, FetchSandbox builds a bounded environment around it. Clock, event order, and data state are controlled within the declared scope. Coverage outside that boundary stays unverified.
- Agent writes the fix
An Asana card is handed to a coding agent. The agent reads the PRD and opens a pull request. FetchSandbox has not verified anything yet. The two roles stay separate.
- Review and checks run
Code review and sandbox verification are separate jobs. CodeRabbit reviews the pull request. FetchSandbox replays two failure scenarios against the proposed revision. Both have to pass. Neither replaces the other.
- Defect caught, agent repairs
The defective revision fails the duplicate-event check. The release is blocked. The agent makes a repair. The same checks run again against the new revision. The rule does not change.
- High-risk billing change waits for named manager approval
Billing changes are flagged high-risk in policy. Deployment waits for a named manager to approve the exact commit and image digest. A post-deploy check runs after deploy. Wiring to your pipeline is a setup step.
- New failure joins check set
When a new failure surfaces, the process restarts at the rule layer. A regression check is proposed, reviewed, and versioned before joining the set. FetchSandbox never adds expectations on its own.
FROM THE BUILDERS
What builders told us.
We started with integration testing. This is the feedback we got.
Connecting real testing scenarios, webhooks, retries, async state, via MCP straight into Cursor/Claude Code instead of a separate dashboard is the part that actually saves time day to day.
A claim that's falsifiable by the customer in 30 seconds. The before/after receipt is the trust unlock.
Baking the precondition into the scenario spec means the test is deterministic, and you can actually explain why it failed. The right call for v1.
The reliability check moves inside the agent's loop instead of being bolted on after, not a human remembering to click replay.
Ask only when undoing the wrong branch would cost you, and say what you already ruled out, so the question reads as work done, not helplessness.
The chaos-order webhook testing is a nice touch, exactly the kinds of issues that are hard to reproduce consistently.
This earned its keep on day one. I pointed it at my invoice.paid handler and the duplicate-webhook run called my exact sin, dedup was a check-then-add, not an atomic primitive.
“Not gonna pretend it's there when it's not”, most people would've said “yes, roadmap!” and I'd have quietly closed the tab. It moved you up my list.
This truly is the Postman for testing integrations.
The worst production bugs look completely fine until a second system reacts, your code passes, the API returns 200, and three hours later a duplicate webhook fires and someone gets charged twice.
VERIFICATION FACTORY / PILOT DESIGN
How team knowledge improves future coverage
Incidents, support issues, requirements, and telemetry are the starting point for new checks. Each proposed rule is reviewed and versioned by your team before it runs against any change. Approved expectations stay stable until your team changes them. Each approved case adds to future coverage. Scope outside the declared sandbox boundary stays unverified. This pilot is being shaped for Python services, monorepos, microservices, and async jobs and queues, with coverage defined per team rather than applied universally.
HOW WE WORK / FOUR STEPS
What must never break?
Let’s put it to the test.
An invariant is a rule that must stay true, even when a dependency fails or a request runs twice.
- 01 / THE RULE
Define your invariants
Tell us what must stay correct. A retry must not charge twice. A query must not lose records. We agree on rules we can check.
- 02 / THE ENVIRONMENT
Connect your system
We work with an isolated version of your app and its dependencies. Together, we decide what we can control and observe.
- 03 / THE FAILURE
Inject failure scenarios
Within that scope, we test what happens when requests time out, jobs retry, or two operations happen together. Then we check the result.
- 04 / THE RECEIPT
Fix it. Prove it.
If something breaks, fix it and run the checks again. The receipt shows the version tested, what happened, and what remains unverified.
Start with your riskiest change. We’ll build the verification around your system. The free integration engine is available today. Try the free engine
01 / START WITH THE BEHAVIOR
Different changes.
One question: does it still work?
Your team changes the software. You need to know it still works. Front end, back end, database, background jobs, or the whole application. Start with what must stay correct. Test what could break it. See the evidence before you ship.
Background job
A worker restarts after writing a result. The same item arrives again. Does it create a second effect?
Explore this example 02Database change
A query changes. Some old records have missing fields. Can you still read every account, with the right owner?
Explore this example 03Application change
A change lands in your front end, back end, or another part of the application. When an operation repeats before it finishes, does the system still produce one result?
Explore this exampleJobs, queries, components, connected services. We’re extending the engine with teams who need to trust these changes. Each pilot starts with clear rules, an isolated environment, and evidence everyone can inspect.
02 / EVIDENCE YOU CAN INSPECT
The failure. The fix.
The receipt that proves it.
See the rule, the failure, and the result. Switch between the original version and the proposed fix. This is an example, not a measured run.
jobs/reconcile.ts
A retry should not do the work twice.
await writeResult(item);await writeOnce(item.id, result);2 effects for the same record
The original version writes twice after restart. The candidate uses the record identity to preserve one effect.
03 / CLOSE THE LOOP
Let agents move fast.
Give them a clear next step.
More code means more changes to trust. Give your team and its agents a clear record of what broke, what changed, and what was checked.
Try the free engineAgree on a rule and how to check it.
Choose the inputs and failures to test.
Check the evidence. Say when something wasn’t observed.
Run the relevant checks on the repaired version.
BEFORE WE TALK
A few questions you might have.
We already have CI and tests. What does FetchSandbox add?
Keep them. A green build tells you those checks passed. We focus on agreed rules under controlled failures: what happens when a job retries, a response arrives late, or work overlaps? The receipt shows the behavior we observed, rather than a claim that everything works.
What can you verify today?
Use the free engine today to check supported integrations against controlled provider twins. For your team’s services, jobs, database changes, and other systems, we build a verification pilot around the behavior that matters to you. We agree on the environment and acceptance checks together.
What is an invariant? Who defines it?
It’s a rule that must stay true. A payment retry must not send two receipts. A database change must preserve who owns each record. Your team knows the requirement; we work with you to turn it into a check with an observable result.
Will you test failures in our production system?
We start in an isolated test environment and agree on what the test can affect. Integration runs use temporary twin credentials. We don’t need live billing enabled to test receipt behavior, and temporary test credentials should be removed when testing is finished.
What does the receipt actually prove?
It shows what passed, what failed, and what could not be checked for the recorded run. Results apply to the tested version, scenarios, and observation windows. Missing evidence stays unverified. A passing run is not a guarantee that every production failure is covered.
How does our team get started?
Start with a change you’re worried about: a PR, a query, a job, or a service. Tell us what must keep working. We’ll agree on a small test scope, the environment, and the evidence needed to judge it. You can also try the free integration engine through MCP today.
BUILD WITH US
Bring the change
you’re nervous to ship.
Tell us what changed and what you’re worried might break.
We’ll work out what to test and what evidence you need.