Test and review

Check a candidate against the original issue, plan, implementation, and previous findings.

Review starts with context. The reviewer reads the original issue, acceptance criteria, plan, implementation explanation, previous attempts, and test output. It then examines the exact candidate commit. A candidate is the committed revision proposed for this issue, identified by its SHA.

Check engineering decisions

The review prompt and selected skills should name the project’s expectations. “Review this code” leaves too much unstated. Ask for evidence against the accepted scope, including the reasons behind the first implementation.

To use an approved project skill, import skills from @foundry.sh/marbles and add skills: skills.none().add("./skills/review") to the reviewer declaration. That directory must contain SKILL.md. Select skills for the review’s remit and retain their version with the evidence; use skill evaluations before changing them.

Decision	What to check
Add a dependency	Is it needed? Does the repository already supply it? Check its version, lockfile, license, maintenance status, and transitive dependencies.
Create a package	Is there a real owner and reusable responsibility? Check its exports, dependency direction, build, and tests. A folder alone may be enough.
Change a module	Does its interface keep complexity out of callers? Include ordering rules, errors, and compatibility in the review.
Change domain rules	Do names match the domain? Are invariants enforced where the state changes?
Add tests	Do they exercise the required result through the module’s interface, including failure cases? Do they fail for the defect being fixed?

Check the plan’s promised commands against the actual evidence. The example below runs installation and bun test; add the project’s lint, typecheck, build, or other required checks before relying on it for delivery. Missing browser or service evidence stays missing even when a model says the code looks correct.

Run the verifier

Create .foundry/marbles/_factory/verify.ts:

Complete verify.ts

import { agent, Output, step } from "@foundry.sh/marbles";
import {
  chronicle,
  recordEvent,
  recordSchema,
  requireRecord,
  reviewSchema,
  workInput,
} from "./record";

const reviewer = agent({
  prompt:
    "Review the exact candidate commit against the original issue, acceptance criteria, plan, implementation explanation, and prior findings. Read repository instructions and relevant callers. Check the code yourself. Examine new packages and dependencies for necessity, ownership, version and lockfile consistency. Check module interfaces, domain invariants, failure handling, and whether tests exercise the required results. For each criterion, name concrete evidence or mark it unmet. Mark promised but unperformed checks as missing evidence. Each blocker needs a file or record reference, a triggering condition, and a consequence. Separate defects from preferences and identify findings the revision already resolved. Do not invent requirements. Do not edit or commit.",
});
export const verify = step("factory-verify")
  .input(workInput)
  .output(recordSchema)
  .do(async ({ input: { work }, agents, workspaces, exec, run }) => {
    const { change } = work;
    if (
      change &&
      work.verification?.revision === change.revision &&
      work.status !== "implemented"
    ) {
      return work;
    }
    if (!change || work.status !== "implemented") {
      throw new Error("An implemented candidate is required.");
    }
    try {
      return await workspaces.current.git.withWorktree(
        { base: change.revision },
        async () => {
          const install = await exec(["bun", "install", "--frozen-lockfile"], {
            check: false,
          });
          const tests =
            install.exitCode === 0
              ? await exec(["bun", "test"], { check: false })
              : {
                  exitCode: -1,
                  stderr: "Tests not run because installation failed.",
                  stdout: "",
                };
          const output = JSON.stringify({ install, tests }, null, 2);
          const checked = await recordEvent(
            work,
            run.id,
            "test output",
            output,
            {
              [`attempts/${work.attempts}/tests.json`]: output,
            }
          );
          const session = await agents.session(
            reviewer,
            work.reviewer ? { session: work.reviewer } : {}
          );
          const reply = await session.generate(
            [
              work.issue.body,
              chronicle(checked),
              `Review ${work.base}..${change.revision}. Initial implementation and all later attempts are in the history.`,
              `Implementation explanation: ${change.summary}`,
            ].join("\n\n"),
            { output: Output.object({ schema: reviewSchema }) }
          );
          const review = reviewSchema.parse(reply.output);
          const head = (await exec(["git", "rev-parse", "HEAD"])).stdout.trim();
          const dirty = (
            await exec(["git", "status", "--porcelain"])
          ).stdout.trim();
          if (head !== change.revision || dirty) {
            throw new Error(
              "Verification changed the checkout. Its evidence cannot be accepted."
            );
          }
          const testsPassed = install.exitCode === 0 && tests.exitCode === 0;
          const meetsCriteria = work.acceptance.every((criterion) =>
            review.criteria.some(
              (item) =>
                item.criterion === criterion && item.met && item.evidence.trim()
            )
          );
          return recordEvent(
            {
              ...checked,
              reviewer: session.ref,
              status:
                testsPassed && meetsCriteria && review.blockers.length === 0
                  ? "verified"
                  : "needs-changes",
              verification: { review, revision: change.revision, testsPassed },
            },
            run.id,
            "review",
            JSON.stringify(review, null, 2)
          );
        }
      );
    } catch (error) {
      // Read the latest record so a failure after testing keeps its output.
      const latest = await requireRecord(work.issue.number);
      await recordEvent(
        { ...latest, status: "blocked" },
        run.id,
        "verification failed",
        String(error)
      );
      throw error;
    }
  });

Replace factory.ts with:

import { workflow } from "@foundry.sh/marbles";
import { implement } from "./_factory/implement";
import { intake } from "./_factory/intake";
import { plan } from "./_factory/plan";
import { issueInput } from "./_factory/record";
import { verify } from "./_factory/verify";

workflow("factory")
  .input(issueInput)
  .do(({ input }) =>
    verify({ work: implement({ work: plan({ work: intake({}, input) }) }) })
  );

Read the result

The verifier records install and test output even when a command fails. It returns verified only when those commands pass, the review supplies positive evidence for every criterion, and there are no review blockers. Otherwise it returns needs-changes. That run completed its assessment; it did not approve the code.

A model’s assessment is evidence for the human decision. Adapt the commands to the project’s actual checks. A successful bun test cannot prove a browser interaction that the suite never exercises.

For several reviewers, use the PR review process to assign different risks and synthesize their findings. This sample has one reviewer. Adding more sessions without a synthesis step only adds more output.

Keep review history

This is the persistent reviewer inside the factory. The work log stores one review session reference per issue, so later attempts can revisit unresolved findings. Each turn also receives the written history, including the initial implementation. The transcript is not the only record.

Changing the checkout during verification rejects the evidence. Prompts ask the agent to leave files alone; they do not enforce read-only access. Use the session controls appropriate to your environment.

Next: Revise the candidate.