Run an on-call shift

Investigate incidents, name an owner, and leave evidence for the next shift.

An on-call shift watches for incidents and keeps a person responsible for the response. In this local setup, incident issues are the intake source. An agent investigates; the primary and backup own acknowledgement and action.

Create an incident label in GitHub. Put symptoms, timestamps, affected services, and links to available evidence in each incident issue. This chapter uses the same per-issue artifact as implementation and review.

Use the GitHub repository and CLI setup from Take an issue and create record.ts from Keep a work log before adding the on-call modules. You do not need an implemented candidate to investigate an incident.

Assign coverage

Create .foundry/on-call.json in the project. Update it at each shift change:

{
  "primary": "Maya",
  "backup": "Alex",
  "runbook": "https://YOUR-TEAM.example/runbooks/service"
}

Use your team’s actual names and runbook URL. The scheduled patrol reads this file on each run. Changing it does not transfer responsibility until the next person acknowledges the handoff.

Investigate and record

Create .foundry/marbles/_factory/on-call.ts:

Complete on-call.ts

import { agent, Output, step } from "@foundry.sh/marbles";
import { z } from "zod";
import {
  baseBranch,
  chronicle,
  issueInput,
  issueSchema,
  newRecord,
  readRecord,
  recordEvent,
  repository,
  requireRecord,
} from "./record";

export const coverageSchema = z.object({
  backup: z.string().min(1),
  primary: z.string().min(1),
  runbook: z.string().url(),
});
const assessment = z.object({
  evidence: z.array(z.string()),
  nextAction: z.string(),
  severity: z.enum(["critical", "high", "normal"]),
});
const investigator = agent({
  prompt:
    "Investigate this incident using the issue, local code, and work log. Report evidence, severity, and the next action. Do not change files, deploy, or claim a service recovered without evidence.",
});
export const onCall = step("factory-on-call")
  .input(issueInput.extend({ coverage: coverageSchema }))
  .do(async ({ input, exec, workspaces, agents, report, run }) => {
    const response = await exec([
      "gh",
      "issue",
      "view",
      String(input.issue),
      "--repo",
      repository,
      "--json",
      "number,title,body,url,updatedAt,state",
    ]);
    const issue = issueSchema.parse(JSON.parse(response.stdout));
    await exec(["git", "fetch", "origin"]);
    const base = (
      await exec(["git", "rev-parse", `origin/${baseBranch}`])
    ).stdout.trim();
    const work = (await readRecord(input.issue)) ?? newRecord(issue, base);
    if (issue.state === "CLOSED") {
      if (work.status === "closed") {
        return work;
      }
      return recordEvent(
        { ...work, status: work.change ? work.status : "closed" },
        run.id,
        "incident closed",
        issue.url
      );
    }
    const marker = JSON.stringify({
      coverage: input.coverage,
      updatedAt: issue.updatedAt,
    });
    if (
      work.events.some(
        (event) => event.stage === "on-call observed" && event.detail === marker
      )
    ) {
      return work;
    }
    const result = await workspaces.current.git.withWorktree(
      { base },
      async () => {
        const session = await agents.session(investigator);
        return session.generate(
          `${issue.body}\n\n${chronicle(work)}\n\nRunbook: ${input.coverage.runbook}`,
          {
            output: Output.object({ schema: assessment }),
          }
        );
      }
    );
    const finding = assessment.parse(result.output);
    const next = await recordEvent(
      work,
      run.id,
      "on-call assessment",
      JSON.stringify(
        {
          ...input.coverage,
          ...finding,
          acknowledgeBy: new Date(Date.now() + 15 * 60_000).toISOString(),
          revision: base,
        },
        null,
        2
      )
    );
    report.result({
      body: `${input.coverage.primary} owns this incident. Backup: ${input.coverage.backup}.\n\n${finding.nextAction}`,
      title: `${finding.severity}: issue #${input.issue}`,
    });
    return recordEvent(next, run.id, "on-call observed", marker);
  });

step("factory-handoff")
  .input(
    issueInput.extend({ note: z.string().min(1), owner: z.string().min(1) })
  )
  .do(async ({ input, run }) =>
    recordEvent(
      await requireRecord(input.issue),
      run.id,
      "on-call handoff",
      `${input.owner}: ${input.note}`
    )
  );

Create .foundry/marbles/on-call.ts:

import "./_factory/on-call";

Launch factory-on-call with an issue and coverage, or let the scheduled patrol supply them. The agent reads the current remote revision in a temporary checkout. Its record includes the observed commit, severity, evidence, next action, primary, backup, and acknowledgement deadline. An unchanged issue and coverage reuse the recorded assessment.

The issue artifact retains that assessment and its evidence. A separate report.result(...) Feed entry names the severity, owner, backup, and next action so a person can respond. The Feed entry is a persisted message; subsequent steps read the issue artifact for the incident’s full history.

Acknowledge and hand off

Record who has taken responsibility and what they will do:

marbles roll factory-handoff --input '{"issue":42,"owner":"Maya","note":"Acknowledged. Investigating the timeout regression; next update at 14:30 UTC."}'
marbles roll factory-handoff --input '{"issue":42,"owner":"Alex","note":"Taking over. Reproduction and logs are attached to the issue; awaiting the database owner."}'

The patrol flags an assessment without a later acknowledgement after 15 minutes and names the backup in its report. That interval is a sample policy. Use your incident policy, and contact the backup through your team’s normal channel. This example reports locally; it does not page anyone or provision an on-call rotation service.

After mitigation, record the evidence and remaining work, close the incident through your normal process, and run factory-on-call again to record its closed state. If a code change is needed, add acceptance criteria and send the issue through the implementation cycle. The investigation stays in its log.

Next: Watch repository and dependency changes.