Sheet 04Run state machine & scoringScale: NTS

One JSON document, read and rewritten

A run is a single file: runs/<runId>/run.json. Every mutating operation loads it, edits it in memory, and writes the whole thing back. That design keeps the state machine legible and auditable, and it is also the reason several fields in the type never acquire a value — nothing is watching the document except the five methods that write it.

The document

RunState (types.ts:186–200) has thirteen fields. Four of them — goal, milestones, evalSelection, budget — are written once at createRun() and, except for milestone status, never touched again. Two arrays grow: attemptRecords and attempts. One array, evalValidationRecords, is initialised empty and stays empty.

Fig. 4.1 Run state — entity relationships

Entity relationship diagram of the persisted RunState document RunState sits at the centre with thirteen fields. It composes exactly one GoalContract, one EvalSelection, one RunBudget and optionally one FrozenEvalSnapshot, and holds three arrays: Milestone, AttemptRecord and AttemptScore, plus an EvalValidationRecord array that is always empty. GoalContract embeds a doneWhen object with minScore, requiredGatesPass and noRegressions. EvalSelection embeds full EvaluatorManifest copies rather than identifiers. AttemptRecord optionally embeds an AttemptScore and declares worktreePath, baseRef and headRef fields that no code assigns. Cardinalities are marked on each edge and entities with unwritten fields are drawn with dashed outlines. RunState — 13 fields types.ts:186 id run_<nanoid 10> goal GoalContract milestones Milestone[] evalSelection EvalSelection phase RunPhase budget RunBudget frozenEvalSnapshot? evalValidationRecords[] attemptRecords[] · attempts[] status · createdAt · updatedAt GoalContract id · goal · domain deliverables[] · constraints[4] requiredProofs: Proof[] doneWhen.minScore 0.75 | 0.85 doneWhen.requiredGatesPass never read doneWhen.noRegressions never read createdAt Milestone id m0 … m5 title · description requiredProofs: Proof[] weight never read status only "pending" -> "passed" EvalSelection domain · requiredProofs[] evaluators: EvaluatorManifest[] uncoveredProofs: Proof[] full manifest copies, not ids RunBudget maxAttempts 50 maxWallMinutes 360 minScoreDelta 0.01 stagnationLimit 8 requireHumanPromotion true written at createRun, read by nothing FrozenEvalSnapshot id evalsnap_<nanoid 10> evaluatorIds: string[] sorted publicManifestHash 8 hex chars hiddenManifestHash? never set frozenAt · ownerRole "judge" notes: string[] AttemptRecord id attempt_<nanoid 10> milestoneId? status 7 values, 4 ever assigned worktreePath? never set baseRef? headRef? never set openedAt · closedAt? · score? AttemptScore attemptId · runId · milestoneId? score: number 0…1 gatesPassed · noRegressions proofScores: Partial<Record> failures: string[] decision keep | revert | needs_review EvalValidationRecord mutationFailurePassed hiddenSeedSplit antiHardcodeCheck artifactEvidenceCaptured approved · notes[] · createdAt array is always length 0 1 0..n 1 1 0..1 0..n 0..n 0 score? 0..1 whole document rewritten on every mutation StateStore.saveRun — state.ts:14 no locking, no append log, no revision history WRITE SITES createRun daedalus.ts:59 freezeEvals daedalus.ts:84 openAttempt daedalus.ts:100 scoreRunAttempt daedalus.ts:143 4 write sites, 1 read site (loadRun) SOLID = fields that receive values · DASHED / amber = declared in types.ts with no assignment anywhere in the repository
Fig. 4.1 — Storing whole EvaluatorManifest objects rather than identifiers is the right call for auditability: a run’s record of what it was graded against cannot be invalidated by later edits to evaluatorRegistry.ts. The amber fields are the design’s forward declarations — worktree isolation, hidden-eval hashing, and the eval-critic record are all named in the type system before any of them has an implementation.

Fig. 4.2 RunPhase — declared vs. reachable

State machine of RunPhase showing which of the thirteen declared phases are actually assigned The RunPhase union declares thirteen values. Six are reachable: goal_created set by createRun, evals_frozen set by freezeEvals, attempt_opened set by openAttempt, and accepted, rejected or scored set by scoreRunAttempt according to the decision. Each transition is labelled with the method and line that performs it. Seven further phases — evals_proposed, evals_validated, agent_running, eval_running, rolled_back, promoted and blocked — are drawn below in a separate unreachable band with dashed outlines, because no assignment statement in the repository ever produces them. A separate lane shows RunState.status, which moves from planned to running to complete, where complete requires every milestone passed and the latest score at or above the goal threshold. REACHABLE PHASES — 6 of 13 goal_created createRun · ts:44 freezeEvals() evals_frozen daedalus.ts:82 openAttempt() attempt_opened daedalus.ts:97 scoreRunAttempt() decision daedalus.ts:139 accepted rejected scored keep needs_review openAttempt() again — no guard, no limit DECLARED BUT UNREACHABLE — types.ts:116–129 evals_proposed evals_validated agent_running eval_running rolled_back promoted blocked 1 These seven are listed verbatim in docs/ARCHITECTURE.md:38–52 as the “authoritative local state machine”. Grepping for phase assignments returns exactly three statements. RunState.status — a second, independent lane "planned" openAttempt "running" scoreRunAttempt "complete" "paused" "failed" declared, never assigned 2 2 · status becomes "complete" only when every milestone is "passed" AND the latest attempt score ≥ goal.doneWhen.minScore (daedalus.ts:141–142). Because the condition is re-evaluated on every score, a later low-scoring attempt sets it back to "running". SOLID = assigned somewhere in the repository · HATCHED = union member with no assignment site
Fig. 4.2 — The phase union is a design document written in TypeScript. That is not a criticism of the type; declaring the intended machine before building it is reasonable, and the union makes the gap legible. But a consumer reading run.phase today will only ever see six of the thirteen values, and cannot distinguish “the agent is running” from “the attempt is open”.

Freezing a manifest

freezeEvals() sorts the selected evaluator identifiers, hashes a three-key object, and stamps the result onto the run. The hash function is 8 lines of arithmetic at the bottom of the same file.

packages/core/src/daedalus.ts · lines 187–194verbatim
187function stableHash(value: unknown): string {
188  const json = JSON.stringify(value);
189  let hash = 0;
190  for (let index = 0; index < json.length; index += 1) {
191    hash = (hash * 31 + json.charCodeAt(index)) >>> 0;
192  }
193  return hash.toString(16).padStart(8, "0");
194}

This is a 32-bit djb-style multiplicative hash, not a cryptographic digest. It is deterministic and adequate for “did this identifier list change”, and it is not adequate for tamper detection — node:crypto is already available in the runtime and is not used. Note also what is hashed: domain, the sorted evaluatorIds, and requiredProofs. The evaluator contentspublicTests, hiddenTests, mutationTests — are not part of the input, so an evaluator can be rewritten entirely without changing the frozen hash.

For the build me Doom run, the input to the hash is fully determined by the compile stage, so the value is reproducible: with evaluatorIds = ["browser-game", "playwright-web", "shell-core"] and the nine browser_game proofs, publicManifestHash is b5f2049e on every machine and every run.

Fig. 4.3 Scoring & the keep / revert decision

How scoreAttempt turns submitted proof scores into a keep, revert or needs_review decision Submitted proof scores are clamped to the range zero to one, with NaN mapped to zero and any proof absent from the submission treated as zero. The clamped values for the required proofs are summed and divided by the count of required proofs, giving an unweighted mean. That score is compared against the goal threshold to produce gatesPassed. The decision is then a two-level conditional: keep when gates passed and no regressions, otherwise revert when the failures array is non-empty, otherwise needs_review. A truth table below enumerates all eight combinations. A worked example scores milestone m0 of the Doom run, where boots equals one and builds equals zero point seven produce a mean of exactly zero point eight five, precisely on the threshold, and therefore keep. scoring.ts:12–32 proofScores Partial<Record<Proof, number>> supplied by the caller never measured by Daedalus clamp(v) NaN → 0 min(1, max(0, v)) absent proof → 0 score Σ clamp(proofScores[p]) required.length unweighted; required.length===0 → 0 gatesPassed score ≥ goal.doneWhen.minScore 0.85 for a classified domain 0.75 for "unknown" gatesPassed && noRegressions ? "keep" true failures.length > 0 ? false "revert" "needs_review" DECISION TABLE — all 8 cases gates noReg fails decision T T 0 keep T T >0 keep T F 0 needs_review T F >0 revert F T 0 needs_review F T >0 revert F F 0 needs_review F F >0 revert 1 1 · Row 2 is worth noting: when the gates pass and there are no regressions, a non-empty failures list is ignored. The decision short-circuits before ever looking at it. WORKED EXAMPLE — scoring milestone m0 of the Doom run milestone m0.requiredProofs = ["boots", "builds"] → scoringGoal = { ...run.goal, requiredProofs: m0.requiredProofs } submitted { boots: 1, builds: 0.7 } score = (1 + 0.7) / 2 = 0.85 minScore = 0.85 gatesPassed = true noRegressions defaults to true failures = [] → decision "keep" → m0.status = "passed" → phase = "accepted" submitted { boots: 1 } score = (1 + 0) / 2 = 0.50 gatesPassed = false failures = [] → decision "needs_review" → m0 stays "pending" → phase = "scored" → the run cannot advance
Fig. 4.3 — The whole scorer is 21 lines. Its most consequential property is the one drawn at the far left of the input: proofScores is data the caller hands in. The threshold comparison at scoring.ts:17 always uses goal.doneWhen.minScore, even when the goal object has been narrowed to a single milestone’s proofs at daedalus.ts:119 — so a one-proof milestone effectively demands a raw score of 0.85 on that single proof.

Budgets are declared and never checked

docs/ARCHITECTURE.md:100–111 makes the case for budgets plainly: “Without budgets, ‘run until done’ becomes either infinite looping or premature acceptance.” The RunBudget object is duly constructed for every run.

RunBudget defaults — daedalus.ts:45–51 — and where each is enforced
FieldDefaultIntended effectEnforcement site
maxAttempts50stop opening attemptsnone
maxWallMinutes360abandon the run after six hoursnone
minScoreDelta0.01ignore improvements below the noise floornone
stagnationLimit8give up after eight flat attemptsnone
requireHumanPromotiontrueblock automatic promotionnone

Searching the TypeScript sources for budget returns the field declaration at types.ts:192, the object literal at daedalus.ts:45, and a handful of unrelated hits on the performance_budget proof name. openAttempt() pushes a new record unconditionally; there is no comparison against run.budget.maxAttempts anywhere. The values are correct and inert — they will be meaningful the moment something reads them, and today a caller can open a five-hundredth attempt without complaint.

The store

StateStore is 39 lines and does three things: resolve a run directory, write JSON, read JSON. Its interesting part is the path guard.

packages/core/src/state.ts · lines 25–39verbatim
25  runDir(runId: string): string {
26    assertRunId(runId);
27    const dir = resolve(this.runsDir, runId);
28    if (!dir.startsWith(`${this.runsDir}`)) {
29      throw new Error(`Invalid run path for ${runId}`);
30    }
31    return dir;
32  }
33}
34
35export function assertRunId(runId: string): void {
36  if (!/^run_[A-Za-z0-9_-]+$/.test(runId)) {
37    throw new Error(`Invalid run id: ${runId}`);
38  }
39}

Two independent checks — a strict identifier pattern and a resolved-prefix comparison — for a value that arrives from an MCP tool call and becomes a filesystem path. The smoke test exercises it directly with core.loadRun("../bad"). This is the most defensive code in the repository, and appropriately so: runId is the only user-controlled string that reaches node:fs.

The root comes from process.env.DAEDALUS_HOME ?? process.cwd() (state.ts:9), which is why docs/MCP_USAGE.md insists on setting DAEDALUS_HOME when the server is launched from a different working directory — otherwise runs are written relative to wherever the MCP host happened to start.

A run, end to end

Commander CLI — create-run · freeze-evals · open-attempt · next
$ daedalus init
Initialized Daedalus state directories.
# mkdir -p runs/ artifacts/ research/repos/          (cli/index.ts:16–21)

$ daedalus create-run "build me Doom"
{
  "id": "run_kM3vQz8pTa",
  "goal": { /* GoalContract, domain browser_game */ },
  "milestones": [ /* m0 … m5 */ ],
  "evalSelection": { /* 3 evaluators, 0 uncovered */ },
  "phase": "goal_created",
  "budget": {
    "maxAttempts": 50, "maxWallMinutes": 360, "minScoreDelta": 0.01,
    "stagnationLimit": 8, "requireHumanPromotion": true
  },
  "evalValidationRecords": [],
  "attemptRecords": [],
  "attempts": [],
  "status": "planned",
  "createdAt": "…", "updatedAt": "…"
}
# written to $DAEDALUS_HOME/runs/run_kM3vQz8pTa/run.json

$ daedalus freeze-evals run_kM3vQz8pTa
{
  "id": "evalsnap_7QbN2xLw0d",
  "evaluatorIds": ["browser-game", "playwright-web", "shell-core"],
  "publicManifestHash": "b5f2049e",
  "frozenAt": "…",
  "ownerRole": "judge",
  "notes": []
}
# phase: goal_created -> evals_frozen

$ daedalus open-attempt run_kM3vQz8pTa m0
{
  "id": "attempt_Yh6rL0sVpE",
  "milestoneId": "m0",
  "status": "opened",
  "openedAt": "…"
}
# phase: evals_frozen -> attempt_opened ; status: planned -> running

$ daedalus next run_kM3vQz8pTa
Implement milestone m0: Project boots. Create a runnable project with a clean
build/start path. Required proofs: boots, builds. Do not edit frozen evaluators
unless explicitly assigned the eval-writer role.

Identifiers are nanoid placeholders and timestamps are elided; every other value is determined by the source. The publicManifestHash is the real computed value for this goal. The next text is assembled at daedalus.ts:156–164 — the trailing failure clause is dropped by .filter(Boolean) when no attempt has been scored yet.

Note that daedalus has no score command. Scoring is reachable only through the MCP tool daedalus_score_attempt, which is a clear statement of intent: the CLI is for humans inspecting state, and the loop is expected to be driven by an agent through MCP.

Sheet notes

  1. scoreRunAttempt() does not require a prior openAttempt(). If the supplied attemptId matches no record, upsertAttemptRecord() fabricates one with openedAt === closedAt (daedalus.ts:177–184). If no attemptId is supplied at all, a fresh attempt_<nanoid> is minted.
  2. Every scored attempt is appended to run.attempts as well as being upserted into run.attemptRecords, so a completed run holds two parallel views of the same history — one flat score list and one record list.
  3. Scoring an unknown milestoneId throws Unknown milestone id: … (daedalus.ts:117). Scoring with no milestoneId is legal and grades against the whole goal’s required proofs.
  4. A milestone can only be marked passed by a scoring call that names it. Scoring the goal as a whole leaves every milestone "pending", which means status can never reach "complete" without per-milestone scoring.
  5. There is no delete, archive or list operation. StateStore exposes saveRun, loadRun and runDir only; enumerating runs means reading the directory yourself.