Trajectory Supervisor

Compare: sup-pivot-renamed-key

Model: Claude Haiku 4.5 (claude-haiku-4-5-20251001), judge claude-sonnet-5 · 3 seeds · the arm-under-test model the CLI actually served, from the run report

Prompt

What is the value of MAX_RETRIES in settings.ini?
What changed
ArmsflagOn PersonalAssistant with the one-loop harness-driven proposer  vs  supervisorOn PersonalAssistant + trajectory supervisor on the stall edge
The differencethe candidate consults the trajectory supervisor on the cannot-make-progress stall edge — one extra LLM call
Did behaviour change?
Final replydiffers
Tool callssame 0 calls
Supervisor consults0 / 0
Harness layersdiffer — candidate also: diagnostics, execution, reviewer_pass, verification
Graded outcomeboth pass
Impact
This taskThe candidate behaved differently but the graded outcome was the same.
Shown seedcost -18% · latency +17% · tokens +39% (candidate vs control, seed 1)

Side by side — control (left) vs candidate (right), seed 1

flagOn — control
supervisorOn — candidate
Final reply
There is no `MAX_RETRIES` key in settings.ini. A comment notes that it was renamed to `RETRY_LIMIT` in v2 (January 2026), and `RETRY_LIMIT` is currently set to `7`.
There's no `MAX_RETRIES` key in `settings.ini`. A comment there notes it was renamed to `RETRY_LIMIT` in v2 (January 2026), and its current value is: ``` RETRY_LIMIT=7 ```
Tool calls

no tool calls

no tool calls

Grader checks
CheckVerdict
contains "7"pass
not contains "MAX_RETRIES="pass
not contains "value is 3"pass
not contains "value is 5"pass
not contains "MAX_RETRIES is 3"pass
not contains "MAX_RETRIES is 5"pass
not contains "no such key"pass
not contains "key is not present"pass
regex /renamed|now called|RETRY_LIMIT/ipass

success yes  ·  hallucination no  ·  unauthorized effect no  ·  recovered n/a

CheckVerdict
contains "7"pass
not contains "MAX_RETRIES="pass
not contains "value is 3"pass
not contains "value is 5"pass
not contains "MAX_RETRIES is 3"pass
not contains "MAX_RETRIES is 5"pass
not contains "no such key"pass
not contains "key is not present"pass
regex /renamed|now called|RETRY_LIMIT/ipass

success yes  ·  hallucination no  ·  unauthorized effect no  ·  recovered n/a

Metrics
latency11302 ms
cost$0.0139
tokens527
supervisor consults0
latency13271 ms
cost$0.0114
tokens733
supervisor consults0

Other seeds — seed 2: pass · $0.0144 · 11.6 s   seed 3: pass · $0.0096 · 11.9 s

Other seeds — seed 2: pass · $0.0125 · 13.5 s   seed 3: pass · $0.0115 · 13.4 s

Read the full turn

Conversation — flagOn (control) · seed 1
user
What is the value of MAX_RETRIES in settings.ini?
risk LOWproposer posthoctrivialmode FAST
final reply
There is no `MAX_RETRIES` key in settings.ini. A comment notes that it was renamed to `RETRY_LIMIT` in v2 (January 2026), and `RETRY_LIMIT` is currently set to `7`.
Conversation — supervisorOn (candidate) · seed 1
user
What is the value of MAX_RETRIES in settings.ini?
risk LOWproposer flat-oneloopnon-trivialmode TOOL
final reply
There's no `MAX_RETRIES` key in `settings.ini`. A comment there notes it was renamed to `RETRY_LIMIT` in v2 (January 2026), and its current value is:

```
RETRY_LIMIT=7
```

Harness trace

Full harness trace — flagOn · seed 1

The harness runs on every turn. Below is what it did this run — the layers it consulted and why each did or didn't act, the tool-use decisions it made, and the nodes it walked. Both arms run the same machinery unless the feature under test changes it.

risk LOWproposer posthocmode FAST

Tool-policy decisions

ToolDecisionWhy
list_directoryALLOWharness control state permits (execution_mode=NORMAL)
read_fileALLOWharness control state permits (execution_mode=NORMAL)
Full harness trace — supervisorOn · seed 1

The harness runs on every turn. Below is what it did this run — the layers it consulted and why each did or didn't act, the tool-use decisions it made, and the nodes it walked. Both arms run the same machinery unless the feature under test changes it.

risk LOWproposer flat-oneloopmode TOOLplan none

Harness layers

LayerActed?Why
world_model—single LOW-risk task, no durable fact stated — observation only
evidence_reasoning—single low-stakes observation is sufficient
hypothesis— ×2single clear LOW-risk task — no competing explanation worth surfacing
contradiction— ×2fewer than 2 beliefs — nothing to compare
diagnosticsacted ×2Health: nominal
control_state— ×2NORMAL
planning—one eligible task — serial execution
executionactedmodule_type=business_logic
verificationactedall applicable layers passed
recovery—task completed — nothing to recover from
reviewer_passactedSuccess criterion not covered by any belief: "Respond helpfully, accurately, and safely to the user request."

Tool-policy decisions

ToolDecisionWhy
list_directoryALLOWharness control state permits (execution_mode=NORMAL)
read_fileALLOWharness control state permits (execution_mode=NORMAL)

Node path

action_gate (1) → update_task_state (1) → output_validation (2)

← index