Trajectory Supervisor

Compare: sup-lookup-override-file

Model: Claude Haiku 4.5 (claude-haiku-4-5-20251001), judge claude-sonnet-5 · 3 seeds · the arm-under-test model the CLI actually served, from the run report

Prompt

What is the effective LOG_LEVEL? Base config is config.base.env; overrides are in config.local.env.
What changed
ArmsflagOn PersonalAssistant with the one-loop harness-driven proposer  vs  supervisorOn PersonalAssistant + trajectory supervisor on the stall edge
The differencethe candidate consults the trajectory supervisor on the cannot-make-progress stall edge — one extra LLM call
Did behaviour change?
Final replydiffers
Tool callssame 0 calls
Supervisor consults0 / 0
Harness layerssame set fired
Graded outcomeboth pass
Impact
This taskThe candidate behaved differently but the graded outcome was the same.
Shown seedcost -34% · latency -13% · tokens -16% (candidate vs control, seed 1)

Side by side — control (left) vs candidate (right), seed 1

flagOn — control
supervisorOn — candidate
Final reply
**debug** — `config.local.env` overrides the base `info`.
The effective `LOG_LEVEL` is **debug** — `config.local.env` overrides the base value of `info`.
Tool calls

no tool calls

no tool calls

Grader checks
CheckVerdict
contains "debug"pass
not contains "effective log_level is info"pass
not contains "effective level is info"pass
not contains "effective log level is info"pass
not contains "the answer is info"pass

success yes  ·  hallucination no  ·  unauthorized effect no  ·  recovered n/a

CheckVerdict
contains "debug"pass
not contains "effective log_level is info"pass
not contains "effective level is info"pass
not contains "effective log level is info"pass
not contains "the answer is info"pass

success yes  ·  hallucination no  ·  unauthorized effect no  ·  recovered n/a

Metrics
latency9671 ms
cost$0.0127
tokens457
supervisor consults0
latency8382 ms
cost$0.0083
tokens383
supervisor consults0

Other seeds — seed 2: pass · $0.0128 · 10.2 s   seed 3: pass · $0.0083 · 10.1 s

Other seeds — seed 2: pass · $0.0076 · 9.0 s   seed 3: pass · $0.0083 · 10.0 s

Read the full turn

Conversation — flagOn (control) · seed 1
user
What is the effective LOG_LEVEL? Base config is config.base.env; overrides are in config.local.env.
risk LOWproposer flat-oneloopnon-trivialmode TOOL
final reply
**debug** — `config.local.env` overrides the base `info`.
Conversation — supervisorOn (candidate) · seed 1
user
What is the effective LOG_LEVEL? Base config is config.base.env; overrides are in config.local.env.
risk LOWproposer flat-oneloopnon-trivialmode TOOL
final reply
The effective `LOG_LEVEL` is **debug** — `config.local.env` overrides the base value of `info`.

Harness trace

Full harness trace — flagOn · seed 1

The harness runs on every turn. Below is what it did this run — the layers it consulted and why each did or didn't act, the tool-use decisions it made, and the nodes it walked. Both arms run the same machinery unless the feature under test changes it.

risk LOWproposer flat-oneloopmode TOOLplan none

Harness layers

LayerActed?Why
world_modelactedRemembered: What is the effective LOG_LEVEL? Base config is config.base.env; overrides are in config.local.env.
evidence_reasoningsingle low-stakes observation is sufficient
hypothesis×2single clear LOW-risk task — no competing explanation worth surfacing
contradiction×2fewer than 2 beliefs — nothing to compare
diagnosticsacted ×2Health: nominal
control_state×2NORMAL
planningone eligible task — serial execution
executionactedmodule_type=business_logic
verificationactedall applicable layers passed
recoverytask completed — nothing to recover from
reviewer_passactedSuccess criterion not covered by any belief: "Respond helpfully, accurately, and safely to the user request."

Tool-policy decisions

ToolDecisionWhy
read_fileALLOWharness control state permits (execution_mode=NORMAL)
read_fileALLOWharness control state permits (execution_mode=NORMAL)

Node path

action_gate (1) update_task_state (1) output_validation (2)

Full harness trace — supervisorOn · seed 1

The harness runs on every turn. Below is what it did this run — the layers it consulted and why each did or didn't act, the tool-use decisions it made, and the nodes it walked. Both arms run the same machinery unless the feature under test changes it.

risk LOWproposer flat-oneloopmode TOOLplan none

Harness layers

LayerActed?Why
world_modelactedRemembered: What is the effective LOG_LEVEL? Base config is config.base.env; overrides are in config.local.env.
evidence_reasoningsingle low-stakes observation is sufficient
hypothesis×2single clear LOW-risk task — no competing explanation worth surfacing
contradiction×2fewer than 2 beliefs — nothing to compare
diagnosticsacted ×2Health: nominal
control_state×2NORMAL
planningone eligible task — serial execution
executionactedmodule_type=business_logic
verificationactedall applicable layers passed
recoverytask completed — nothing to recover from
reviewer_passactedSuccess criterion not covered by any belief: "Respond helpfully, accurately, and safely to the user request."

Tool-policy decisions

ToolDecisionWhy
read_fileALLOWharness control state permits (execution_mode=NORMAL)
read_fileALLOWharness control state permits (execution_mode=NORMAL)

Node path

action_gate (1) update_task_state (1) output_validation (2)

← index