metrics: what the month cost in turns, commands and seconds
the standings say where we finished; this page says what it took. every number below is counted from our own logs: the race session log, the flag ledger, the saved timed runs, the intel and mail the agents exchanged, and the transcripts of every agent that ran. the numbers are aggregated by day, lane and role; no agent, runner or model is named.
the month in eight numbers
514flagssingles, time trials and masteries
8:58median day: release to board completefastest 0:10
5.1kagent turns on the clock42k more off it
16.4btokens, every runnerin + cached + out
42kshell commands logged
22kintel rows shared
29solver crash rows
1.6kweb lane gm callslogged from day 20
an agent turn is one model reply; a shell command is one logged $ line in the race session log; 'on the clock' means between a day's 09:00 et release and its last flag.
the clock
the race is from the 09:00 et release to the team's last flag of the day. the non-web lanes (warmups, metadata, mobile, script, binary) ran on the solver pipeline with a fixer per lane; web ran its own machinery.
release to the last flag, per day
non-web lanes done
web done
the scale stops at 20 minutes; a hollow dot is a day beyond it (hover for the time). day 1 opened late (first start 11:28) and is plotted from its own first start; every other day counts from the 09:00 et release. the web lane was first played on release day on day 19; the earlier web challenges were cleared as a backlog and count on the day they were won.
seconds from release to the single's flag, lane by day
tint by quintile: the lightest fifth of cells are the fastest singles, the darkest fifth the slowest. hover a cell for the time. a dashed cell is a day that lane had no flag recorded.
lane
median single
median time trial
median mastery
best single
warmups
0:02
0:05
0:10
0:01 (day 20)
metadata
0:28
1:01
1:04
0:01 (day 11)
mobile
0:03
0:05
0:08
0:01 (day 18)
script
0:04
0:07
0:12
0:02 (day 11)
binary
0:42
0:58
1:04
0:02 (day 11)
web
6:02
8:32
10:51
1:04 (day 30)
timed tiers
a time trial or mastery attempt runs against a deadline the game master sets when the clock starts; the runner solves every answer set in parallel and submits once.
383timed runs saved
50smedian deadline given
1.8smedian seconds used
83%runs that came back correct
timed-tier runs per day, by outcome
correct
incorrect
expired
crash
who took the flag
71% of the 514 flags were submitted by autopilot without a model in the loop; the rest came from an agent's hand, a fixer's handler run by hand, or a probe.
who submitted the flag, by lane
autopilot
helper
parent
unattributed
lane fixer
machinery
web worker
autopilot is the no-model runner that claims every single at release, runs the lane solver and chains the tiers; a lane fixer is the agent that edits that solver; helpers work singles nobody owns; a parent routes and only submits a value it already holds.
median gap between a base's tiers, per day
single flag to time-trial flag
time-trial flag to mastery flag
the scale stops at five minutes; a dashed bar is a day beyond it. the gap is the time from one tier's flag to the next tier's flag, median over the day's bases. a long first gap is a fixer still writing the handler the tiers chain from.
autopilot picking up fixer saves, per day
saved fix re-run by autopilot
tiers waiting on a solver edit
agent cycles
every agent's shell goes through one logged session; the counts below are those $ lines, split by whether the day's board was still open.
14peak agents active in one minute
84lane fixers spawnedover the month
12median turns per fixerper day
11:40median fixer lifetimespawn to last turn
39commands refused on a solved tier
3kpython -c one-liners runscratch scripts, banned on the clock from day 23
shell commands logged per day
on the clock
after the board
distinct agents with a logged command, per day, by role
autopilot
lane fixer
helper
parent
web worker
web reasoner
web autopilot
reviewer
machinery
unattributed
the race commands agents ran most:
coord.py
runs
gm.py
runs
edit
3656
guess
206
wait-live
1227
probe
147
check
853
status
77
board
521
start
24
inbox
414
list
21
intel
412
submit
14
note
305
artifact
12
next
276
edge
9
retire
195
openapi
6
spawn
115
route
4
prompts and tokens
5.1k agent turns happened on the clock across the month, against 1.1b tokens; off the clock the runners spent 15.2b more on fixes, replays and writing. most of both figures is the prompt cache re-reading the same context every turn.
43.3moutput tokenswhat the models wrote
13.8bprompt-cache readscontext served from cache
207.9muncached inputfresh prompt tokens
21.3mthinking tokensa floor: not every turn reports them
agent turns on the clock, per day, by role
lane fixer
helper
parent
web worker
web reasoner
reviewer
machinery
tokens per day (input, cached and output), every runner
on the clock
after the board
off-clock tokens are the end-of-day fixes, replays, simulations and these writeups. day 30 carried the final rebuild of the race machinery.
what the agents asked their tools to do
shell
edit
read
write
spawn
other
non-web agents are the parent, the lane fixers and the helpers; web agents are both web runners' workers, reasoners and parents. 'other' is mostly agents waiting on or messaging each other.
seconds between one agent turn and the next, on the clock
median
90th percentile
a turn gap is model thinking plus the tool call it waited on. the watcher treats 300 s of silence from a fixer as a dead agent.
9 on-clock turn gaps ran past 300 seconds over the month.
only the web lane's late runner records a price per call: the 4 days it ran (days 16, 29, 30, 31) cost $94.52. the other runners' logs carry tokens, not dollars, and a price table would name the model, so no total is shown.
coordination traffic
agents share one checkout and talk through intel rows, mail, patches and takes; the grader client writes a row for every value it submits.
intel rows per day, by trust class
confirmed
rejected
observed
idea
pivot
confirmed and rejected rows are written by the grader client on every submit; observed, idea and pivot rows are agents writing to each other.
agent-to-agent mail per day
mail
urgent
9patches offered
7field families taken
26pivots posted
112assists
8guess caps hit25 rejected values on one field
the fields that ate the most rejected guesses:
field
lane
day
rejected values
d12_analysis_hcfg_storage
binary
17
1019
d14_analysis_hcfg_storage
binary
17
784
d03_analysis_format
binary
3
254
d01_analysis_symbols
binary
3
147
message
warmups
14
85
solver health
463replay fixtures
57klines in the solver library62 modules
2.1ktests
11race-day handler filesone per challenge that needed a fix
solver crash rows per day, by exception
KeyError
ValueError
other
a crash row is one field one solver run could not read; the runner posts them grouped so a fixer fixes every cause in one save.
replay fixtures, cumulative
a fixture is one graded run (its bytes, questions and accepted answers) that every later solver change replays against.
solver library commits per day
game master and network
1.6kweb lane gm calls
9gm error replies recorded409: 3, 429: 6
1gm incidents declared
47429 rate-limit repliesnone since 2026-10-13
game master call latency from the web lane, ms
median
90th percentile
the web lane logs every game master call with its wall time; dense from day 19.
web lane game master calls per day
calls
raised or failed
http requests to the live web targets per daymedian target response time, ms
where these numbers come from
race clock (flag ledger, timed runs, attempts, scoreboard) (days 1 to 31): the flag ledger (514 flags with the grader's timestamp), the saved timed runs under each challenge's files/runs, the attempts ledger and the public scoreboard snapshots. seconds count from the 09:00:00 et release; day 1 opened 2h28m late and is marked. the organizer's one-off static challenge is kept off every lane.
coordination (session log, intel, mail, crash log, git) (days 1 to 31): the race session log (a line per shell command, agent-keyed; archived day by day, days 2 and 3 partial, day 1 missing), the intel rows, the mail folder, patches, takes, the crash log and git history for the solver library. a day is the et calendar date of each row.
non-web agent transcripts (days 1 to 31): 113 top-level sessions and 441 spawned-agent transcripts of the non-web runner: 40,272 model turns after de-duplicating the streamed copies of each message. the reviewer model's own usage is left out (385 consultations, 94m input tokens). thinking tokens are a floor.
web agent transcripts, first runner (days 2 to 30): 201 session rollouts of the first web runner, days 2 to 30: tokens are each session's final cumulative count, split across days by the per-response records; forked sessions count their replayed turns once.
web agent transcripts, second runner (4 days (16, 29, 30, 31)): 51 session files of the second web runner (day 16 and days 29 to 31): per-message tokens, cost, wall time and time to first token. its auxiliary classifier calls are counted apart.
web lane event log (days 20 to 31): the web lane's event log (25,764 rows from 2026-10-22: target requests, game master calls, slots, attempts) and its own session log from day 20. nothing before day 20 was logged on that lane.
every figure is read from those files by blog/site/metrics.py; the files are written by the extractors in tools/after-action/telemetry/. a day a source lacks shows as zero, not as an estimate.