metrics: what the month cost in turns, commands and seconds

the standings say where we finished; this page says what it took. every number below is counted from our own logs: the race session log, the flag ledger, the saved timed runs, the intel and mail the agents exchanged, and the transcripts of every agent that ran. the numbers are aggregated by day, lane and role; no agent, runner or model is named.

the month in eight numbers

  • 514flagssingles, time trials and masteries
  • 8:58median day: release to board completefastest 0:10
  • 5.1kagent turns on the clock42k more off it
  • 16.4btokens, every runnerin + cached + out
  • 42kshell commands logged
  • 22kintel rows shared
  • 29solver crash rows
  • 1.6kweb lane gm callslogged from day 20

an agent turn is one model reply; a shell command is one logged $ line in the race session log; 'on the clock' means between a day's 09:00 et release and its last flag.

the clock

the race is from the 09:00 et release to the team's last flag of the day. the non-web lanes (warmups, metadata, mobile, script, binary) ran on the solver pipeline with a fixer per lane; web ran its own machinery.

release to the last flag, per day
0:005:0010:0015:0020:001: non-web lanes done 3h04m (off the scale)2: non-web lanes done 18:473: non-web lanes done 14:264: non-web lanes done 11:065: non-web lanes done 3:356: non-web lanes done 1:457: non-web lanes done 4:448: non-web lanes done 12:439: non-web lanes done 4:1810: non-web lanes done 1:4111: non-web lanes done 0:1012: non-web lanes done 1:0613: non-web lanes done 1:5514: non-web lanes done 4:3615: non-web lanes done 2:3116: non-web lanes done 1:0217: non-web lanes done 1h45m (off the scale)18: non-web lanes done 0:4519: non-web lanes done 1:0420: non-web lanes done 1:0821: non-web lanes done 0:5622: non-web lanes done 3:5923: non-web lanes done 8:5824: non-web lanes done 1:0425: non-web lanes done 0:2426: non-web lanes done 3:1827: non-web lanes done 0:3128: non-web lanes done 31:20 (off the scale)29: non-web lanes done 1:1630: non-web lanes done 4:1031: non-web lanes done 3:4719: web done 17:5620: web done 12:5121: web done 27:16 (off the scale)22: web done 10:4023: web done 8:3224: web done 3:4725: web done 6:1526: web done 19:2227: web done 34:43 (off the scale)28: web done 3:4529: web done 34:39 (off the scale)30: web done 1:2331: web done 10:51135791113151719212325272931
  • non-web lanes done
  • web done

the scale stops at 20 minutes; a hollow dot is a day beyond it (hover for the time). day 1 opened late (first start 11:28) and is plotted from its own first start; every other day counts from the 09:00 et release. the web lane was first played on release day on day 19; the earlier web challenges were cleared as a backlog and count on the day they were won.

seconds from release to the single's flag, lane by day
12345678910111213141516171819202122232425262728293031warmupswarmups, 31: 2h30mwarmups, 31: 10:25warmups, 31: 0:26warmups, 31: 0:58warmups, 31: 0:02warmups, 31: 0:01warmups, 31: 0:05warmups, 31: 0:04warmups, 31: 0:04warmups, 31: 0:04warmups, 31: 0:01warmups, 31: 0:02warmups, 31: 0:02warmups, 31: 0:03warmups, 31: 0:02warmups, 31: 0:03warmups, 31: 0:42warmups, 31: 0:01warmups, 31: 0:26warmups, 31: 0:01warmups, 31: 0:02warmups, 31: 0:02warmups, 31: 0:01warmups, 31: 0:21warmups, 31: 0:01warmups, 31: 0:02warmups, 31: 0:02warmups, 31: 0:03warmups, 31: 0:02warmups, 31: 0:02warmups, 31: 0:02metadatametadata, 31: 2h30mmetadata, 31: 9:06metadata, 31: 0:28metadata, 31: 1:12metadata, 31: 0:02metadata, 31: 1:04metadata, 31: 1:05metadata, 31: 1:06metadata, 31: 0:26metadata, 31: 1:08metadata, 31: 0:01metadata, 31: 1:03metadata, 31: 1:26metadata, 31: 0:18metadata, 31: 0:23metadata, 31: 0:05metadata, 31: 0:18metadata, 31: 0:18metadata, 31: 0:02metadata, 31: 0:38metadata, 31: 0:30metadata, 31: 0:02metadata, 31: 0:15metadata, 31: 0:02metadata, 31: 0:18metadata, 31: 1:28metadata, 31: 0:03metadata, 31: 8:16metadata, 31: 0:03metadata, 31: 0:56metadata, 31: 1:11mobilemobile, 31: 2h48mmobile, 31: 9:56mobile, 31: 0:30mobile, 31: 0:49mobile, 31: 1:16mobile, 31: 0:02mobile, 31: 0:04mobile, 31: 0:04mobile, 31: 0:04mobile, 31: 0:04mobile, 31: 0:01mobile, 31: 0:02mobile, 31: 0:02mobile, 31: 0:02mobile, 31: 0:02mobile, 31: 0:03mobile, 31: 0:19mobile, 31: 0:01mobile, 31: 0:02mobile, 31: 0:02mobile, 31: 0:01mobile, 31: 0:02mobile, 31: 0:02mobile, 31: 0:02mobile, 31: 0:02mobile, 31: 0:29mobile, 31: 0:02mobile, 31: 14:12mobile, 31: 0:17mobile, 31: 0:03mobile, 31: 0:03scriptscript, 31: 2h45mscript, 31: 12:35script, 31: 7:18script, 31: 3:27script, 31: 3:29script, 31: 1:27script, 31: 2:25script, 31: 2:07script, 31: 0:05script, 31: 0:05script, 31: 0:02script, 31: 0:03script, 31: 0:03script, 31: 0:04script, 31: 0:03script, 31: 0:05script, 31: 0:03script, 31: 0:03script, 31: 0:05script, 31: 0:02script, 31: 0:02script, 31: 0:04script, 31: 0:03script, 31: 0:04script, 31: 0:04script, 31: 0:03script, 31: 0:04script, 31: 0:04script, 31: 0:04script, 31: 0:04script, 31: 0:03binarybinary, 31: 2h43mbinary, 31: 7:10binary, 31: 9:37binary, 31: 2:31binary, 31: 0:28binary, 31: 1:40binary, 31: 0:05binary, 31: 1:54binary, 31: 1:03binary, 31: 1:02binary, 31: 0:02binary, 31: 0:06binary, 31: 0:05binary, 31: 0:05binary, 31: 1:02binary, 31: 0:04binary, 31: 1h41mbinary, 31: 0:03binary, 31: 0:54binary, 31: 0:42binary, 31: 0:22binary, 31: 0:08binary, 31: 0:04binary, 31: 0:07binary, 31: 0:06binary, 31: 1:02binary, 31: 0:09binary, 31: 13:37binary, 31: 0:07binary, 31: 3:58binary, 31: 1:08

tint by quintile: the lightest fifth of cells are the fastest singles, the darkest fifth the slowest. hover a cell for the time. a dashed cell is a day that lane had no flag recorded.

lane median single median time trial median mastery best single
warmups 0:02 0:05 0:10 0:01 (day 20)
metadata 0:28 1:01 1:04 0:01 (day 11)
mobile 0:03 0:05 0:08 0:01 (day 18)
script 0:04 0:07 0:12 0:02 (day 11)
binary 0:42 0:58 1:04 0:02 (day 11)
web 6:02 8:32 10:51 1:04 (day 30)

timed tiers

a time trial or mastery attempt runs against a deadline the game master sets when the clock starts; the runner solves every answer set in parallel and submits once.

  • 383timed runs saved
  • 50smedian deadline given
  • 1.8smedian seconds used
  • 83%runs that came back correct
timed-tier runs per day, by outcome
0510152013: correct 103: incorrect 53: expired 134: correct 104: incorrect 45: correct 1056: correct 107: correct 107: crash 378: correct 108: incorrect 28: crash 19: correct 109: crash 1910: correct 1011: correct 101112: correct 1013: correct 101314: correct 1014: incorrect 115: correct 101516: correct 1017: correct 91718: correct 1019: correct 101920: correct 1320: crash 421: correct 102122: correct 1223: correct 1023: incorrect 123: crash 72324: correct 1024: crash 225: correct 1325: crash 22526: correct 1326: crash 227: correct 1427: crash 22728: correct 1428: crash 429: correct 1329: incorrect 129: crash 12930: correct 1431: correct 1431: crash 231
  • correct
  • incorrect
  • expired
  • crash

who took the flag

71% of the 514 flags were submitted by autopilot without a model in the loop; the rest came from an agent's hand, a fixer's handler run by hand, or a probe.

who submitted the flag, by lane
warmupswarmups: autopilot 73 (78%)78%warmups: helper 10 (11%)11%warmups: parent 5 (5%)warmups: unattributed 3 (3%)warmups: lane fixer 2 (2%)metadatametadata: autopilot 52 (56%)56%metadata: helper 24 (26%)26%metadata: lane fixer 7 (8%)8%metadata: parent 7 (8%)8%metadata: unattributed 3 (3%)mobilemobile: autopilot 72 (77%)77%mobile: helper 8 (9%)9%mobile: parent 6 (6%)6%mobile: lane fixer 4 (4%)mobile: unattributed 3 (3%)scriptscript: autopilot 69 (74%)74%script: helper 12 (13%)13%script: parent 8 (9%)9%script: unattributed 3 (3%)script: machinery 1 (1%)binarybinary: autopilot 60 (65%)65%binary: helper 21 (23%)23%binary: parent 7 (8%)8%binary: unattributed 3 (3%)binary: lane fixer 2 (2%)webweb: autopilot 38 (81%)81%web: web worker 7 (15%)15%web: parent 2 (4%)
  • autopilot
  • helper
  • parent
  • unattributed
  • lane fixer
  • machinery
  • web worker

autopilot is the no-model runner that claims every single at release, runs the lane solver and chains the tiers; a lane fixer is the agent that edits that solver; helpers work singles nobody owns; a parent routes and only submits a value it already holds.

median gap between a base's tiers, per day
0:001:152:303:455:001: single flag to time-trial flag 18:04 (off the scale)1: time-trial flag to mastery flag 2:5212: single flag to time-trial flag 1:112: time-trial flag to mastery flag 1:143: single flag to time-trial flag 0:203: time-trial flag to mastery flag 0:2534: single flag to time-trial flag 0:374: time-trial flag to mastery flag 0:335: single flag to time-trial flag 0:075: time-trial flag to mastery flag 0:0256: single flag to time-trial flag 0:036: time-trial flag to mastery flag 0:027: single flag to time-trial flag 0:037: time-trial flag to mastery flag 0:0678: single flag to time-trial flag 4:368: time-trial flag to mastery flag 0:059: single flag to time-trial flag 0:029: time-trial flag to mastery flag 0:05910: single flag to time-trial flag 0:0210: time-trial flag to mastery flag 0:0111: single flag to time-trial flag 0:0211: time-trial flag to mastery flag 0:021112: single flag to time-trial flag 0:0212: time-trial flag to mastery flag 0:0213: single flag to time-trial flag 0:0313: time-trial flag to mastery flag 0:021314: single flag to time-trial flag 0:0214: time-trial flag to mastery flag 0:0415: single flag to time-trial flag 0:0315: time-trial flag to mastery flag 0:031516: single flag to time-trial flag 0:0316: time-trial flag to mastery flag 0:0617: single flag to time-trial flag 0:2717: time-trial flag to mastery flag 0:021718: single flag to time-trial flag 0:0418: time-trial flag to mastery flag 0:0219: single flag to time-trial flag 0:0319: time-trial flag to mastery flag 0:031920: single flag to time-trial flag 0:0320: time-trial flag to mastery flag 0:0521: single flag to time-trial flag 0:0321: time-trial flag to mastery flag 0:032122: single flag to time-trial flag 0:0422: time-trial flag to mastery flag 0:0723: single flag to time-trial flag 1:1323: time-trial flag to mastery flag 0:022324: single flag to time-trial flag 0:0524: time-trial flag to mastery flag 0:0425: single flag to time-trial flag 0:0325: time-trial flag to mastery flag 0:042526: single flag to time-trial flag 0:0426: time-trial flag to mastery flag 0:0827: single flag to time-trial flag 0:0327: time-trial flag to mastery flag 0:062728: single flag to time-trial flag 0:1128: time-trial flag to mastery flag 0:0729: single flag to time-trial flag 0:0429: time-trial flag to mastery flag 0:082930: single flag to time-trial flag 0:0430: time-trial flag to mastery flag 0:0831: single flag to time-trial flag 0:0731: time-trial flag to mastery flag 0:0731
  • single flag to time-trial flag
  • time-trial flag to mastery flag

the scale stops at five minutes; a dashed bar is a day beyond it. the gap is the time from one tier's flag to the next tier's flag, median over the day's bases. a long first gap is a fixer still writing the handler the tiers chain from.

autopilot picking up fixer saves, per day
01.22.53.851: saved fix re-run by autopilot 01: tiers waiting on a solver edit 012: saved fix re-run by autopilot 02: tiers waiting on a solver edit 03: saved fix re-run by autopilot 03: tiers waiting on a solver edit 034: saved fix re-run by autopilot 04: tiers waiting on a solver edit 05: saved fix re-run by autopilot 05: tiers waiting on a solver edit 056: saved fix re-run by autopilot 06: tiers waiting on a solver edit 07: saved fix re-run by autopilot 07: tiers waiting on a solver edit 078: saved fix re-run by autopilot 08: tiers waiting on a solver edit 09: saved fix re-run by autopilot 09: tiers waiting on a solver edit 0910: saved fix re-run by autopilot 010: tiers waiting on a solver edit 011: saved fix re-run by autopilot 011: tiers waiting on a solver edit 01112: saved fix re-run by autopilot 012: tiers waiting on a solver edit 013: saved fix re-run by autopilot 013: tiers waiting on a solver edit 01314: saved fix re-run by autopilot 014: tiers waiting on a solver edit 015: saved fix re-run by autopilot 015: tiers waiting on a solver edit 21516: saved fix re-run by autopilot 016: tiers waiting on a solver edit 017: saved fix re-run by autopilot 117: tiers waiting on a solver edit 41718: saved fix re-run by autopilot 018: tiers waiting on a solver edit 119: saved fix re-run by autopilot 019: tiers waiting on a solver edit 11920: saved fix re-run by autopilot 020: tiers waiting on a solver edit 121: saved fix re-run by autopilot 121: tiers waiting on a solver edit 12122: saved fix re-run by autopilot 122: tiers waiting on a solver edit 023: saved fix re-run by autopilot 323: tiers waiting on a solver edit 02324: saved fix re-run by autopilot 124: tiers waiting on a solver edit 125: saved fix re-run by autopilot 425: tiers waiting on a solver edit 02526: saved fix re-run by autopilot 126: tiers waiting on a solver edit 127: saved fix re-run by autopilot 327: tiers waiting on a solver edit 02728: saved fix re-run by autopilot 228: tiers waiting on a solver edit 329: saved fix re-run by autopilot 529: tiers waiting on a solver edit 12930: saved fix re-run by autopilot 230: tiers waiting on a solver edit 031: saved fix re-run by autopilot 331: tiers waiting on a solver edit 231
  • saved fix re-run by autopilot
  • tiers waiting on a solver edit

agent cycles

every agent's shell goes through one logged session; the counts below are those $ lines, split by whether the day's board was still open.

  • 14peak agents active in one minute
  • 84lane fixers spawnedover the month
  • 12median turns per fixerper day
  • 11:40median fixer lifetimespawn to last turn
  • 39commands refused on a solved tier
  • 3kpython -c one-liners runscratch scripts, banned on the clock from day 23
shell commands logged per day
02.5k5k7.5k10k12: after the board 1853: on the clock 5613: after the board 54634: on the clock 6054: after the board 8555: on the clock 1525: after the board 62356: on the clock 1196: after the board 4477: on the clock 1757: after the board 89878: on the clock 4208: after the board 1.5k9: on the clock 1649: after the board 2k910: on the clock 15310: after the board 2.1k11: on the clock 411: after the board 9031112: on the clock 4812: after the board 49113: on the clock 10213: after the board 8891314: on the clock 20814: after the board 1.1k15: on the clock 4515: after the board 1.1k1516: on the clock 1316: after the board 77617: on the clock 85217: after the board 1.4k1718: on the clock 2218: after the board 87219: on the clock 3719: after the board 8671920: on the clock 6320: after the board 54321: on the clock 4721: after the board 5842122: on the clock 13522: after the board 57123: on the clock 25723: after the board 9802324: on the clock 3624: after the board 46525: on the clock 3925: after the board 8102526: on the clock 18026: after the board 82327: on the clock 4227: after the board 6232728: on the clock 15728: after the board 3.4k29: on the clock 49829: after the board 1.2k2930: on the clock 9530: after the board 7.5k31: on the clock 34231: after the board 87331
  • on the clock
  • after the board
distinct agents with a logged command, per day, by role
05010015020012: lane fixer 12: helper 12: parent 12: machinery 32: unattributed 13: helper 43: parent 23: machinery 103: unattributed 134: autopilot 64: helper 74: parent 24: reviewer 14: machinery 34: unattributed 15: autopilot 55: helper 85: parent 25: machinery 35: unattributed 156: autopilot 56: helper 96: parent 26: machinery 16: unattributed 17: autopilot 67: helper 77: parent 27: reviewer 17: machinery 27: unattributed 178: autopilot 58: helper 98: parent 28: machinery 78: unattributed 19: autopilot 59: helper 89: parent 29: reviewer 29: machinery 169: unattributed 1910: autopilot 510: helper 910: parent 210: reviewer 110: machinery 1210: unattributed 1311: autopilot 1111: helper 811: parent 211: reviewer 111: machinery 711: unattributed 131112: autopilot 1112: helper 812: parent 212: reviewer 112: machinery 512: unattributed 713: autopilot 1113: helper 813: parent 213: reviewer 113: machinery 613: unattributed 111314: autopilot 1114: helper 814: parent 214: reviewer 214: machinery 414: unattributed 815: autopilot 1115: lane fixer 515: parent 215: reviewer 115: machinery 615: unattributed 41516: autopilot 1116: lane fixer 516: parent 216: reviewer 116: machinery 816: unattributed 917: autopilot 1217: lane fixer 517: helper 1217: parent 217: web worker 117: reviewer 117: machinery 817: unattributed 81718: autopilot 1218: lane fixer 518: parent 218: web worker 118: reviewer 118: machinery 718: unattributed 219: autopilot 1219: lane fixer 519: parent 219: web worker 819: reviewer 119: machinery 1019: unattributed 51920: autopilot 1220: lane fixer 520: parent 220: web worker 320: web autopilot 220: reviewer 220: machinery 520: unattributed 121: autopilot 1221: lane fixer 521: helper 121: parent 221: web worker 421: web autopilot 421: reviewer 121: machinery 521: unattributed 42122: autopilot 1222: lane fixer 522: parent 222: web worker 322: web autopilot 322: reviewer 322: machinery 622: unattributed 323: autopilot 1223: lane fixer 523: parent 223: web worker 323: web autopilot 423: reviewer 223: machinery 1223: unattributed 12324: autopilot 1224: lane fixer 524: parent 224: web worker 324: web autopilot 324: reviewer 224: machinery 725: autopilot 1225: lane fixer 525: parent 225: web worker 625: web autopilot 425: reviewer 125: machinery 625: unattributed 52526: autopilot 1226: lane fixer 526: parent 226: web worker 626: web reasoner 326: web autopilot 326: reviewer 126: machinery 626: unattributed 727: autopilot 1227: lane fixer 527: parent 227: web worker 527: web autopilot 527: reviewer 227: machinery 527: unattributed 242728: autopilot 1228: lane fixer 428: helper 328: parent 228: web worker 328: web autopilot 328: reviewer 328: machinery 2028: unattributed 1129: autopilot 1229: lane fixer 529: helper 329: parent 329: web worker 529: web autopilot 429: reviewer 329: machinery 829: unattributed 32930: autopilot 1230: lane fixer 730: parent 330: web worker 330: web reasoner 330: web autopilot 330: machinery 5730: unattributed 1331: autopilot 1231: lane fixer 531: parent 231: web worker 331: web reasoner 331: web autopilot 331: machinery 931
  • autopilot
  • lane fixer
  • helper
  • parent
  • web worker
  • web reasoner
  • web autopilot
  • reviewer
  • machinery
  • unattributed

the race commands agents ran most:

coord.py runs gm.py runs
edit 3656 guess 206
wait-live 1227 probe 147
check 853 status 77
board 521 start 24
inbox 414 list 21
intel 412 submit 14
note 305 artifact 12
next 276 edge 9
retire 195 openapi 6
spawn 115 route 4

prompts and tokens

5.1k agent turns happened on the clock across the month, against 1.1b tokens; off the clock the runners spent 15.2b more on fixes, replays and writing. most of both figures is the prompt cache re-reading the same context every turn.

  • 43.3moutput tokenswhat the models wrote
  • 13.8bprompt-cache readscontext served from cache
  • 207.9muncached inputfresh prompt tokens
  • 21.3mthinking tokensa floor: not every turn reports them
agent turns on the clock, per day, by role
02505007501k1: helper 781: parent 8412: lane fixer 142: helper 1942: parent 492: machinery 63: helper 2443: parent 1123: machinery 1934: helper 2054: parent 344: reviewer 55: helper 1205: parent 1356: helper 806: parent 187: helper 1357: parent 3878: helper 2408: parent 629: helper 1109: parent 159: reviewer 1910: helper 8610: parent 141112: helper 3012: parent 213: helper 3913: parent 121314: helper 16214: parent 3414: reviewer 515: lane fixer 3515: parent 141516: lane fixer 1316: parent 317: lane fixer 27417: helper 44617: parent 19617: web worker 917: machinery 181718: lane fixer 1518: parent 519: lane fixer 2819: parent 619: web worker 11920: lane fixer 5420: parent 6020: web worker 2421: lane fixer 2321: parent 621: web worker 232122: lane fixer 3622: parent 5022: web worker 1122: web reasoner 822: reviewer 823: lane fixer 12323: parent 3023: web worker 1023: web reasoner 82324: lane fixer 2724: parent 624: web worker 225: lane fixer 1525: parent 1025: web worker 525: reviewer 32526: lane fixer 5326: parent 7926: web worker 2326: web reasoner 227: lane fixer 1427: parent 2227: web worker 662728: lane fixer 8228: parent 10428: web worker 928: machinery 929: lane fixer 2529: helper 1429: parent 2529: web worker 342930: lane fixer 4530: parent 3830: web worker 5630: web reasoner 331: lane fixer 4631: parent 29531: web worker 17331: web reasoner 3431
  • lane fixer
  • helper
  • parent
  • web worker
  • web reasoner
  • reviewer
  • machinery
tokens per day (input, cached and output), every runner
01.2b2.5b3.8b5b1: on the clock 51.9m1: after the board 37.2m12: on the clock 33.4m2: after the board 90.6m3: on the clock 81.4m3: after the board 212.1m34: on the clock 62.7m4: after the board 414.5m5: on the clock 10.5m5: after the board 194.4m56: on the clock 7.6m6: after the board 133.4m7: on the clock 15.2m7: after the board 336.8m78: on the clock 39.6m8: after the board 586m9: on the clock 13.7m9: after the board 564.8m910: on the clock 12m10: after the board 614.9m11: on the clock 67k11: after the board 259.4m1112: on the clock 3m12: after the board 147.4m13: on the clock 7.9m13: after the board 344.7m1314: on the clock 34.6m14: after the board 568.9m15: on the clock 5.5m15: after the board 454.5m1516: on the clock 1.7m16: after the board 328.8m17: on the clock 209.3m17: after the board 707.3m1718: on the clock 2.1m18: after the board 310.9m19: on the clock 3m19: after the board 725.1m1920: on the clock 63.5m20: after the board 379.5m21: on the clock 5.9m21: after the board 307.7m2122: on the clock 52.1m22: after the board 553.2m23: on the clock 49.2m23: after the board 460.1m2324: on the clock 7.2m24: after the board 303.9m25: on the clock 27.5m25: after the board 559m2526: on the clock 24.2m26: after the board 524.1m27: on the clock 37.6m27: after the board 320m2728: on the clock 39.2m28: after the board 1.2b29: on the clock 125.5m29: after the board 933.1m2930: on the clock 15m30: after the board 2.6b31: on the clock 89.1m31: after the board 143.2m31
  • on the clock
  • after the board

off-clock tokens are the end-of-day fixes, replays, simulations and these writeups. day 30 carried the final rebuild of the race machinery.

what the agents asked their tools to do
non-web agentsnon-web agents: shell 37k (83%)83%non-web agents: edit 3.1k (7%)7%non-web agents: read 2.1k (5%)non-web agents: write 826 (2%)non-web agents: spawn 444 (1%)non-web agents: other 1.1k (2%)web agentsweb agents: shell 17k (87%)87%web agents: edit 516 (3%)web agents: read 778 (4%)web agents: write 84 (0%)web agents: spawn 77 (0%)web agents: other 1.2k (6%)
  • shell
  • edit
  • read
  • write
  • spawn
  • other

non-web agents are the parent, the lane fixers and the helpers; web agents are both web runners' workers, reasoners and parents. 'other' is mostly agents waiting on or messaging each other.

seconds between one agent turn and the next, on the clock
0:000:120:250:380:501: median 0:212: median 0:063: median 0:074: median 0:055: median 0:056: median 0:047: median 0:068: median 0:059: median 0:0510: median 0:0411: median 0:0012: median 0:0413: median 0:0614: median 0:0415: median 0:0516: median 0:0517: median 0:1118: median 0:0519: median 0:0420: median 0:0621: median 0:0522: median 0:0723: median 0:0624: median 0:0525: median 0:0526: median 0:1027: median 0:1128: median 0:0929: median 0:0630: median 0:0731: median 0:061: 90th percentile 0:422: 90th percentile 0:163: 90th percentile 0:224: 90th percentile 0:115: 90th percentile 0:126: 90th percentile 0:077: 90th percentile 0:108: 90th percentile 0:279: 90th percentile 0:1510: 90th percentile 0:0711: 90th percentile 0:0012: 90th percentile 0:0813: 90th percentile 0:1114: 90th percentile 0:1115: 90th percentile 0:1916: 90th percentile 0:1317: 90th percentile 0:5018: 90th percentile 0:0819: 90th percentile 0:1020: 90th percentile 0:1821: 90th percentile 0:1122: 90th percentile 0:2923: 90th percentile 0:3424: 90th percentile 0:1325: 90th percentile 0:0726: 90th percentile 0:4327: 90th percentile 0:3728: 90th percentile 0:4629: 90th percentile 0:1030: 90th percentile 0:1531: 90th percentile 0:20135791113151719212325272931
  • median
  • 90th percentile

a turn gap is model thinking plus the tool call it waited on. the watcher treats 300 s of silence from a fixer as a dead agent.

9 on-clock turn gaps ran past 300 seconds over the month.

only the web lane's late runner records a price per call: the 4 days it ran (days 16, 29, 30, 31) cost $94.52. the other runners' logs carry tokens, not dollars, and a price table would name the model, so no total is shown.

coordination traffic

agents share one checkout and talk through intel rows, mail, patches and takes; the grader client writes a row for every value it submits.

intel rows per day, by trust class
05k10k15k20k13: confirmed 2263: rejected 4503: observed 93: idea 734: confirmed 1584: rejected 304: observed 105: confirmed 3575: observed 656: confirmed 1026: rejected 56: observed 87: confirmed 2467: rejected 147: observed 778: confirmed 4818: rejected 298: observed 169: confirmed 1229: observed 7910: confirmed 12010: rejected 110: observed 1010: idea 111: confirmed 211112: confirmed 2112: observed 413: confirmed 2113: rejected 513: observed 81314: confirmed 5814: rejected 8514: observed 1615: confirmed 17815: rejected 2315: observed 51516: confirmed 2716: rejected 4416: observed 217: confirmed 1.2k17: rejected 1.9k17: observed 4917: pivot 141718: confirmed 2018: observed 419: confirmed 24219: observed 151920: confirmed 26920: rejected 220: observed 3920: pivot 121: confirmed 26621: observed 2421: idea 22122: confirmed 13222: rejected 2222: observed 922: pivot 323: confirmed 11k23: rejected 3023: observed 3423: pivot 12324: confirmed 2524: observed 2225: confirmed 2625: observed 2425: pivot 22526: confirmed 34226: observed 2926: pivot 127: confirmed 3027: observed 4127: idea 227: pivot 12728: confirmed 2.4k28: rejected 2828: observed 3129: confirmed 3029: rejected 129: observed 2229: pivot 12930: confirmed 44830: rejected 1230: observed 31630: idea 130: pivot 131: confirmed 53531: rejected 731: observed 2731: idea 131: pivot 131
  • confirmed
  • rejected
  • observed
  • idea
  • pivot

confirmed and rejected rows are written by the grader client on every submit; observed, idea and pivot rows are agents writing to each other.

agent-to-agent mail per day
025507510012: mail 253: mail 503: urgent 234: mail 84: urgent 555: mail 55: urgent 1056: mail 16: urgent 177: mail 27: urgent 2978: mail 28: urgent 449: mail 89: urgent 5910: mail 410: urgent 61112: mail 112: urgent 113: mail 613: urgent 11314: mail 2014: urgent 181516: urgent 317: mail 617: urgent 211718: urgent 119: mail 119: urgent 31920: mail 2320: urgent 4921: mail 721: urgent 62122: mail 522: urgent 823: mail 323: urgent 372324: mail 824: urgent 1325: mail 325: urgent 122526: urgent 2227: mail 627: urgent 452728: mail 628: urgent 1729: mail 1029: urgent 212930: mail 3330: urgent 631: mail 531: urgent 2431
  • mail
  • urgent
  • 9patches offered
  • 7field families taken
  • 26pivots posted
  • 112assists
  • 8guess caps hit25 rejected values on one field

the fields that ate the most rejected guesses:

field lane day rejected values
d12_analysis_hcfg_storage binary 17 1019
d14_analysis_hcfg_storage binary 17 784
d03_analysis_format binary 3 254
d01_analysis_symbols binary 3 147
message warmups 14 85

solver health

  • 463replay fixtures
  • 57klines in the solver library62 modules
  • 2.1ktests
  • 11race-day handler filesone per challenge that needed a fix
solver crash rows per day, by exception
02.557.51013579111314: KeyError 21516: KeyError 11718: KeyError 119: KeyError 41921: KeyError 42122: KeyError 72324: KeyError 225: KeyError 12526: ValueError 127: KeyError 12728: ValueError 429: KeyError 12931
  • KeyError
  • ValueError
  • other

a crash row is one field one solver run could not read; the runner posts them grouped so a fixer fixes every cause in one save.

replay fixtures, cumulative
01252503755001: replay fixtures 52: replay fixtures 113: replay fixtures 264: replay fixtures 445: replay fixtures 616: replay fixtures 767: replay fixtures 918: replay fixtures 1069: replay fixtures 12410: replay fixtures 14111: replay fixtures 15812: replay fixtures 17513: replay fixtures 19214: replay fixtures 20815: replay fixtures 22316: replay fixtures 23817: replay fixtures 25318: replay fixtures 26819: replay fixtures 28320: replay fixtures 29821: replay fixtures 31322: replay fixtures 32823: replay fixtures 34324: replay fixtures 35825: replay fixtures 37326: replay fixtures 38827: replay fixtures 40328: replay fixtures 41829: replay fixtures 43330: replay fixtures 44831: replay fixtures 463135791113151719212325272931

a fixture is one graded run (its bytes, questions and accepted answers) that every later solver change replays against.

solver library commits per day
02550751001: 012: 03: 034: 75: 256: 47: 378: 239: 29910: 3311: 81112: 513: 121314: 2215: 141516: 1017: 171718: 2019: 101920: 1221: 62122: 1723: 132324: 925: 102526: 727: 52728: 1729: 102930: 5631: 131

game master and network

  • 1.6kweb lane gm calls
  • 9gm error replies recorded409: 3, 429: 6
  • 1gm incidents declared
  • 47429 rate-limit repliesnone since 2026-10-13
game master call latency from the web lane, ms
01.2k2.5k3.8k5k1: median 02: median 03: median 04: median 05: median 06: median 07: median 08: median 09: median 010: median 011: median 012: median 013: median 014: median 015: median 016: median 017: median 018: median 019: median 020: median 021: median 022: median 475.923: median 443.824: median 580.925: median 482.426: median 528.427: median 496.928: median 1k29: median 647.330: median 578.731: median 467.11: 90th percentile 02: 90th percentile 03: 90th percentile 04: 90th percentile 05: 90th percentile 06: 90th percentile 07: 90th percentile 08: 90th percentile 09: 90th percentile 010: 90th percentile 011: 90th percentile 012: 90th percentile 013: 90th percentile 014: 90th percentile 015: 90th percentile 016: 90th percentile 017: 90th percentile 018: 90th percentile 019: 90th percentile 020: 90th percentile 021: 90th percentile 022: 90th percentile 3.2k23: 90th percentile 1.2k24: 90th percentile 1.8k25: 90th percentile 2.9k26: 90th percentile 1.1k27: 90th percentile 1.6k28: 90th percentile 3.2k29: 90th percentile 1.6k30: 90th percentile 2.4k31: 90th percentile 1.5k135791113151719212325272931
  • median
  • 90th percentile

the web lane logs every game master call with its wall time; dense from day 19.

web lane game master calls per day
01252503755001357911131517192122: calls 10322: raised or failed 1423: calls 12123: raised or failed 212324: calls 7924: raised or failed 1025: calls 6425: raised or failed 52526: calls 12026: raised or failed 1327: calls 13427: raised or failed 242728: calls 20028: raised or failed 5329: calls 18829: raised or failed 142930: calls 29130: raised or failed 5431: calls 6231: raised or failed 131
  • calls
  • raised or failed
http requests to the live web targets per day
01.2k2.5k3.8k5k1: 012: 03: 034: 05: 056: 07: 078: 09: 0910: 011: 01112: 013: 01314: 015: 01516: 017: 01718: 019: 01920: 021: 02122: 45823: 1862324: 17125: 3782526: 15227: 3252728: 76329: 3532930: 1.2k31: 3.3k31
median target response time, ms
0501001502001: median 02: median 03: median 04: median 05: median 06: median 07: median 08: median 09: median 010: median 011: median 012: median 013: median 014: median 015: median 016: median 017: median 018: median 019: median 020: median 021: median 022: median 1.623: median 113.724: median 114.425: median 7.826: median 102.527: median 107.528: median 132.529: median 133.330: median 103.931: median 87.5135791113151719212325272931

where these numbers come from

every figure is read from those files by blog/site/metrics.py; the files are written by the extractors in tools/after-action/telemetry/. a day a source lacks shows as zero, not as an estimate.

{generated by a clanker}