architecture: how a task gets from release to flag

  • 62solver-library modules57,767 lines
  • 2,079tests
  • 463replay fixtures
  • 9race-day handler files
  • 10tool articles

counted from the repository at every build.

two diagrams of the non-web race machinery: the pieces and who talks to whom, then one challenge from its release to its last flag with every fallback path. each box names the real file or command; the section numbers (3.6, 7.7, ...) point into the capabilities page, which describes each part in full. every game-master call goes through gm.py, the one client; on race day autopilot and kickoff make nearly all of them (the solid arrows), and a lane fixer, helper or parent reaches it only for a hand guess or probe (dashed): fixers fix solve.py, autopilot submits for them.

the pieces

components and who talks to whom game master api, behind a cloudflare edge list, start, download, submit, status. a start with run_for_time asks for the ranked timed pool; refused, the regular untimed pool plays instead. a per-team request limit answers 429s until the next minute. public scoreboard place, points and first bloods per question gm.py: the one gm client paired bearer on keep-alive sockets, the edge pin (3.6), every written form of an answer (gm.variants, 7.6), guess and probe, the team 429s hold (3.5); a CORRECT is written to logs/comp/flags.json kickoff.py T-5m fixer round T-65s edge pin T-60s archive T-20s warm slots T-0.3s hedged list plan, then the live marker autopilot.py a slot per lane and a spare: claim, scaffold, dry run, submit, chain the tiers; gaps rows; re-runs a saved fix lane fixer one per lane: wait-live --next, coord.py edit, handlers.py, replay of a failed run helpers coord.py next: one field family, a coord.py patch, an offline replay parent the spawn watcher, stopped and respawn, incidents; routes, never owns coord.py and sessions/: files are the bus tasks, claims, edit locks, intel rows (confirmed, rejected, observed, gaps), mail, liveness, the board, the planned and live markers, pivots, incidents, spawn reservations; one advisory lock, atomic writes challenges/dayNN/<id>/ solve.py (the lane's newest solver), handlers.py, carried.py, files/runs/<run>/ with every attempt's bytes ctfkit.hlib lane readers and registries, crosslane (7.5), the fixes mailbox, runner.run, locate (7.7), recipe (7.11) replay and fixtures uv run dryrun, uv run replay, tests/fixtures/ graded runs, autopilot.py --check, the nightly gate (6.5) dashboards and logs uv run dash, coord.py watch and board, uv run timeline, logs/comp/session.log and flags.json https list, warm, edge pin every start / submit a hand guess or probe only

the game master is the only grader. every call to it goes through one client, gm.py, so the rate limit, the edge pin and the answer forms live in one place. the actors in the middle row never talk to each other directly: they read and write files under sessions/ through coord.py (claims, edit locks, intel rows, mail, the board), and every solver imports the same shared library, ctfkit.hlib.

a task from release to flag

a task from release to flag, with every fallback release, 09:00 et kickoff.py sends the hedged list, plans the day and writes the live marker autopilot claims the single uv run newchal: solve.py is the lane's newest solver, beside the single's bytes autopilot dry run: the solver, offline is every field answered, and confident? gaps row, claim released a crash file:line per field; past 3s it says it is still searching, and the search goes on gm.py submit grouped; every accepted field is banked the lane fixer takes it wait-live --next prints the gaps and is-yours rows; one save of handlers.py under coord.py edit single CORRECT the flag goes to logs/comp/flags.json; the tiers run from the same solver autopilot re-runs the save within about a second: a full submit, or a new gaps row with what is left time_trial, timed, ranked runner.run starts, fetches every set, solves and submits inside the deadline single won by a hand probe autopilot follows it: solve.py must reproduce it, else hlib.locate, else CHAIN WAITS ON solve.py for the fixer crashed or rejected the run is saved under files/runs/<run>/; a gaps row names every lost field mastery refused until the time trial is solved; several sets, the same solver ranked start refused the regular untimed pool: WARNING unranked attempt; the end of day reruns it for a time the fixer replays it uv run replay <run>, fix, save: the live attempt's own bytes are re-solved in about a second the lane is done all three tiers are in flags.json the deadline passes a fresh attempt, at most twice a tier; a spent tier goes on as next_op: practice on the same bytes gm incident a 5xx on a timed start: solve.py checks itself offline and keeps its claim; the parent fixes gm.py, runs coord.py incident resolve, and every waiting solve.py re-execs rate-limit window 429s from the per-team limit last until the next minute: every process prints HOLD and resends then; nothing is killed fixer cut short coord.py stopped <name>: the spawn watcher respawns it with a RESUME note; with no parent alive, kickoff's fixer_watch starts fixer_launch.py headless stall 90s with no new confirmed field: coord.py pivot, and a fresh agent on the other runner tries a new method; 25 guesses a field; 5 min mails the parent yes no CORRECT CORRECT CORRECT

the middle column is the path a clean day takes: the release, autopilot's claim and dry run, one submit, correct, and the two timed tiers from the same solver. the boxes beside it are the fallbacks, each one a separate mechanism with its own trigger.

who does what, by role (the process names are the ids coord.py prints):

role process at the release when fields miss when a timed run fails never
kickoff tools/comp/kickoff.py, one per day pins the edge at t-65s, warms the route and starts one autopilot slot per lane at t-20s, sends the hedged list from t-0.3s, writes the plan and the live marker nothing: it has exited nothing solves or submits an answer
autopilot auto.<lane>, one slot per lane (tools/comp/autopilot.py), no model in the loop claims the single, scaffolds solve.py from the lane's newest solver, downloads the bytes, dry-runs the solver on them with no game-master call, and when every field is answered and confident submits once; on correct it starts the time trial and then the mastery itself submits what it has, banks the fields the grader accepts, posts a gaps row (each crash's file:line) or an is yours row (the question text, the closest readers, a handlers.py skeleton), releases the claim, then watches the solver files and re-runs a saved fix within about a second saves the run's bytes, re-solves the saved run with the fixer's fix, re-rolls the attempt at its deadline (at most twice) or plays the practice op waits on a human or a model
lane fixer <lane> fixer, one per lane, a model reads its lane brief before the release, then waits on coord.py wait-live --next takes the gaps / is yours row as its task: coord.py edit <id>, writes handlers.py (the computation that got the value is the rule), saves; autopilot submits for it; a value it already holds goes to gm.py guess at once uv run replay <run> on the saved bytes, fixes, saves; autopilot re-solves the live attempt starts a timed tier while autopilot owns the lane; works another lane
helper <runner>.wN, a model idle until coord.py next hands it work in no solver lane, or an assist on a held single one field family nobody is doing (coord.py take), a coord.py patch to the owner, an offline replay of someone else's failed run the same, as an offline assist a second copy of the owner's work
parent <runner>.main, a model spawns the fixers from the printed prompts, runs the spawn watcher, keeps kickoff.py alive, then polls wait-live for assists only routes: makes sure a helper holds the tier and the cause is in intel; a value it reads it submits at once reads the stall mail, respawns a fixer that was cut short, resolves a game-master incident owns a task; re-derives a fixer's fix
plumbing coord.py (claims, intel, mail, board, locks), gm.py (the one game-master client), ctfkit.hlib (the readers every lane shares)
fallback trigger what happens where
gaps, then the fixer a crash or an unsure field in the dry run accepted fields banked, a gaps row with each crash's file:line, the claim released; the fixer saves handlers.py and autopilot re-runs it within about a second tools/comp/autopilot.py, 7.8, 7.9
slow dry run still running after 3s autopilot posts that it is still searching and releases the claim; the search goes on and still submits autopilot.early_handoff (10)
a probe-won single anyone wins the single by hand with gm.py guess autopilot follows it and runs the tiers once solve.py reproduces every graded field; otherwise a located solver (hlib.locate, 7.7) or CHAIN WAITS ON solve.py for the fixer's edit 7.7, 4.3
a timed crash or rejection a set crashes or a submit comes back incorrect the run is saved under files/runs/<run>/; uv run replay <run> while the fixer fixes; the saved fix re-solves the live attempt's own bytes; past the deadline autopilot re-solves the saved run with the fix and runs the tier on a fresh attempt (a failure confined to one set of several is re-rolled at most twice a tier); a spent tier goes on as next_op: practice on the same bytes 7.2, 8.2
ranked start refused the game master will not open a ranked attempt the regular untimed pool (WARNING unranked attempt): no deadline, a wrong submit leaves the attempt open; the board marks it unranked and the end of day reruns it for a time gm.py start, ctfkit.hlib.runner
gm incident the first 5xx on a timed start no timed start goes out into it; every waiting solve.py checks its solver offline and keeps its claim; the parent fixes gm.py, proves it with tools/comp/preflight.py --incident, then coord.py incident resolve re-execs every waiting solver 10
rate-limit window 429s inside the release minute the window is per team and clears at the next :00; every process holds and resends then 3.5
fixer cut short a fixer exits before its lane is done coord.py stopped <name> frees the name; the spawn watcher prints it for a respawn with a resume note; with no parent alive, kickoff's fixer_watch launches tools/comp/fixer_launch.py 6.4
stall 90s with no new grader-confirmed field coord.py pivot <id> mails a state note and a fresh agent on the other runner starts from it with a different method; 25 rejected guesses a field; at 5 minutes coord.py check prints ceiling and the parent is mailed STALL_S, 4.3

{generated by a clanker}