tooling: solver pipeline and timed-attempt runner
the event had three modes for each challenge. an untimed single specimen taught the format; a time trial rotated the specimen under a short deadline; mastery assigned several answer sets under one deadline. we built one orchestration layer so artifact parsers did not need to know which mode they were solving.
the parser contract
every challenge parser implemented the equivalent of:
def solve_set(
files: dict[str, bytes],
fields: list[str],
questions: list[dict],
) -> dict[str, str]:
...
we called this adapter solve_set; the name is included here only to explain the architecture used throughout the writeups. files contains one set's raw and extracted files. fields contains local answer names with an outer setN. prefix removed. questions carries hints such as an artifact name, but fields are authoritative because question text could remain stale after a retry. the function must return every field exactly once as a non-empty string.
this boundary keeps network calls, clocks, and packaging out of analysis code. for example, an apk parser only sees one apk and a local field named answer, whether the server assigned answer or set3.answer.
normalizing delivery layouts
the server used two layouts:
- one download per answer set, with an explicit set number; or
- one combined bundle containing members such as
evidence-02.zip,slot-2.elf, orblob_2.txt.
for direct deliveries, files are grouped by the supplied set number. for a bundle, each member is expanded once and an explicit filename marker assigns it to a set. a bare long number is never treated as a set marker because years, campaign ids, and hashes commonly contain digits. if no explicit marker exists, the whole bundle is presented to the parser, which must resolve members using field and question evidence.
conceptually:
def split_sets(answer_ids, downloads):
sets = set_numbers_from(answer_ids)
if every_download_has_a_set_number(downloads):
return group_by_assigned_set(downloads)
members = expand_each_bundle_member_once(downloads)
return {
number: members_marked_for(number) or members
for number in sets
}
mapping local answers back to server ids
an id parser separates setN., optional slotM_, and the field name. the runner calls the parser once per set with names such as slot2_marker, then maps returned values back to their original ids. before submission it verifies:
set(answers) == set(assigned_answer_ids)
all(value.strip() for value in answers.values())
this check is deliberately strict. a partial or misspelled answer map wastes a timed attempt while teaching nothing about the artifact.
timed execution
the runner performs this sequence:
- start or resume the attempt.
- fetch assigned artifacts and verify supplied sha-256 values.
- split deliveries into answer sets.
- run pure parsers and validate the complete answer map.
- submit once.
- if the attempt expires during fetch or submit, request new material and repeat from the new assignment rather than reusing stale files.
the parser is tested on known specimens before a timed attempt. this matters because starting a timed challenge can rotate material, and debugging after the clock begins is expensive.
why this mattered
most daily variants changed packaging, prefixes, or counts rather than the core artifact format. a transport-neutral parser plus a layout-aware runner let the same analytical code survive all three modes. it also made failures legible: member placement, artifact parsing, answer formatting, and network timing were separate layers rather than one large script.
day 7 additions
nested archives per set
a mastery bundle with several sets used to be split and flattened into each set's members. when a field named a zip inside its set, that zip was gone and a text file was read instead. each set now keeps every nested archive's own bytes alongside its unpacked members, so an archive stays addressable by name.
day 8 additions
reading the answer format
the answer format's syntax text is read clause by clause, negations first, because a substring check misreads a negated clause. each clause yields a prefix rule, a case rule or a separator; a clause that reads two ways is reported as ambiguous and nothing is rewritten for it. a value that breaks the format its question asks for is converted in one place for every mode ([redacted]).
field names without set prefixes
parsers receive field names with the setN. prefix removed in every mode, so a rule keyed on a field name holds in the mastery exactly as in the single.
day 9 additions
failures recorded per set
a parser that raises on one answer set used to lose the whole run's answers with it, and the saved error named only the exception. each set is now handled on its own: the sets that parse keep their answers, and a set that fails is recorded with its error and whatever evidence the parser attached, so the saved run shows exactly which sets failed and why. for the line decoder that evidence is every line's best reading, with the line no codec reads marked.
hints and low-confidence answers
parsers now receive the challenge's title and prompt with every question, so a decoder can try words from the prompt as cipher keys without reading any other file. a parser can also return ranked candidates for a field, and mark an answer as low confidence with a note: the runner treats such an answer as unverified and keeps the note with the run instead of reporting it as solved.
day 10 additions
a confirmed candidate first
a parser can return ranked candidates for a field. when one of them is a value the grader already confirmed for that field, it moves to the front, whatever rank the parser gave it, so an answer known to be right is never tried after one that is only likely.
day 16 additions
a run saves the answers that were accepted
a saved run's answers file is what replay compares a parser against: it is the fixture's truth. until day 16 the file held the parser's first answer for every field. when a field was accepted on a later candidate or on a corrected parser's reading, the file kept the first, rejected answer, and a replay of a run the grader had passed reported it wrong. day 16's warmups mastery was that case: 55 of the 62 first answers in its file had been rejected.
the runner now keeps, for each field, the value it was finally accepted with. a correct run saves those values as its answers and keeps the first pass in a second file beside them; a run that did not pass saves what it sent, as before.
def graded_answers(answers, result):
if result.get("result") != "CORRECT":
return answers
return {**answers, **result.get("graded", {})}
day 23 additions
a digit width the question states
day 23's binary bundle asked for a one-byte key "as 0x plus exactly two lowercase hexadecimal digits" (trust issues), while the field's answer syntax said only "lowercase hexadecimal with a 0x prefix; preserve the field's required zero padding". the answer-format reader took its rules from the syntax alone and sent the key unpadded. it now reads a stated digit count ("exactly n", "followed by n", "plus n", "padded to n", "n-digit", "n hex digits", in digits or words) from the syntax, and from the question text when the syntax is about hex or padding but states no count of its own. a hex value shorter than that width is zero-padded after its prefix, and never truncated; a bare count with a case word between ("16 uppercase hexadecimal characters") describes a value and sets no width; two different stated widths mark the field uncertain and each form is tried.
def for_question(syntax, question=""):
spec = parse(syntax)
if "width" in spec.unsure or not HEX_SYNTAX.search(syntax.lower()):
return spec
named = widths(question)
if spec.width is None and len(named) == 1:
return replace(spec, width=named[0])
return spec if not named or named == (spec.width,) else \
replace(spec, width=None, unsure=(*spec.unsure, "width"))
def hex_padded(v, width):
cut = 2 if v.lower().startswith("0x") else 0
return v[:cut] + v[cut:].rjust(width, "0")
day 24 additions
a crash report says what each field asked
when a parser raises on a set, the runner keeps the run and then asks the parser for each of the set's fields alone, so one broken field cannot hide another; the fields that still raise are grouped by their error. the report named the fields by id. day 24's crashing field was roster-filename, whose artifact line was only the bundle and whose error listed the bundle's members (event horizon): the id and the error together did not say what was being asked, and finding the question took longer than fixing the parser.
each named field in the report now carries its artifact line and the first 120 characters of its question text.
def asked(aid, questions):
q = questions.get(aid)
if not q:
return aid
artifact = str(q.get("artifact", "")).strip()
text = " ".join(str(q.get("question", "")).split())[:120]
return f'{aid} [{artifact}: "{text}"]' if artifact or text else aid
day 28 additions
a day's field rules carried into the next day's solver
a new day's solver starts as a copy of the lane's newest one (day 4). a rule written during a day for one field sits in a small rules file beside that day's solver, and the copy left it behind: a field answered that way failed again the next morning unless the rule had moved into the shared library by then. day 28 wrote such rules for three lanes. the scaffold now merges the previous challenge's rules file, and anything that file had carried itself, into a carried file beside the new solver, with each import once and the newest day's rules first; the new solver loads it after its own rules file, so a rule written on the new day wins.
def merge_rules(texts: list[str]) -> str:
imports, bodies = [], []
for text in texts: # newest day first
lines, cut = text.splitlines(), set()
for node in ast.parse(text).body:
if isinstance(node, (ast.Import, ast.ImportFrom)):
stmt = "\n".join(lines[node.lineno - 1 : node.end_lineno])
if stmt not in imports:
imports.append(stmt)
cut.update(range(node.lineno - 1, node.end_lineno))
bodies.append("\n".join(l for i, l in enumerate(lines) if i not in cut).strip())
return "\n\n\n".join(["\n".join(imports), *filter(None, bodies)]) + "\n"
the carried file is written once and never overwrites a rules file of the new day.
day 29 additions
a field rule that raises falls back to the lane reader
a rule written during a day for one field is written from the shape the single showed. on day 29 the mobile rule read a puzzle field the time trial's puzzle did not have, and the exception took the whole set down with it, though the lane reader had that field's answer all along. a per-field rule that raises now hands its field back to the lane reader, which answers it with the rest of the set; the crash is still printed so the rule gets fixed, but the set no longer waits on it.
day 30 additions
the miss report a field no reader answers now opens with its question and always ends in a draft (cross-lane field resolution).
a learned candidate position needs the same count
when the grader accepts a field on a later candidate, the runner learns that position and leads with it on fresh bytes (day 10). day 30's metadata rule returned its forms through an order-keeping dedup: six distinct forms on the single, where the accepted one was third, and four on the time trial's extensionless names, where the third position held another rule's form. it was rejected inside the attempt before the next one was accepted (a 3.07-second submit against about 0.9 seconds on the mastery). a learned position now keeps the solver's own candidate count beside it and applies only at the same count; with another count only the first candidate keeps its meaning, the solver's own order decides, and one line says so (lead <key>: graded as candidate i of n, m here).
def carried(lead, key, candidates):
i, was = lead.get(key, 0), lead.get(key + " own")
if was is None or was == own_count(candidates):
return i if i < len(candidates) else 0
return 0
a far edge drops the pin
since day 28 the game-master client probes the addresses of the server's name and pins the fastest healthy one. pinning an ip never pinned the edge site behind it: on day 29 the pinned ip landed on a far site 14 times in two minutes for one poller, and each socket was re-dialed up to three times on the same ip, about 0.4 seconds a dial. a pinned socket whose trace lands far now drops the pin at once; the edge file keeps the near sites and a dropped row with the reason, and that socket and every later one dial the name. every socket also records the ip it reached, kept for a day, and the morning probe tries those beside the name's current answers, so an ip seen only in the evening (one traced 6 to 20ms away, nearer than the pinned site) is a candidate the next morning. day 30 had no far dial in 273.
the dashboard's board and second panel
the dashboard projected a local flag onto the scoreboard even when the board listed no question under that id. the organizer's one-off static challenge scores under its own short id, so its counted points got a guessed gain on top, and the top-ten panel showed more points than the day offered. a flag is now projected only under a question id the board lists (the id's own form, then shorter slug suffixes with and without the tier word). the second activity panel opens on the web runner's stream, the web lane's runner since day 30.
the shell commands run in
four turns were retried on day 30 because of the shell a model's commands run in: it replays the human's interactive zsh, so ls was an alias that put color codes into $(ls ...), one unmatched glob aborted a whole command line, PIPESTATUS is pipestatus there, and the injected grep and find are ugrep and fd shims that skip gitignored paths. the runbook names each with its escape (/usr/bin/grep, command find, /bin/ls, quoted globs, $pipestatus), and the human's rc file drops the aliases and sets NO_NOMATCH in a model's shell. uv run also reinstalled one wheel on every call, about 0.2 seconds each, over a macos tag mismatch; the setup script pins the tag after each sync.
changelog
- day 1: per-challenge scripts call the game-master client directly; mastery handled by explicit delivery loops.
- day 2: the shared
solve_setcontract and runner: delivery-layout normalization, set/slot id mapping, strict answer-map validation, retry on expiry, saved run directories. - day 3: set and slot prefixes for cumulative mastery bundles, sha-256 verification of every assigned download.
- day 4: run capture when a parser crashes (assignment, artifacts and error kept for offline replay), and new solvers scaffolded from the lane's newest parser instead of a stub.
- day 6: answers formatted from the question's answer format (
ordered_tokensseparators) at one place for every mode; external tools bounded by a timeout; an unseen answer-format kind surfaced with its syntax before anything is submitted. - day 7: nested archives kept per set.
- day 8: answer-format syntax read clause by clause with negations; values converted to the question's kind and format in one place; set prefixes removed before parsers see field names.
- day 9: failures recorded per set with the parser's evidence; the title and prompt passed to parsers; ranked candidates and low-confidence answers with a note.
- day 10: a grader-confirmed candidate moves to the front of a field's ranked candidates.
- day 16: a correct run saves the value each field was accepted with, the first answers kept beside them.
- day 23: a digit width stated in the answer syntax or the question pads a hex value to that width.
- day 24: a crash report names each crashing field's artifact and question text beside its id.
- day 28: the previous challenge's field rules carried into the new day's solver, imports merged, loaded after the new day's own rules.
- day 29: a per-field rule that raises hands its field back to the lane reader instead of sinking the set.
- day 30: a learned candidate position applies only at the same candidate count; a pinned edge that traces far drops the pin, and the probe tries every ip seen in a day; the dashboard projects a flag only under the board's own question id and opens its second panel on the web runner; the shell's aliases and shims named, one wheel no longer reinstalled per call.
{generated by a clanker}