tooling: solver pipeline and timed-attempt runner

the event had three modes for each challenge. an untimed single specimen taught the format; a time trial rotated the specimen under a short deadline; mastery assigned several answer sets under one deadline. we built one orchestration layer so artifact parsers did not need to know which mode they were solving.

the parser contract

every challenge parser implemented the equivalent of:

def solve_set(
    files: dict[str, bytes],
    fields: list[str],
    questions: list[dict],
) -> dict[str, str]:
    ...

we called this adapter solve_set; the name is included here only to explain the architecture used throughout the writeups. files contains one set's raw and extracted files. fields contains local answer names with an outer setN. prefix removed. questions carries hints such as an artifact name, but fields are authoritative because question text could remain stale after a retry. the function must return every field exactly once as a non-empty string.

this boundary keeps network calls, clocks, and packaging out of analysis code. for example, an apk parser only sees one apk and a local field named answer, whether the server assigned answer or set3.answer.

normalizing delivery layouts

the server used two layouts:

for direct deliveries, files are grouped by the supplied set number. for a bundle, each member is expanded once and an explicit filename marker assigns it to a set. a bare long number is never treated as a set marker because years, campaign ids, and hashes commonly contain digits. if no explicit marker exists, the whole bundle is presented to the parser, which must resolve members using field and question evidence.

conceptually:

def split_sets(answer_ids, downloads):
    sets = set_numbers_from(answer_ids)
    if every_download_has_a_set_number(downloads):
        return group_by_assigned_set(downloads)

    members = expand_each_bundle_member_once(downloads)
    return {
        number: members_marked_for(number) or members
        for number in sets
    }

mapping local answers back to server ids

an id parser separates setN., optional slotM_, and the field name. the runner calls the parser once per set with names such as slot2_marker, then maps returned values back to their original ids. before submission it verifies:

set(answers) == set(assigned_answer_ids)
all(value.strip() for value in answers.values())

this check is deliberately strict. a partial or misspelled answer map wastes a timed attempt while teaching nothing about the artifact.

timed execution

the runner performs this sequence:

  1. start or resume the attempt.
  2. fetch assigned artifacts and verify supplied sha-256 values.
  3. split deliveries into answer sets.
  4. run pure parsers and validate the complete answer map.
  5. submit once.
  6. if the attempt expires during fetch or submit, request new material and repeat from the new assignment rather than reusing stale files.

the parser is tested on known specimens before a timed attempt. this matters because starting a timed challenge can rotate material, and debugging after the clock begins is expensive.

why this mattered

most daily variants changed packaging, prefixes, or counts rather than the core artifact format. a transport-neutral parser plus a layout-aware runner let the same analytical code survive all three modes. it also made failures legible: member placement, artifact parsing, answer formatting, and network timing were separate layers rather than one large script.

day 7 additions

nested archives per set

a mastery bundle with several sets used to be split and flattened into each set's members. when a field named a zip inside its set, that zip was gone and a text file was read instead. each set now keeps every nested archive's own bytes alongside its unpacked members, so an archive stays addressable by name.

day 8 additions

reading the answer format

the answer format's syntax text is read clause by clause, negations first, because a substring check misreads a negated clause. each clause yields a prefix rule, a case rule or a separator; a clause that reads two ways is reported as ambiguous and nothing is rewritten for it. a value that breaks the format its question asks for is converted in one place for every mode ([redacted]).

field names without set prefixes

parsers receive field names with the setN. prefix removed in every mode, so a rule keyed on a field name holds in the mastery exactly as in the single.

day 9 additions

failures recorded per set

a parser that raises on one answer set used to lose the whole run's answers with it, and the saved error named only the exception. each set is now handled on its own: the sets that parse keep their answers, and a set that fails is recorded with its error and whatever evidence the parser attached, so the saved run shows exactly which sets failed and why. for the line decoder that evidence is every line's best reading, with the line no codec reads marked.

hints and low-confidence answers

parsers now receive the challenge's title and prompt with every question, so a decoder can try words from the prompt as cipher keys without reading any other file. a parser can also return ranked candidates for a field, and mark an answer as low confidence with a note: the runner treats such an answer as unverified and keeps the note with the run instead of reporting it as solved.

day 10 additions

a confirmed candidate first

a parser can return ranked candidates for a field. when one of them is a value the grader already confirmed for that field, it moves to the front, whatever rank the parser gave it, so an answer known to be right is never tried after one that is only likely.

day 16 additions

a run saves the answers that were accepted

a saved run's answers file is what replay compares a parser against: it is the fixture's truth. until day 16 the file held the parser's first answer for every field. when a field was accepted on a later candidate or on a corrected parser's reading, the file kept the first, rejected answer, and a replay of a run the grader had passed reported it wrong. day 16's warmups mastery was that case: 55 of the 62 first answers in its file had been rejected.

the runner now keeps, for each field, the value it was finally accepted with. a correct run saves those values as its answers and keeps the first pass in a second file beside them; a run that did not pass saves what it sent, as before.

def graded_answers(answers, result):
    if result.get("result") != "CORRECT":
        return answers
    return {**answers, **result.get("graded", {})}

day 23 additions

a digit width the question states

day 23's binary bundle asked for a one-byte key "as 0x plus exactly two lowercase hexadecimal digits" (trust issues), while the field's answer syntax said only "lowercase hexadecimal with a 0x prefix; preserve the field's required zero padding". the answer-format reader took its rules from the syntax alone and sent the key unpadded. it now reads a stated digit count ("exactly n", "followed by n", "plus n", "padded to n", "n-digit", "n hex digits", in digits or words) from the syntax, and from the question text when the syntax is about hex or padding but states no count of its own. a hex value shorter than that width is zero-padded after its prefix, and never truncated; a bare count with a case word between ("16 uppercase hexadecimal characters") describes a value and sets no width; two different stated widths mark the field uncertain and each form is tried.

def for_question(syntax, question=""):
    spec = parse(syntax)
    if "width" in spec.unsure or not HEX_SYNTAX.search(syntax.lower()):
        return spec
    named = widths(question)
    if spec.width is None and len(named) == 1:
        return replace(spec, width=named[0])
    return spec if not named or named == (spec.width,) else \
        replace(spec, width=None, unsure=(*spec.unsure, "width"))

def hex_padded(v, width):
    cut = 2 if v.lower().startswith("0x") else 0
    return v[:cut] + v[cut:].rjust(width, "0")

day 24 additions

a crash report says what each field asked

when a parser raises on a set, the runner keeps the run and then asks the parser for each of the set's fields alone, so one broken field cannot hide another; the fields that still raise are grouped by their error. the report named the fields by id. day 24's crashing field was roster-filename, whose artifact line was only the bundle and whose error listed the bundle's members (event horizon): the id and the error together did not say what was being asked, and finding the question took longer than fixing the parser.

each named field in the report now carries its artifact line and the first 120 characters of its question text.

def asked(aid, questions):
    q = questions.get(aid)
    if not q:
        return aid
    artifact = str(q.get("artifact", "")).strip()
    text = " ".join(str(q.get("question", "")).split())[:120]
    return f'{aid} [{artifact}: "{text}"]' if artifact or text else aid

day 28 additions

a day's field rules carried into the next day's solver

a new day's solver starts as a copy of the lane's newest one (day 4). a rule written during a day for one field sits in a small rules file beside that day's solver, and the copy left it behind: a field answered that way failed again the next morning unless the rule had moved into the shared library by then. day 28 wrote such rules for three lanes. the scaffold now merges the previous challenge's rules file, and anything that file had carried itself, into a carried file beside the new solver, with each import once and the newest day's rules first; the new solver loads it after its own rules file, so a rule written on the new day wins.

def merge_rules(texts: list[str]) -> str:
    imports, bodies = [], []
    for text in texts:                         # newest day first
        lines, cut = text.splitlines(), set()
        for node in ast.parse(text).body:
            if isinstance(node, (ast.Import, ast.ImportFrom)):
                stmt = "\n".join(lines[node.lineno - 1 : node.end_lineno])
                if stmt not in imports:
                    imports.append(stmt)
                cut.update(range(node.lineno - 1, node.end_lineno))
        bodies.append("\n".join(l for i, l in enumerate(lines) if i not in cut).strip())
    return "\n\n\n".join(["\n".join(imports), *filter(None, bodies)]) + "\n"

the carried file is written once and never overwrites a rules file of the new day.

day 29 additions

a field rule that raises falls back to the lane reader

a rule written during a day for one field is written from the shape the single showed. on day 29 the mobile rule read a puzzle field the time trial's puzzle did not have, and the exception took the whole set down with it, though the lane reader had that field's answer all along. a per-field rule that raises now hands its field back to the lane reader, which answers it with the rest of the set; the crash is still printed so the rule gets fixed, but the set no longer waits on it.

day 30 additions

the miss report a field no reader answers now opens with its question and always ends in a draft (cross-lane field resolution).

a learned candidate position needs the same count

when the grader accepts a field on a later candidate, the runner learns that position and leads with it on fresh bytes (day 10). day 30's metadata rule returned its forms through an order-keeping dedup: six distinct forms on the single, where the accepted one was third, and four on the time trial's extensionless names, where the third position held another rule's form. it was rejected inside the attempt before the next one was accepted (a 3.07-second submit against about 0.9 seconds on the mastery). a learned position now keeps the solver's own candidate count beside it and applies only at the same count; with another count only the first candidate keeps its meaning, the solver's own order decides, and one line says so (lead <key>: graded as candidate i of n, m here).

def carried(lead, key, candidates):
    i, was = lead.get(key, 0), lead.get(key + " own")
    if was is None or was == own_count(candidates):
        return i if i < len(candidates) else 0
    return 0

a far edge drops the pin

since day 28 the game-master client probes the addresses of the server's name and pins the fastest healthy one. pinning an ip never pinned the edge site behind it: on day 29 the pinned ip landed on a far site 14 times in two minutes for one poller, and each socket was re-dialed up to three times on the same ip, about 0.4 seconds a dial. a pinned socket whose trace lands far now drops the pin at once; the edge file keeps the near sites and a dropped row with the reason, and that socket and every later one dial the name. every socket also records the ip it reached, kept for a day, and the morning probe tries those beside the name's current answers, so an ip seen only in the evening (one traced 6 to 20ms away, nearer than the pinned site) is a candidate the next morning. day 30 had no far dial in 273.

the dashboard's board and second panel

the dashboard projected a local flag onto the scoreboard even when the board listed no question under that id. the organizer's one-off static challenge scores under its own short id, so its counted points got a guessed gain on top, and the top-ten panel showed more points than the day offered. a flag is now projected only under a question id the board lists (the id's own form, then shorter slug suffixes with and without the tier word). the second activity panel opens on the web runner's stream, the web lane's runner since day 30.

the shell commands run in

four turns were retried on day 30 because of the shell a model's commands run in: it replays the human's interactive zsh, so ls was an alias that put color codes into $(ls ...), one unmatched glob aborted a whole command line, PIPESTATUS is pipestatus there, and the injected grep and find are ugrep and fd shims that skip gitignored paths. the runbook names each with its escape (/usr/bin/grep, command find, /bin/ls, quoted globs, $pipestatus), and the human's rc file drops the aliases and sets NO_NOMATCH in a model's shell. uv run also reinstalled one wheel on every call, about 0.2 seconds each, over a macos tag mismatch; the setup script pins the tag after each sync.

changelog

{generated by a clanker}