metadata: 31 days of one new property a day
the metadata lane asked for properties of one archive member, read from its raw bytes. each day added one property: hashes and magic bytes on a text note first, then image, pdf, zip, pe, .net, installer, shortcut, office, mail, media, apk, jar, sqlite, event-log, hive, prefetch, capture, disk and certificate fields, and on day 30 a relation across a whole corpus. the single stayed one field on most days while the timed tiers turned cumulative. the solver became a property registry: header fields by structured name, record readers whose keys answer by their words, and a relation family for corpus questions.
- 31challengesone a day, days 1 to 31
- 93flagssingle, time trial and mastery
- 28 smedian singletime trial 1:01, mastery 1:04; release to flag
- 56%flags by autopilot52 of 93
- 363inventory items74 graded, 289 not yet graded
- 8 of 27covered at releasedays 5 to 31; 18 partial, 1 cold start
- autopilot
- lane fixer
- helper
- parent
- other
the scale stops at ten minutes; a dashed bar is a day beyond it (hover for the time). a bar's colour is the role on the flag: autopilot claims each single at release and runs the lane solver, a lane fixer edits that solver, a helper works a single nobody owns, a parent routes and submits only a value it already holds. day 1 opened at 11:28, so its bar counts from 09:00 and includes the late start.
how the family evolved
answer ids are <member stem>-<property>; every value comes from the raw bytes of one archive member, and the member's extension can lie. a day added one property and a family lasted one to three days.
days 1 to 3, one text note. filename, size and three digests (hash to ride), the first 16 bytes as hex (looks can be deceiving), whether a utf-8 byte-order mark is present (bom voyage).
days 4 to 7, a member family a day. each new member came under a dNN-<family>-NNN stem: jpeg frame dimensions (picture this), png ihdr dimensions (chunk norris), the pdf header version (portable deception) and the entry count a nested zip's end record claims (seven-zip, no relation).
days 8 and 9, a cumulative single. the single carried every property so far and added the names a zip's central directory stores (what's in the box) and the pe coff machine field (built different). from day 9 a field named by structure, <family>-<struct>-<field>-<width>-<format>, answers from the struct table.
days 10 to 16, windows and office records. the pe entry point rva (where do we begin), the version resource's original filename (what's my name again), a .net module's mvid (managed expectations), an installer's product code with its braces (product placement), a shortcut's target (the shortcut home), an office document's external relationship targets (a word to the wise) and a mail message id (return to sender). from day 15 the time trial turned cumulative, 20 fields and one more a day.
days 17 to 23, media and other containers. an audio stream's codec in ffprobe's words (hear me out), a video's dimensions composed from two ffprobe keys (frame of reference), an elf's machine (little endian on the prairie), the cpu types a mach-o advertises (universal appeal), an android package name (package deal), a jar's main class (main character energy) and a sqlite page size, where the solver's first reading was the file size (page me maybe).
days 24 to 29, forensic records. an event log's record count (event horizon), a registry hive's root key (hive mind), a compressed windows 10 prefetch file's executable name (before you go), a capture's packet count (packet loss prevention), a disk image's partition starts (where it all began) and a certificate's fingerprint (trust issues). day 28 was the lane's one cold start, 8:16 to the single.
days 30 and 31, relations. groups of byte-identical files in a corpus zip (seeing double), the one graded member of a relation family the prompt announced, then an execution chain walked through six file formats (no artifact left behind). the day 31 time trial asked 36 fields.
the property each single asked first, by day:
| day | property asked on the single | time trial fields | mastery sets | post |
|---|---|---|---|---|
| 1 | filename, md5, sha1, sha256, size-bytes |
- | - | hash to ride |
| 2 | magic-prefix-hex-16 |
- | - | looks can be deceiving |
| 3 | text-utf8-bom-present |
1 | 2 | bom voyage |
| 4 | jpeg-sof-dimensions-pixels |
1 | 2 | picture this |
| 5 | png-ihdr-dimensions-pixels |
1 | 2 | chunk norris |
| 6 | pdf-header-version |
1 | 2 | portable deception |
| 7 | zip-eocd-total-entries |
1 | 2 | seven-zip, no relation |
| 8 | zip-central-directory-member-names (12 fields) |
12 | 5 | what's in the box |
| 9 | pe-coff-machine-u16-hex (13 fields) |
13 | 5 | built different |
| 10 | pe-address-of-entry-point-rva-hex |
1 | 2 | where do we begin |
| 11 | pe-version-original-filename |
1 | 2 | what's my name again |
| 12 | dotnet-mvid-guid |
1 | 2 | managed expectations |
| 13 | msi-product-code-guid |
1 | 2 | product placement |
| 14 | lnk-target-path-windows |
1 | 2 | the shortcut home |
| 15 | ooxml-external-relationship-targets |
20 | 5 | a word to the wise |
| 16 | email-message-id |
21 | 5 | return to sender |
| 17 | media-primary-audio-codec-name |
22 | 5 | hear me out |
| 18 | media-primary-video-dimensions |
23 | 5 | frame of reference |
| 19 | elf-e-machine-u16-hex |
24 | 5 | little endian on the prairie |
| 20 | macho-cpu-type-set |
25 | 5 | universal appeal |
| 21 | apk-package-name |
26 | 5 | package deal |
| 22 | jar-main-class |
27 | 6 | main character energy |
| 23 | sqlite-page-size-bytes |
28 | 7 | page me maybe |
| 24 | evtx-record-count |
29 | 8 | event horizon |
| 25 | registry-hive-root-name |
30 | 9 | hive mind |
| 26 | prefetch-executable-name |
31 | 10 | before you go |
| 27 | pcap-packet-count |
32 | 11 | packet loss prevention |
| 28 | disk-partition-start-lba-sequence |
33 | 12 | where it all began |
| 29 | x509-sha256-fingerprint |
34 | 13 | trust issues |
| 30 | relation-identical-sha256-groups |
35 | 14 | seeing double |
| 31 | graph-execution-chain-artifact-ids |
36 | 15 | no artifact left behind |
coverage
this lane's row of the coverage grid. autopilot answered 8 of the 27 singles from day 5 on with no gaps row; the table lists the rest. closed in runs from autopilot's gaps row to the single's flag.
- covered at release
- partial, under 3 min
- cold start, 3 to 15 min
- long, over 15 min
- no release pass or unknown
hover a cell for the time, the role on the flag and what was missing. grey on days 1 to 4: the release pass started on day 5.
| day | at release | what was missing | closed in |
|---|---|---|---|
| 6 | partial | 1 field (d06-pdf): no reader for d06-pdf-001-pdf-header-version |
1:03, helper |
| 7 | partial | 1 field (d07-zip): no reader for zip-eocd-total-entries |
1:01, helper |
| 8 | partial | 2 fields (d07-zip, evidence-note): no reader for zip-central-directory-member-names |
1:03, helper |
| 9 | partial | 1 field (d09-pe): no reader for pe-coff-machine-u16-hex |
0:22, helper |
| 10 | partial | 1 field (d09-pe): no reader for pe-address-of-entry-point-rva-hex |
1:04, helper |
| 12 | partial | 1 field (d12-dotnet): no reader for dotnet-mvid-guid |
1:01, helper |
| 13 | partial | 1 field (d13-msi): no reader for msi-product-code-guid |
1:24, parent |
| 14 | partial | 1 field (d14-lnk): no reader for lnk-target-path-windows |
0:15, helper |
| 15 | partial | 1 field (d15-ooxml): no reader for ooxml-external-relationship-targets |
0:20, lane fixer |
| 17 | partial | 1 field (d17-audio): no reader for media-primary-audio-codec-name |
0:15, lane fixer |
| 18 | partial | 1 field (d18-video): KeyError: "media-primary-video-dimensions: the video stream has no 'dimensions'" |
0:16, lane fixer |
| 20 | partial | 1 field (d20-macho): no reader for macho-cpu-type-set |
0:36, lane fixer |
| 21 | partial | 1 field (d21-apk): no reader for apk-package-name |
0:27, lane fixer |
| 23 | partial | 1 field (d23-sqlite): no crash; the solver's 12288 was rejected |
0:13, autopilot |
| 25 | partial | 1 field (d25-registry): no reader for registry-hive-root-name |
0:15, autopilot |
| 26 | partial | 1 field (d26-prefetch): no reader for prefetch-executable-name |
1:25, autopilot |
| 28 | cold start | 1 field (d28-disk): no reader for disk-partition-start-lba-sequence |
8:12, lane fixer |
| 30 | partial | 1 field (d30-cluster): no reader for relation-identical-sha256-groups |
0:52, autopilot |
| 31 | partial | 1 field (d31-case): no reader for graph-execution-chain-artifact-ids |
1:07, lane fixer |
the tooling that grew for it
the lane's tool page, archive and metadata analysis, opens with what the solver did by day 3 and adds a section for every day its code changed:
- day 3: prepared format readers
- day 4: jpeg frame dimensions, image dimensions in any common format
- day 5: day 5 additions
- day 6: day 6 additions
- day 7: raw header fields, matched by name, zip census, nested archives as members
- day 8: central directory names, a guess must cover every word, bounded decompression
- day 9: header fields by structured name, loud failures
- day 10: shorter structured names, the pe version resource, executable labels and zip provenance
- day 11: the rest of the pe, comparing a pe's names, archives
- day 12: .net identity by name
- day 13: windows installer databases, fields across members
- day 14: one-record readers, mapped by the property's words
- day 15: day 15 additions
- day 16: day 16 additions
- day 17: media streams by ffprobe's own keys
- day 18: composites, near misses and the keys a stream has, video containers in the sniff, composites of any reader's keys
- day 19: a header field named without a format word
- day 20: a field over every record: set values, the bytes pick the struct
- day 21: an android package as a record
- day 22: the jar record family
- day 23: a field id splits at its stem first, the sqlite record, every key by its words, a database embedded past offset 0, member-name lists in two orders
- day 24: a member named by the leading tokens of its stem, the event-log record, read as a whole log
- day 25: the hive record's whole-tree keys, one word over: a property's synonym
- day 26: compressed prefetch: lzxpress huffman in python, the prefetch record
- day 28: a disk record for partitioned images, the partition an offset in the question names, filesystem types and container headers, header-first lookups, the winzip-aes key through hashlib
- day 29: six more record readers
- day 30: the relation family, semantic keys, near keys and crops, capture hosts and one tool budget, memos keyed by digest
every post, in order
times run from the 09:00 et release to each tier's flag; day 1 opened at 11:28 and counts from 09:00.
| day | post | single | time trial | mastery |
|---|---|---|---|---|
| 1 | hash to ride | 2h30m | 2h57m | 3h00m |
| 2 | looks can be deceiving | 9:06 | 10:43 | 14:37 |
| 3 | bom voyage | 0:28 | 0:38 | 1:03 |
| 4 | picture this | 1:12 | 2:34 | 3:07 |
| 5 | chunk norris | 0:02 | 0:53 | 0:55 |
| 6 | portable deception | 1:04 | 1:15 | 1:17 |
| 7 | seven-zip, no relation | 1:05 | 1:08 | 2:13 |
| 8 | what's in the box | 1:06 | 5:42 | 5:44 |
| 9 | built different | 0:26 | 1:03 | 1:05 |
| 10 | where do we begin | 1:08 | 1:39 | 1:41 |
| 11 | what's my name again | 0:01 | 0:04 | 0:05 |
| 12 | managed expectations | 1:03 | 1:05 | 1:06 |
| 13 | product placement | 1:26 | 1:54 | 1:55 |
| 14 | the shortcut home | 0:18 | 0:57 | 0:58 |
| 15 | a word to the wise | 0:23 | 1:14 | 1:17 |
| 16 | return to sender | 0:05 | 0:08 | 0:11 |
| 17 | hear me out | 0:18 | 0:44 | 0:47 |
| 18 | frame of reference | 0:18 | 0:43 | 0:45 |
| 19 | little endian on the prairie | 0:02 | 0:05 | 0:07 |
| 20 | universal appeal | 0:38 | 0:56 | 0:58 |
| 21 | package deal | 0:30 | 0:54 | 0:56 |
| 22 | main character energy | 0:02 | 0:06 | 0:09 |
| 23 | page me maybe | 0:15 | 1:13 | 1:15 |
| 24 | event horizon | 0:02 | 1:01 | 1:04 |
| 25 | hive mind | 0:18 | 0:20 | 0:24 |
| 26 | before you go | 1:28 | 1:30 | 1:34 |
| 27 | packet loss prevention | 0:03 | 0:05 | 0:11 |
| 28 | where it all began | 8:16 | 8:28 | 8:33 |
| 29 | trust issues | 0:03 | 0:07 | 0:11 |
| 30 | seeing double | 0:56 | 1:00 | 1:04 |
| 31 | no artifact left behind | 1:11 | 3:43 | 3:47 |
{generated by a clanker}