metadata: 31 days of one new property a day

the metadata lane asked for properties of one archive member, read from its raw bytes. each day added one property: hashes and magic bytes on a text note first, then image, pdf, zip, pe, .net, installer, shortcut, office, mail, media, apk, jar, sqlite, event-log, hive, prefetch, capture, disk and certificate fields, and on day 30 a relation across a whole corpus. the single stayed one field on most days while the timed tiers turned cumulative. the solver became a property registry: header fields by structured name, record readers whose keys answer by their words, and a relation family for corpus questions.

  • 31challengesone a day, days 1 to 31
  • 93flagssingle, time trial and mastery
  • 28 smedian singletime trial 1:01, mastery 1:04; release to flag
  • 56%flags by autopilot52 of 93
  • 363inventory items74 graded, 289 not yet graded
  • 8 of 27covered at releasedays 5 to 31; 18 partial, 1 cold start
metadata: release to the single's flag, per day
0:002:305:007:3010:00day 1: 2h30m, unattributed (off the scale)1day 2: 9:06, parentday 3: 0:28, parent3day 4: 1:12, helperday 5: 0:02, autopilot5day 6: 1:04, helperday 7: 1:05, helper7day 8: 1:06, helperday 9: 0:26, helper9day 10: 1:08, helperday 11: 0:01, autopilot11day 12: 1:03, helperday 13: 1:26, parent13day 14: 0:18, helperday 15: 0:23, lane fixer15day 16: 0:05, autopilotday 17: 0:18, lane fixer17day 18: 0:18, lane fixerday 19: 0:02, autopilot19day 20: 0:38, lane fixerday 21: 0:30, lane fixer21day 22: 0:02, autopilotday 23: 0:15, autopilot23day 24: 0:02, autopilotday 25: 0:18, autopilot25day 26: 1:28, autopilotday 27: 0:03, autopilot27day 28: 8:16, lane fixerday 29: 0:03, autopilot29day 30: 0:56, autopilotday 31: 1:11, lane fixer31
  • autopilot
  • lane fixer
  • helper
  • parent
  • other

the scale stops at ten minutes; a dashed bar is a day beyond it (hover for the time). a bar's colour is the role on the flag: autopilot claims each single at release and runs the lane solver, a lane fixer edits that solver, a helper works a single nobody owns, a parent routes and submits only a value it already holds. day 1 opened at 11:28, so its bar counts from 09:00 and includes the late start.

how the family evolved

answer ids are <member stem>-<property>; every value comes from the raw bytes of one archive member, and the member's extension can lie. a day added one property and a family lasted one to three days.

days 1 to 3, one text note. filename, size and three digests (hash to ride), the first 16 bytes as hex (looks can be deceiving), whether a utf-8 byte-order mark is present (bom voyage).

days 4 to 7, a member family a day. each new member came under a dNN-<family>-NNN stem: jpeg frame dimensions (picture this), png ihdr dimensions (chunk norris), the pdf header version (portable deception) and the entry count a nested zip's end record claims (seven-zip, no relation).

days 8 and 9, a cumulative single. the single carried every property so far and added the names a zip's central directory stores (what's in the box) and the pe coff machine field (built different). from day 9 a field named by structure, <family>-<struct>-<field>-<width>-<format>, answers from the struct table.

days 10 to 16, windows and office records. the pe entry point rva (where do we begin), the version resource's original filename (what's my name again), a .net module's mvid (managed expectations), an installer's product code with its braces (product placement), a shortcut's target (the shortcut home), an office document's external relationship targets (a word to the wise) and a mail message id (return to sender). from day 15 the time trial turned cumulative, 20 fields and one more a day.

days 17 to 23, media and other containers. an audio stream's codec in ffprobe's words (hear me out), a video's dimensions composed from two ffprobe keys (frame of reference), an elf's machine (little endian on the prairie), the cpu types a mach-o advertises (universal appeal), an android package name (package deal), a jar's main class (main character energy) and a sqlite page size, where the solver's first reading was the file size (page me maybe).

days 24 to 29, forensic records. an event log's record count (event horizon), a registry hive's root key (hive mind), a compressed windows 10 prefetch file's executable name (before you go), a capture's packet count (packet loss prevention), a disk image's partition starts (where it all began) and a certificate's fingerprint (trust issues). day 28 was the lane's one cold start, 8:16 to the single.

days 30 and 31, relations. groups of byte-identical files in a corpus zip (seeing double), the one graded member of a relation family the prompt announced, then an execution chain walked through six file formats (no artifact left behind). the day 31 time trial asked 36 fields.

the property each single asked first, by day:

day property asked on the single time trial fields mastery sets post
1 filename, md5, sha1, sha256, size-bytes - - hash to ride
2 magic-prefix-hex-16 - - looks can be deceiving
3 text-utf8-bom-present 1 2 bom voyage
4 jpeg-sof-dimensions-pixels 1 2 picture this
5 png-ihdr-dimensions-pixels 1 2 chunk norris
6 pdf-header-version 1 2 portable deception
7 zip-eocd-total-entries 1 2 seven-zip, no relation
8 zip-central-directory-member-names (12 fields) 12 5 what's in the box
9 pe-coff-machine-u16-hex (13 fields) 13 5 built different
10 pe-address-of-entry-point-rva-hex 1 2 where do we begin
11 pe-version-original-filename 1 2 what's my name again
12 dotnet-mvid-guid 1 2 managed expectations
13 msi-product-code-guid 1 2 product placement
14 lnk-target-path-windows 1 2 the shortcut home
15 ooxml-external-relationship-targets 20 5 a word to the wise
16 email-message-id 21 5 return to sender
17 media-primary-audio-codec-name 22 5 hear me out
18 media-primary-video-dimensions 23 5 frame of reference
19 elf-e-machine-u16-hex 24 5 little endian on the prairie
20 macho-cpu-type-set 25 5 universal appeal
21 apk-package-name 26 5 package deal
22 jar-main-class 27 6 main character energy
23 sqlite-page-size-bytes 28 7 page me maybe
24 evtx-record-count 29 8 event horizon
25 registry-hive-root-name 30 9 hive mind
26 prefetch-executable-name 31 10 before you go
27 pcap-packet-count 32 11 packet loss prevention
28 disk-partition-start-lba-sequence 33 12 where it all began
29 x509-sha256-fingerprint 34 13 trust issues
30 relation-identical-sha256-groups 35 14 seeing double
31 graph-execution-chain-artifact-ids 36 15 no artifact left behind

coverage

this lane's row of the coverage grid. autopilot answered 8 of the 27 singles from day 5 on with no gaps row; the table lists the rest. closed in runs from autopilot's gaps row to the single's flag.

metadata: the single at release, by day
12345678910111213141516171819202122232425262728293031metadataday 1: no release pass, single at 2h30m (unattributed). no release pass yet: autopilot's first release pass was day 5day 2: no release pass, single at 9:06 (parent). no release pass yet: autopilot's first release pass was day 5day 3: no release pass, single at 0:28 (parent). no release pass yet: autopilot's first release pass was day 5day 4: no release pass, single at 1:12 (helper). no release pass yet: autopilot's first release pass was day 5 (on day 4 it first ran at 11:44:40, after the singles)day 5: covered, single at 0:02 (autopilot). nothing: autopilot answered every field at releaseday 6: partial, single at 1:04 (helper). 1 field (d06-pdf): no reader for d06-pdf-001-pdf-header-versionday 7: partial, single at 1:05 (helper). 1 field (d07-zip): no reader for zip-eocd-total-entriesday 8: partial, single at 1:06 (helper). 2 fields (d07-zip, evidence-note): no reader for zip-central-directory-member-namesday 9: partial, single at 0:26 (helper). 1 field (d09-pe): no reader for pe-coff-machine-u16-hexday 10: partial, single at 1:08 (helper). 1 field (d09-pe): no reader for pe-address-of-entry-point-rva-hexday 11: covered, single at 0:01 (autopilot). nothing: autopilot answered every field at releaseday 12: partial, single at 1:03 (helper). 1 field (d12-dotnet): no reader for dotnet-mvid-guidday 13: partial, single at 1:26 (parent). 1 field (d13-msi): no reader for msi-product-code-guidday 14: partial, single at 0:18 (helper). 1 field (d14-lnk): no reader for lnk-target-path-windowsday 15: partial, single at 0:23 (lane fixer). 1 field (d15-ooxml): no reader for ooxml-external-relationship-targetsday 16: covered, single at 0:05 (autopilot). nothing: autopilot answered every field at releaseday 17: partial, single at 0:18 (lane fixer). 1 field (d17-audio): no reader for media-primary-audio-codec-nameday 18: partial, single at 0:18 (lane fixer). 1 field (d18-video): KeyError: "media-primary-video-dimensions: the video stream has no 'dimensions'"day 19: covered, single at 0:02 (autopilot). nothing: autopilot answered every field at releaseday 20: partial, single at 0:38 (lane fixer). 1 field (d20-macho): no reader for macho-cpu-type-setday 21: partial, single at 0:30 (lane fixer). 1 field (d21-apk): no reader for apk-package-nameday 22: covered, single at 0:02 (autopilot). nothing: autopilot answered every field at releaseday 23: partial, single at 0:15 (autopilot). 1 field (d23-sqlite): no crash; the solver's 12288 was rejectedday 24: covered, single at 0:02 (autopilot). nothing: autopilot answered every field at releaseday 25: partial, single at 0:18 (autopilot). 1 field (d25-registry): no reader for registry-hive-root-nameday 26: partial, single at 1:28 (autopilot). 1 field (d26-prefetch): no reader for prefetch-executable-nameday 27: covered, single at 0:03 (autopilot). nothing: autopilot answered every field at releaseday 28: cold start, single at 8:16 (lane fixer). 1 field (d28-disk): no reader for disk-partition-start-lba-sequenceday 29: covered, single at 0:03 (autopilot). nothing: autopilot answered every field at releaseday 30: partial, single at 0:56 (autopilot). 1 field (d30-cluster): no reader for relation-identical-sha256-groupsday 31: partial, single at 1:11 (lane fixer). 1 field (d31-case): no reader for graph-execution-chain-artifact-ids
  • covered at release
  • partial, under 3 min
  • cold start, 3 to 15 min
  • long, over 15 min
  • no release pass or unknown

hover a cell for the time, the role on the flag and what was missing. grey on days 1 to 4: the release pass started on day 5.

day at release what was missing closed in
6 partial 1 field (d06-pdf): no reader for d06-pdf-001-pdf-header-version 1:03, helper
7 partial 1 field (d07-zip): no reader for zip-eocd-total-entries 1:01, helper
8 partial 2 fields (d07-zip, evidence-note): no reader for zip-central-directory-member-names 1:03, helper
9 partial 1 field (d09-pe): no reader for pe-coff-machine-u16-hex 0:22, helper
10 partial 1 field (d09-pe): no reader for pe-address-of-entry-point-rva-hex 1:04, helper
12 partial 1 field (d12-dotnet): no reader for dotnet-mvid-guid 1:01, helper
13 partial 1 field (d13-msi): no reader for msi-product-code-guid 1:24, parent
14 partial 1 field (d14-lnk): no reader for lnk-target-path-windows 0:15, helper
15 partial 1 field (d15-ooxml): no reader for ooxml-external-relationship-targets 0:20, lane fixer
17 partial 1 field (d17-audio): no reader for media-primary-audio-codec-name 0:15, lane fixer
18 partial 1 field (d18-video): KeyError: "media-primary-video-dimensions: the video stream has no 'dimensions'" 0:16, lane fixer
20 partial 1 field (d20-macho): no reader for macho-cpu-type-set 0:36, lane fixer
21 partial 1 field (d21-apk): no reader for apk-package-name 0:27, lane fixer
23 partial 1 field (d23-sqlite): no crash; the solver's 12288 was rejected 0:13, autopilot
25 partial 1 field (d25-registry): no reader for registry-hive-root-name 0:15, autopilot
26 partial 1 field (d26-prefetch): no reader for prefetch-executable-name 1:25, autopilot
28 cold start 1 field (d28-disk): no reader for disk-partition-start-lba-sequence 8:12, lane fixer
30 partial 1 field (d30-cluster): no reader for relation-identical-sha256-groups 0:52, autopilot
31 partial 1 field (d31-case): no reader for graph-execution-chain-artifact-ids 1:07, lane fixer

the tooling that grew for it

the lane's tool page, archive and metadata analysis, opens with what the solver did by day 3 and adds a section for every day its code changed:

every post, in order

times run from the 09:00 et release to each tier's flag; day 1 opened at 11:28 and counts from 09:00.

day post single time trial mastery
1 hash to ride 2h30m 2h57m 3h00m
2 looks can be deceiving 9:06 10:43 14:37
3 bom voyage 0:28 0:38 1:03
4 picture this 1:12 2:34 3:07
5 chunk norris 0:02 0:53 0:55
6 portable deception 1:04 1:15 1:17
7 seven-zip, no relation 1:05 1:08 2:13
8 what's in the box 1:06 5:42 5:44
9 built different 0:26 1:03 1:05
10 where do we begin 1:08 1:39 1:41
11 what's my name again 0:01 0:04 0:05
12 managed expectations 1:03 1:05 1:06
13 product placement 1:26 1:54 1:55
14 the shortcut home 0:18 0:57 0:58
15 a word to the wise 0:23 1:14 1:17
16 return to sender 0:05 0:08 0:11
17 hear me out 0:18 0:44 0:47
18 frame of reference 0:18 0:43 0:45
19 little endian on the prairie 0:02 0:05 0:07
20 universal appeal 0:38 0:56 0:58
21 package deal 0:30 0:54 0:56
22 main character energy 0:02 0:06 0:09
23 page me maybe 0:15 1:13 1:15
24 event horizon 0:02 1:01 1:04
25 hive mind 0:18 0:20 0:24
26 before you go 1:28 1:30 1:34
27 packet loss prevention 0:03 0:05 0:11
28 where it all began 8:16 8:28 8:33
29 trust issues 0:03 0:07 0:11
30 seeing double 0:56 1:00 1:04
31 no artifact left behind 1:11 3:43 3:47

{generated by a clanker}