live web target capture: bounded requests and offline answers
the web lane receives temporary target origins instead of downloadable artifacts. the live collector turns each assigned origin into a small response bundle, then the ordinary solver reads the requested value from that bundle. every web challenge post uses this pipeline: back up a minute, permission to slip and query me this (day 19); administratively speaking and stay in your directory (day 20); reflect upon this, session impression and stored for later (day 21); forged with care and uploading expectations (day 22); ask again later, posted too much and template temper (day 23); fetch quest and the api has opinions (day 24); reset expectations, resolver revolt and shell company (day 25); serialized offender and trust the webhook (day 26); double spent, forwarded misgivings and socket to me (day 27); fail-open friday and if a breach falls in a log (day 28); internal affairs and trust the build (day 29); and pass the identity and queue the confusion (day 30).
input model
an assignment contains one or more explicit http or https target urls. each target gets a separate client, cookie jar, origin boundary and attempt deadline. credentials in a url are rejected. redirects must remain on the assigned scheme, host and port. ordinary requests use bounded GET and POST; a clue-directed stream can use a bounded websocket exchange.
the collector holds each target to its attempt deadline instead of response-count or response-size caps: a redirect is followed only while it stays on the assigned origin, at most sixteen targets capture concurrently, and the event-stream reader is bounded on its own (an idle timeout, an event count and a byte cap). it does not inherit proxy settings or attach game-master credentials to target requests.
parsing and capture algorithm
the first response is parsed with a deliberately small html parser. it records links, external script urls, forms, input names, visible text and comments. signals on that page rank the known recipes separately for every target:
scores = {
"backup": words.count("robots.txt") + words.count("backup"),
"idor": words.count("reference_only") + words.count("/api/history"),
"sqli": words.count("search") + words.count("lookup"),
}
this per-target choice matters in mastery because one bundle can contain different vulnerability families. a challenge title is only a fallback when a sparse page gives no useful signal.
the collector records the response body, status, safe response headers, elapsed time and recipe stage. request metadata records parameter names, not their values. the captured json becomes the artifact used by the offline answer step.
validation and ambiguity
every requested url is resolved against the current response and checked against the assigned origin before it is sent. backup links, scripts, feeds, object routes and redirects all pass the same check. a response that tries to leave the origin fails loudly.
the offline step maps a capture to a question by its answer id or target number. it accepts exactly one requested value in that target's responses. zero values produce a compact diagnostic with attempted recipes, forms, same-origin links, api routes and statuses. multiple values are rejected rather than choosing one by position.
integration
each target is captured concurrently, with one persistent connection per target. the output ordering remains target order even though network work runs in parallel. the solver then reads only saved bytes:
captures = capture_targets(assignment, challenge)
answers = {question.id: answer(captures, question) for question in questions}
a new vulnerability family can install one challenge-local recipe through the custom hook. that keeps ungraded active behavior out of the shared collector while preserving all origin, size, method and time limits.
limitations
the shared recipes cover the evidenced families described below. the html parser does not execute javascript, and the collector does not crawl arbitrary links. an application that requires a browser runtime, a cross-origin flow or a method beyond the bounded interface needs a challenge-local rule and an explicit review of the safety boundary.
day 20 additions
the first live families showed why a recipe must discover a route from the current target instead of carrying a path from a previous one. the collector now permits a challenge-local recipe to inspect a page's same-origin links or a json clue, follow the route the target advertises, and run only when that target has the expected feature. this matters in mastery, where one bundle can mix vulnerability families.
if viewer := find_viewer_link(page):
capture(get_same_origin(viewer.path, params=viewer.proof_request))
elif clues_path := find_clues_link(page):
clues = parse_json(get_same_origin(clues_path))
capture(get_same_origin(clues[route_key]))
a response timeout now retains the responses captured before the failure, together with the error and target index. when several proofs share a bundle, the offline step can validate the unaffected target responses independently; a failed target is never silently treated as an empty proof. the known recipes still obey the assigned-origin and request bounds described above.
day 21 additions
the completed reflected-xss, stored-xss and session targets share one clue endpoint, but the clue schema selects a different bounded flow. sparse pages in those known families can omit the clue link, so the challenge family enables one request to the already observed clue path; other targets still need target-local evidence or a known-family fallback.
for a reflected callback, the collector uses the clue's route and nonce to build the q value, checks that the response contains the payload verbatim, then sends the callback request with same-origin Origin and Referer headers. it does not treat a reflected string alone as success.
the stored flow reads openapi only after the moderator-data and receipt-template clues match. a unique non-receipt api POST receives the note, and the response's detail url becomes the only polling target. receipt polling waits up to 60 seconds after submission, but still shares the target's 16-response limit and the gm attempt deadline. ordinary target work keeps its 25-second budget.
the session flow sends the clue's value as sid to its feature route, then reuses the isolated cookie jar on the protected route. a separate case-portal fallback reads the authenticated activity feed and requests only reference-only or other-owner case ids. both preserve per-target cookies and same-origin checks, so a mixed mastery bundle can use different recipes without sharing state.
if "callback_nonce" in clues:
verify_raw_reflection_then_request_callback(clues)
elif "moderator_data_route" in clues and "receipt_route_template" in clues:
submit_note_and_poll_its_detail_receipt(clues)
elif "session_hint" in clues:
dispatch_with_sid_then_request_protected_route(clues)
day 22 additions
day 22 added two target-local web flows. for a preference form, the collector now reads the method, action, hidden controls, and selected option from the current page. it submits those observed values and derives the request context headers from the target's own form data before following the same-origin evidence link.
the upload flow reads the current feature route from the target's clue response, uses the upload structure described by its guide, and follows the preview path returned by that upload. neither route nor preview path is carried between target instances.
each saved capture stays associated with its target and question. this keeps proofs from unrelated mastery targets from being combined; a returned proof is treated as opaque response data and left unchanged.
form = parse_observed_form(page)
submit_observed_controls(form)
follow_same_origin_evidence(form.response_link)
clues = read_target_route_map(page)
upload = send_documented_upload(clues.feature_route, guide)
follow_same_origin_preview(upload.preview_path)
day 23 additions
three more families use the clue document as a route map. the rendering target names a feature route and a framework; the collector sends its expression as a query value and reads the rendered response. the delayed query target names a feature route and a result route; a json write is followed by a read using the same target session. the profile target also names two routes, but first requires an exact administrator-required audit response before a synthetic update. a different audit response skips the update.
if framework == "Spring Boot":
read_rendered_feature(clues.feature_route)
elif framework == "Sinatra":
store_condition_then_read_result(clues.feature_route, clues.secondary_route)
elif framework == "NestJS" and audit_requires_administrator(clues.secondary_route):
update_synthetic_profile_then_read_audit(clues.feature_route, clues.secondary_route)
each route is checked against the current assigned origin. the proof comes from the final response of that target and remains attached to its target number and question, including in a mixed mastery bundle.
day 24 additions
an api target can disclose its own inventory route and accepted inputs for a feature request. the collector reads that inventory from the current assigned origin, confirms the feature route matches the clue, and tries only the advertised combinations under a fixed cap. it ranks the observed working pair first and still requires proof text in the returned body.
a target-side fetch is gated by two responses from the same target: the feature's absolute-url requirement and the clue-named route's internal-only response. only then does the collector encode that route with the assigned container's loopback listener into one feature request. it does not scan ports, hosts, or paths.
a sparse page that says a note will be opened by a moderator in a browser can identify the stored-moderator family even when the page omits its clue link. the collector then reads the established clue endpoint once and continues only if that response supplies the expected route keys.
mastery target mapping now uses the assignment's ordered answer list before question ids. this preserves the target-to-answer pairing when question ids describe a different ordering after retry.
day 25 additions
the first capture now follows a clue's example request when the feature says its path input is required. it records the example output, then makes one harmless identity check only when the example response identifies a command executor. this gives a later solver the response pair needed to determine whether the input crossed a shell boundary. neither a successful status nor a directory listing is treated as a proof.
two more bounded flows follow target-local documents. a recovery guide supplies the request and redeem routes, parameter names and account records; the collector obtains a token for a listed self-service account and tests redemption against a listed protected account. a graphql clue supplies the endpoint and protected reference; the collector sends a json POST and records both data and errors. the graphql fallback currently knows the field names observed on day 25, so a changed schema needs discovery from that target before it can be answered.
if clue_has_example_path(clues) and feature_requires_path(capture):
capture_example_then_identity_check(clues)
elif recovery_guide_is_complete(clues):
request_token_then_redeem_for_listed_account(clues)
elif graphql_reference_is_complete(clues):
query_current_endpoint_and_record_errors(clues)
each flow remains within the assigned origin and keeps its output attached to that target's question in a mixed bundle.
day 26 additions
the clue reader gained bounded flows for a webhook event and a state-format document. the webhook request uses the event type and owner scope supplied by the current clue under the route's accepted alias names. the state flow decodes the current format response, changes a role in the structured object, and re-encodes it for the feature route. neither flow carries a route or role value from a previous target.
clues = read_clues(target)
if event_type_and_scope(clues):
follow_event_body_aliases(clues)
elif state_format_route(clues):
follow_current_state_format(clues)
day 27 additions
a clue-directed stream can now use websocket. the client validates the same-origin upgrade, sends a masked json frame with the clue's stream reference, joins response continuation fragments, and bounds close and timeout behavior before looking for a proof in the complete message. a plain get cannot stand in for that protocol exchange.
stream = websocket_connect(clues["feature_route"], params=current_reference(clues))
stream.send_json({"stream_reference": current_reference(clues)})
message = reassemble_fragments(stream)
proof = one_proof(message)
day 28 additions
two additional clue-following shapes now run in the shared collector. a policy-format response supplies the current query parameter and prefix for a bounded feature request. an export clue supplies a feature route and a secondary status route; the collector reads a generated id, the report schema, and event records before the follow-up. both shapes are guarded by response structure and the assigned-origin boundary, so a different page does not trigger them merely because its challenge title is familiar.
clues = read_clues(target)
if policy_format_route(clues):
follow_documented_policy_format(clues)
elif feature_and_status_routes(clues):
follow_export_and_audit_report(clues)
the saved captures are replayed with rotated routes and ids to check that the method follows target-local evidence, including in a mixed mastery bundle.
a mixed mastery variant also exposed an older file-counter target whose clue named an upload schema, preview guide and render route. the collector now reads that guide for the objective field, submits one bounded upload, and follows only the preview path returned by that target. this path is enabled by all three clue keys, so unrelated upload forms do not trigger it.
a rails one-credit variant requires two simultaneous redemption posts on the same session. the collector first checks the clue-provided status route for one remaining credit, races two posts to the feature route, then reads status again for proof. the guard runs before any serial post can consume the credit.
day 29 additions
two clue-following shapes from the day's new classes are now named in the shared collector: an update-inspector flow that reads the clue-named policy and feed routes and posts the signed manifest with only its hook field swapped to the restricted name the policy lists, and a case-fetch flow that composes the loopback service url a topology route discloses with a cross-role case id from the directory route, passed through the feature route's own named parameter.
a generic bounded fallback now covers renamed variants of these classes: it follows every same-origin route the clue document names, then sends what those bodies disclose through the feature route in ranked shapes -- the records bare or wrapped, a privilege-swap variant for a post-only inspector, a composed fetch target for a parameter route, a base64-json state blob with its role scalar swapped to a value the same blob discloses, and a re-read of the followed routes after a post, since some classes only show the proof there. every candidate is a fact the target itself disclosed, same-origin only, under a fixed request budget.
for route in clue_named_routes(clues):
bodies[route] = get(route)
for shape in ranked_shapes(bodies, plain_reply):
post_or_param(feature_route, shape)
if proof():
return
the first-failure packet a no-proof capture carries now includes a compact clue-route disclosure, so the reviewing solver starts from every route body's keys and scalar values instead of re-deriving them.
day 30 additions
the day's two novel classes are deterministic families in the shared collector now, so a renamed variant is a sub-second win rather than a per-target recipe. the delegated-identity boundary: an access-denied or forbidden reply whose error names identity or delegation, or a clue document naming an identity-directory route; the directory discloses the accepted identity, and the sweep carries each disclosed value under the observed header first, then the standard trust headers and the close spellings, feature route before secondary, inside a fixed budget of header-value pairs.
for header in identity_headers():
for value in disclosed_identities():
reply = get(feature_route, headers={header: value})
if proof(reply):
return
the queue-then-audit flow: a 405 post_required route; the post prepends the restricted owner shapes the target itself discloses, collects id-like scalars from each job response, then reads the clue-named audit route per id under its own 400-disclosed query parameter, with one forwarded-for retry on a denied read. both classes are pinned on capture-faithful local targets that replay the day's exchanges, and the matcher beats them end to end with no per-target recipe.
the loop around the collector gained its own liveness: a ten-second classifier reads each racer's record beat, last transcript line and process state -- never a target request -- and respawns a stalled or dead one immediately, never under a live attempt; the runner's readiness check also stopped refusing a ranked start over a proofless non-graded set.
changelog
- day 19: added assigned-origin validation, persistent per-target sessions, bounded response capture, per-target recipe ranking, concurrent mastery capture, offline answer mapping and failure fingerprints.
- day 20: target-local route discovery and applicability guards for viewer and clue flows; partial capture retained on a target read failure and unaffected bundle responses validated independently.
- day 21: clue-selected reflected callback, moderator receipt, and session-hint flows; bounded asynchronous receipt polling and a case-portal fallback for heterogeneous mastery targets.
- day 22: added observed-form handling, per-target evidence mapping, and a clue-directed upload-to-preview flow for the new target families.
- day 23: added clue-directed template, delayed query, and guarded profile flows with target-local routes and proof extraction.
- day 24: added clue-driven api inventory requests, evidence-gated internal fetch, ordered target-to-answer mapping, and a target-text cue for sparse moderator-review pages.
- day 25: captured clue example and command-boundary responses, added guide-driven recovery and target-local graphql flows, and retained graphql errors for inspection.
- day 26: added clue-derived webhook event aliases and encoded state-format flows.
- day 27: added same-origin websocket capture with fragmented-frame reassembly.
- day 28: added guarded policy-format, export-audit, upload-preview and one-credit race flows with saved-capture and local-http regressions.
- day 29: named the update-inspector and case-fetch flows, added the generic bounded fallback for renamed clue-routed classes, and carried a compact clue-route disclosure in the first-failure packet.
- day 30: named the delegated-identity sweep and queue-then-audit families, pinned both on capture-faithful local targets, and added the racer liveness classifier and the fail-soft readiness check.
{generated by a clanker}