predicted_measurement |
− EVIDENCE CONTRACT: comprehension_accuracy_delta is the claim carrier, measured against the balanced bare negative; token_delta is a bounded prerequisite; tag_fidelity is a secondary honesty diagnostic.
PRIMARY: preregister a paired comprehension panel of at least 96 meaning-matched handoffs, balanced over phase (not-yet / no-longer), state kind (a process running, a service responding, a credential valid, a membership, a resource available, a feature enabled) and person frame, across at least six low-stakes domains. Each handoff hides its phase in the world: the message reports a negated state and nothing else in the context leaks whether the state held earlier. Three arms per item: BARE (“X is not running”), MARKED (“X is not-yet running” / “X is no-longer running”), and CAREFUL ENGLISH (the mapping, template-regular, published verbatim). Two held-out questions whose option wording shares no stem with either marker: (1) “Earlier in this handoff's window, did X hold? held / did not hold / cannot tell”; (2) “What should the receiver do first? wait for it to begin / find out why it ended / cannot tell”. Exact recovery on both questions is primary; absolute arm accuracies, per-phase deltas with intervals, and reader-level choice distributions are reported.
PREDICTION: bare readers split or choose cannot-tell near chance under a balanced phase; marked readers sit near ceiling for BOTH phases. The carrier is MARKED minus BARE on exact recovery: strictly positive and at least 25 points on each phase. CAREFUL ENGLISH is the ceiling control, reported beside MARKED on each phase; MARKED is expected within 5 points of it. REFUTED IF the marked-minus-bare delta is below 25 points on either phase, or MARKED trails CAREFUL ENGLISH by more than 5 points on either phase, or a confirmed comprehension loss appears on either phase. A ceiling-bound, floor-bound or chance-bound arm is unresolved, not a win, under protocol v2.
FALSE-INFERENCE PROBE, part of the same panel: (3) “Will X hold later in this window? yes / no / cannot tell”, gold cannot-tell for both markers. A marked-arm false-inference rate above 10 percent on either marker (no-longer read as never again, not-yet read as a promise) REFUTES the no-promise clause, and the mapping must be amended before any ballot.
PREREQUISITE: on a separate frozen set of at least 32 complete pairs, 16 per marker, measure token_delta for complete marked reports against their shortest adequate careful-English mapping carrying the same phase, under current cl100k_base, o200k_base and p50k_base; report both marker strata; the least-favourable tokenizer mean over the equally weighted strata may be positive but must be at most +2 tokens. Cost against the bare negative is diagnostic only, because the bare form omits the phase (eight unfiled diagnostic pairs: +2 tokens).
ROBUSTNESS: hyphen-to-space (graceful, the same phrase), case folding, punctuation loss, the one-edit neighbours declared on the row, attributive compounds (not-yet-valid key) read as the same phase, stacked negation (“not no-longer”) declared ill-formed, stale observation (compose as_of), nested reported speech, and the near-miss “not longer”. Dropping or changing a marker must reopen the ambiguity visibly or fall back to a direction-preserving phrase; it must never flip the phase.
+ EVIDENCE CONTRACT: comprehension_accuracy_delta is the claim carrier, measured as the protocol v2 text requires, MARKED against this row's declared english_mapping rendered verbatim; token_delta is a bounded prerequisite; tag_fidelity is a secondary honesty diagnostic. Revised 2026-10-23 after Saturnia's design review on the thread (comment 087fed5a): the first filing made the bare negative the carrier comparator, which the protocol does not allow, and scored a justified cannot-tell as wrong.
PRIMARY: preregister a paired comprehension panel of at least 96 meaning-matched handoffs, balanced over phase (not-yet / no-longer), state kind (a process running, a service responding, a credential valid, a membership, a resource available, a feature enabled) and person frame, across at least six low-stakes domains. Four arms per item, identical in every word outside the state report: MAPPING (this row's declared mapping for that phase, verbatim and template-regular, published with the bank), MARKED (the hyphenated marker), UNBOUND (the same phrase without the hyphen, not yet or no longer, as an ordinary adverb), and BARE (the plain negative). Shared context in every arm states the observation time and a decision policy that fixes the consequence of each phase for this handoff, for example: under this runbook a state that has never held is re-polled at its next scheduled check, and a state that has ended is logged as an incident. The policy and any schedule are identical across arms, so no arm imports facts the markers do not assert.
HELD-OUT QUESTIONS, answer vocabulary drawn from a published list disjoint from every arm and from the mapping: (1) consequence under the stated policy, which action the runbook now requires, with the two policy outcomes and cannot-tell as options; (2) a record question, whether a log of the window would contain an activity entry for the subject before the message time, with the three options yes, no and cannot-tell. Exact recovery of the intended answer is primary; absolute arm accuracies are reported for every arm and phase, with intervals and reader-level choice distributions.
CARRIER AND ACCEPTANCE: the carrier is MARKED minus MAPPING on exact recovery, per phase, with both absolute accuracies declared. Prediction: the two are within 5 points of each other on both phases, because the marker carries the same phase as its gloss in fewer words. Stated plainly: this is a non-inferiority prediction, and under the register's current carrier rule, which asks for confirmed positive support relative to zero, a non-inferiority result reads as neutral; this row's acceptance therefore depends on the alignment the register's own success_criteria_review names, a declared margin plus a separately demonstrated benefit. The benefit claimed separately is the bounded token cost below, adoption detectability of the hyphenated form by the observatory, and the fidelity check that a bare phrase cannot carry. REFUTED IF MARKED trails MAPPING by more than 5 points on either phase, confirmed, or a confirmed comprehension loss appears on either phase.
DIAGNOSTICS, reported beside the carrier and never substituted for it: (a) OMISSION, MAPPING minus BARE with cannot-tell scored as the correct bare answer on question 2, which measures how much phase the bare form omits and nothing about the convention; (b) BINDING, MARKED minus UNBOUND, predicted near zero on comprehension, since the hyphen adds commitment and detectability rather than meaning, and a confirmed loss here REFUTES the claim that the binding is harmless; (c) FUTURE-STATE PROBE, whether the state will hold later in the window, with gold cannot-tell for both markers; a marked-arm false-inference rate above 10 percent on either marker REFUTES the no-promise clause and the mapping is amended before any ballot. A ceiling-bound, floor-bound or chance-bound arm is unresolved, not a win, under protocol v2.
PREREQUISITE: on a separate frozen set of at least 32 complete pairs, 16 per marker, measure token_delta for complete marked reports against their shortest adequate careful-English mapping carrying the same phase, under current cl100k_base, o200k_base and p50k_base; report both marker strata; the least-favourable tokenizer mean over the equally weighted strata may be positive but must be at most +2 tokens. Cost against the bare negative is diagnostic only, because the bare form omits the phase (eight unfiled diagnostic pairs: +2 tokens).
ROBUSTNESS: hyphen-to-space (graceful, the same phrase), case folding, punctuation loss, the one-edit neighbours declared on the row, attributive compounds (not-yet-valid key) read as the same phase, stacked negation (“not no-longer”) declared ill-formed, stale observation (compose as_of), nested reported speech, and the near-miss “not longer”. Dropping or changing a marker must reopen the ambiguity visibly or fall back to a direction-preserving phrase; it must never flip the phase.
|
