english_mapping |
− "<subject> should-as-rule <predicate>" = "a norm that applies here — a policy, spec, schedule, promise, or good practice — calls for <subject> <predicate>; whether it actually happens, or happened, is a separate question the speaker is not settling." "<subject> should-as-forecast <predicate>" = "from how things normally go, the speaker expects <subject> <predicate>; no norm is invoked and nothing is recommended — if it fails to hold, the speaker's picture of the system was wrong; whether some norm also required it, and was broken, is a separate question this sentence does not settle." Both drop in where bare 'should' sits and ride the modal through negation and contraction: "shouldn't-as-rule <predicate>" = the norm calls for NOT <predicate>; "shouldn't-as-forecast <predicate>" = the speaker expects <predicate> not to hold. Perfect and past compose unchanged: "should-as-rule have completed by 02:10" = the schedule required completion by 02:10. When both claims are meant, mark each clause: "it should-as-rule have run, and I judge it should-as-forecast did". Lossless round-trip: "the backup should-as-forecast have finished" ⇄ "I expect the backup finished, going by its normal behaviour (whether anything required it to is a separate question)". Out of scope: conditional-inversion 'should' ("should the deploy fail, page me") is a different construction (= 'if'), and RFC 2119 SHOULD inside normative spec sentences keeps its BCP 14 meaning. Bare 'should' remains legal English: mark the modal when the fork is load-bearing.
+ "<subject> should-as-rule <predicate>" = "a norm that applies here — a policy, spec, schedule, promise, or good practice — calls for <subject> <predicate>; whether it actually happens, or happened, is a separate question the speaker is not settling." "<subject> should-as-forecast <predicate>" = "from how things normally go, the speaker expects <subject> <predicate>; no norm is invoked and nothing is recommended — if it fails to hold, the speaker's picture of the system was wrong; whether some norm also required it, and was broken, is a separate question this sentence does not settle." Both drop in where bare 'should' sits and ride the modal through negation and contraction: "shouldn't-as-rule <predicate>" = the norm calls for NOT <predicate>; "shouldn't-as-forecast <predicate>" = the speaker expects <predicate> not to hold. Perfect and past compose unchanged: "should-as-rule have completed by 02:10" = the schedule required completion by 02:10. When both claims are meant, mark each clause: "it should-as-rule have run, and I judge it should-as-forecast have run". Lossless round-trip: "the backup should-as-forecast have finished" ⇄ "I expect the backup finished, going by its normal behaviour (whether anything required it to is a separate question)". Out of scope: conditional-inversion 'should' ("should the deploy fail, page me") is a different construction (= 'if'), and RFC 2119 SHOULD inside normative spec sentences keeps its BCP 14 meaning. Bare 'should' remains legal English: mark the modal when the fork is load-bearing.
|
predicted_measurement |
− PRIMARY (claim carrier, comprehension_accuracy_delta): the marked forms preserve the meaning of their complete careful-English mapping, within 5 pp, on two held-out questions whose vocabulary is disjoint from the mapping's, each question naming the only evidence its answer may use. Each item is an operational context stating what is known about the world (whether the event happened, and whether anything required it, or that this is not stated) plus one sentence about the event; arms are the marked form and its complete careful-English expansion, read cold by two qualified readers from two model families at pinned digests. Q1 (sentence only): which does the SENTENCE assert, reading nothing else: (R) something required the event; (F) the writer predicts the event, and this sentence does not assert a requirement; (N) the sentence does not say which. Q2 (context and sentence both shown): what does the stated CONTEXT establish about compliance, answering from the context's statements only: a requirement is stated and was missed; a requirement is stated and was met; no requirement is stated, so compliance is unknown. Inputs shown, exactly: Q1 shows the sentence alone and no context; Q2 shows the context and the sentence together, so the arms differ on both questions and a Q2 difference between arms is interference, the marker moving a reader's reading of the world. With the sentence absent from Q2 the arms would receive identical inputs and Q2 could not lose. The sentence's force is never scored as a fact about the world, and the world is never scored as the sentence's commitment. Load-bearing strata, every one required for settlement: rule items (Q1 gold R), forecast items (Q1 gold F), and OVERLAP items, forecast-marked sentences in a context that states a requirement (Q1 gold F; Q2 gold: requirement stated and missed, or met, as the context says), at least 48 items each, intended readings balanced and the complement's tense, aspect, person and stativity balanced across the readings (checked before unblinding; this fork's known confound). Q2 gold is fixed by the context and identical across arms, so Q2 preservation measures whether the marker leads a reader to infer world facts the context did not state. DIAGNOSTIC arm, reported beside the primary and outside the scalar: bare 'should', Q1 gold N, so a bare reader who answers cannot-tell is scored correct; forced to choose R or F, bare readers are predicted to split near chance. Readers qualified within 7 days on target-independent controls; calibration gate 0.5 planted gap before any real cell; per-stratum 2,000-draw item bootstrap intervals beside the point. PREDICTION: marked minus careful English within [-5, +5] pp on Q1 and Q2 in all three load-bearing strata, lower interval bound above -10 pp. REFUTED IF: any load-bearing stratum's marked arm sits more than 10 pp below careful English on either question with all gates passing; or fewer than 144 admissible items with both readings live can be authored, which would show context already fixes the force. Prerequisite token_delta at most 0 against the same complete mapping (confirmed -11 on the predecessor's inputs; re-run on the successor's bank).
+ PRIMARY (claim carrier, comprehension_accuracy_delta): the marked forms preserve the meaning of their complete careful-English mapping, within 5 pp, on two held-out questions whose vocabulary is disjoint from the mapping's, each question naming the only evidence its answer may use. Each item is an operational context stating what is known about the world (whether the event happened, and whether anything required it, or that this is not stated) plus one sentence about the event; arms are the marked form and its complete careful-English expansion, read cold by two qualified readers from two model families at pinned digests. Q1 (sentence only): which does the SENTENCE assert, reading nothing else: (R) the writer invokes a norm calling for the event, of whatever strength the context gives it, and the sentence itself claims no strength; (F) the writer predicts the event and invokes no norm; (N) the sentence does not say which. Q2 (context and sentence both shown): what does the stated CONTEXT establish about compliance, answering from the context's statements only: a requirement is stated and was missed; a requirement is stated and was met; only a recommendation is stated, so an omission is not a breach; no norm is stated, so compliance is unknown. Contexts that state a norm state its strength in words, required or recommended but optional, and rule items carry both strengths, so a marker that silently adds mandatory force loses on Q2. Inputs shown, exactly: Q1 shows the sentence alone and no context; Q2 shows the context and the sentence together, so the arms differ on both questions and a Q2 difference between arms is interference, the marker moving a reader's reading of the world. With the sentence absent from Q2 the arms would receive identical inputs and Q2 could not lose. The sentence's force is never scored as a fact about the world, and the world is never scored as the sentence's commitment. Load-bearing strata, every one required for settlement: rule items (Q1 gold R), forecast items (Q1 gold F), and OVERLAP items, forecast-marked sentences in a context that states a requirement (Q1 gold F; Q2 gold: requirement stated and missed, or met, as the context says), allocated 96 rule, 48 forecast and 48 overlap items, disjoint, so each marked form appears in 96 items; every stratum is reported with its own interval and must settle on its own, and the scalar is the unweighted mean of the three stratum deltas, equal weight per stratum, so a pooled mean cannot hide the overlap stratum; each stratum delta is the mean of that stratum's Q1 delta and Q2 delta, so each of the six stratum-by-question cells carries one sixth of the scalar, and because the forecast form appears in the forecast and overlap strata it carries two thirds of the scalar by this choice, stated before any outcome: the overlap stratum is the case the fork exists for and is not down-weighted to equalise the forms; the scalar is a summary and the gate is each stratum on each question on its own interval; intended readings balanced and the complement's tense, aspect, person and stativity balanced across the readings (checked before unblinding; this fork's known confound). Q2 gold is fixed by the context and identical across arms, so Q2 preservation measures whether the marker leads a reader to infer world facts the context did not state. DIAGNOSTIC arm, reported beside the primary and outside the scalar: bare 'should', Q1 gold N, so a bare reader who answers cannot-tell is scored correct; forced to choose R or F, bare readers are predicted to split near chance. Readers qualified within 7 days on target-independent controls; calibration gate 0.5 planted gap before any real cell; per-stratum 2,000-draw item bootstrap intervals beside the point. PREDICTION: marked minus careful English within [-5, +5] pp on Q1 and Q2 in all three load-bearing strata, lower interval bound above -10 pp. REFUTED IF: any load-bearing stratum's marked arm sits more than 10 pp below careful English on either question with all gates passing; or fewer than 192 admissible items with both readings live can be authored, which would show context already fixes the force. Prerequisite token_delta at most 0 against the same complete mapping, declared as a bounded prerequisite on the tokenizer population cl100k_base, o200k_base and p50k_base (confirmed -11 on the predecessor's inputs; re-run on the successor's bank).
|