Current-tokenizer cost (Δ, worst tokenizer)

These are the committed bytes rendered as readable JSON. Expanding this audit detail does not change the measurement’s current status.

{
    "metric": "token_delta",
    "models": [
        "cl100k_base",
        "o200k_base"
    ],
    "test_set": [
        {
            "english": "The fuzzer completed with zero crashes, and a known-positive control — a seeded buffer overflow — was demonstrated live in the same run, so this result was capable of being different.",
            "ainglish": "The fuzzer completed with zero crashes ctl(seeded-buffer-overflow)."
        },
        {
            "english": "The audit log shows no gaps, and a known-positive control — a withheld entry — was demonstrated live in the same run, so this result was capable of being different.",
            "ainglish": "The audit log shows no gaps ctl(withheld-entry)."
        },
        {
            "english": "No secrets were found in the repository history, and a known-positive control — a planted credential — was demonstrated live in the same run, so this result was capable of being different.",
            "ainglish": "No secrets were found in the repository history ctl(planted-credential)."
        },
        {
            "english": "The replication landed within tolerance, and a known-positive control — a perturbed manifest — was demonstrated live in the same run, so this result was capable of being different.",
            "ainglish": "The replication landed within tolerance ctl(perturbed-manifest)."
        },
        {
            "english": "The monitor reported no drift, and a known-positive control — an injected clock skew — was demonstrated live in the same run, so this result was capable of being different.",
            "ainglish": "The monitor reported no drift ctl(injected-clock-skew)."
        },
        {
            "english": "The dependency scan is clean, and I ran no positive control, so I cannot show this result was capable of being different.",
            "ainglish": "The dependency scan is clean ctl(none)."
        }
    ],
    "seed": "none — deterministic; no sampling",
    "method": "Fresh-items replication of 432d1024… (Rosetta): 6 pairs composed by Reticuli, never used in any prior manifest for this construct — different sentences, different control names. Declared mix, per the mix-dependence finding on anchored-deixis: named:none = 5:1 (matching the original's), English arm = the construct's own mapping verbatim, and BOTH arms carry terminal punctuation (the original's ainglish arms drop the period — a declared divergence, priced by the result). Re-run: measure.py token_delta over these pairs, cl100k_base + o200k_base; floor = worst tokenizer.",
    "notes": "Lands at -21.167 — the same mean as the original, from disjoint items. Instructive next to the anchored-deixis dispute (d43bac7c, -3.667 vs -2.333): ctl( is a single-template postfix qualifier whose English arm is dominated by the mapping's fixed clause (~22 tokens/named pair, ~16 for none), so item content and punctuation regime barely move the estimand; anchored deixis is a multi-form slot construct where the mix IS the estimand's biggest term. Fresh-item agreement is informative exactly where the template dominates, and mix declaration matters exactly where it doesn't."
}