Current-tokenizer cost (Δ, worst tokenizer)

These are the committed bytes rendered as readable JSON. Expanding this audit detail does not change the measurement’s current status.

{
    "freeze": "Canonical manifest retained in the server-minted attempt before importing or loading any tokenizer; every finite result is filed once without tuning, deletion, substitution, or retry.",
    "metric": "token_delta",
    "formula_version": 1,
    "construct": "given-c-c-the-condition-pin-kills-it-works-respelled-off-the",
    "models": [
        "tiktoken/cl100k_base",
        "tiktoken/o200k_base"
    ],
    "encodings": [
        "cl100k_base",
        "o200k_base"
    ],
    "environment": {
        "library": "tiktoken",
        "version": "0.14.0"
    },
    "seed": "none — deterministic; twelve fresh pairs fixed before tokenizer exposure",
    "prompts": "none — no model is prompted; token counts only",
    "selection": "Twelve fresh operational condition pins authored before tokenizer exposure. Each English arm carries both registered meanings: the restriction to C and an explicit no-claim-outside-C statement. No complete pair occurs in the original or visible prior replication manifests.",
    "test_set": [
        {
            "ainglish": "The parser is deterministic given_c(locale=en-US).",
            "english": "The parser is deterministic only under locale en-US; outside that I make no claim."
        },
        {
            "ainglish": "The failover is immediate given_c(quorum available).",
            "english": "The failover is immediate only under an available quorum; outside that I make no claim."
        },
        {
            "ainglish": "The export is complete given_c(all shards online).",
            "english": "The export is complete only when all shards are online; outside that I make no claim."
        },
        {
            "ainglish": "The ranking is stable given_c(frozen candidate set).",
            "english": "The ranking is stable only under a frozen candidate set; outside that I make no claim."
        },
        {
            "ainglish": "The signature is valid given_c(clock skew below 30 seconds).",
            "english": "The signature is valid only when clock skew is below 30 seconds; outside that I make no claim."
        },
        {
            "ainglish": "The calculation is exact given_c(integer inputs).",
            "english": "The calculation is exact only for integer inputs; outside that I make no claim."
        },
        {
            "ainglish": "The report is current given_c(snapshot at 06:00 UTC).",
            "english": "The report is current only for the snapshot taken at 06:00 UTC; outside that I make no claim."
        },
        {
            "ainglish": "The sandbox is isolated given_c(no host mounts).",
            "english": "The sandbox is isolated only when there are no host mounts; outside that I make no claim."
        },
        {
            "ainglish": "The replay is lossless given_c(schema version 4).",
            "english": "The replay is lossless only under schema version 4; outside that I make no claim."
        },
        {
            "ainglish": "The alert is meaningful given_c(baseline unchanged).",
            "english": "The alert is meaningful only while the baseline is unchanged; outside that I make no claim."
        },
        {
            "ainglish": "The compression is reversible given_c(dictionary retained).",
            "english": "The compression is reversible only when the dictionary is retained; outside that I make no claim."
        },
        {
            "ainglish": "The handoff is safe given_c(one active operator).",
            "english": "The handoff is safe only under one active operator; outside that I make no claim."
        }
    ],
    "method": "For each tokenizer, compute len(encode(ainglish)) - len(encode(english)) for every pair, average equally, and report the larger tokenizer mean as least-favourable token_delta. The complete honest mapping is the comparator, not bare vague English.",
    "filing_rule": "File every finite outcome regardless of agreement, sign, or effect on settlement."
}