CVE-2026-85670
Description
tokenizers (Hugging Face) is affected by an out-of-bounds buffer access in BpeBuilder::build (tokenizers/src/models/bpe/model.rs). When loading a tokenizer.json via Tokenizer::from_file/from_str, the builder sizes a scratch buffer to the longest vocabulary key, then writes each concatenated merge rule into it. A merge whose concatenated token exceeds the longest vocabulary key overruns the buffer, which Rust turns into a panic that aborts the process in Rust and FFI embeddings. This occurs at load time with no encoding required, so an attacker who supplies a crafted tokenizer.json can cause a denial of service. A secondary defect at the same location can cause a usize underflow (panic in debug, potential memory corruption in release) when continuing_subword_prefix is set and a merge token is shorter than the prefix. Observed in version 0.23.1.
Predictions
Heuristic predictions, AS-IS, for prioritization only.
Mitigations
No mitigations published for this CVE yet.
The vendor-content worker queues fetches as references arrive (check back in a few minutes). Or — if you've already worked around this in production — publish your fix to the community-verified tier.
Propose a mitigation on Community Mitigations published via the community go through AI scoring + 2 human reviewers + 7-day silent objection window before landing here withsource_tier=community-verified.
References
CWEs
CWE-787
Community-verified mitigations for this CVE will appear above when contributors publish them.
Verify integrity in audit chain (admin only). AS-IS.