perf: accelerate uchardet language and candidate analysis

[PyYoshi]

Summary

• cache repeated language-model code-point lookups per detector
• cache MBCS candidate analysis, avoid duplicate known-language candidates, and reuse SBCS filtering storage
• add direct C API benchmark and exact candidate-output comparison tools to the maintained uchardet fork
• publish native and Python-level results while preserving the public cChardet/uchardet APIs

The uchardet changes are in PyYoshi/uchardet@7993e0a. No public C or Python API is removed or changed.

Performance

Direct uchardet C API benchmark, 158 bundled fixtures preloaded before timing (Clang Release, median of 200 samples):

Fresh detector	Baseline	Tuned	Improvement
Whole file	63.456 ms	28.868 ms	54.5%
64-byte chunks	66.057 ms	31.991 ms	51.6%

On 19 charset-normalizer sample files, the native median fell from 19.777 ms to 8.694 ms (56.0%).

A rigorous 166-file / 641,202-byte Python benchmark with native acceleration enabled measured:

Detector	Serial	4 threads
cChardet + tuned uchardet	55.9 ms	30.4 ms
chardet 7.6.0 (mypyc)	125 ms	194 ms
charset-normalizer 3.5.1 (Cython)	88.6 ms	110 ms

The same benchmark using the sibling projects pure fallback paths is documented separately so charset-normalizer Cython and chardet mypyc builds are not conflated with their Python fallbacks.

Compatibility and validation

• all 153 enabled uchardet CTest cases pass
• candidate count, order, encoding, language, and confidence are byte-identical to baseline on all 158 fixtures with whole-file and 64-byte feeds
• identical results on the charset-normalizer samples and 100 random byte inputs at chunk sizes 1, 7, 64, and 1,024
• exported uchardet C symbol set and public header signatures are unchanged
• canonical uchardet accuracy scores are unchanged
• Python tests pass on 3.11, 3.12, 3.13, and 3.14 (28 tests each)
• Ruff and strict mypy pass
• Clang Static Analyzer reports no bugs
• ASan/UBSan pass all 153 tests and byte-by-byte corpus feeding (LeakSanitizer disabled because the environment runs under ptrace)

Notes

The benchmark intentionally calls the public uchardet C API directly and preloads input data, avoiding Python and filesystem overhead while tuning the native implementation. The Python benchmark remains as an end-to-end comparison among cChardet, chardet, and charset-normalizer.