[wleeaf]

Q	A
Branch?	1.x
Bug fix?	yes
New feature?	no
Deprecations?	no
Issues	-
License	MIT

mb_strwidth() counts a character as two columns when it matches a hard-coded list of ranges. That list predates the emoji and most of the additions to EastAsianWidth.txt, so the polyfill and ext/mbstring disagree on many characters:

mb_strwidth('😀⌚日㉈');
// ext/mbstring: 7
// polyfill:     6, because 😀 and ⌚ count as one column and ㉈ (East Asian Ambiguous) counts as two

This mostly shows up as misaligned columns in CLI tables and progress output that contain emoji, when mbstring is not installed.

ext/mbstring treats the W and F code points of EastAsianWidth.txt as wide. A new Compiler::eastAsianWidthRegexp() builds the same set from that file, and update-unidata.php downloads the file and writes Resources/unidata/eastAsianWidthRegexp.php. mb_strwidth() loads the regexp like mb_convert_case() loads titleCaseRegexp. The committed file comes from Unicode 17.0.0. With it, a scan of every code point gives the same width as PHP 8.5's mb_strwidth() (0 differences, against 115 differing ranges before).

The new assertion in testStrwidth() calls the polyfill directly. It fails without the change and passes with it. ./phpunit tests/Mbstring passes, including the ill-formed UTF-8 cases.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

[nicolas-grekas]

Would Unicode 18 give the same map?

[wleeaf]

Almost: Unicode 18.0.0 only adds to it. Its EastAsianWidth.txt has 990 more W/F code points than 17.0.0 and none fewer, and all 990 are characters first assigned in 18.0 (e.g. U+18CD6..U+18CDA, U+18E00..U+19191, U+191A0..U+191D2, U+1B123..U+1B128, U+1F6D9, U+1FA8B..U+1FA8D). Every code point that is W or F in 17.0.0 is still W or F in 18.0.0.

I used 17.0.0 because that is what PHP 8.5's mbstring uses: with it, an all code point scan of mb_strwidth() gives 0 differences against native 8.5. With 18.0.0 the polyfill would count those 990 new characters as 2 where native 8.5 counts 1. If you'd rather track the latest Unicode, I can regenerate it from 18.0.0, it's the same script.

[nicolas-grekas]

Thank you @wleeaf.