Skip to content

Fix toSnakeCase/toCamelCase silently dropping non-Latin identifiers - #6144

Open
nexrall wants to merge 1 commit into
drizzle-team:betafrom
nexrall:fix/casing-unicode-6082
Open

Fix toSnakeCase/toCamelCase silently dropping non-Latin identifiers#6144
nexrall wants to merge 1 commit into
drizzle-team:betafrom
nexrall:fix/casing-unicode-6082

Conversation

@nexrall

@nexrall nexrall commented Aug 19, 2026

Copy link
Copy Markdown

Summary

toSnakeCase/toCamelCase's word-splitting regex only matched ASCII letters and digits ([\da-z], [A-Z]). String.prototype.match() silently discards any character that doesn't match one of the pattern's alternatives, so Korean, Japanese, Cyrillic, and other non-Latin characters were dropped entirely rather than left unchanged — not merely miscased.

For snakeCase.table() this means a Korean column name like 칼럼명 becomes an empty string, which drizzle-kit generate turns into an invalid empty SQL identifier ( ``), and the resulting migration fails to apply.

Closes #6082

Root cause

// before
.match(/[\da-z]+|[A-Z]+(?![a-z])|[A-Z][\da-z]+/g) ?? []

None of the three alternatives matches a Hangul syllable, CJK ideograph, or even accented Latin (é, ñ) — so .match() simply omits those characters from the result array instead of keeping them as their own word.

Fix

Generalized the word-boundary regex with Unicode property escapes instead of ASCII character classes:

const WORD_PATTERN = /[\p{Ll}\d]+|\p{Lu}+(?!\p{Ll})|\p{Lu}[\p{Ll}\d]+|\p{Lo}+/gu;
  • \p{Ll} / \p{Lu} extend the existing lower/upper-case logic to any script with case distinctions (Latin, Cyrillic, Greek, etc.), so accented Latin (café, Résumé) is preserved instead of stripped.
  • \p{Lo} is a new alternative for caseless scripts (CJK ideographs, Hangul syllables) that have no upper/lower distinction to split words on.

All existing ASCII test cases keep producing byte-identical output — this only changes behavior for input that was previously being silently corrupted.

Input Before After
칼럼명 '' (bug) 칼럼명
日本語Test test (bug, dropped 日本語) 日本語_test
café caf (bug, dropped é) café
drizzleORM drizzle_orm drizzle_orm (unchanged)
drizzleOrmAndKit drizzle_orm_and_kit drizzle_orm_and_kit (unchanged)

Test plan

  • Added regression tests to tests/casing/casing.test.ts covering the exact Korean case from the issue, a mixed Japanese/Latin case, and an accented-Latin non-regression case.
  • Verified red→green: with the fix stashed, exactly the 3 new tests fail (for the right reason — dropped characters) while all 6 pre-existing tests still pass; restoring the fix turns all 9 green.
  • pnpm vitest run tests/casing/casing.test.ts — 9/9 passing.
  • pnpm vitest run tests/ (drizzle-orm package) — 967/970 passing. The 3 failures are in tests/sql-builder.test.ts and are pre-existing on unmodified beta HEAD — verified by stashing this fix entirely and re-running, which reproduces the same 3 failures unrelated to casing.ts.
  • tsc --noEmit — no new errors. The one pre-existing error (types-bench.ts, missing a generated scaffold file) reproduces identically on unmodified beta HEAD.
  • End-to-end verification through the real snakeCase.table() pipeline (not just the unit-level regex): built the package locally and ran the exact schema from the issue through snakeCase.table()getTableConfig(). Before the fix, the Korean column resolves to '' (bug reproduced exactly as reported); after the fix, it resolves to 칼럼명, matching the output of an unaffected sqliteTable() call with the same column name.

Why this targets beta

The issue's repro uses snakeCase.table(), which only exists on the 1.0.0-rc line (beta) — it isn't present on main (0.45.x). The underlying toSnakeCase/toCamelCase regex bug is present on both branches, but only beta can run the exact reproduction from the issue.

The word-splitting regex in toSnakeCase/toCamelCase only matched ASCII
letters and digits ([\da-z], [A-Z]). Array.prototype.match() silently
discards any character that doesn't match one of the pattern's
alternatives, so Korean, Japanese, Cyrillic, and other non-Latin
characters were dropped entirely rather than left unchanged - not
merely miscased.

For snakeCase.table() this means a Korean column name like 칼럼명
becomes an empty string, which drizzle-kit generate turns into an
invalid empty SQL identifier (`` `` ``), and the resulting migration
fails to apply.

Fix: generalize the word-boundary regex with Unicode property escapes
(\p{Ll}, \p{Lu}, \p{Lo}) instead of ASCII character classes:
- \p{Ll}/\p{Lu} extend the existing lower/upper-case logic to any
  script with case distinctions (Latin, Cyrillic, Greek, etc.), so
  accented Latin (café, Résumé) is preserved instead of stripped.
- \p{Lo} is added for caseless scripts (CJK ideographs, Hangul
  syllables) that have no upper/lower distinction to split words on.

All existing ASCII test cases keep producing identical output.
Verified end-to-end (not just the unit-level regex) by building the
package and running the exact schema from the issue through
snakeCase.table() -> getTableConfig(): the Korean column resolves to
'칼럼명' instead of ''.

Fixes drizzle-team#6082
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant