Skip to content

DSK-658 - Support non ascii - #96

Merged
OleksiiShvachenko merged 2 commits into
spark2from
feature/DSK-658-support-non-ascii
Aug 28, 2026
Merged

DSK-658 - Support non ascii#96
OleksiiShvachenko merged 2 commits into
spark2from
feature/DSK-658-support-non-ascii

Conversation

@OleksiiShvachenko

@OleksiiShvachenko OleksiiShvachenko commented Aug 27, 2026

Copy link
Copy Markdown

DSK-658: mailcore paths are UTF-8 (fileSystemRepresentation), but the Windows layer used narrow CRT calls (fopen_s, _unlink, _mkdir, stat) that resolve paths through the ANSI codepage. Any path with characters outside the codepage — e.g. a temp dir under a non-ASCII user profile — failed to open, breaking RFC822 temp files (MCOErrorFile) and stalling Outbox.

Convert UTF-8 to UTF-16 and call _wfopen_s/_wunlink/_wmkdir/_wstat64i32, falling back to the narrow call when the input is not valid UTF-8. Also route MCDataStreamDecoder and MCCertificateUtils through MCWin32.h so their fopen calls hit the wrapper.


Note

Medium Risk
Changes core Win32 path handling for fopen, unlink, mkdir, and stat across the library via header macros; incorrect UTF-8 handling or stat layout assumptions could cause subtle file failures on Windows.

Overview
Fixes Windows file I/O when mailcore paths are UTF-8 (fileSystemRepresentation) but narrow CRT calls (fopen_s, _unlink, _mkdir, stat) interpret them via the ANSI codepage, which broke temp RFC822 files and Outbox on non-ASCII profile paths.

win32_fopen now converts paths with MultiByteToWideChar(CP_UTF8, …) and opens via _wfopen_s, with fallback to the existing narrow fopen_s when conversion fails or the mode string is too long. New win32_unlink, win32_mkdir, and win32_stat follow the same pattern using _wunlink, _wmkdir, and _wstat64i32.

MCWin32.h (source and public include) redefines unlink, mkdir, and stat to those wrappers (alongside the existing fopen macro), adds io.h/wchar.h, and declares the new helpers. MCDataStreamDecoder.cpp and MCCertificateUtils.cpp include MCWin32.h first so their fopen calls use the wrapper.

Reviewed by Cursor Bugbot for commit 72c919b. Bugbot is set up for automated code reviews on this repo. Configure here.

DSK-658: mailcore paths are UTF-8 (fileSystemRepresentation), but the
Windows layer used narrow CRT calls (fopen_s, _unlink, _mkdir, stat)
that resolve paths through the ANSI codepage. Any path with characters
outside the codepage — e.g. a temp dir under a non-ASCII user profile —
failed to open, breaking RFC822 temp files (MCOErrorFile) and stalling
Outbox.

Convert UTF-8 to UTF-16 and call _wfopen_s/_wunlink/_wmkdir/_wstat64i32,
falling back to the narrow call when the input is not valid UTF-8.
Also route MCDataStreamDecoder and MCCertificateUtils through MCWin32.h
so their fopen calls hit the wrapper.
@OleksiiShvachenko
OleksiiShvachenko force-pushed the feature/DSK-658-support-non-ascii branch from 5d96348 to 4cc613c Compare August 27, 2026 15:20
lxbndr
lxbndr previously approved these changes Aug 27, 2026

@lxbndr lxbndr left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Testing is problematic and comes ugly. It would become legacy very fast.

UPD. see tests were reverted

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 4cc613c. Configure here.

Comment thread src/core/basetypes/MCWin32.h
corecrt_io.h declares POSIX-compat unlink; when io.h is included after
MCWin32.h, the unlink macro rewrites that declaration into
extern "C" mailcore::win32_unlink and clang-cl rejects the language
linkage mismatch. Pre-include io.h (and wchar.h for _wstat64i32) so all
CRT declarations of the wrapped names precede the macros, matching how
direct.h and sys/stat.h are already handled.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@OleksiiShvachenko
OleksiiShvachenko merged commit e521317 into spark2 Aug 28, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants