Fix rtl string qa NULL-deref crashes (test bug + source hardening)#486
Open
leginee wants to merge 2 commits into
Open
Fix rtl string qa NULL-deref crashes (test bug + source hardening)#486leginee wants to merge 2 commits into
leginee wants to merge 2 commits into
Conversation
DamjanJovanovic
requested changes
Jun 21, 2026
|
|
||
| /* ======================================================================= */ | ||
| /* NULL-pointer guards */ | ||
| /* */ |
Contributor
There was a problem hiding this comment.
Please remove these NULL-pointer guards in release mode for better performance, and only use them in debug builds.
Contributor
Author
There was a problem hiding this comment.
do you think their performance impact is that visible?
Error case 1 of the test migration: ~40 *_000 cases in the rtl string
qa suites passed NULL into C string functions that dereference it
(e.g. rtl_str_compare(NULL, NULL)), causing 0xC0000005 AVs. These were
dormant under the old dmake build and only surface now that the tests
actually run. NULL violates the functions' documented contract
("must be null-terminated"), so the defect was in the tests, not the
(correct, unchanged) source. Fixed both sides for defense in depth.
Tests (qa/rtl/ostring/rtl_str.cxx, rtl_string.cxx,
qa/rtl/oustring/rtl_ustr.cxx):
- Rewrote the UB NULL-deref cases as contract-respecting boundary
tests (empty string, prefix/ordering-sign < 0 / > 0), which also
closes a previously-untested coverage gap (result sign was never
asserted).
- Added real assertions to the safe NULL-with-length-0 cases, which
document the length-bounded functions' tolerance as a regression
guard.
Source (rtl/source/strtmpl.c, ustring.c):
- Added entry-point NULL guards: OSL_PRECOND (diagnoses misuse loudly
in non-product builds, compiles out in product builds) plus a
defined empty-string fallback so the library never dereferences
NULL. Guards sit at function entry, outside the per-character loops,
so string-processing throughput is unchanged.
- strtmpl.c: one edit covers both the sal_Char and sal_Unicode
instantiations. getLength is the choke point (guarding it
transitively protects hashCode, lastIndexOf*, indexOfStr, trim);
compare/compareIgnoreAsciiCase/indexOfChar/replaceChar/
toAscii{Lower,Upper}Case/valueOfChar guarded directly.
- ustring.c: guarded the 6 mixed UTF-16/ASCII comparison helpers;
length-bounded args clamp the length to 0 to avoid NULL+0 pointer
arithmetic.
BUILD.bazel: rtl_str/rtl_ustr/rtl_string removed from the
"known upstream failures" notes; they now pass.
Verified: sal3.dll rebuilds (both template instantiations) and
rtl_str / rtl_ustr / rtl_string all pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2c29486 to
8f1ce3d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is the first out of 3 fixes from test migration. The original test did not make sense to me. I added the test for NULL pointers in the code itself as guards. And turned the tests into boundry tests.
The code is ai generated. best is to have more eyes to check it.
commit message for conveniance
Error case 1 of the test migration: ~40 *_000 cases in the rtl string qa suites passed NULL into C string functions that dereference it (e.g. rtl_str_compare(NULL, NULL)), causing 0xC0000005 AVs. These were dormant under the old dmake build and only surface now that the tests actually run. NULL violates the functions' documented contract ("must be null-terminated"), so the defect was in the tests, not the (correct, unchanged) source. Fixed both sides for defense in depth.
Tests (qa/rtl/ostring/rtl_str.cxx, rtl_string.cxx, qa/rtl/oustring/rtl_ustr.cxx):
Source (rtl/source/strtmpl.c, ustring.c):
BUILD.bazel: rtl_str/rtl_ustr/rtl_string removed from the "known upstream failures" notes; they now pass.
Verified: sal3.dll rebuilds (both template instantiations) and rtl_str / rtl_ustr / rtl_string all pass.