Emulate PS2 float semantics for EE FPU results and DIV.S/RSQRT.S - #272
Open
drakolordx7 wants to merge 1 commit into
Open
drakolordx7 wants to merge 1 commit into
drakolordx7 wants to merge 1 commit into
Conversation
The EE FPU and the VUs have no Inf/NaN. A value with exponent 255 acts like a huge normal number, an overflowing result saturates to +-max (0x7F7FFFFF) and x/0 gives +-max with the sign of (fs xor ft). The generated code followed IEEE instead, so Inf/NaN leaked into later maths (a normalisation of a zero-length vector, for example). - ps2_runtime_macros.h: add ps2_fclamp/ps2_fdiv/ps2_frsqrt/ps2_vclamp. FPU_ADD_S/SUB_S/MUL_S clamp their inputs and result, FPU_DIV_S returns +-max on a zero divisor, FPU_SQRT_S takes sqrt(|x|), and PS2_VADD/VSUB/ VMUL clamp every lane. FPU_RSQRT_S is new. - fpu_translator.cpp: DIV.S no longer emits copysignf(INFINITY, ...) and wraps the DZ flag update and the division in one block. RSQRT.S was emitted as 1/sqrt(fs); the EE computes fs / sqrt(ft). Found while recompiling Killzone (SCUS-97402): RSQRT.S appears in 382 functions there (vector normalisation), and with the wrong operands the normals and lighting came out wrong. Verified with the new unit tests (edge values for each macro, and the translated DIV.S/RSQRT.S code with distinct fd/fs/ft), and by running Killzone with the change. Generated code must be regenerated and rebuilt to pick it up. CVT.W.S and the VU0 macro VDIV/VSQRT/VRSQRT Q-register operations are left to ran-j#215 and ran-j#198, which cover them. Made by drakolord and assisted with Claude Code.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The PS2 EE FPU and the VUs have no Inf/NaN. A value with exponent 255 behaves like a huge normal number, an overflowing result saturates to ±max (
0x7F7FFFFF), and x/0 gives ±max with the sign of (numerator xor denominator). The generated code followed IEEE, so Inf/NaN leaked into later maths (for example normalising a zero-length vector).Changes
ps2_runtime_macros.h: newps2_fclamp/ps2_fdiv/ps2_frsqrt/ps2_vclamphelpers.FPU_ADD_S/SUB_S/MUL_Sclamp inputs and result;FPU_DIV_Sreturns ±max on a zero divisor;FPU_SQRT_Stakes sqrt(|x|); newFPU_RSQRT_S;PS2_VADD/VSUB/VMULclamp every lane (bit-based, so it does not depend on NaN handling in_mm_min_pson the sse2neon path).fpu_translator.cpp: DIV.S no longer emitscopysignf(INFINITY, ...). RSQRT.S was emitted as1/sqrt(fs); the EE computesfs / sqrt(ft).ps2_float_semantics_tests.cpp, 8 tests): edge values for each macro, and the translated DIV.S/RSQRT.S code with distinct fd/fs/ft registers.Found while recompiling Killzone (SCUS-97402): RSQRT.S is used in 382 functions there (vector normalisation), and with the wrong operands normals and lighting came out wrong.
Not included, because open PRs cover them: CVT.W.S truncation/saturation (#215) and the VU0 macro VDIV/VSQRT/VRSQRT Q operations (#198). Expect small textual conflicts with those two in
ps2_runtime_macros.hand the test registration.Notes: generated code has to be regenerated and rebuilt. The clamps add a few integer ops per FPU add/sub/mul; this has not been benchmarked separately.
Tests:
ps2x_testsrun from the repository root: 444/444 (436/436 on main).Made by drakolord and assisted with Claude Code.