Repository navigation
fix: two SIGFPE crashes, an unbounded retry loop, and a sub-minimum GNFS abort - #98
Merged
Merged
Conversation
print_factors() builds prod from the factor list and divides flist->N by it without checking. A zero-valued factor in the list makes prod zero, and GMP raises SIGFPE from mpz_tdiv_r: the program dies with no diagnostic at all. That is exactly how the pp1/trial operand misplacement presented itself - an uninitialized operand reached new_factorization(), which recorded it as a factor, and the crash surfaced here: bbuhrow#5 __gmpz_tdiv_r bbuhrow#6 print_factors (fobj=...) at factor/factor_common.c:1431 Report the bogus list and skip the remainder bookkeeping instead, so a bad factor list becomes a visible error message rather than a signal. An empty list is not affected (its product is 1), which is what the surrounding "product of factors doesn't equal input" check below already relies on. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Arguments are placed according to the declared arity in function_nargs[]: with
arity na and k arguments supplied, the first argument lands in operands[na-k]
and the last in operands[na-1]. Two cases disagreed with the table.
pp1 was declared with arity 3 while its case handles at most two arguments, so
the shift was one slot off for every call shape:
pp1(n, bases) -> arguments in operands[1], operands[2]; operands[0] untouched
pp1(n) -> argument in operands[2]; operands[0] and [1] untouched
The case reads operands[0] and operands[1], so new_factorization() was handed
an uninitialized mpz (0) and print_factors() then divided by the zero product
of the resulting factor list. GMP raises SIGFPE, so every pp1 call died with
exit 136 before doing any work:
printf 'pp1(18446744073709551617)\n' | yafu -> SIGFPE
Declaring arity 2 puts both call shapes into the slots the case reads: a
two-argument call now fills operands[0] and operands[1], and a one-argument
call fills operands[1], which is what the one-argument branch already used.
trial had the mirror-image bug in its one-argument branch only: arity is 2, so
a single argument sits in operands[1], but that branch passed operands[0] to
new_factorization() (while correctly taking operands[1] for div_obj). Use
operands[1] for both. The crash was the same:
printf 'trial(15)\n' | yafu -> SIGFPE
Verified:
pp1(19327352823000000088046829527, 2) -> 2147483647 * 9000000000000000041
trial(15) -> 15 = 3 * 5
trial(3000018000027, 3000000) -> 3 * 1000003 * 1000003
trial(18446744073709551617) -> reports C20, no longer dies
(274177 is above the default
trial bound of 10000)
pp1 output stays probabilistic - a p+1 curve can degenerate to gcd == n - so a
single-base call may legitimately report nothing.
Document the placement rule above the table so the next case does not have to
rediscover it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
check_specialcase() finishes inputs below 115 bits with tiny-qs, and when that
leaves a composite behind it retries tiny-ecm in an unbounded loop:
do { found = getfactor_tecm(fobj->qs_obj.gmp_n, f1, 0, &lcg); ... }
while (!found);
Tiny-ecm cannot always succeed. When every factor of the remaining composite
is small enough that its curve group order divides the stage-1 exponent, each
curve collapses to gcd == n, which the caller correctly rejects as trivial, and
no amount of retrying will do better. 15 is such a case, and the loop spun on
it at 100% CPU forever - gdb sampling put it in ecm_stage2 -> mulmod128 with a
modulus of 15:
printf 'siqs(15)\n' | yafu -> never returns
Bound the retry at 100 attempts and finish with deterministic methods when
tiny-ecm gives up: trial division to 1e5 via zTrial(), then Pollard rho via
brent_loop() for anything still composite. Both append to fobj->factors and
reduce their own gmp_n the same way tiny-ecm did, so the surrounding code needs
no change. A prime remainder is added to the factor list directly.
Verified:
siqs(15) -> 15 = 3 * 5
siqs(21) -> 21 = 3 * 7
siqs(18446744073709551617) -> 274177 * 67280421310721
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The polynomial-search parameter tables begin at 80 digits (params_deg4 in
nfs_poly.c) and the search compares log10(n) against that bound, so an input
needs at least 81 decimal digits. Below that get_default_params() returns
without filling anything in, leaving stage2_norm at zero, and the msieve code
underneath answers with
error: stage 2 size bound not provided
and exit(-1), killing the program - from an explicit nfs() call on a number
that SIQS factors in a fraction of a second:
printf 'nfs(<60 digits>)\n' | yafu -> exit 255
nfs() now checks the size against NFS_GNFS_POLY_MIN_DIGITS before starting a
GNFS job and hands anything smaller to SIQS, the same fallback the crossover
check below it already used. The crossover is about which method is faster;
this is about which method is possible at all, so it is checked first:
nfs: input of size 60 is below the GNFS polynomial-search minimum of 81
digits, using siqs
... = 618970019642690137449562111 * 162259276829213363391578010288127
do_msieve_polyselect() additionally zero-initializes its parameter struct and
returns with a message if the table left it empty. That keeps the same abort
from coming back through any other route into poly search, and covers the far
end of the range too - the struct was uninitialized stack memory before, so the
guard could not have been relied on.
GNFS itself is unaffected: an 83-digit job still completes end to end
(polynomial search -> lattice sieving -> msieve filtering -> LA -> sqrt),
verified in 69-114 seconds depending on which parameter table is in use.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The guard prints "degree-%d table starts at %.0f digits" using pref_degree, which can be 5 or 6, but always takes the bound from params_deg4. Select the table by pref_degree so the message matches the degree it names.
Owner
|
Thank you for these fixes! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Four independent input-handling defects, each of which crashes or hangs yafu. They are unrelated to each other, so they are easy to review one at a time.
1.
pp1andtrialread the wrong operand slotsArguments are placed according to the declared arity in
function_nargs[]: with aritynaandkarguments supplied, the first argument lands inoperands[na-k]and the last inoperands[na-1]. Two cases disagreed with that table.pp1was declared with arity 3 while its case handles at most two arguments, so the shift was one slot off for every call shape. The case readsoperands[0]andoperands[1], sonew_factorization()was handed an uninitializedmpz(0), andprint_factors()then divided by the zero product of the resulting factor list. GMP raises SIGFPE, so everypp1call died with exit 136:Declaring arity 2 puts both call shapes into the slots the case reads.
trialhad the mirror-image bug in its one-argument branch only: arity is 2, so a single argument sits inoperands[1], but that branch passedoperands[0]tonew_factorization():The placement rule is now documented above the table so the next case does not have to rediscover it.
2.
print_factorsdivides by a zero factor productprint_factors()buildsprodfrom the factor list and dividesflist->Nby it without checking. A zero-valued factor makesprodzero, and GMP raises SIGFPE frommpz_tdiv_r, taking the program down with no diagnostic at all. That is exactly how thepp1/trialmisplacement above presented itself - an uninitialized operand reachednew_factorization(), was recorded as a factor, and the crash surfaced here:The bogus list is now reported and the remainder bookkeeping is skipped, so a bad factor list becomes a visible error message rather than a signal. An empty list is unaffected (its product is 1), which is what the surrounding check already relies on.
3. Unbounded tiny-ecm retry in the siqs special-case path
check_specialcase()finishes inputs below 115 bits with tiny-qs, and when that leaves a composite behind it retries tiny-ecm in an unbounded loop. Tiny-ecm cannot always succeed: when every factor of the remaining composite is small enough that its curve group order divides the stage-1 exponent, each curve collapses togcd == n, which the caller correctly rejects as trivial.15is such a case, and the loop spun on it at 100% CPU forever:The retry is now bounded at 100 attempts, after which the job finishes with deterministic methods: trial division to 1e5 via
zTrial(), then Pollard rho viabrent_loop()for anything still composite. Both append tofobj->factorsand reduce their owngmp_nthe same way tiny-ecm did, so the surrounding code needs no change. A prime remainder is added to the factor list directly.4. Inputs below the GNFS polynomial-search minimum
The polynomial-search parameter tables begin at 80 digits (
params_deg4innfs_poly.c) and the search compareslog10(n)against that bound, so an input needs at least 81 decimal digits. Below thatget_default_params()returns without filling anything in, leavingstage2_normat zero, and the msieve code underneath answers witherror: stage 2 size bound not providedandexit(-1)- killing the program from an explicitnfs()call on a number that SIQS factors in a fraction of a second:nfs()now checks the size againstNFS_GNFS_POLY_MIN_DIGITSbefore starting a GNFS job and hands anything smaller to SIQS, the same fallback the crossover check below it already used. The crossover is about which method is faster; this is about which method is possible at all, so it is checked first:do_msieve_polyselect()additionally zero-initializes its parameter struct and returns with a message if the table left it empty. That keeps the same abort from coming back through any other route into poly search, and covers the far end of the range too - the struct was uninitialized stack memory before, so a guard on it could not have been relied on.Testing
Built from source on Linux (GCC 16,
make yafu) and checked with:pp1output stays probabilistic - a p+1 curve can degenerate togcd == n- so a single-base call may legitimately report nothing.GNFS itself is unaffected: an 83-digit job still completes end to end (polynomial search -> lattice sieving -> msieve filtering -> LA -> sqrt):
I am putting together a separate change that adds a regression harness, so the reproductions above are manual for now. Happy to split this into four PRs if that is easier to review.