Skip to content

Mldsa mem - #51

Closed
Frauschi wants to merge 11 commits into
masterfrom
mldsa_mem
Closed

Frauschi wants to merge 11 commits into
masterfrom
mldsa_mem

Conversation

@Frauschi

Copy link
Copy Markdown
Owner

Description

Please describe the scope of the fix or feature addition.

Fixes zd#

Testing

How did you test?

Checklist

  • added tests
  • updated/added doxygen
  • updated appropriate READMEs
  • Updated manual and documentation

Comment thread wolfcrypt/test/test.c Outdated
}
#endif

/* Cross-implementation signature KAT. ML-DSA signing is deterministic given

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we already have KAT tests for ML-DSA in the repo? Why these new ones?

Comment thread wolfssl/wolfcrypt/dilithium.h Outdated

/* Canonical option implications. These derive one canonical option from
* another and must apply whether or not the legacy name gates are enabled. */
#ifdef WOLFSSL_MLDSA_SIGN_SMALLEST_MEM

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these new ones should go to the new wc_mldsa.h header? This is only the legacy compatibility header for the old naming.

Comment thread wolfssl/wolfcrypt/settings.h Outdated
(defined(HAVE_ED448) && defined(HAVE_ED448_KEY_IMPORT)) || \
(defined(HAVE_CURVE448) && defined(HAVE_CURVE448_KEY_IMPORT)) || \
defined(HAVE_FALCON) || defined(HAVE_DILITHIUM) || \
defined(WOLFSSL_HAVE_MLDSA) || \

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of adding WOLFSSL_HAVE_MLDSA in addition to the old HAVE_DILITHIUM, shouldn't we replace it? I think the compat gating in dilitihium.h should then catch this and define WOLFSSL_HAVE_MLDSA automatically when HAVE_DILITHIUM is defined.

Small memory signing kept vector w1 as k full polynomials although it is
only ever consumed in its encoded form, once by the commit hash and once
by MakeHint, which only tests each coefficient for zero. Keep the encoded
w1 and build it a polynomial at a time from a single scratch polynomial
of w. This also removes the separate w1 encode buffer that was allocated
and freed on every rejection attempt.

Add WOLFSSL_MLDSA_SIGN_SMALLEST_MEM. It generates matrix A a column at a
time so that one polynomial of y is held instead of the whole vector and
is transformed once rather than once per row, decomposes w into w0 in
place, and regenerates y for the z calculation. Peak signing heap drops
by about half against the small memory path and signing is quicker as
ML-DSA-87 does 7 forward transforms of y per attempt instead of 56. It
cannot be combined with the PRECALC options, and WOLFSSL_MLDSA_SMALL_MEM_
POLY64 has no effect on signing in this mode.

Small memory key generation held t as a full vector for one final vector
encode. Encode t a polynomial at a time and overlay t and the single
decoded s2 polynomial on the s2 vector, which is dead once s2 has been
encoded into the private key.

Peak signing heap, in bytes:

                before    after   smallest
  ML-DSA-44      16201    13897       9801
  ML-DSA-65      21321    16969      11849
  ML-DSA-87      27465    21321      14153

Peak key generation heap falls from 14153, 19273 and 25417 bytes to
10057, 13129 and 17225 bytes. Signatures are unchanged.

Problems found while making these changes are fixed here too:

- mldsa_vec_expand_mask dispatched to AVX2 generators that only handle
  dimensions 4, 5 and 7, and silently left y untouched for any other
  dimension. Fall through to the C implementation instead.
- WOLFSSL_MLDSA_SIGN_SMALL_MEM_PRECALC_A greater than k indexed A, w0
  and w1 past their allocations, as the loops used the raw macro rather
  than min(macro, k).
- WOLFSSL_MLDSA_SIGN_CHECK_W0 passed two arguments to the three argument
  mldsa_vec_check_low, so the option never compiled with small memory
  signing.
- WOLFSSL_MLDSA_VERIFY_SMALLEST_MEM did not build alongside small memory
  signing, as mldsa_vec_ntt_full and mldsa_vec_check_low were unused.
- WOLFSSL_MLDSA_VERIFY_SMALLEST_MEM sized z at l polynomials while only
  ever using one.
Two combinations of the ML-DSA cache options did not work.

WC_MLDSA_CACHE_PUB_VECTORS declares a cached t1 vector in wc_MlDsaKey and
WOLFSSL_MLDSA_VERIFY_NO_MALLOC declares a pinned t1 scratch polynomial in
the same structure, so enabling both failed to compile with a duplicate
member. Rename the pinned scratch to vt1.

The matrix A cache regression test asserts that signing populates key->a.
That holds for the full signing implementation, which is what the test was
written for, but the small memory implementations stream matrix A rather
than caching it, so they leave key->a as NULL and the test failed. Run the
test only when small memory signing is off. WC_MLDSA_CACHE_PRIV_VECTORS
enables WC_MLDSA_CACHE_MATRIX_A, so it saw the same failure.
WOLFSSL_MLDSA_VERIFY_SMALLEST_MEM defined WOLFSSL_MLDSA_VERIFY_NO_MALLOC,
so streaming vector z a polynomial at a time was only available with the
verify buffers pinned against the key for the life of the key. The two are
independent, so let them be selected independently. The pinned buffers are
still available by defining WOLFSSL_MLDSA_VERIFY_NO_MALLOC as well.

Peak verify heap with allocated buffers, in bytes:

               small mem   smallest mem
  ML-DSA-44         8777           5705
  ML-DSA-65         9801           5705
  ML-DSA-87        12105           5961

Structure sizes when the buffers are pinned instead are 20176 bytes for
small memory verify and 14032 bytes for smallest memory verify.

Also document WOLFSSL_MLDSA_VERIFY_SMALLEST_MEM in the option list, which
it was missing from.
Small memory signing walked matrix A a row at a time and copied and
transformed the polynomial of y it needed for every element of the row.
Each polynomial of y was therefore transformed k times with an identical
result: 156 transforms per ML-DSA-65 signature where 31 are needed.

Walk the matrix a column at a time instead. Every row of the matrix uses
the same column of y, so the transform is done once and the products are
accumulated down the column into w0, which is already a vector of k
polynomials. Inverting, decomposing and encoding then run as a second
pass over the rows, with w0 replaced in place and the scratch polynomial
holding w1.

Vector y stays resident, so unlike the smallest memory implementation
this does not have to regenerate it, and it keeps generating y as a whole
vector so the assembly generators are still used.

Memory is unchanged: w0 was already held for the hints and the row
accumulator becomes the scratch the transform runs in.
WOLFSSL_MLDSA_SMALL_MEM_POLY64 no longer applies to signing as a 64-bit
accumulator would now be needed for every row of w, which also returns
the 2KB it was allocating.

Signatures are unchanged. Signing is 12 to 29 percent quicker with the C
code and 6 to 12 percent quicker with AVX-512, which makes it faster than
the smallest memory implementation on every target rather than only where
the assembly runs.
Signing no longer uses a 64-bit accumulator, so the option now does
nothing unless key generation or verification is also built for small
memory. Say so where the option is documented and in the ChangeLog, as
a build that sets it alongside only WOLFSSL_MLDSA_SIGN_SMALL_MEM used to
get something for it and now does not.
Signing with WOLFSSL_MLDSA_SIGN_SMALL_MEM_PRECALC_A sized w for maxK
polynomials, but two helpers ignore their dimension for ML-DSA-44: the q88
decompose kernels take no dimension argument at all, and mldsa_vec_encode_w1()
bounds its q88 loops by PARAMS_ML_DSA_44_K. With PRECALC_A set to 1,
decomposing w ran past it into the first polynomial of matrix A, which has to
survive every rejection restart, so signing returned success with a signature
the verifier rejects. Size w on params->k and honour the caller's dimension in
mldsa_vec_encode_w1().

Align the preprocessor guards of the vector helpers with their call sites. The
PRECALC_A path drives seven whole-vector helpers whose guards had drifted, so
introduce MLDSA_SIGN_VEC_HELPERS and use it consistently, move the PRECALC and
PRECALC_A implications into dilithium.h so every translation unit agrees, and
match mldsa_rej_ntt_poly() and mldsa_expand_a() to their callers. Reject a
PRECALC_A value below one, which underflowed the allocation size.

Return NOT_COMPILED_IN from wc_CheckPrivateKey() for ML-DSA when
WOLFSSL_MLDSA_CHECK_KEY is not defined, matching what the RSA arm already does.
The call to wc_MlDsaKey_CheckKey() was unguarded while the function itself is
compiled out, so WOLFSSL_MLDSA_NO_CHECK_KEY builds failed to compile.

Reduce w before the inverse transform on both column accumulating paths now
that the 64-bit accumulator is gone, and bind the two expansions of y under
WC_MLDSA_FAULT_HARDEN.

Add functional and -Wconversion CI rows for the two configurations this branch
enables but nothing built: SIGN_SMALLEST_MEM, and VERIFY_SMALLEST_MEM with
malloc.
Signing retries until a candidate passes four infinity norm checks.
mldsa_check_low() stopped at the first out of range coefficient, so its running
time revealed which coefficient failed. Replace it with a branchless
mldsa_check_low_ct() at every signing check site. This costs nothing: the
accepting path already scanned all 256 coefficients, so only a failing check
changes, and ML-DSA-44 signing measures no slower than before.

The rejecting index itself carries no key information for z and w0-cs2. Both
are masked by the uniform vector y, and the count of y values that push a
coefficient out of range is 2*beta+1 whatever the secret contributes, so every
coefficient rejects with the same probability. The reference implementation
relies on the same argument. Those loops therefore keep their early exit.

ct0 is different: it is c*t0 with no mask, so the index of a failing polynomial
does depend on t0. Check every polynomial of ct0 rather than stopping at the
first failure. This is free in practice because that check rarely rejects. The
smallest and small memory signers compute ct0 in the same row loop as w0-cs2,
so they accumulate the ct0 result in a flag of its own: the loop still stops at
the first rejecting w0-cs2 row, but a failing ct0 row no longer ends it, and the
hint is made for every row the loop reaches whatever ct0 decided.

Move the canonical option implications out of the legacy name gate region of
dilithium.h. WOLFSSL_NO_DILITHIUM_LEGACY_GATES suppresses legacy to canonical
name translation, but it was also suppressing SIGN_SMALLEST_MEM, PRECALC and
PRECALC_A implying WOLFSSL_MLDSA_SIGN_SMALL_MEM, so those builds silently got
the full memory signer.

Derive WC_ENABLE_ASYM_KEY_IMPORT and WC_ENABLE_ASYM_KEY_EXPORT from
WOLFSSL_HAVE_MLDSA rather than the legacy HAVE_DILITHIUM, so an ML-DSA build that
opts out of the legacy gates still gets the RFC 5958 helpers it calls. settings.h
maps HAVE_DILITHIUM onto WOLFSSL_HAVE_MLDSA unconditionally before either use, so
the canonical name alone covers both spellings.

Make WOLFSSL_NO_MALLOC select the small memory verify along with
WOLFSSL_MLDSA_VERIFY_NO_MALLOC. The pinned verify buffers only exist under the
small memory verify, so the no-malloc option alone selected nothing and every
verification failed with MEMORY_E. Set only the canonical names there and gate
on WOLFSSL_HAVE_MLDSA; dilithium.h mirrors them onto the legacy names for code
that still reads those.

Zeroize the fault hardening checksums of the signing mask, and narrow the
mldsa_vec_check_low() guard away from the smallest memory signer that does not
call it.

Wipe the key's SHAKE object when the smallest memory signer returns. Its last
use of the object regenerates a polynomial of y, so signing left rho'' in the
key: the whole of it could be read back from the object on the generic path,
and on the Intel path the Keccak state it leaves is a permutation of it. The
other signers finish by hashing the public commitment. Every later use sets
the object up afresh, so only the device id needs restoring.

Add a signing KAT to testwolfcrypt. ML-DSA signing is deterministic given the
key seed and the signing seed, so every signer must produce the same bytes; the
test compares a SHAKE-256 digest of the signature with the default signer's.
tests/api already holds full signing KATs, but unit.test is not built in the
--enable-cryptonly configurations that every smallest memory and PRECALC_A CI
row uses, so without this nothing checked that those signers produce the FIPS
204 signature rather than merely one that verifies.

Add CI rows for the configurations none of this was covered by: the legacy gate
opt out, and smallest memory signing with the y and w0 checks.
Narrowing the range check gates left three configurations unbuildable.

mldsa_vec_check_low() and mldsa_vec_check_low_c() lost their last caller when
signing moved to the constant time helpers, but their gate still carried the
sign clause, so --enable-mldsa=make,sign and VERIFY_SMALLEST_MEM paired with the
default signer compiled a static function with no references. The hardened
CFLAGS put -Wunused-function after the -Wno-unused in AM_CFLAGS, so that is a
build failure rather than a warning. Reduce both gates to the verify condition,
and gate the constant time vector form on the two arms that actually call it.

The MC/DC whitebox test had its inner guard updated to match the library but
not its outer guard or its call site, so SIGN_SMALLEST_MEM with NO_VERIFY
referenced a function that is no longer compiled. Mirror the library condition
in both places.

The pq-all row named mldsa-smallest-checks-no-verify set VERIFY_SMALLEST_MEM
rather than NO_VERIFY, so it neither matched its name nor reached the case it
was added for.

Bind the two derivations of the signing mask unconditionally. The smallest
memory signer is the only path that derives y twice per rejection round, and the
check that the second derivation matches the first was gated on
WC_MLDSA_FAULT_HARDEN. A glitch in the second derivation yields z computed from
one mask against a commitment computed from another, which is the shape a fault
attack on the private vector needs. Compile the check always and accumulate the
checksum with FNV-1a so a single altered coefficient changes the whole result.
Signing in that mode is 2.6 percent slower; no other path derives y twice.

Scrub the caller's signature buffer when signing fails. The commit hash and each
accepted polynomial of z are written into it as they are produced, but the round
is only accepted after later checks, so an error return could leave part of a
signature behind.

The fault hardening check added to the small memory column walk tested the loop
variable against its own bound, which can never hold, and sat after the copy it
was meant to guard. Validate the derived pointer before the access instead.

Range check WOLFSSL_MLDSA_SIGN_SMALL_MEM_PRECALC_A so that a definition with no
value produces the intended error rather than a preprocessor syntax error, move
the canonical option implications inside the ML-DSA guard, default the ML-DSA
operations when --enable-mldsa names only a parameter set, and add CI rows for
sign only, smallest memory verify with the default signer, and small memory
signing with only the w0 check.
Moving the canonical option implications inside the WOLFSSL_HAVE_MLDSA guard
made the wc_MlDsaKey layout depend on include order. dilithium.h has a single
include and it sits after that guard, so a translation unit that reaches
dilithium.h before any settings-bearing header evaluates the guard with
WOLFSSL_HAVE_MLDSA still undefined and never derives
WOLFSSL_MLDSA_VERIFY_SMALL_MEM. The verify scratch tail of the key structure is
gated on that macro together with WOLFSSL_MLDSA_VERIFY_NO_MALLOC, so the library
would be built with the full structure while an application that includes
dilithium.h first gets one without it, and verification then writes past the end
of the caller's key. Derive the implications unconditionally again, and do it in
wc_mldsa.h straight after its include of dilithium.h rather than in the legacy
compatibility shim: every route to the key structure passes through that point,
and the implications have to outlive the shim.

Checksum the signing mask with rotation and exclusive-or rather than a
multiplicative hash. The coefficients are secret, and a multiply chain over them
is not constant time on cores with an operand dependent multiplier, which is
what the smallest memory option targets. It also hands power analysis a clean
per-coefficient hypothesis. Rotating keeps a changed coefficient's position
significant, which is all the fault check needs.

Clear the rows of w that PRECALC_A does not multiply into. The ML-DSA-44
decompose kernels take no dimension and touch all k rows, so the rows above the
pre-calculated ones were read before being written.

Cover the constant time range checks in the ML-DSA whitebox test: both
boundaries of the accepted range, and a bad coefficient in the last position of
a polynomial and in the last polynomial of a vector, which only fail because
neither helper exits early.
Three whole-vector helpers are only called by the full-vector key generation,
but their guards still admitted WOLFSSL_MLDSA_MAKE_KEY_SMALL_MEM, and
mldsa_vec_ntt_full still carried a WOLFSSL_MLDSA_SIGN_SMALL_MEM_PRECALC clause
with no call site behind it. Two configurations therefore compiled a static
function with no references: MAKE_KEY_SMALL_MEM with NO_CHECK_KEY and the
smallest memory sign and verify (mldsa_vec_invntt_full, mldsa_matrix_mul, and
mldsa_vec_red once WOLFSSL_MLDSA_SMALL is added), and NO_VERIFY or
VERIFY_SMALLEST_MEM with SIGN_SMALL_MEM_PRECALC (mldsa_vec_ntt_full). Neither
built at all before the guard rework, so this finishes that work rather than
fixing a regression. Give matrix_mul, invntt_full and vec_red the
MAKE_KEY_SMALL_MEM exclusion their siblings already have, and reduce the
ntt_full signing clause to MLDSA_SIGN_VEC_HELPERS, which already spells the
set of call sites that exist.

Accumulate the WOLFSSL_MLDSA_SIGN_CHECK_W0 result in both small memory signers
instead of leaving the loop on the first rejecting row. w0 is LowBits(A o y)
and so is derived from the secret mask, exactly as the y check immediately
above it is, and that check already runs every column for this reason.

Read the cached public vector in the small memory verify. Widening the
mldsa_vec_decode_t1 guard let WC_MLDSA_CACHE_PUB_VECTORS build alongside the
small memory verify, but nothing consumed the cache: key->pubVecSet was only
tested by the full-vector verifier, so every public key import paid an
allocation of s2Sz and a full decode and transform that was then discarded and
re-done a polynomial at a time. The cached vector is already in the form the
row loop wants.

Mix the coefficient index into mldsa_poly_checksum. Rotation alone has period
32 over 256 coefficients, so a fault applying the same delta to coefficients
32 apart cancelled. Single coefficient faults were always caught, which is
what the white-box test drives.

Match the white-box guard for wb_poly_checksum to the definition guard of the
function it calls, as the three neighbouring guards in that file already do.

Compile test_wolfSSL_X509_check_private_key_mldsa in a WOLFSSL_MLDSA_NO_CHECK_KEY
build and assert that the pair is rejected there, rather than gating the test
off and leaving the NOT_COMPILED_IN arm of wc_CheckPrivateKey with no coverage.

Drop the key's caches when the small memory key generation replaces a key. The
default key generation clears aSet, privVecsSet and pubVecSet on success, but
the small memory arm never did, so a key generated into an object that had
imported or used another key kept that key's matrix A and vectors, and its
signatures failed to verify. The full-vector verifier has the same exposure on
master; reading the cache in the small memory verify widened it. Rework the
signing KAT to generate the key into a fresh object and into one that already
signed and verified with a different key, and require both to produce the same
signature and to verify it. A sign and verify on the reused object alone does
not catch this, because the signer and the verifier read the same stale caches
and agree. The comparison needs no known answer, so it also runs for the draft
and CHECK_Y/CHECK_W0 signers, which only skip the digest check.

Add a pq-all row for cached matrix A on the default signer: the existing row
pairs the cache with PRECALC_A, which implies the small memory signer and
preprocesses away the key->a and aSet assertions the test exists for. Correct
that row's comment to say what it does cover. Add another row that builds every
cache with the small memory key generation, the only CI configuration where the
comparison reaches the fix.
mldsa_vec_expand_mask_c() owns a 680 byte scratch that it allocates, zeroizes
and frees on every call, which is an XMALLOC/XFREE pair under
WOLFSSL_SMALL_STACK. Expanding the whole vector in one call amortised that over
l polynomials, but the smallest memory signer expands one polynomial at a time
and calls it twice per polynomial - once to build w and once to regenerate y
for z - so the cost is paid 2*l times per rejection attempt. The l == 1
argument also bypasses every assembly dispatch arm, so that path was always
going to land in the C implementation anyway.

Split the per-polynomial work into mldsa_expand_mask_poly(), which takes the
scratch from the caller, and let the signer pass the z polynomial: z is not
live at either call site, being written by the multiply that follows, and it is
larger than MLDSA_MAX_V. ML-DSA-44 signing drops from 42 allocations per
signature to 7. Throughput is unchanged, since the SHAKE work dominates.

Read the cached public vector in place when verifying. The small memory verify
copied one already decoded and transformed polynomial out of the cache into w,
then immediately overwrote w with the result of the pointwise multiply. Point
the multiply at the cache instead. No functional change, one buffer less
touched per polynomial.

Compile mldsa_vec_expand_mask(), mldsa_vec_expand_mask_c() and the AVX2 and
AVX-512 mask generators only when the smallest memory signer is not selected.
That signer no longer calls them, and SIGN_SMALLEST_MEM implies SIGN_SMALL_MEM,
whose signer is the only other caller, so every smallest memory build warned
about an unused static function and an in-tree -Werror build failed. Move the
ExpandMask algorithm comment back onto the vector function it documents. The
AVX-512 generator's dimension gate only ever turned away that single
polynomial, so drop it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant