fix: NEWOBJ_EX objects re-encode as copyreg.__newobj_ex__ REDUCE (#35)

[jensens]

Fixes #35

Built on #56 (fix/reduce-build-newobj) and targeting main so CI runs: the diff shows #56's four commits plus four of its own. Merge #56 first and this shrinks to its own commits; merging this one first brings #56 along.

The problem

A protocol 4 pickle creating an object with NEWOBJ_EX (__getnewargs_ex__ with keyword arguments) decoded into a Reduce whose args were a dict {"@args": ..., "@kwargs": ...}, and re-encoding wrote GLOBAL <dict> NEWOBJ, which CPython rejects (NEWOBJ args argument must be a tuple); with a following BUILD the object came back without its constructor arguments. ZODB writes protocol 3, so only pickle_to_json of protocol 4 pickles met it.

Design (a fork the issue left open, decided in the plan)

Instead of a new kwargs key, the decoder turns cls args kwargs NEWOBJ_EX into copyreg.__newobj_ex__(cls, args, kwargs) with newobj: false: the callable CPython's own object.__reduce_ex__ returns for __getnewargs_ex__ objects (the pickler only rewrites it into NEWOBJ_EX at protocol 4). It is a plain REDUCE for every writer, reader and encoder, loads to cls.__new__(cls, *args, **kwargs) on any Python 3.4 or newer, and a following BUILD becomes its state. The reviewer checked the ruling against both CPython unpicklers: the REDUCE form is never less permissive than NEWOBJ_EX and exactly as permissive as the pure-Python unpickler.

The encoders also translate the shape 1.6.x wrote for these objects (the two-key @args/@kwargs dict as args, with or without a newobj key, and nested in an instance's @args/@state state) into the same call, so old pickle_to_json output becomes loadable. A dict is never a valid REDUCE/NEWOBJ argument, so nothing loadable is misread.

Evidence

• 242 Rust and 402 Python tests; the Python tests round-trip __getnewargs_ex__ classes through every path at protocol 4, with and without state, with true empty args and kwargs (a custom __reduce_ex__, the only way CPython emits them), through ZODB records written by one pickler (shared memo), and load both legacy shapes.
• The 1,692-record sample produces identical PG JSON, dict-path output and re-encoded pickles.
• A fresh reviewer ran 21 __getnewargs_ex__ shapes at protocols 2 to 5 through five paths against pickle.loads of the source with the C and the pure-Python unpickler, nested NEWOBJ_EX 1,100 deep (clean error, no crash), and the 1.6.1 wheel's actual output. Its one Important finding (the 1.6.1 shape has no newobj key, the translation was gated on it) is the last commit.

Assisted-by: Claude Fable 5.1