Remove pure Python packages from constraints (and distribution)

[juntyr]

• asciitree: Publish wheels? mbr/asciitree#18
• covjsonkit: Publish wheel to PyPi ecmwf/covjsonkit#68
• earthkit-data: decide on pyfdb dependency support
  • the error message is nicer if pyfdb is not installed at all than if the FDB5 library is missing
  • could mock pyfdb using micropip.add_mock_package("pyfdb", "0.1.1", modules={"pyfdb": "raise ModuleNotFoundError('pyfdb')"}), but the error message would still not be great
  • since supporting pyfdb is currently not possible, the missing library error is the best we can do
• earthkit-regrid: replace patch with a default config file: Cache configuration without threads ecmwf/earthkit-regrid#64
• ecmwf-api-client: Publish wheels to PyPi ecmwf/ecmwf-api-client#23
• ecmwf-opendata: Publish wheels to PyPi ecmwf/ecmwf-opendata#62
• filelock: Support fcntl check on Emscripten tox-dev/filelock#398
• findlibs: upgrade from v0.0.5 to v0.1.0: Fix ctypes.util.find_library usage ecmwf/findlibs#16
• fsspec: upstream patch, though this is blocked on JSPI becoming a standard
• gribscan: remove need for the patch, will likely still be non-pure though (to package the shared lib) AECCodec usage inside gribscan gribscan/gribscan#61
• hda: Publish wheels to PyPi ecmwf/hda#34
• ipython: unlikely until we can insert runtime patches in a different way Add bootstrapCode config option jupyterlite/pyodide-kernel#172
• jupyterlite-cors: private implementation detail
• jupyterlite-preload: private implementation detail
• ldcpy: Remove nodejs dependency in requirements.txt NCAR/ldcpy#348
• micropip: would need to inject into the lockfile before there is access to micropip
• multidict: optional C extension for speed, could use the pure Python version
• multiurl: Publish wheel to PyPi ecmwf/multiurl#3
• packaging: feat: add handling for Emscripten wheels tags per PEP 783 pypa/packaging#804
• pdbufr: upgrade from v0.11.0 to v0.12.2
• pyicon-diagnostics: https://gitlab.dkrz.de/m300602/pyicon/-/issues/14
• pyodc: needs to be patched to exlude codc, wheel published since v1.5.0, 32bit build support ecmwf/odc#27
  • from v1.6.0, the package now depends on the odclib and eckitlib packages, which include the prebuilt shared library dependencies
  • earthkit-data's check for codc vs pyodc also works if codc is technically available even though the odccore library is missing
  • odclib as an optional dependency ecmwf/pyodc#38
• pyodide-http: unlikely, since the patch depends on jupyterlite-cors
• yarl: optional C extension for speed, could use the pure Python version
• zarr: upstream patch, though this is blocked on JSPI becoming a standard

[agriyakhetarpal]

Hi @juntyr!

• zarr: upstream patch, though this is blocked on JSPI becoming a standard

Done in zarr-developers/zarr-python#1903 and merged today via pyodide/pyodide-recipes#584. It won't work in Firefox of course, no JSPI there.

I guess we should do fsspec next? Could you please share your patch for it or point me to your existing work, since you mention "upstream patch" in the point above, suggesting that you have one that can be upstreamed? I was unable to find it in the recipes here or in https://github.com/juntyr/pyodide-recipes/tree/3d8c652bb97c652cf7b04041eabbc34ec8b5a907/packages/fsspec.

[juntyr]

Cool, thanks @agriyakhetarpal for your work!

Here are my patches for Zarr (https://github.com/climet-eu/pyodide-recipes/blob/596d402fd12d0ce7522804915761987a1718f224/packages/zarr/patches/0001-syncify-awaitable.patch) and fsspec (https://github.com/climet-eu/pyodide-recipes/blob/596d402fd12d0ce7522804915761987a1718f224/packages/fsspec/patches/0001-syncify-awaitable.patch).

The main difference is that they come from a time when JSPI support was still experimental and so involved a hacky fallback where I patched asyncio to transform everything possible into a future that could just be polled once to synchronously force evaluation and then used that to syncify. But hopefully that won’t be needed any longer soon and we can just used JSPI / tell users that it’s JSPI or no syncifying async.

[agriyakhetarpal]

Nice, thanks for the links! Yeah, those definitely look like early-JSPI era 😄 I guess it only works for coroutines that complete synchronously in a single step. I was mainly going to use the patches to see if I have prior art to base fsspec support on, but this will require more work. I'll come around to this sometime later, but I cannot know when. :)