[JuhoErvasti]

Description

Add a new core module parallel.py to handle multithreaded geometry processing in parallel using ThreadPoolExecutor.

Design remarks

On import parallel.py instantiates a _ThreadManager singleton instance which manages a ThreadPoolExecutor. This removes some overhead of spawning threads multiple times per a process run. By default it uses one thread per logical core on the CPU, but this can be configured by environment variable or explicitly setting the worker count with set_num_workers, which takes precedence over env variable.

Parallel processing is exposed mainly through a ParallelGeometryProcess object, which is meant to be instantiated from the in_parallel function. This allows one to write code and chain operations like:

gdf.geometry = (
    in_parallel(gdf.geometry) # turns to ParallelGeometryProcess
    .buffer(0.5)
    .intersection(other_geom)
    .buffer(-0.5)
    .resolve_parallel() # -> turns back to GeoSeries
)

I'm not a 100% happy with this, but this felt like the best solution, first I had just something like

gdf.geometry = parallel_process(gdf.geometry, shapely.buffer, 0.5)

but chaining operations this way makes for really unreadable and awkward code. Pandas also supports defining a custom property accessor like

gdf.geometry = gdf.geometry.parallel.buffer(0.5)

but this way you don't get any type information or autocomplete in an IDE.

Multithreading improves performance on intermediate size datasets mostly only with heavier operations like buffer/intersection etc. With cheap operations like area() etc. it's typically not even worth it to parallelize. This is why I've only defined some methods for ParallelGeometryProcess, but you can use other shapely functions too by passing it in run(), I think if necessary we can also add more methods as we go.

This PR doesn't yet actually use the parallel processing anywhere.

Obviously it's hard to pinpoint a clear speed-up because really it depends a lot on the type of data and the vertex count of geometries (and worker count). In the included benchmarks we do get a decent speed-up (3-5x) especially in the larger datasets and heavier operations like buffering or intersections; with small datasets and cheap operations it either breaks even or is slower.

Type of change

• Bug fix

• New feature

• Other

• Non-breaking change

• Breaking change

Developer checklist

• A CHANGELOG entry has been included (only when change is visible to users)