Provide a shared memory option when creating an instance of the Requests class.
This would allow Requests to make GRPC requests using shared memory, as shown in the example here: https://github.com/triton-inference-server/client/blob/main/src/python/examples/simple_grpc_shm_client.py.
With non-uniform batch sizes we need to allocate new shm resources for each request. Otherwise, these can be reused. We can provide a flag to indicate whether batch size is non-uniform.
Provide a shared memory option when creating an instance of the Requests class.
This would allow Requests to make GRPC requests using shared memory, as shown in the example here: https://github.com/triton-inference-server/client/blob/main/src/python/examples/simple_grpc_shm_client.py.
With non-uniform batch sizes we need to allocate new shm resources for each request. Otherwise, these can be reused. We can provide a flag to indicate whether batch size is non-uniform.