Skip to content

Reusing context for compression #7

Description

@nono303

Actually, ZSTD_createCCtx() is called on compress_filter (output filter registered function)
This implies that the zstd context is created and destroyed on each request - which might not be optimized for performance.

https://facebook.github.io/zstd/zstd_manual.html#chapter4

When compressing many times,
  it is recommended to allocate a context just once,
  and re-use it for each successive compression operation.
  This will make workload friendlier for system's memory.
  Note : re-using context is just a speed / resource optimization.
         It doesn't change the compression ratio, which remains identical.
  Note 2 : In multi-threaded environments,
         use one different context per thread for parallel execution.

Also Cf. facebook/zstd#2107

So, my question is, could the zstd context can be instantiated one time by "worker ❓" and reused for all compression process.
⚠ Must be thread safe in case of concurrent compression in a given worker

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesthelp wantedExtra attention is needed

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions