Skip to content

storage(tigris): cold acquire buffers the whole archive in RAM with no in-flight dedup #487

Description

@euxaristia

Summary

repo_store::acquire's cache-miss path (crates/gitlawb-node/src/git/repo_store.rs:180-193) runs tigris.exists + tigris.download per concurrent caller with no in-flight dedup, and download buffers the entire archive in memory (crates/gitlawb-node/src/git/tigris.rs:127-134, resp.body.collect().into_bytes(); the upload side likewise builds the whole tar.zst in RAM, :86-93). The publish_lock serializes only the final swap (#343). The anonymous read handlers that reach acquire (tree/blob/changelog REST routes) have no timeout clamp or admission, unlike git_info_refs/upload-pack//ipfs.

Impact

N concurrent cold hits on a Tigris-backed node produce N parallel full-archive downloads and N temp-dir unpacks: memory and bandwidth multiply by archive size (hundreds of MB) with no per-IP brake. Extends the #343 admission family with a cold-path amplification member.

Remediation

  1. Single-flight the cold acquire per repo (in-flight map or per-repo permit).
  2. Stream the download to disk instead of buffering, and cap archive size.

Proposed labels: kind:security, crate:node, subsystem:storage.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    crate:nodegitlawb-node — the serving node and REST APIkind:securityVulnerability fix or hardeningsev:highMajor break or real security/trust risk, no easy workaroundsubsystem:apiNode REST API request/response surfacesubsystem:storageBlob/object store, Arweave, IPFS, archives

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions