Bound wire-declared sizes and decompressed output during parsing

[wwbmmm]

What problem does this PR solve?

Problem Summary:

Wire input to bRPC parsers contains lengths/counts that are declared
by the peer. Several parsers used those values to size buffers or
drive allocations before the corresponding bytes were known to exist,
and decompressors never bounded their output. With untrusted clients
this lets a small amount of traffic trigger disproportionate memory
usage.

What is changed and the side effects?

Changed:

• Decompression (gzip/zlib/snappy) is now bounded: a new
  -max_decompressed_body_size flag (default: 32x
  -max_body_size, 0 = use the default) caps how much a single body
  may decompress to. Bodies that exceed the cap are rejected with a
  warning log instead of being fully materialized.
• AMF (rtmp) string reading grows the output as bytes arrive in
  64 KB chunks instead of allocating the declared length up front,
  so a truncated message no longer forces the full allocation.
• RTMP chunk parsing now rejects declared message lengths above
  -max_body_size and discards a partially assembled body when a
  new message header arrives mid-message.
• The mcpack2pb object iterator validates the wire-declared field
  count against the remaining bytes (each field head is at least 2
  bytes) and generated array-parse code caps its Reserve() size.

Side effects:

• Performance effects: negligible; the new bounds only cut
  allocations that were already on the path to failure.
• Breaking backward compatibility: none. Messages within
  -max_body_size are unaffected unless they also exceed the new
  decompression cap; operators with legitimately larger compressed
  bodies can raise -max_decompressed_body_size.

Check List:

• Make sure your changes are compilable.
• Tests are added in test/brpc_snappy_compress_unittest.cpp,
  test/brpc_rtmp_unittest.cpp, test/brpc_mcpack2pb_unittest.cpp,
  test/brpc_sofa_pbrpc_protocol_unittest.cpp.
• Follow Contributor Covenant Code of Conduct.

[Copilot]

Pull request overview

Adds protections against oversized wire-declared values and decompression output across bRPC parsers, with regression tests.

Changes:

• Adds configurable gzip, zlib, and Snappy decompression limits.
• Bounds AMF and RTMP parsing and resets incomplete bodies.
• Validates mcpack counts and caps generated reservations.
• Adds tests for oversized and truncated inputs.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Summary / final review status
test/brpc_sofa_pbrpc_protocol_unittest.cpp	Tests oversized SOFA metadata. No final comments.
test/brpc_snappy_compress_unittest.cpp	Tests decompression caps. Moderate: use FlagSaver for global flag mutations.
test/brpc_rtmp_unittest.cpp	Tests AMF and RTMP bounds. Moderate: use FlagSaver to restore max_body_size safely.
test/brpc_mcpack2pb_unittest.cpp	Tests mcpack count validation. No final comments.
src/mcpack2pb/parser.h	Adds bounded reservation support. No final comments.
src/mcpack2pb/parser-inl.h	Validates object item counts. Critical: check the size before cut_packed_pod reads ItemsHead.
src/mcpack2pb/generator.cpp	Applies bounded reservations to generated parsers. No final comments.
src/brpc/policy/snappy_compress.cpp	Bounds Snappy decompression output. No final comments.
src/brpc/policy/rtmp_protocol.cpp	Bounds RTMP message lengths and resets partial bodies. No final comments.
src/brpc/policy/gzip_compress.cpp	Bounds gzip/zlib decompression output. No final comments.
src/brpc/compress.h	Declares decompression configuration. Critical: include gflags/gflags_declare.h before DECLARE_uint64.
src/brpc/compress.cpp	Defines decompression-size configuration and limits. No final comments.
src/brpc/amf.cpp	Reads AMF strings incrementally. No final comments.

Suppressed comments (1)

src/mcpack2pb/parser-inl.h:160

• item_count is peer-controlled, so this rejection path is reachable from a malformed request. CHECK(false) is fatal in bRPC, which aborts the process instead of returning a bad stream; the newly added regression test also terminates here rather than passing. Use non-fatal logging followed by set_bad() so the parser rejects the input without taking down the server.

        CHECK(false) << "inconsistent item_count(" << items_head.item_count
                     << ") and value_size(" << size << ")";
        return set_bad();

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 3 comments.

Suppressed comments (3)

src/mcpack2pb/parser-inl.h:163

• The new inconsistent-count branch is also driven entirely by peer-controlled bytes, but CHECK(false) is fatal here. The regression test's item_count = 0x7fffffff input will abort at this line before it == NULL can be observed, and a remote client can use the same packet to crash the process; log the parse error and then call set_bad() without a fatal check.

        CHECK(false) << "inconsistent item_count(" << items_head.item_count
                     << ") and value_size(" << size << ")";

test/brpc_rtmp_unittest.cpp:697

• The added RTMP test returns at the preceding oversized-length check, so it never exercises the new partial-body reset in either the type-0 or type-1 branch. Add a case that feeds a partial message followed by a new header and verifies the stale bytes are discarded; otherwise this security-sensitive reassembly behavior is untested.

TEST(RtmpTest, chunk_stream_rejects_message_length_over_max_body_size) {

test/brpc_snappy_compress_unittest.cpp:272

• This assertion only checks that decompression eventually returns false; it would still pass if the full 64-KB body were materialized before failure. Please assert that out remains within the configured cap, and add rejection coverage for the separate Message/Zlib paths, since the new Message limiter and shared Zlib output path are otherwise untested.

    ASSERT_FALSE(brpc::policy::GzipDecompress(gzipped, &out));

[chenBright]

LGTM