tonic/client: support immediate cancellation of streams without sending End-of-Stream

[arjan-bal]

Motivation

Tonic's client streaming and bidirectional (bidi) streaming APIs do not currently support immediate request cancellation. Because outbound request streams are infallible, completing them automatically transmits a DATA frame with an End-of-Stream (EOS) flag, which signals a normal, graceful termination. An RST_STREAM frame is only sent by the client after the request stream terminates and the corresponding response stream is subsequently dropped.

This delayed teardown creates a problem: the server may interpret the EOS flag as a graceful finish and process or commit incomplete payloads before it ever receives the RST_STREAM frame. To prevent potential data corruption, Tonic requires a mechanism to abort and tear down the HTTP/2 stream immediately without sending a misleading EOS flag.

Solution

Introduce a paired CancellationHandle to coordinate immediate, client-side cancellation of outbound request streams.

Core Changes

• CancellationHandle (User-Facing): Users can create a handle by calling request.cancellation_handle(). Invoking .cancel() on this handle triggers an immediate, non-graceful abort.
• The CancellationHandle wraps a CancellationToken which is extracted from the request extensions by Tonic's client dispatcher to actively track the cancellation state of the active stream.
• Immediate Abort in Encoder : EncodeBody::poll_frame actively checks the CancellationListener. If cancellation is flagged, it immediately returns an error mapped with H2Reason::CANCEL. This bypasses the empty DATA frame with EOS and immediately triggers an HTTP/2 level RST_STREAM.

Alternative

Use the Encoder API to inject an error causing hyper to cancel the stream: #2788

This seems like a mis-use of the codec API and causes the RST_STREAM code to be internal.

[arjan-bal]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces client-side cancellation support in tonic by adding a CancellationHandle to Request and propagating it to abort outbound streams with an HTTP/2 RST_STREAM frame. The feedback highlights three critical compilation issues: http::Extensions does not have a get_or_insert method, the std::future::Future trait needs to be imported to call .poll(), and unstable let-chains (&& let) are used which will fail on stable Rust.

[dfawley]

Looks good, but I'm curious about the feature changes and also you had mentioned needing to manage the waker before, but I don't see anything about that in this PR. Thanks!

[dfawley]

What is the significance of the diffs in this file? I think these are essentially equivalent before/after? Isn't tonic's "h2" feature independent from this one?

[arjan-bal]

The cargo-udeps check in CI fails when a feature shares the same name as an optional dependency (see this failure example).

I previously encountered a similar issue when introducing a feature flag for the optional tower dependency. We resolved it by explicitly declaring a feature flag with the same name as the optional dependency (see PR #2432). While the exact root cause is not fully clear, it likely stems from how Cargo implicitly defines feature flags for optional dependencies, which confuses cargo-udeps during unused dependency detection.

[arjan-bal]

I've made h2 a required dependency in tonic now, making the changes unecessary.

[dfawley]

I'm still not sure I follow why examples needed to change, though? Didn't you need to set the h2 flag in tonic, not change this?

[arjan-bal]

If you revert examples/Cargo.toml and run the following, you'll see a similar failure to the udeps CI:

$ git revert examples/Cargo.toml
$ cargo udeps --no-default-features --features h2 -p tonic
error: package `examples v0.0.0 (HOME/grpc-rust/examples)` does not have feature `h2`

help: an optional dependency with that name exists, but the `features` table includes it with the "dep:" syntax so it does not have an implicit feature with that name
Dependency `h2` would be enabled by these features:
        - `streaming`

Adding a feature in the tonic crate interacts with the examples crate which uses the dep: syntax to specify an optional dependency on h2. I couldn't figure out why this happens, but adding a feature named h2 in the examples crate works around the issue.

[arjan-bal]

After discussing this with Gemini, the issue seems to be due to the behavior of cargo metadata (which cargo-udeps uses internally) in workspaces.

cargo metadata --no-default-features --features X

The above command will:

• Succeed if a feature named X is either present or entirely absent in a crate.
• Fail if a feature named X is absent, but an optional dependency dep:X exists.

So cargo metadata --no-default-features --features h2 fails, while cargo metadata --no-default-features --features server succeeds.

[dfawley]

What's the purpose of this source having an H2 error in it? Is this needed? It only happens if the (newly added) h2 feature is set, or if "server" is set...why doesn't client-side get it always?

It seems like we could just eliminate it since we already set status to cancelled, but maybe I'm missing an important use case?

[arjan-bal]

What's the purpose of this source having an H2 error in it?

When poll_frame returns an error, hyper calls on_user_error to send a RST_STREAM. This, in turn, calls h2_reason, which inspects the error's cause chain for an h2::Reason, defaulting to INTERNAL if none is found. By explicitly setting h2::Reason::CANCEL as the source in the Tonic status here, we guarantee that the RST code received by the server is CANCEL rather than INTERNAL.

I wasn't sure initially why the h2 dependency was optional and gated behind the server feature flag, so I kept the new client-side usages behind a feature flag as well to maintain that behavior. However, upon closer consideration, I suspect it was made optional to prevent cargo-udeps from flagging it as unused when it was only referenced in server code. I changed h2 to be a a required dependency now to ensure the correct RST code will be sent every time.

[dfawley]

Thanks for the explanation. So if you aren't using hyper then this does nothing? So should this be predicated on "channel"? But also if you aren't using hyper then how do we express stream cancellation to the transport properly (whatever that would be)? Something here seems like it needs to be understood a little better, I think.

[LucioFranco]

This feels correct to me (for how we designed this code), h2/hyper is optional but the one that has the best support. So if you are not using hyper then you have to figure it out yourself. We need to pass the source through as a "flag" for hyper/h2 so it knows how to respond but that has to tie into that crate somehow. So this is a way for us to support it.

[arjan-bal]

Looks good, but I'm curious about the feature changes and also you had mentioned needing to manage the waker before, but I don't see anything about that in this PR. Thanks!

Initially, I had implemented something similar to a CancellationToken from scratch. It was lock-free and assumed only a single task would poll for cancellation, meaning it only held the latest waker and used atomics to update it. This assumption of a single polling task becomes invalid when we propagate the cancellation state through Extensions, which are cloneable. Because the same extensions can be sent with multiple Requests, storing only the latest waker would fail to handle the cancellation of multiple requests correctly.

To address this, I switched to using CancellationTokens from tokio-util (which rely on Tokio's Notify internally). When we call CancellationToken::cancelled_owned(), it returns a WaitForCancellationFutureOwned that implements Future and wraps an underlying Notified struct. The Notified future updates the current waker when polled. Under the hood, each task obtains its own Notified object which refers to a shared linked-list node containing its latest waker. This enables concurrent cancellation across multiple requests sharing cloned Extensions.

[LucioFranco]

LGTM left you to suggestions

[LucioFranco]

prob dont need this feature in examples?

[LucioFranco]

I wonder if using something like https://doc.rust-lang.org/stable/std/macro.cfg_select.html would make this cleaner to read