Add HTTP body capture for WSGI by anuraaga · Pull Request #5272 · open-telemetry/opentelemetry-python-contrib

anuraaga · 2026-10-30T03:33:55Z

Description

I am helping implement HTTP body capture. This PR defines shared helpers and uses them in WSGI. After this PR, I would send PRs using the shared helpers for ASGI, tornado, aiohttp.

#5215

I have taken some liberties from the proposals in the issue. I think separating request and response body size limits is useful, requests are often untrusted and usually have far different size than response. It matches a similar concept as the separate header allowlists for header capture. And when separating, I don't see much need for the enabled flag at all since it duplicates =0, a 1KB default seems quite arbitrary and needing to be changed almost all the time, so it' seems reasonable to instead just make the user specify always. So I implemented as a single knob captured_request_body_size defaulting to 0, disabled. Seems to do the job in the easiest way for users

/cc @xrmx

Type of change

Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

I am a human and not a bot
I have run tox -e precommit
I have run tox -e typecheck
I have run the relevant tests and linting tox environments
This requires the following PR in core:

opentelemetry-pr-dashboard · 2026-10-30T03:34:47Z

Pull request dashboard status

Waiting on the author · refreshed 2026-11-11 09:32 UTC

Respond to 7 review items (e.g. link a commit, explain why not, ask a follow-up):

Inline threads: 1, 2, 3, 4, 5, 6, 7

Status above doesn't look right?

Just replied or pushed? Anything around or after the refresh time above may not be picked up yet — give it a few minutes.
Should this be with reviewers? Comment /dashboard route:reviewers to route it to them.
Anything wrong — including the routing? Report it with what you expected; it helps us improve the dashboard.

anuraaga · 2026-10-30T03:34:48Z

+)
+
+
+class CapturingStream(wrapt.ObjectProxy):


WSGI is very old and I know is used widely with functions not defined in any PEP. I have tried to support common extensions while being as safe as possible here

anuraaga · 2026-10-30T03:37:00Z

+        end = min(position + len(data), self._max_size)
+        self._data[position:end] = data[: end - position]
+
+    def to_attribute_value(self) -> str | bytes | None:


I was a bit surprised by the string/bytes polymorphic type for the attribute. I don't know if there is any similar one in semconv? It puts a lot of work in SDKs since it's not that trivial to know if it's text or not, we also have to work around the unicode truncation caused by max size - it feels like SDKs "doing too much". It feels like all of this should be left to backends? In that case, it could also be possible to capture content-encoding bodies since the backend should be able to support decoding them.

@SylvainJuge what do you think? Could it make sense to reconsider this part of semconv?

I agree with you, here trying to capture it as text in the SDK/instrumentation level adds quite a lot of complexity, this also shows in the WIP java implementation open-telemetry/opentelemetry-java-instrumentation#20620

requires to implement a heuristic to detect textual content, which may also rely on HTTP headers or not

requires to deal with unicode truncation detection

may also imply some overhead as conversion from/to bytes to string is required

The history and discussion of open-telemetry/semantic-conventions#3521 drifted in this direction because:

using text makes it easier for humans to consume

using text makes implementation of sanitization easier

As you wrote, using the same semconv attribute for something that is either a binary array or text makes also the consumption harder as the format is not known in advance and depends on the context (it may vary from HTTP request to request).

I will open an issue in semconv so we can further discuss improvements we could make here.

@anuraaga I've opened open-telemetry/semantic-conventions#4307 to try to clarify/simplify this.

I think doing it in the SDK has some advantages for the end-users, however if this proves too complex to implement we should definitely consider alternatives, feel free to comment on the issue above to continue the discussion.

opentelemetry-pr-dashboard · 2026-11-09T15:29:11Z

Hi @anuraaga — just a friendly reminder that this pull request is waiting on you. The dashboard status comment has the open items and is kept current.

Replying is enough to hand it off — answer, explain why no change is needed, or ask a follow-up. The dashboard routes it onward once nothing on the list is waiting on you.
To hand it back for any other reason, including the dashboard getting this wrong, comment /dashboard route:reviewers.

anuraaga · 2026-11-09T15:30:35Z

Don't think this has anything pending for me - @xrmx can you take a look? We'll probably make changes to the spec to simplify it, but it probably makes sense to follow up after those changes to update this instrumentation

Copilot

🟡 Changes recommended

Uncaught codec errors can disrupt request handling and span cleanup, while the pinned SDK cannot preserve captured binary bodies as byte-valued span attributes.

3 open issues

What changed in this PR

Adds opt-in HTTP request body capture to WSGI instrumentation, with shared helpers intended for reuse by other HTTP instrumentations.

Changes:

Adds bounded buffering and stream proxying to capture bytes read by the application.
Integrates capture with WSGI span finalization, disabled by default and in stable-only HTTP mode.
Adds documentation, tests, a changelog entry, and the wrapt dependency.

File	Description
`uv.lock`	Records the new utility dependency on `wrapt`.
`util/opentelemetry-util-http/tests/test_body_capture.py`	Tests buffering, decoding, limits, and stream proxy behavior.
`util/opentelemetry-util-http/src/opentelemetry/util/http/body_capture.py`	Adds shared body buffering and capture helpers.
`util/opentelemetry-util-http/pyproject.toml`	Declares the `wrapt` dependency.
`instrumentation/opentelemetry-instrumentation-wsgi/tests/test_wsgi_middleware.py`	Tests capture configuration and WSGI integration.
`instrumentation/opentelemetry-instrumentation-wsgi/src/opentelemetry/instrumentation/wsgi/__init__.py`	Documents and integrates request body capture.
`.changelog/5272.added`	Announces opt-in request body capture.

_{Review effort: Balanced}

xrmx · 2026-11-11T09:30:19Z

+        return
+    body = body_buffer.to_attribute_value()
+    if body is not None:
+        span.set_attribute(HTTP_REQUEST_BODY_CONTENT, body)


Should be handled by #5323

xrmx · 2026-11-11T09:31:27Z

+        except (LookupError, UnicodeDecodeError):
+            return data


This will require a pylint warning suppression # pylint: disable=broad-exception-caught

+    def _self_readinto(self, target: bytearray | memoryview) -> int | None:
+        count = self.__wrapped__.readinto(target)
+        if count:
+            self._self_capture(memoryview(target)[:count])


xrmx · 2026-11-10T14:34:19Z

+
+# TODO: use the semconv constant once the semconv package is generated from a
+# semantic conventions release that includes this attribute (> 1.44.0).
+HTTP_REQUEST_BODY_CONTENT: Final = "http.request.body.content"


We now have it in opentelemetry.semconv._incubating.attributes.http_attributes

xrmx · 2026-11-10T14:49:52Z

+) -> BodyBuffer | None:
+    """Returns a buffer for a body, or ``None`` if it should not be captured.
+
+    Bodies with a content coding are not captured.


Suggested change

Bodies with a content coding are not captured.

Bodies with a content encoding are not captured.

xrmx · 2026-11-10T15:50:02Z

+        return
+    body = body_buffer.to_attribute_value()
+    if body is not None:
+        span.set_attribute(HTTP_REQUEST_BODY_CONTENT, body)


Should we also try to set the content type and content-encoding here? These are opt-in but otherwise it would be pretty hard for backends to know how to handle the data for non-text data. This would probably require an update on the semconv side as well.

Add HTTP body capture for WSGI

9bd6396

anuraaga requested a review from a team as a code owner October 30, 2026 03:33

otelbot-python Bot added this to Python PR digest Oct 30, 2026

opentelemetry-pr-dashboard Bot mentioned this pull request Oct 30, 2026

Pull Request Dashboard #4959

Open

anuraaga commented Oct 30, 2026

View reviewed changes

changelog

bc20f2b

SylvainJuge mentioned this pull request Nov 4, 2026

enhance or simplify http.{request,response}.body.content open-telemetry/semantic-conventions#4307

Open

xrmx requested a balanced review from Copilot November 10, 2026 14:29

Copilot started reviewing on behalf of xrmx November 10, 2026 14:29

View session

Copilot AI reviewed Nov 10, 2026

View reviewed changes

xrmx reviewed Nov 10, 2026

View reviewed changes

	Bodies with a content coding are not captured.
	Bodies with a content encoding are not captured.

		)


		class CapturingStream(wrapt.ObjectProxy):

Conversation

anuraaga commented Oct 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

Checklist:

Uh oh!

opentelemetry-pr-dashboard Bot commented Oct 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull request dashboard status

Uh oh!

anuraaga Oct 30, 2026

Choose a reason for hiding this comment

Uh oh!

anuraaga Oct 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SylvainJuge Nov 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SylvainJuge Nov 4, 2026

Choose a reason for hiding this comment

Uh oh!

opentelemetry-pr-dashboard Bot commented Nov 9, 2026

Uh oh!

anuraaga commented Nov 9, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

xrmx Nov 11, 2026

Choose a reason for hiding this comment

Uh oh!

xrmx Nov 11, 2026

Choose a reason for hiding this comment

Uh oh!

xrmx Nov 10, 2026

Choose a reason for hiding this comment

Uh oh!

xrmx Nov 10, 2026

Choose a reason for hiding this comment

Uh oh!

xrmx Nov 10, 2026

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

anuraaga commented Oct 30, 2026 •

edited

Loading

opentelemetry-pr-dashboard Bot commented Oct 30, 2026 •

edited

Loading

anuraaga Oct 30, 2026 •

edited

Loading

SylvainJuge Nov 2, 2026 •

edited

Loading