Skip to content

Fix parse() raising TypeError on text-mode file-like objects - #594

Open
afonsojanu wants to merge 1 commit into
kurtmckee:releasesfrom
afonsojanu:fix/stringio-input-encoding
Open

afonsojanu wants to merge 1 commit into
kurtmckee:releasesfrom
afonsojanu:fix/stringio-input-encoding

Conversation

@afonsojanu

Copy link
Copy Markdown

Related to #427.

_open_resource() returns whatever a file-like object's .read() gives back untouched. For io.BytesIO that's bytes, which is fine, but for io.StringIO it's a plain str, and convert_to_utf8() always matches its input against a bytes regex (RE_XML_PI_ENCODING), so it blows up with:

TypeError: cannot use a bytes pattern on a string-like object

The parse() docstring actually recommends wrapping an untrusted string in io.StringIO or io.BytesIO, so this hits anyone following that advice with the text variant. Passing a plain str directly already works, since that path already encodes to utf-8 a few lines further down — this just applies the same treatment when the data comes from a text-mode file-like object.

Fix is a small change in _open_resource(): if the object we read from a file-like isn't bytes, encode it, matching the existing plain-string branch. Added two tests: one directly on _open_resource(), one going through parse() with an io.StringIO. Confirmed both fail with the original TypeError without this change and pass with it; full existing suite (4335 tests) still passes.

Targeting releases rather than main since this is the 6.0.x maintenance branch where the bug actually lives — I noticed main already handles this correctly after the _to_in_memory_file() rework, so this is just a small backport for people on the current stable line.

_open_resource() returned a text-mode file-like object's data untouched,
but convert_to_utf8() always matches it against a bytes regex, so
passing an io.StringIO ends up raising TypeError instead of parsing.
The docstring for parse() even suggests wrapping untrusted strings in
io.StringIO, which is exactly the case that breaks.

This encodes the read data to utf-8 when it comes back as str, mirroring
what already happens a few lines down for plain string input.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants