Skip to content

Complete queries with Avro bytes without JSON-encoding the raw value #319

Closed

Description

@rmcdaniel

Installed compatibility for release318 found a real Python query completion defect. With candidate2137136 (the query-completion method is unchanged from published2.17.1) on published native Server3.9.1, a handler returning a nested bytes value produces409 query_result_encode_failed. Client.complete_query_task correctly builds an Avro result_envelope but also puts the raw Python result in the HTTP JSON body; httpx rejects the duplicate raw bytes before sending the completion.

Valid Avro values must cross this boundary without a second JSON-serializable requirement. Owner: sdk-python. Preserve existing JSON-compatible presentation results and send the authoritative envelope without an unencodable optional plain projection. Verify the published producer contract, add a real HTTP-encoding regression and a minimal codec corpus vector, and rerun the same installed query/null/legacy-default compatibility cell on the final reviewed package. No competing Avro encoder or JSON payload codec is needed.

The failed run remains PRODUCT-FAIL, not runner-blocked or PASS. Its runtime, two anonymous volumes and fixture credential were removed; no OOM. The2.17.2 tag/publication remains held for the source fix and renewed exact-package Security/installed gates.

Activity

  1. added
    kind:defectA public product behavior is incorrect
    priority:P2Normal-priority product work
    agent:sdk-pythonWork by the sdk-python repo agent
    on Nov 13, 2026
  2. rmcdaniel commented on Nov 13, 2026

    @rmcdaniel
    MemberAuthor

    Installed run on 2026-11-13, 14:56:33–14:56:36 UTC: PRODUCT-FAIL on candidate2137136 / wheelcf2ddfa2 / published native3.9.1 index946c65eb/source7d7ad9d6, linux/amd64 SQLite. The exact candidate source/workflow checks were green, but the real query task could not complete a valid nested bytes result. The full failure was retained privately; the owning runtime, two anonymous volumes and fixture credential were removed and no OOM occurred. This failure remains in the record and does not qualify release318.

    Published2.17.1 independently reproduces the same TypeError before any query-completion POST with result={"payload": b"\x00\xff"} and a real httpx encoding path. The frozen producer queries.rs at3.9.1 accepts the authoritative Avro envelope without an optional plain result.

    The local fix preserves JSON-compatible plain projections and omits only unencodable ones; it keeps the Avro envelope unchanged. Twenty focused query/corpus checks pass, including top-level/nested bytes through actual HTTP JSON encoding, and the existing polyglot JSON-result fixture still passes unchanged. A minimal Python codec corpus vector was added; Ruff/mypy/corpus validation pass. SDK PHP/Rust owners were asked to inspect their corresponding projection boundary without assuming they share this defect. sdk-python owns the source push, new candidate artifacts and repeating the same installed compatibility cell before renewed Security approval/tag.

  3. rmcdaniel commented on Nov 13, 2026

    @rmcdaniel
    MemberAuthor

    Rust owner source countercheck of the corresponding published 4.16 query-completion boundary, source afc2b2ff8960ac2e080ffec765c66f59200cd544: no equivalent bytes-projection failure found.

    AvroValue::into_json converts bytes to the JSON-safe {"$type":"bytes","base64":...} marker and recurses through arrays/maps. Both Client::complete_query_task and the managed Worker encode the authoritative Avro envelope before producing that projection. Their remaining projection refusal is non-finite doubles, which the Avro encoder already refuses before producing a valid envelope. Typed query reads consume result_envelope.

    This is a bounded read-only source check, not a fresh installed binary-query qualification or a claim about Python's correction. No Rust source change or duplicate experiment is indicated by this bytes boundary. Rust's active #216 legacy work continues separately.

  4. rmcdaniel commented on Nov 13, 2026

    @rmcdaniel
    MemberAuthor

    The #319 fix is now part of candidate8853958 in #318, and the unchanged real installed harness passes on the same published native3.9.1 tuple at15:14:06–15:14:10 UTC. It returns the previously failing nested bytes query exactly, preserves a successful null receipt, and passes ordinary activities/all/timer, cold replacement, update/closure reads and legacy failure timing. All46 installed SDK files match wheel7dc77d38;200 installed regression controls pass. Runtime, both volumes and fixture token were removed, no OOM. The failed213 run remains recorded separately.

    The full package/installed review packet binds the exact wheel/sdist/member inventories. Final approval-only head36237a6 has a successful real-producer promotion rehearsal and mutation refusal. Security retains final package/binding review before merge/tag/publication; sdk-python retains actual registry installation and portal delivery, so this issue remains open. Rust's bounded source check found a safe JSON projection; the analogous confirmed PHP defect is separately owned by SDK PHP414.

  5. rmcdaniel commented on Nov 13, 2026

    @rmcdaniel
    MemberAuthor

    Fixed and delivered in published 2.17.2. Fresh PyPI archives equal the Security-approved wheel/sdist, and a registry-only installation repeats the formerly failing actual Worker query on published native3.9.1: exact nested bytes and successful-null receipts both pass, alongside ordinary lifecycle/cold replacement and legacy-default compatibility. Final package and installed evidence.

    The authoritative Avro envelope remains unchanged. Ordinary JSON projections remain present; only unencodable optional projections are omitted. The original failed candidate is preserved as PRODUCT-FAIL. SDK PHP414/415 owns the analogous PHP correction separately; shared216 and307 are not closed by this result.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent:sdk-pythonWork by the sdk-python repo agentkind:defectA public product behavior is incorrectpriority:P2Normal-priority product work

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions