You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix: MapStreamQuery serializes items with the payload serializer, breaking NDJSON/SSE framing and the JSON contract across formats and TFMs #846
As a developer exposing stream queries via MapStreamQuery, I want every stream format (NDJSON and SSE) to use the same valid framing and the same JSON contract as the rest of my Minimal API, so that clients can parse streamed items reliably and receive the same property names regardless of format or target framework.
Problem
MapStreamQuery injects IPayloadSerializer (src/NetEvolve.Pulse.AspNetCore/EndpointRouteBuilderExtensions.cs:234) and uses it for NDJSON on all TFMs (:249) and for SSE on net8.0/net9.0 (:258). On net10.0, SSE goes through TypedResults.ServerSentEvents(items) (:255) instead.
1. Framing breaks when the serialized item contains newlines.
Pre-net10 SSE (:374-377) writes data: once, then the whole output of payloadSerializer.SerializeToBytes(item), then the suffix. The payload is never split per line.
NDJSON (:399-401) writes the serialized bytes followed by \n, with no check that the payload itself contains no newline.
The default configuration is not affected, because the default JsonSerializerOptions has WriteIndented = false. The bug appears as soon as an application sets services.Configure<JsonSerializerOptions>(o => o.WriteIndented = true) (the options SystemTextJsonPayloadSerializer reads, src/NetEvolve.Pulse/Serialization/SystemTextJsonPayloadSerializer.cs:34-37) or registers a custom IPayloadSerializer that emits newlines:
NDJSON: each item spans several lines, so no single line is a valid JSON text and NDJSON readers fail.
SSE (net8/9): the client takes only { as the event data. The following lines (for example "OrderId": 1,) have no data: prefix and are dropped as unknown fields.
2. Different JSON contracts depending on format and TFM.
SystemTextJsonPayloadSerializer uses plain IOptions<JsonSerializerOptions>, falling back to new JsonSerializerOptions(). That means PascalCase and none of the application's HttpJsonOptions converters or naming policy. MapQuery, MapCommand and net10 SSE (ServerSentEventsResult) serialize with HttpJsonOptions, which use web defaults (camelCase). For the same DTO:
Endpoint / format
net8.0 / net9.0
net10.0
MapQuery / MapCommand
{"orderId":1}
{"orderId":1}
MapStreamQuery SSE
{"OrderId":1}
{"orderId":1}
MapStreamQuery NDJSON
{"OrderId":1}
{"OrderId":1}
A client reading SSE therefore sees the casing change when the server moves between net8/9 and net10. There is also a smaller difference: net10 ServerSentEvents writes a string item raw, while the pre-net10 SSE and NDJSON paths write it as a quoted JSON string.
Using IPayloadSerializer here was a deliberate choice in #681 ("so streamed items honor the app's configured Pulse serializer"), but no ADR records it. The existing tests stream only primitives (int in tests/.../EndpointRouteBuilderIntegrationTests.cs via MapStreamQuery<NumbersStreamQuery, int>, and string in the unit and correlation tests). Neither the framing nor the casing issue is covered.
Not covered by #794 / #802: those cover only the inspector endpoints and the Azure Queue Storage envelope.
Specification
NDJSON spec §3.1/§3.2: "The JSON texts MUST NOT contain newlines or carriage returns." Each JSON text is terminated by \n.
WHATWG HTML §9.2.6 Interpreting an event stream: for a field named data, the value plus a LF is appended to the data buffer. Any other field name "is ignored". A line without a colon uses "the whole line as the field name". Multi-line data therefore needs a data: prefix on every line.
ASP.NET Core Minimal APIs: JSON options: "By default, Minimal API apps use Web defaults options during JSON serialization." ServerSentEventsResult (ASP.NET Core 10) resolves IOptions<Microsoft.AspNetCore.Http.Json.JsonOptions> and its SseFormatter prefixes every line with data:.
Requirements
Choose one JSON contract for MapStreamQuery that applies to every format and TFM. Recommended: align with MapQuery, MapCommand and net10 SSE by serializing with IOptions<Microsoft.AspNetCore.Http.Json.JsonOptions>.Value.SerializerOptions, copied once with WriteIndented = false. This explicitly reverses the IPayloadSerializer choice from refactor(serialization): use IPayloadSerializer instead of raw System.Text.Json #681.
If IPayloadSerializer is kept instead: prefix every output line with data: for pre-net10 SSE, and make sure an NDJSON item never contains \n or \r. Document the resulting casing difference from net10 SSE and MapQuery.
NDJSON output must be valid NDJSON regardless of the application's indentation settings.
Pre-net10 SSE output must deliver the complete item as event data (single-line JSON, or data: per line like SseFormatter).
Keep NativeAOT and trimming compatibility (resolve type info through the options' TypeInfoResolverChain, no new trim warnings).
Record the decision in an ADR under decisions/, and add a breaking-change note to the NetEvolve.Pulse.AspNetCore README. Moving NDJSON and pre-net10 SSE to HttpJsonOptions changes the casing existing clients receive (PascalCase to camelCase).
Acceptance Criteria
A failing test is added first: a multi-property DTO streamed with WriteIndented = true configured produces invalid NDJSON and a truncated pre-net10 SSE event.
NDJSON output with indentation enabled contains exactly one valid JSON text per line.
Pre-net10 SSE output with indentation enabled delivers the full item as data to a spec-compliant parser.
The same DTO has identical property casing across MapQuery, MapStreamQuery NDJSON and MapStreamQuery SSE on net8.0, net9.0 and net10.0 (tests per TFM).
The string item handling difference between net10 SSE and the other paths is covered by a test and either aligned or documented.
An ADR and a README breaking-change note document the chosen contract.
The NativeAOT publish (aot.yml) passes with -warnaserror.
User Story
As a developer exposing stream queries via
MapStreamQuery, I want every stream format (NDJSON and SSE) to use the same valid framing and the same JSON contract as the rest of my Minimal API, so that clients can parse streamed items reliably and receive the same property names regardless of format or target framework.Problem
MapStreamQueryinjectsIPayloadSerializer(src/NetEvolve.Pulse.AspNetCore/EndpointRouteBuilderExtensions.cs:234) and uses it for NDJSON on all TFMs (:249) and for SSE on net8.0/net9.0 (:258). On net10.0, SSE goes throughTypedResults.ServerSentEvents(items)(:255) instead.1. Framing breaks when the serialized item contains newlines.
:374-377) writesdata:once, then the whole output ofpayloadSerializer.SerializeToBytes(item), then the suffix. The payload is never split per line.:399-401) writes the serialized bytes followed by\n, with no check that the payload itself contains no newline.The default configuration is not affected, because the default
JsonSerializerOptionshasWriteIndented = false. The bug appears as soon as an application setsservices.Configure<JsonSerializerOptions>(o => o.WriteIndented = true)(the optionsSystemTextJsonPayloadSerializerreads,src/NetEvolve.Pulse/Serialization/SystemTextJsonPayloadSerializer.cs:34-37) or registers a customIPayloadSerializerthat emits newlines:{as the event data. The following lines (for example"OrderId": 1,) have nodata:prefix and are dropped as unknown fields.2. Different JSON contracts depending on format and TFM.
SystemTextJsonPayloadSerializeruses plainIOptions<JsonSerializerOptions>, falling back tonew JsonSerializerOptions(). That means PascalCase and none of the application'sHttpJsonOptionsconverters or naming policy.MapQuery,MapCommandand net10 SSE (ServerSentEventsResult) serialize withHttpJsonOptions, which use web defaults (camelCase). For the same DTO:MapQuery/MapCommand{"orderId":1}{"orderId":1}MapStreamQuerySSE{"OrderId":1}{"orderId":1}MapStreamQueryNDJSON{"OrderId":1}{"OrderId":1}A client reading SSE therefore sees the casing change when the server moves between net8/9 and net10. There is also a smaller difference: net10
ServerSentEventswrites astringitem raw, while the pre-net10 SSE and NDJSON paths write it as a quoted JSON string.Using
IPayloadSerializerhere was a deliberate choice in #681 ("so streamed items honor the app's configured Pulse serializer"), but no ADR records it. The existing tests stream only primitives (intintests/.../EndpointRouteBuilderIntegrationTests.csviaMapStreamQuery<NumbersStreamQuery, int>, andstringin the unit and correlation tests). Neither the framing nor the casing issue is covered.Not covered by #794 / #802: those cover only the inspector endpoints and the Azure Queue Storage envelope.
Specification
\n.data, the value plus a LF is appended to the data buffer. Any other field name "is ignored". A line without a colon uses "the whole line as the field name". Multi-line data therefore needs adata:prefix on every line.ServerSentEventsResult(ASP.NET Core 10) resolvesIOptions<Microsoft.AspNetCore.Http.Json.JsonOptions>and itsSseFormatterprefixes every line withdata:.Requirements
MapStreamQuerythat applies to every format and TFM. Recommended: align withMapQuery,MapCommandand net10 SSE by serializing withIOptions<Microsoft.AspNetCore.Http.Json.JsonOptions>.Value.SerializerOptions, copied once withWriteIndented = false. This explicitly reverses theIPayloadSerializerchoice from refactor(serialization): use IPayloadSerializer instead of raw System.Text.Json #681.IPayloadSerializeris kept instead: prefix every output line withdata:for pre-net10 SSE, and make sure an NDJSON item never contains\nor\r. Document the resulting casing difference from net10 SSE andMapQuery.data:per line likeSseFormatter).TypeInfoResolverChain, no new trim warnings).decisions/, and add a breaking-change note to theNetEvolve.Pulse.AspNetCoreREADME. Moving NDJSON and pre-net10 SSE toHttpJsonOptionschanges the casing existing clients receive (PascalCase to camelCase).Acceptance Criteria
WriteIndented = trueconfigured produces invalid NDJSON and a truncated pre-net10 SSE event.datato a spec-compliant parser.MapQuery,MapStreamQueryNDJSON andMapStreamQuerySSE on net8.0, net9.0 and net10.0 (tests per TFM).stringitem handling difference between net10 SSE and the other paths is covered by a test and either aligned or documented.aot.yml) passes with-warnaserror.