You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
perf(export): streaming chunked JSON export without intermediate Map and list allocations #1535
In DataExportService.formatJson (lib/shared/services/data_export_service.dart), exporting table data creates heavy memory pressure:
Allocates a new Map<String, Object?> for every row (100,000 maps for 100,000 rows).
Collects all maps into a list of 100,000 items.
Encodes the whole structure via JsonEncoder.convert, creating a single massive string in RAM before writing to disk, leading to potential OOM on large datasets.
Solution
Stream JSON chunks directly into StringSink / IOSink using zero-allocation chunked formatting: buffer.write('{\n "'), buffer.write(col), buffer.write('": '), _writeJsonValue(buffer, val).
Eliminate all intermediate Map objects and the outer list.
Benefits
0 intermediate Map allocations during JSON export.
Flat memory footprint regardless of row count, avoiding Out-Of-Memory errors on large exports.
Save-to-file already streams: DataExportService.writeToSink -> _writeJsonObjectArray encodes one row at a time and yields every 500 rows. No full list of maps and no giant string on that path.
The clipboard path (formatAsync -> formatJson) needs the full string anyway (OS clipboard API), and it already runs in an isolate.
Bench, 50k rows x 20 cols: JSON via all-maps + one string is ~259 ms on one isolate, which is why it is already off the UI thread.
Activity
ZhuchkaTriplesix commentedon Oct 12, 2026
Closing after measuring against the current code.
DataExportService.writeToSink->_writeJsonObjectArrayencodes one row at a time and yields every 500 rows. No full list of maps and no giant string on that path.formatAsync->formatJson) needs the full string anyway (OS clipboard API), and it already runs in an isolate.Nothing user-visible is left to win here.