Issue 173 bytea jdbctype

[stalep]

No description provided.

[stalep]

Allocation profiling results: boot-time-verbose import (50 runs, batch=5, cores=4, 8GB heap)

Profiled with async-profiler alloc event at 30-second intervals during a boot-time-verbose legacy import. Three runs compared:

Baseline (no fixes — JSONB, sql/sqlall nodes, String encoding)

Snapshot	Total	byte[]	StringBuilder	PG wire	UTF-8 encode	String decode	WorkQueue	JqParse
30s	9,785 MB	6,980	2,547	1,177	895	1,053	371	1,888
70s	11,719 MB	6,687	2,607	971	838	971	1,551	1,773

Result: OOM at ~50s, ~4,200 values processed before crash.

With #154 only (sql→jq, but still JSONB + String encoding)

Snapshot	Total	byte[]	StringBuilder	PG wire	UTF-8 encode	String decode	WorkQueue	JqParse
30s	14,607 MB	7,903	2,221	1,912	873	1,909	95	3,434

Result: OOM at ~60s, ~10,150 values. Faster throughput but higher peak allocation from in-memory jq evaluation.

With all fixes including #173 BYTEA + JqValueJdbcType

Snapshot	Total	byte[]	StringBuilder	PG wire	UTF-8 encode	String decode	WorkQueue	JqParse
30s	16,258 MB	8,183	1	1,510	0	4	755	2,747
70s	18,011 MB	9,394	0	1,274	0	1	786	2,388

Result: Completes successfully in 105s, 28,200 values. No OOM.

Summary of #173 impact

Encoding step	Baseline (MB/10s)	With #173 (MB/10s)	Reduction
StringBuilder (toJsonString → serialize)	2,547	1	99.96%
String.encodeUTF8 (JDBC param encoding)	895	0	100%
Encoding.decode (JDBC result decoding)	1,053	4	99.6%

The serializeToBytes() near-zero-alloc path and getBytes()/setBytes() JDBC operations completely eliminate the String intermediaries that were the primary allocation bottleneck. For deferred-from-bytes JqString values (the common case for data flowing through parse→jq→serialize), the appendToBytes() path does zero-copy raw byte copying from the source array.

Performance comparison

Metric	Baseline	All fixes + #173
Values at 30s	1,413	7,477
Final values	~4,200 (OOM)	28,200 (complete)
Time	OOM at ~50s	105s (clean exit)
Throughput	~84 vals/sec	~268 vals/sec
OOM?	Yes	No

3.2x throughput improvement and the import actually completes on 8GB heap for the first time.