Commit ee61151
committed
core: drop non-blocking DNS resolver from S3ClientCache
S3ClientCache opted the AWS SDK Netty async client into a non-blocking
DNS resolver, gated on a hardcoded probe for the unrelocated class
io.netty.resolver.dns.DnsAddressResolverGroup. On the Spark 4 / Core ETL
driver bundle this probe is misleading: Spark and Hadoop put the
unrelocated class on the classpath, so the probe returns true, but the
active AWS SDK comes from iceberg-aws-bundle, whose NettyNioAsyncHttpClient
-> AwaitCloseChannelPoolMap -> BootstrapProvider calls DnsResolverLoader,
which is relocated to org.apache.iceberg.aws.shaded.io.netty.resolver.dns
-- a class the bundle omits. Enabling non-blocking DNS then fails when the
first connection pool/bootstrap is created.
Remove the setting entirely and always use the SDK default (blocking)
resolver. The benefit is negligible for this workload: the S3 backup
reader goes through a single (or few) bucket host(s) per job with
long-lived cached clients, keep-alive, and connection pooling, so DNS is
resolved at connection creation and then cached by the JVM/Netty --
steady-state lookups are rare. Dropping it removes the relocation/
missing-class fragility for no measurable loss and shrinks the surface
area to test across the OSS, aggregate, and prod-parity packaging shapes.
This deletes the class-name constant, the availability field and probe
helpers, the conditional builder branch, and the dns=nonBlocking cache-key
suffix, and removes the now-defunct test plus its classpath helper. Netty
relocation in the driver bundles remains correct and is retained.
Revisit only if a future workload reads across many distinct S3 hosts or
runs where DNS is slow.1 parent 0a5245d commit ee61151
2 files changed
Lines changed: 2 additions & 53 deletions
File tree
- cassandra-analytics-core/src
- main/java/org/apache/cassandra/spark/data
- test/java/org/apache/cassandra/spark/data
Lines changed: 2 additions & 33 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
68 | 68 | | |
69 | 69 | | |
70 | 70 | | |
71 | | - | |
72 | | - | |
73 | 71 | | |
74 | 72 | | |
75 | 73 | | |
| |||
221 | 219 | | |
222 | 220 | | |
223 | 221 | | |
224 | | - | |
| 222 | + | |
225 | 223 | | |
226 | 224 | | |
227 | 225 | | |
| |||
295 | 293 | | |
296 | 294 | | |
297 | 295 | | |
298 | | - | |
299 | | - | |
300 | | - | |
301 | | - | |
302 | | - | |
303 | | - | |
304 | | - | |
305 | | - | |
306 | | - | |
307 | | - | |
308 | 296 | | |
309 | 297 | | |
310 | 298 | | |
| |||
317 | 305 | | |
318 | 306 | | |
319 | 307 | | |
320 | | - | |
| 308 | + | |
321 | 309 | | |
322 | 310 | | |
323 | 311 | | |
324 | | - | |
325 | 312 | | |
326 | 313 | | |
327 | 314 | | |
328 | 315 | | |
329 | 316 | | |
330 | 317 | | |
331 | | - | |
332 | | - | |
333 | | - | |
334 | | - | |
335 | | - | |
336 | | - | |
337 | | - | |
338 | | - | |
339 | | - | |
340 | | - | |
341 | | - | |
342 | | - | |
343 | | - | |
344 | | - | |
345 | | - | |
346 | | - | |
347 | | - | |
348 | | - | |
349 | 318 | | |
350 | 319 | | |
351 | 320 | | |
| |||
Lines changed: 0 additions & 20 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
223 | 223 | | |
224 | 224 | | |
225 | 225 | | |
226 | | - | |
227 | | - | |
228 | | - | |
229 | | - | |
230 | | - | |
231 | | - | |
232 | | - | |
233 | 226 | | |
234 | 227 | | |
235 | 228 | | |
| |||
395 | 388 | | |
396 | 389 | | |
397 | 390 | | |
398 | | - | |
399 | | - | |
400 | | - | |
401 | | - | |
402 | | - | |
403 | | - | |
404 | | - | |
405 | | - | |
406 | | - | |
407 | | - | |
408 | | - | |
409 | | - | |
410 | | - | |
411 | 391 | | |
0 commit comments