Optimize intensity matching - #236
Conversation
trautmane
left a comment
There was a problem hiding this comment.
Looks good to me! My first instinct is to consolidate the caching additions with the other implementations but it is not that important. Let's leave this as-is for now and come back to consolidate later if we think consolidation (or improvement of the older cache stuff) is worth the effort.
|
Thanks for your comments, @trautmane! In response to them, I renamed On a related note, I also deprecated the "old" intensity matching code that we haven't used in a while (see here). I agree that we should consolidate the caching implementations at some point. It was not immediately clear to me how to do that, since |
This PR slightly shifts the level of caching from raw tiles to downscaled and filtered or even fully rendered tiles. The first one is a zero-tradeoff improvement: instead of the full-scale tile + mask, the downsampled versions are stored, reducing per-tile effort and cache pressure. Therefore, it is the new default. The second one trades memory for computational resources: the full rendered bounding box is stored, making the first render somewhat expensive, and the following ones essentially free. It should therefore be used cautiously (especially for ic3d processes, where the full tile needs to be rendered anyway).
@trautmane This introduces a hand-rolled cache that was modeled after
ImageProcessorCache. If you think it's worth doing that, I'm happy to try to consolidate all the different caching mechanisms. Also, imglib2 images are used instead of ImageJ image processors to enable zero-copy views for the fully-cached implementation.Previous state
IntensityMatcherrendered both tiles of every overlapping pair from scratch, on every call, viaRender.render(...)into ImageJFloatProcessor/ColorProcessorrasters sized to just the overlap box. The only cache in the path was a sharedImageProcessorCacheof raw, full-resolution source images (pre-filter, pre-downsample, pre-mesh) — so filtering, downsampling, and mesh-mapping were all redone for every pair a tile appears in, even though a tile typically overlaps many same-layer and cross-layer neighbors.Two options now
Rendering is factored into
TileRenderer, split into two independent halves:loadSource— pair-independent: load the full-res source, apply the tile's filter, downsample to its mipmap level →DownsampledSource.renderBox— pair-dependent: transform the downsampled source into the requested world-space box (using a mesh transform) →RenderedTile(image / weight / coefficient-label rasters).Two strategies decide what to cache between calls, selected by the new
--cacheRenderedTilesflag:TightBoxTileRenderer(default)CachedTileRenderer(--cacheRenderedTiles)DownsampledSourceper tileRenderedTile(whole bounding box) per tileViews.interval) of the cached renderBoth share a small generic LRU,
TileCache<T>, keyed by tile spec and weight-bounded in kilobytes by the existing--maxPixelCacheGb(adapted fromImageProcessorCache's eviction logic). As before, same-layer and cross-layer matching share one renderer/cache when their render scales are equal.New / modified classes
New:
TileRenderer— abstract base defining theloadSource/renderBoxsplit and the shared mesh-mapping logic.TightBoxTileRenderer— caches only the downsampled source; re-meshes each overlap box.CachedTileRenderer— caches the full per-tile render; serves overlaps as zero-copy crops.TileCache<T>— generic per-tile-id, kilobyte-bounded LRU cache used by both renderer strategies.DownsampledSource— the cacheable pair-independent render result (filtered/downsampled image, mask, mipmap level).RenderedTile— the image/weight/coefficient rasters as imglib2 views, replacing the previous raw ImageJ processors.Modified:
IntensityMatcher— no longer renders directly; picks a same-/cross-layerTileRendererand iterates the resulting imglib2 rasters in lockstep instead of indexing ImageJ processors by hand.AffineIntensityCorrectionBlockWorker— replaced the single sharedImageProcessorCachewith a kilobyte budget passed down toIntensityMatcher, which now owns its own per-scaleTileCacheinstances.FIBSEMIntensityCorrectionParameters— carries the newcacheRenderedTilesflag through to the matcher.AlgorithmicIntensityAdjustParameters— adds the--cacheRenderedTilesCLI flag.Validation and benchmarking
The three methods give exactly the same results, see results obtained with the:
--cacheRenderedTilesflag enabledRuntimes with the processes can be seen below, where 10 layers of the w61_s109 multi-sem stack were corrected with ic3d. Matching times were recorded, clustered into two clusters (roughly cached and uncached), and the median of both clusters is shown, as well as overall runtime and cache miss overhead. The first image shows the situation with an overly large cache (100GB), the second with a somewhat tight cache (1GB). It looks like 10GB should be sufficient for multi-sem stacks for all three methods. In the case where the cache is sufficiently large, the new methods speed up the process by ~3x and ~5x, respectively.
FYI @StephanPreibisch