Skip to content

Fix redundant direct I/O tail rewrite in WritableFileWriter - #15083

Open
taimoor789 wants to merge 1 commit into
facebook:mainfrom
taimoor789:fix-12168-direct-io-redundant-tail-write
Open

Fix redundant direct I/O tail rewrite in WritableFileWriter#15083
taimoor789 wants to merge 1 commit into
facebook:mainfrom
taimoor789:fix-12168-direct-io-redundant-tail-write

Conversation

@taimoor789

Copy link
Copy Markdown

Summary

Under O_DIRECT, WriteDirect()/WriteDirectWithChecksum() pad an unaligned tail with zeros, write it, and park the tail bytes at the front of buf_ via RefitTail() so a future Append() can glue more data onto them before the next write. Neither function marked that parked tail as already written, so a later Flush(), Sync(), or Close() call with no intervening Append() would re-issue a PositionedAppend() for the same already-written bytes -- not a correctness issue, since the same bytes just get rewritten at the same offset, but wasted I/O.

This adds a direct_io_buf_is_clean_tail_ flag: set after RefitTail() when the buffer holds only an already-written tail, cleared the moment Append() adds new data. Flush()'s direct-I/O path now also checks this flag and skips the redundant write.

Fixes #12168, following the fix direction @ajkr suggested on the issue: track whether the buffer is dirty or clean and skip the write when it's clean.

Test Plan

Added WritableFileWriterTest.DirectIOSkipsRedundantTailRewrite (util/file_reader_writer_test.cc), which counts PositionedAppend() calls across Flush()/Sync()/Close() sequences with no intervening Append() and asserts no redundant write occurs, plus that appending more data still glues onto the tail correctly and final file content is byte-correct. Fails on unfixed code, passes with the fix. Full file_reader_writer_test suite (114 tests) and make check pass; make check-sources clean.

Under O_DIRECT, WriteDirect() pads an unaligned tail with zeros,
writes it, and parks the tail bytes in buf_ for a future Append() to
glue onto. It never marked that tail as already written, so a later
Flush(), Sync(), or Close() with no intervening Append() would
re-issue the same write. Track when buf_ holds only such a clean tail
and skip the redundant write. Fixes facebook#12168.
@meta-cla meta-cla Bot added the CLA Signed label Aug 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

WritableFileWriter::WriteDirect's buf_.RefitTail() causes redundant disk write when an immediate Flush() is invoked right after.

1 participant