[frame looping] Update and optimize buffer state restoration

frame_loop_buffers_content.md

@@ -1,14 +1,14 @@
 # Buffer Content Restoration during Frame Looping
 
-During frame looping, GFXReconstruct replays a specific range of frames repeatedly. Since GPU and compute shader workloads can mutate buffer contents (e.g., storage buffers, uniform texel buffers) during loop iterations, these mutations must be reverted at the loop boundary to prevent rendering drift or verification mismatch.
+During frame looping, GFXReconstruct replays a specific range of frames repeatedly. Since GPU and compute shader workloads can mutate buffer contents (e.g., storage buffers, uniform texel buffers, copy destinations) during loop iterations, these mutations must be reverted at the loop boundary to prevent rendering drift or verification mismatch.
 
-This document describes the design for backing up and restoring Vulkan buffer contents across loop iterations.
+This document describes the design for tracking, backing up, and restoring Vulkan buffer contents across loop iterations.
 
 ---
 
-## Proposed Design: Shadow Buffer Backup and Restore
+## Design: Lazy Shadow Buffer Backup and Restore
 
-To restore buffer contents at the loop boundary, GFXReconstruct implements a **Shadow Buffer** allocation and copy mechanism.
+To restore buffer contents at the loop boundary, GFXReconstruct implements a **Lazy Shadow Buffer** allocation and copy mechanism that mirrors the design used for images.
 
 ### 1. Enabling Copy Capability on All Buffers
 Per the Vulkan specification, a buffer cannot be the source or destination of a transfer copy command unless it was created with `VK_BUFFER_USAGE_TRANSFER_SRC_BIT` and `VK_BUFFER_USAGE_TRANSFER_DST_BIT`.
@@ -19,39 +19,54 @@ To ensure we can copy any buffer at runtime, GFXReconstruct intercepts buffer cr
 
 This guarantees that all buffers created during setup or replay can legally participate in shadow copy operations.
 
-### 2. Initial Buffer State Backup (At Loop Start)
-When entering the loop (`OnLoopStart`), before replaying the loop range, GFXReconstruct captures the initial contents of all active buffers:
-
-1.  **Initialize Restoration Resources**: Ensure a dedicated restoration command pool and command buffer (`restoration_command_buffer_`) are created for the target Vulkan logical device.
-2.  **Pipeline Barrier Injection**: Begin recording the restoration command buffer and insert a execution barrier:
-    *   `srcStageMask = VK_PIPELINE_STAGE_ALL_COMMANDS_BIT`
-    *   `dstStageMask = VK_PIPELINE_STAGE_TRANSFER_BIT`
-    *   `srcAccessMask = VK_ACCESS_MEMORY_WRITE_BIT`
-    *   `dstAccessMask = VK_ACCESS_TRANSFER_READ_BIT`
-    This guarantees any previous GPU writes to the buffers are flushed and visible before we copy them.
-3.  **Iterate and Create Shadow Buffers**: Iterate through all tracked buffers in the `VulkanObjectInfoTable`. For each active buffer:
-    *   Verify the buffer's device allocator is valid and that its size is greater than zero.
-    *   Create a corresponding GPU-only **Shadow Buffer** (`shadow_buffer`) with identical size and `VK_BUFFER_USAGE_TRANSFER_SRC_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT`.
-    *   Allocate memory and bind it to the shadow buffer handle.
-4.  **Record Copies**: Call `vkCmdCopyBuffer` to copy the original buffer contents to the shadow buffer.
-5.  **Submit and Synchronize**: Close the command buffer, submit it to the active queue, and wait for completion (`vkQueueWaitIdle` or device wait idle). This ensures backup is fully completed on the GPU before replay begins.
-
-### 3. Buffer Content Restoration (At Loop Boundary, Iteration 2+)
+### 2. Tracking Buffer Usage Across Command Buffers
+Instead of snapshotting every buffer created in the entire application (including inactive auxiliary devices or unused buffers), GFXReconstruct tracks which buffers are actually referenced by submitted command buffers:
+
+*   **Command Recording Interception**: Intercepts commands referencing or mutating buffers to populate `VulkanCommandBufferInfo::bound_buffers`:
+    *   `vkCmdBindVertexBuffers`, `vkCmdBindVertexBuffers2`, `vkCmdBindVertexBuffers2EXT` (vertex buffers)
+    *   `vkCmdBindIndexBuffer`, `vkCmdBindIndexBuffer2` (index buffers)
+    *   `vkCmdDrawIndirect`, `vkCmdDrawIndexedIndirect`, `vkCmdDispatchIndirect` (indirect arguments)
+    *   `vkCmdCopyBuffer`, `vkCmdCopyBufferToImage`, `vkCmdCopyImageToBuffer` (source and destination buffers)
+    *   `vkCmdUpdateBuffer`, `vkCmdFillBuffer` (destination buffer)
+    *   `vkCmdPipelineBarrier` (buffer memory barriers)
+*   **Descriptor Set Tracking**: Intercepts `vkCmdBindDescriptorSets` / `vkCmdBindDescriptorSets2` to inspect bound descriptor sets and track all referenced buffer descriptors (storage buffers, uniform buffers, and texel buffer views).
+*   **Secondary Command Buffer Propagation**: When `vkCmdExecuteCommands` is called, secondary command buffers are tracked and their `bound_buffers`, `bound_descriptor_sets`, and layout transitions are propagated to the primary command buffer. All executed secondary command buffers are also recursively traversed when collecting touched buffers.
+
+### 3. Lazy Buffer State Backup (Before Queue Submit in Iteration 1)
+When `vkQueueSubmit` / `vkQueueSubmit2` is called during the first loop iteration (`IsLoopFirstIteration()`):
+
+1.  **Extract Submitted Command Buffers**: GFXReconstruct extracts the command buffers in the submit and calls `CollectTouchedBuffersFromCommandBuffer` to populate `loop_touched_buffers_`.
+2.  **Filter Buffers**: For each touched buffer:
+    *   Verify the buffer has a valid handle, non-zero size, and a valid device allocator.
+    *   Verify the buffer belongs to the submitting queue's logical device (`dev_info->handle == device`).
+    *   Verify the buffer has not already been snapshotted (`shadow_buffers_.find(buffer_id) == shadow_buffers_.end()`).
+3.  **Allocate Shadow Buffers**: For each eligible buffer:
+    *   Create a GPU-only shadow buffer (`shadow_buffer`) with matching size and transfer usage flags.
+    *   Allocate and bind device local memory (`shadow_memory`).
+    *   Track the allocation in `shadow_buffers_`.
+4.  **Record and Submit Snapshot Copies**:
+    *   Record a pipeline barrier ensuring previous writes are complete (`srcStageMask = VK_PIPELINE_STAGE_ALL_COMMANDS_BIT | VK_PIPELINE_STAGE_HOST_BIT`, `dstStageMask = VK_PIPELINE_STAGE_TRANSFER_BIT`).
+    *   Record `vkCmdCopyBuffer` from the original buffer to the shadow buffer.
+    *   Submit the restoration command buffer and wait for idle before the workload command buffers execute.
+
+This guarantees that pristine initial buffer contents are captured just before the GPU first touches them.
+
+### 4. Buffer Content Restoration (At Loop Boundary, Iteration 2+)
 At the end of each iteration (in `ResetLoopBoundary`), before replaying the loop range again:
 
-1.  Record a new set of copy commands onto `restoration_command_buffer_`.
-2.  For each backed up buffer, record a copy back from the shadow buffer to the original buffer:
+1.  Begin recording the restoration command buffer (`restoration_command_buffer_`).
+2.  For each backed up buffer in `shadow_buffers_`, record a copy from the shadow buffer back to the original buffer:
     *   `vkCmdCopyBuffer(shadow_buffer -> original_buffer)`
 3.  Insert a post-transfer barrier to guarantee memory visibility for subsequent stages:
     *   `srcStageMask = VK_PIPELINE_STAGE_TRANSFER_BIT`
     *   `dstStageMask = VK_PIPELINE_STAGE_ALL_COMMANDS_BIT`
     *   `srcAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT`
     *   `dstAccessMask = VK_ACCESS_MEMORY_READ_BIT | VK_ACCESS_MEMORY_WRITE_BIT`
-4.  Submit the restoration command buffer and wait for completion.
+4.  Submit the restoration command buffer and wait for completion (`QueueWaitIdle`).
 
 ---
 
 ## Resource Cleanup
 To prevent memory leaks:
-*   All allocated shadow buffer handles (`shadow_buffer`) and their memory bindings are tracked in a `shadow_buffers_` map.
-*   When the consumer is destroyed (`~VulkanReplayFrameLoopConsumer`), it calls `DestroyShadowBuffers()` to free all allocated memory and destroy all shadow buffer handles.
+*   All allocated shadow buffer handles (`shadow_buffer`) and their memory bindings (`shadow_memory`) are tracked in `shadow_buffers_`.
+*   When the consumer is destroyed (`~VulkanReplayFrameLoopConsumer`), it calls `DestroyShadowBuffers()` to free all allocated memory and destroy all shadow buffer handles via their respective device allocators.

framework/decode/vulkan_object_info.h

@@ -918,6 +918,7 @@ struct VulkanCommandBufferInfo : public VulkanPoolObjectInfo<VkCommandBuffer>
     };
     std::unordered_map<format::HandleId, std::vector<ImageLayoutTransition>> image_layout_barriers;
     std::vector<format::HandleId>                                            bound_descriptor_sets;
+    std::vector<format::HandleId>                                            bound_buffers;
     std::vector<format::HandleId>                                            executed_secondary_command_buffers;
     std::unordered_map<VkPipelineBindPoint, format::HandleId>                bound_pipelines;
     std::vector<uint8_t>                                                     push_constant_data;

framework/decode/vulkan_replay_consumer_base.cpp

@@ -10389,6 +10389,7 @@ void VulkanReplayConsumerBase::ClearCommandBufferInfo(VulkanCommandBufferInfo* c
     command_buffer_info->active_render_pass_attachment_image_view_ids.clear();
     command_buffer_info->dynamic_rendering_image_view_ids.clear();
     command_buffer_info->image_layout_barriers.clear();
+    command_buffer_info->bound_buffers.clear();
     command_buffer_info->bound_pipelines.clear();
     command_buffer_info->push_constant_data.clear();
     command_buffer_info->push_constant_stage_flags     = 0;