refactor!(stump): Move updated data to its own function by Davidson-Souza · Pull Request #81 · mit-dci/rustreexo

Davidson-Souza · 2025-12-31T20:57:06Z

Before this change, modify would return the data needed to update a proof for the new block. This requires additional internal computation and extra allocations. During IBD we may never use this, since proof update is only meant for updating a few blocks worth of changes. Now there's a method specific to pull the modify_data, and modify itself will only return a new Stump.

When refactoring the addition function, I've modified it a little to allow a smarter one with a very nice property: it can pretend that it added a node, but actually represent it as deleted. The goal here is to do a Swift Sync-style protocol where you don't need deletions.

If a txout is already spent, you give an empty hash (BitcoinNodeHash::empty()). This will be exactly equivalent as giving this txo's hash and later on calling delete for with this txo's hash. However, it does this with only one call to modify and doesn't require proofs to achieve that.

This is an API breaking change, as modify now only returns one parameter, otherwise the behavior stays unchanged.

Davidson-Souza · 2026-02-05T15:43:52Z

Pushed 427a309 with minor docs updates suggested by @jaoleal

luisschwab · 2026-02-05T16:10:57Z

@Davidson-Souza do you have a Floresta branch using this PR?

Davidson-Souza · 2026-02-05T17:13:29Z

@luisschwab

@Davidson-Souza do you have a Floresta branch using this PR?

No, waiting on Jose's pr with swift sync stuff

JoseSK999 · 2026-03-19T17:03:30Z

It's time 🎅🏻

luisschwab · 2026-03-19T17:22:15Z

nit: on the commit description: the ! comes after the (*), ie: refactor(stump)!: ..., per the Conventional Commit specification.

JoseSK999 · 2026-03-23T01:31:48Z

        proof: &Proof<Hash>,
-    ) -> Result<(Self, UpdateData<Hash>), StumpError> {
-        let (intermediate, mut computed_roots) = self.remove(del_hashes, proof)?;
+    ) -> Result<Self, StumpError> {


Why no longer return UpdateData, a rebase issue? What is it for btw, I wonder

Before this change, modify would return the data needed to update a proof for the new block. This requires additional internal computation and extra allocations. During IBD we may never use this, since proof update is only meant for updating a few blocks worth of changes. Now there's a method specific to pull the modify_data, and modify itself will only return a new Stump.

luisschwab · 2026-05-12T16:22:29Z

Needs rebase

Davidson-Souza · 2026-05-13T15:16:43Z

Rebased

rustaceanrob · 2026-05-29T18:33:50Z

Unable to offer conceptual review but code LGTM

JoseSK999 · 2026-06-01T11:17:42Z

+    let stump = s.modify(&new_utxos, &[utxos[0]], &p1).unwrap();
+
    // and the proof
+    let update_data = s.get_update_data(&utxos, &[], &Proof::default()).unwrap();


Question: why aren't we passing here the deleted utxos[0] and the proof p1? Maybe a comment about it, it's confusing.

Everywhere else you pass the delete hashes and proof in get_update_data, so it seems wrong

JoseSK999 · 2026-06-01T12:28:58Z

    /// in the path to the root. This function returns the calculated roots and the hashes
    /// that were calculated in the process.


This function no longer returns those hashes no? (forest nodes)

JoseSK999 · 2026-06-01T12:48:00Z

    /// proofs, it doesn't compute the roots after the deletion, only the roots that are
-    /// needed for verification (i.e. the current accumulator).
+    /// needed for verification (i.e. the current accumulator). `calculate_hashes_delete` also
+    /// won't return the hashes that were calculated, we won't need them for updating the


Nit, better?

Suggested change

/// won't return the hashes that were calculated, we won't need them for updating the

/// doesn't return the nodes that were calculated, as we won't need them for updating the

JoseSK999

More nits

JoseSK999 · 2026-06-01T16:05:36Z

-            "29590a14c1b09384b94a2c0e94bf821ca75b62eacebc47893397ca88e3bbcbd7",
-            "4bf5122f344554c53bde2ebb8cd2b7e3d1600ad631c385a5d7cce23c7785459a",
-            "2b77298feac78ab51bc5079099a074c6d789bd350442f5079fcba2b3402694e5",
-            "726fdd3b432cc59e68487d126e70f0db74a236267f8daeae30b31839a4e7ebed",


Only a question, is there a different place where we could repurpose these constants to check the computed hashes?

We use similar tests in several other places, I don't think removing this here would be any problem.

Davidson-Souza · 2026-06-02T17:39:02Z

Applied docs suggestions by @JoseSK999

JoseSK999 · 2026-06-03T13:33:16Z

First 3 comments not addressed yet

Davidson-Souza · 2026-06-03T17:03:18Z

First 3 comments not addressed yet

Shoot, sorry! Should be fixed now

Davidson-Souza · 2026-07-30T19:02:40Z

Applied nits by @JoseSK999

JoseSK999 · 2026-08-01T12:24:30Z

+            if let Some(pos) = computed_roots.iter().position(|(old, _new)| old == root) {
+                let (old_root, new_root) = computed_roots.remove(pos);
+                if old_root == *root {
+                    new_roots.push(new_root);
+                    continue;
+                }
+            }


The if clause is always true, we can do this (same code as in modify)

if let Some(pos) = computed_roots.iter().position(|(old, _new)| old == root) { let (_, new_root) = computed_roots.remove(pos); new_roots.push(new_root); continue; }

JoseSK999 · 2026-08-01T12:42:45Z

                        updated_subtree.insert((pos, to_add));
                        pos = util::parent(pos, after_rows);

                        to_add = AccumulatorHash::parent_hash(&root, &to_add);


Shouldn't we use the match arm here to also account for empty hashes?

This function shouldn't use implicit deletions as we need the proof hashes to update the proof anyway.

JoseSK999 · 2026-08-01T12:51:11Z

+
+            new_roots.push(*root);
+        }
+        let (_, updated, destroyed) = Self::compute_update_data_add(new_roots, utxos, self.leaves);


Shouldn't we also do this here?

// If there are still roots to be added, it means that the proof is invalid // as we should have consumed all the roots. if !computed_roots.is_empty() { return Err(StumpError::InvalidProof(ProofError::RootsMismatch)); } let (_, updated, destroyed) = ...

get_update_data assumes you already validated that block, but I can add it out of caution.

JoseSK999 · 2026-08-01T12:55:05Z

+    let stump = s.modify(&new_utxos, &[utxos[0]], &p1).unwrap();
+
    // and the proof
+    let update_data = s.get_update_data(&utxos, &[utxos[0]], &p1).unwrap();


This was wrong

Suggested change

let update_data = s.get_update_data(&utxos, &[utxos[0]], &p1).unwrap();

let update_data = s.get_update_data(&new_utxos, &[utxos[0]], &p1).unwrap();

JoseSK999 · 2026-08-01T13:17:40Z

+
+/// A type representing what's changed for addition.
+///
+/// It returns the final roots destroyed hashes and  all the new positions created during this add.


Nit: I think order is inaccurate and there's a double space

Before this change, `modify` would return the data needed to update a proof for the new block. This requires additional internal computation and extra allocations. During IBD we may never use this, since proof update is only meant for updating a few blocks worth of changes. Now there's a method specific to pull the modify_data, and `modify` itself will only return a new Stump. When refactoring the addition function, I've modified it a little to allow a smarter one with a very nice property: it can pretend that it added a node, but actually represent it as deleted. The goal here is to do a Swift Sync-style protocol where you don't need deletions. If a txout is already spent, you give an empty hash (BitcoinNodeHash::empty()). This will be exactly equivalent as giving this txo's hash and later on calling delete for with this txo's hash. However, it does this with only one call to modify and doesn't require proofs to achieve that. This is an API breaking change, as modify now only returns one parameter, otherwise the behavior stays unchanged.

Davidson-Souza · 2026-08-04T21:29:19Z

@JoseSK999 done!

Davidson-Souza force-pushed the swift-sync branch from 8eb9f98 to bfd09ed Compare December 31, 2025 20:59

Davidson-Souza marked this pull request as ready for review January 2, 2026 17:40

jaoleal reviewed Jan 6, 2026

View reviewed changes

Comment thread src/accumulator/stump.rs Outdated

Comment thread src/stump/mod.rs

Comment thread src/stump/mod.rs

Davidson-Souza force-pushed the swift-sync branch from 38e545e to 427a309 Compare February 5, 2026 15:42

Davidson-Souza force-pushed the swift-sync branch 4 times, most recently from 78d99d9 to d86c319 Compare March 19, 2026 20:11

JoseSK999 reviewed Mar 23, 2026

View reviewed changes

rustaceanrob mentioned this pull request May 12, 2026

[WIP] feat: assume-valid swift sync getfloresta/Floresta#837

Draft

Davidson-Souza force-pushed the swift-sync branch from d86c319 to df4572f Compare May 13, 2026 15:15

JoseSK999 reviewed Jun 1, 2026

View reviewed changes

Davidson-Souza force-pushed the swift-sync branch from df4572f to 0f90d32 Compare June 2, 2026 17:38

Davidson-Souza force-pushed the swift-sync branch from 0f90d32 to d2c3dbd Compare June 2, 2026 17:50

JoseSK999 mentioned this pull request Jun 3, 2026

[WIP] feat: Assume-Valid SwiftSync with accumulator building getfloresta/Floresta#1115

Draft

Davidson-Souza force-pushed the swift-sync branch from d2c3dbd to 41e1159 Compare June 3, 2026 17:02

lorenzolfm reviewed Jun 9, 2026

View reviewed changes

Comment thread src/stump/mod.rs Outdated

Comment thread src/stump/mod.rs Outdated

Davidson-Souza force-pushed the swift-sync branch from 41e1159 to b97558b Compare July 26, 2026 22:32

JoseSK999 reviewed Jul 28, 2026

View reviewed changes

Comment thread src/proof/mod.rs Outdated

Comment thread src/proof/mod.rs Outdated

Comment thread src/stump/mod.rs Outdated

Comment thread src/stump/mod.rs

Comment thread src/stump/mod.rs Outdated

Comment thread src/stump/mod.rs Outdated

Davidson-Souza force-pushed the swift-sync branch from b97558b to eed1584 Compare July 30, 2026 19:02

JoseSK999 suggested changes Aug 1, 2026

View reviewed changes

Davidson-Souza added 3 commits August 4, 2026 18:22

docs: update documentation for the API

9ade180

docs(examples): update docstring to match new flow

e1afeb7

Davidson-Souza force-pushed the swift-sync branch from eed1584 to e1afeb7 Compare August 4, 2026 21:23

		/// in the path to the root. This function returns the calculated roots and the hashes
		/// that were calculated in the process.

	/// won't return the hashes that were calculated, we won't need them for updating the
	/// doesn't return the nodes that were calculated, as we won't need them for updating the

	let update_data = s.get_update_data(&utxos, &[utxos[0]], &p1).unwrap();
	let update_data = s.get_update_data(&new_utxos, &[utxos[0]], &p1).unwrap();

Uh oh!

Conversation

Davidson-Souza commented Dec 31, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Davidson-Souza commented Feb 5, 2026

Uh oh!

luisschwab commented Feb 5, 2026

Uh oh!

Davidson-Souza commented Feb 5, 2026

Uh oh!

JoseSK999 commented Mar 19, 2026

Uh oh!

luisschwab commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

luisschwab commented May 12, 2026

Uh oh!

Davidson-Souza commented May 13, 2026

Uh oh!

rustaceanrob commented May 29, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JoseSK999 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Davidson-Souza commented Jun 2, 2026

Uh oh!

JoseSK999 commented Jun 3, 2026

Uh oh!

Davidson-Souza commented Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Davidson-Souza commented Jul 30, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

luisschwab commented Mar 19, 2026 •

edited

Loading

Davidson-Souza commented Jun 3, 2026 •

edited

Loading