Repository navigation
WIP - Concurrent psbt rewrite - #78
arminsabouri wants to merge 3 commits into
Conversation
01cd573 to
1b45dc3
Compare
range-diff for force push 01cd573ddd → 1b45dc344a
|
|
|
||
| A *Creator* creating an unordered transaction SHOULD set `PSBT_GLOBAL_SORT_DETERMINISTIC` to enure consistent ordering rules are enforced. | ||
|
|
||
| In applications where this is desirable, the *Creator* should only set `PSBT_GLOBAL_SORT_DETERMINISTIC` to `0x01`, and leave `PSBT_GLOBAL_SORT_SEED` unset. Each replica will locally provide a value for `PSBT_GLOBAL_SORT_SEED` just in time, as a commitment to protocol transcript or something similar. |
There was a problem hiding this comment.
Deterministic flag requires a seed that is left unset
The spec requires PSBT_GLOBAL_SORT_SEED with at least 128 bits of randomness when PSBT_GLOBAL_SORT_DETERMINISTIC is 0x01, but this paragraph tells the Creator to set 0x01 and leave the seed unset, so a validator enforcing the MUST rejects the Creator's intermediate PSBT.
oak · automated · 1b45dc3
There was a problem hiding this comment.
fully deterministic requires a seed at the time of sorting, not throughout
in fungi this seed is a hash of the protocol transcript set by each client independently at the very end
| where `H_s` is a BIP 341 style tagged hash, with the tag preimage being `BIP ???? deterministic ordering` (TODO pick a real domain separator). | ||
|
|
||
| If any sort key is missing, ordering can only proceed after `PSBT_GLOBAL_SORT_SEED` is set. | ||
| If `PSBT_GLOBAL_SORT_SEED` is set and an individual field-level sort seed is set. The inidivudal sort seed will be ignored and replaced by the determinsitic key defined above. |
There was a problem hiding this comment.
Undefined field-level sort seed leaves ordering ambiguous
The new sentence refers to an "individual field-level sort seed" that is never defined; the only per-map ordering fields are PSBT_IN_SORT_KEY and PSBT_OUT_SORT_KEY. If it means the sort key, it contradicts the preceding rule that derived keys are used only when explicit keys are omitted, so replicas can sort the same set differently before clearing PSBT_GLOBAL_TX_UNORDERED.
oak · automated · 1b45dc3
There was a problem hiding this comment.
not sure what "field level sort seed" is
the only way in which the sort seed is used is to derive sort keys by a hash, so there's one sort seed for the entire transaction
|
|
||
| Clearing the `PSBT_GLOBAL_TX_UNORDERED` field indicates the order of the inputs and outputs. Due to the shuffling requirement, the given order will in general not be consistent across replicas if this is done without determining an order, resulting in diverging unsigned transactions. | ||
|
|
||
| For consistent ordering we define `PSBT_IN_SORT_KEY` and `PSBT_OUT_SORT_KEY` fields for inputs and outputs respectively, that may include up to 32 bytes of arbitrary data, used as a sort key for lexicographical sorting. |
There was a problem hiding this comment.
Equal sort keys leave input order undefined
Sort keys are arbitrary bytes with no uniqueness requirement and may be empty, but lexicographic sorting has no tie-break, so replicas can order equal-key maps differently and produce divergent unsigned transactions.
oak · automated · 1b45dc3
There was a problem hiding this comment.
this is a fair point
not likely to matter for cases we actually care about where collisions of 128-256 bit random strings are cryptographically negligible, but this probably should be addressed
stable sort is unfortunately not enough.... tie breaking seems awkward. since we already have related error conditions in sortting (e.g. missing keys if non-deterministic) i think we should just add another restriction that sort keys must be unique?
oakagent
left a comment
There was a problem hiding this comment.
Typos in what this pull request adds:
psbt-concurrent.md
- l.15 - "integrate information add by either" → "integrate information added by either"
- l.15 - "intended for for PSBTs" → "intended for PSBTs"
- l.120 - "the transaction in unordered" → "the transaction is unordered"
- l.128 - "Ordering inputs and ouputs" → "Ordering inputs and outputs"
- l.151 - "inidivudal sort seed" → "individual sort seed"
- l.151 - "determinsitic key" → "deterministic key"
- l.153 - "to enure consistent" → "to ensure consistent"
- l.162 - "This can provides" → "This can provide"
- l.167 - "needed to if replicas" → "needed if replicas"
- l.170 - "its the same" → "it's the same"
- l.170 - "same prodcedure" → "same procedure"
oak · automated
yuval-block
left a comment
There was a problem hiding this comment.
first review pass, approach ACK, i might have some more comments where i'll be a stickler for particular phrasing but on the whole this looks like a big improvement, thanks!
| BIP 370 introduces the notion of a [unique ID](https://github.com/bitcoin/bips/blob/master/bip-0370.mediawiki#user-content-Unique_Identification) which is computed as a hash that commits to UTXOs being spent and the new outputs being created, thereby ensuring PSBTs with the same ID have the same structure, but abstracting away some details of the transaction, thereby introducing a notion of equivalence that is a bit more general. | ||
| Example collaborative protocol (*N* wallets). **Creator** and **Constructor** share an empty PSBT with every wallet. Each wallet *i* runs the same pipeline below in parallel. **Combiner**, **Finalizer**, and **Extractor** fan in once all wallets have signed. | ||
|
|
||
| ```mermaid |
There was a problem hiding this comment.
if this was laid out horizontally i think that would help its readability
also maybe a second diagram that shows how forking and merging of the state is possible would be helpful? this can also remove the "wallet i" caption which is implying that
FWIW there was some pushback on using mermaid diagrams in the BIP 77 PR since that requires client side javascript, i think that is fine if thee diagrams are not essential for reading and we can just add a rendered version, but just wanted to take note of this
There was a problem hiding this comment.
Yeah i see this as a useful guide.
| The effects of a transaction on the distribution of money are determined by its effects the UTXO set: which outputs does it spend and create. However, with regards to money changing hands, we would like to treat equivalent outputs as interchangeable by disregarding the specific outpoints by which specific coins are identified. | ||
|
|
||
| Both notions of identity ensure that PSBTs with the same ID have the same effects, but it is not true that PSBTs with the same effects will have the same ID, since depends on the specific order of the inputs and outputs, among other things. By abstracting over these details we obtain a more general notion of PSBT identity than in BIP 370. The details of how to compute this ID are [provided below](#unique-identifiers). |
There was a problem hiding this comment.
was this deliberately removed? seems like it's worth discussing in which sense are unordered PSBTs a representative of an equivalence class of ordered ones, and since BIP 174 and 370 define a notion of PSBT ID i think the spec would be incomplete without this
There was a problem hiding this comment.
was this deliberately removed?
yes. i had a hard time understanding what this was trying to convey. It seems like the point its trying to convey are already covered in strict combiner section?
| #### Strict Combiner | ||
|
|
||
| BIP 174 specifies that fields with duplicate keys and more than one distinct value may be combined by [picking one of the values arbitrarily](https://github.com/bitcoin/bips/blob/master/bip-0174.mediawiki#handling-duplicated-keys). This is safe within the scope of BIP 174 because any inconsistency will only affect the validity of a transaction. In other words, when choosing one value over another for a duplicate field, although signing may fail to produce a valid signature, no money will be lost because if the IDs are the same, the the effects are identical. | ||
| BIP 174 lets a Combiner resolve a key that has two different values by picking either one. That is safe when every copy shares the same unsigned transaction, because the worst outcome is an invalid signature. Once inputs and outputs can differ between copies, picking one value can silently change what the transaction spends or pays, and the result depends on the order in which copies were combined. |
There was a problem hiding this comment.
"within the scope of bip 174" is a bit more subtle than "shares the same unsigned transaction" because equivocated values may have safety implications... bip 174 stipulates that this could only produce an invalid or valid transaction which is correct in its scope, but for security to carry through more generally may mean more than just "on any equivocation the transaction isn't valid", in particular where some notion of secret sharing is in effect that could potentially lead to subtle bugs like key cancellation, leaking secrets through nonces etc, which is why the strict combiner is a safer baseline (in addition to being a useful definition for our general merging procedure)
| A strict Combiner therefore MUST fail when the same key has conflicting values, whenever bit 0 or bit 1 of `PSBT_GLOBAL_TX_MODIFIABLE` is set. With this rule, the transaction effects of combining a set of PSBTs don't depend on the order or grouping of the combinations, or on whether a PSBT is combined more than once. The following sections define exceptions that keep these properties. | ||
|
|
||
| Unfortunately choosing between conflicting values arbitrarily does not preserve this notion of safety when merging PSBTs where the inputs or outputs of one PSBT don't correspond to the inputs or outputs which occupy the same positions in the second PSBT. For example, if the first inputs of two PSBTs spend different coins, and one PSBT takes precedence, the input in the other transaction will be omitted. If the `PSBT_IN_PREVIOUS_TXID` field of one PSBT overrides the other but this precedence is reversed for `PSBT_IN_OUTPUT_INDEX`, the one remaining input will be corrupted. | ||
| ##### Combining unique PSBTs |
There was a problem hiding this comment.
what is meant by "unique PSBTs"? my only guess is that it's the joinpsbts specific restriction, which we want to relax
There was a problem hiding this comment.
psbts with different txids. Is there a more canonical term for this?
| This ensures that repeated addition of the same output is idempotent, but that if two different distributed constructors both attempt to add an identical output (same `PSBT_OUT_VALUE` and `PSBT_OUT_SCRIPT`), that will still result in two copies of the output as intended. This is important because a transaction with only a single copy will appear correct but end up losing the value of all missing copies to mining fees. | ||
|
|
||
| This establishes a more conservative foundation for ensuring that the effects of a transaction formed by combining a set of PSBTs will always be the same regardless how these PSBTs were combined to obtain a complete transaction. | ||
| // TODO: if we merge two output how do we decide which id identifies the new one with the usmmer value? |
There was a problem hiding this comment.
merging of same script txout is deterministic and applied by or after the Sorter, so unique IDs are no longer required after since by that point txouts would be identified by position
| If the global field `PSBT_GLOBAL_SORT_DETERMINISTIC` is set (unsigned 8 bit integer) to `0x01`, then sort keys MUST NOT be set, and `PSBT_GLOBAL_SORT_SEED` MUST be set and contain at least 128 bits of randomness. | ||
|
|
||
| If `PSBT_GLOBAL_SORT_DETERMINISTIC` is set to `0x00`, explicit sort keys are required to be set and used prior to clearing the `PSBT_GLOBAL_TX_UNORDERED` field and fixing the transaction order. | ||
| // TODO: and sort key must not be set? and you have both PSBT_GLOBAL_SORT_DETERMINISTIC and SORT_SEED? |
There was a problem hiding this comment.
the rationale here is that sorting itself is only done by sort key, fully deterministic sorting treats the sort keys as purely a computed property, but in between there's a potentially useful (for compatibility with lightning interactive-tx, and potentially for coinjoins that also atomically trade ordinals) to have a specified order for some of the inputs/outputs and deterministically order the remaining ones.
so the GLOBAL_SORT_DETERMINSTIC field controls whether we're in this purely-by-explicit-key, purely deterministic, or this mixed mode without defining two or three distinct procedures
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| where `H_s` is a BIP 341 style tagged hash, with the tag preimage being `BIP ???? deterministic ordering` (TODO pick a real domain separator). | ||
|
|
||
| If any sort key is missing, ordering can only proceed after `PSBT_GLOBAL_SORT_SEED` is set. | ||
| If `PSBT_GLOBAL_SORT_SEED` is set, any explicit `PSBT_IN_SORT_KEY` or `PSBT_OUT_SORT_KEY` is ignored and replaced by the deterministic key defined above. |
There was a problem hiding this comment.
Sort-seed rule contradicts explicit-key fallback
Line 151 says any explicit PSBT_IN_SORT_KEY/PSBT_OUT_SORT_KEY is ignored whenever PSBT_GLOBAL_SORT_SEED is set, but line 40 says the seed derives keys only when explicit keys are absent and line 141 derives only omitted keys. With both present, the table keeps explicit keys while line 151 discards them, so replicas can order differently and produce divergent unsigned transactions.
oak · automated · 626af58
|
|
||
| Every receiver whose balance is net negative (they send more than they receive) must wait for confirmation from all receivers for whom it is a sender before it confirms the unique ID to the senders for whom it is a receiver. | ||
|
|
||
| If there is a cyclic payment graph, for example Alice pays Bob, Bob pays Carol, and Carol pays Alice, net positive receivers can safely initiate confirmations, breaking the circular dependency, because senders will sign will only occur after all receivers have confirmed and if all net positive receivers have confirmed then all funds are accounted for. At least one such node must exist in every cyclic payment structure, since the sum of the payments along a cycle will be 0. |
There was a problem hiding this comment.
Confirmation deadlocks in net-negative payment cycle
A net-negative receiver must wait for every receiver it pays, so a payment cycle whose nodes all have negative net balances due to external payments forms a wait cycle. The net-positive receiver outside the cycle confirming does not unblock them, so confirmation never completes.
oak · automated · 626af58
| COMB --> SIGNED | ||
| ``` | ||
|
|
||
| #### Strict Combiner |
There was a problem hiding this comment.
Strict Combiner nested under per-output types
#### Strict Combiner sits under ### New per-output types, so the combiner rules render as a subsection of the per-output types section rather than a top-level specification section.
oak · automated · 626af58






I left both commits so we can see what changed. This ended up being mostly a rewrite of the wording of this BIP to be less jargony and dense. I found the writing to be hard to parse even tho the ideas are quite simple. So no new ideas being presented just a restructing of what was there.
For example, I reconciled all of the background related text into a background subsection.
THere are still many todos and cleanups. But more to rewrite and the flow of ideas can be improved.
i.e:
Any language specific to collaborative tx construction is treated as outside the scope.
Anything that was removed is in scratch section.