Skip to content

Compute (potentially reorging) Settlement Status From Wallet Data - #1692

Closed
DanGould wants to merge 6 commits into
payjoin:masterfrom
DanGould:settlement-status
Closed

DanGould wants to merge 6 commits into
payjoin:masterfrom
DanGould:settlement-status

Conversation

@DanGould

@DanGould DanGould commented Jun 29, 2026 •

Copy link
Copy Markdown
Member

This is following up on #1684 where I figured out Monitor's status can go stale in the case of a reorg. Instead, we can run a classifier that can always classify the status based on mempool+chain state data instead.

This classification, that tracks OutPoint status also makes the #1218 workaround that skips monitoring the status of non-segwit outputs obsolete and can actually monitor non-segwit spending Payjoin cooperative txs.

Sessions still close once a Receiver<Monitor> detects a transaction but the actual SettlementStatus is determined by calling classify on chain state each time.

Co-authored by Claude code

Pull Request Checklist

Please confirm the following before requesting review:

@DanGould
DanGould force-pushed the settlement-status branch from 8832e47 to e7c0dde Compare June 29, 2026 17:03
@coveralls

coveralls commented Jun 29, 2026 •

Copy link
Copy Markdown
Collaborator

Coverage Report for CI Build 29565128648

Coverage increased (+0.3%) to 86.441%

Details

  • Coverage increased (+0.3%) from the base build.
  • Patch coverage: 40 uncovered changes across 2 files (495 of 535 lines covered, 92.52%).
  • 2 coverage regressions across 2 files.

Uncovered Changes

File Changed Covered %
payjoin-cli/src/app/v2/mod.rs 73 37 50.68%
payjoin/src/core/receive/v2/mod.rs 429 425 99.07%
Total (4 files) 535 495 92.52%

Coverage Regressions

2 previously-covered lines in 2 files lost coverage.

File Lines Losing Coverage Coverage
payjoin-cli/src/app/v2/mod.rs 1 57.7%
payjoin/src/core/receive/v2/mod.rs 1 93.48%

Coverage Stats

Coverage Status
Relevant Lines: 16432
Covered Lines: 14204
Line Coverage: 86.44%
Coverage Strength: 335.74 hits per line

💛 - Coveralls

@spacebear21

Copy link
Copy Markdown
Collaborator

IMO such an API change should wait until after 1.0 release and is a candidate for a 2.0 API:

  • Like you said in Rename Receiver<Monitor>::check_for_broadcast to check_for_transaction, etc. #1684: "I doubt that this updated follow-up Receiver change is critically necessary for 1.0. What we've shipped seems good enough for mobile wallets to get status". We already have plenty of demand from mobile wallets and releasing a stable API now seems like a bigger win than pushing that deadline further while we debate the exact monitor shape.
  • We already have a big breaking API rewrite in the works with Non-blocking Interface for Payjoin State Machine  #1446, that would abandon the callback pattern for advancing the state machine in favor of a validator pattern where the typestate machine is advanced explicitly by providing the result of the check explicitly. That seems more inline with the Monitor approach in this PR and those changes can all be bundled into a comprehensive, well thought-out 2.0 follow-up.

In the meantime, I think it's sufficient to add a big disclaimer in the check_for_transaction making it clear that it:

@chavic

chavic commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Field evidence in favor of this direction, from the BTCPay Server payjoin plugin (a settlement-accounting receiver over payjoin-ffi): we independently hit both flaws this PR identifies. Our accounting layer pins the proposal's expected transaction ID at hand-out time for exact reconciliation, which is silently wrong for non-SegWit senders (we currently reject them as policy, ValeraFinebits/btcpayserver-payjoin-plugin#78), and our stored reconciliation verdict shares the stale-on-reorg problem — a re-runnable classify() over contested outpoints would replace both host-side workarounds, and the Cooperative cut-through recognition covers a case txid-pinning can't see at all.

In the meantime, for whatever it's worth to the 1.0 discussion: #1718 proposes the machine-readable version of the disclaimer suggested above — proposal_txid_is_stable() on the fallback-holding states, additive-only and shape-neutral with respect to this PR; it degrades into a convenience if this lands.

@spacebear21 spacebear21 added this to the payjoin-2.0 milestone Jul 9, 2026
@benalleng benalleng added the receive receiving payjoin label Jul 16, 2026
@DanGould DanGould mentioned this pull request Jul 16, 2026
2 tasks done
@DanGould
DanGould force-pushed the settlement-status branch from e7c0dde to 47b49c5 Compare July 16, 2026 16:26
DanGould added 6 commits July 17, 2026 15:38
The persisted SessionEvent format freezes at the payjoin-cli 1.0
database-backwards-compatibility commitment, so reshape the receiver's
terminal SessionOutcome while a format break is still free.

Change Success(Vec<(ScriptBuf, Witness)>) to Success(Txid). The
witness payload was recorded so the receiver could reconstruct the
final Payjoin on the non-segwit path, but payjoin#1218 removed that path
entirely, leaving the enum payload as dead code. The Receiver settlement
classifier (payjoin#1692) will restore non-segwit detection, which would make
the payload live again, but sufficient record of a settlement is the txid
of the transaction that settled it rather than the witness list. The
wallet that observed the spend already holds the full transaction.
Recording Success(Txid) also keeps the terminal variants consistent
where the outcome for the transaction we recognize carries the same
evidence as the one we do not.

Add Other(Txid), mapped to SessionStatus::Failed, for contested
outpoints settled by an unrecognized transaction. Nothing produces it
yet. The reason to reserve it now is API semver, not data
compatibility. 8c92050 deliberately kept SessionOutcome exhaustive.
A wildcard match arm has no meaningful semantics for a terminal outcome,
and the compile error on a new variant is a feature. So adding a
variant after 1.0 would be a semver-breaking change that we want. Naming
Other before the freeze makes the settlement classifier additive by
construction while keeping the enum exhaustive. (Forward
compatibility of the persisted data is explicitly not a goal. See the
persist module docs, we want unknown state to error.)

Update payjoin-cli to match the new shapes exhaustively. no wildcard
arms, per 8c92050 surfacing the settlement txid in the history and
cancel output, and pin the sealed JSON encodings of the terminal
events in a serialization-shape test. StatusText becomes String rather
than &'static str in order to accomodate the txid in format!.
Introduce outpoint-based settlement attribution for a monitored
session. A pure, total classify maps caller-supplied chain facts to a
settlement status without any I/O or persisted events.

Add the compute types SettlementStatus and SettlementOutcome
{Cooperative(Txid), Fallback, Other(Txid)} plus ChainView/OutpointSpend,
and the primitive getters the wallet needs (payjoin_txid,
fallback_txid, contested_outpoints, receiver_input).

classify recognizes a cooperative settlement by either signal: the
spending transaction carries the receiver's contributed input (the
input fingerprint) or, for an output-substitution proposal, reproduces
the proposal's output set (a cut-through). Both are structural, so a
non-SegWit txid change does not hide them; Cooperative therefore
carries the observed settling txid, which in that case differs from
the predicted payjoin_txid and is otherwise unavailable to the caller
of a txid-oblivious classifier. Precedence is
Cooperative > Fallback > Other.

SettlementOutcome is #[non_exhaustive]. This is a deliberate exception
to 8c92050's keep-exhaustive list, judged by that commit's own test:
does a wildcard arm have meaningful semantics? For a *persisted*
terminal outcome it does not (SessionOutcome stays exhaustive). But
SettlementOutcome is a never-persisted classification verdict, and a
finer future label (say, distinguishing a sender re-spend from a
third-party spend) degrades soundly in a wildcard arm to "settled by
something this build cannot label -- surface for review". No session
state is resumed through it, so the failure mode 8c92050 guards
against does not arise. SettlementStatus itself stays exhaustive:
Pending/Detected is a closed shape.
Add Receiver<Monitor>::conclude, which persists the single terminal
Closed(SessionOutcome) event once the caller's confidence bar is met,
mapping Cooperative(txid) -> Success(txid), Fallback ->
FallbackBroadcasted, and Other(txid) -> Other(txid). The persisted
SessionOutcome shape (Success(Txid), the reserved Other(Txid)) was
sealed with the 1.0 format freeze; this wires the settlement
classifier to it and makes the classifier produce the reserved Other
variant, so drop the reservation note from its docs. The Success txid
is the observed settling transaction's, which for a non-SegWit sender
differs from the predicted payjoin txid the receiver could compute on
its own.

Re-express the legacy check_for_transaction over classify via a
txid-only ChainView: each found transaction is classified structurally
before concluding, instead of trusting the txid probe alone. The
consume-self signature and transient-error flow are unchanged.
Add the foreign-facing settlement surface: a ChainBackend trait that
reports each contested outpoint's spend status, the ChainSpend record,
and SettlementStatus/SettlementOutcome uniffi mirrors. Monitor::classify
queries the backend, builds a ChainView, and returns the status without
persisting anything. Cooperative mirrors the core variant's payload,
carrying the observed settling txid as a string.

SettlementOutcome is #[non_exhaustive] in core, so the conversion keeps
a wildcard arm that degrades a finer label this binding predates to
Other until the binding is regenerated.
Drive the monitor step from classify: build a ChainView by querying the
wallet for each contested outpoint (get_raw_transaction_verbose for the
spending tx and its confirmations), then render the status each tick as
Pending / cooperative / fallback / other -- surfacing the settling txid
for cooperative and other spends -- concluding once a configurable
confirmation bar is met. Status lines go through the session-scoped
printer so concurrently resumed sessions stay attributable.

SettlementOutcome is #[non_exhaustive], so the render match keeps a
wildcard arm that shows a generic settled line for any finer label
this build predates.
Spell out the Monitor contract so integrators do not trust an imprecise
0-conf verdict. The proposal and the fallback double-spend the sender's
inputs, so detection is necessary but not sufficient: only confirmation
resolves the race, and confirmation depth is a confidence dial, not a
state transition.

Document the division of labor (library knows the semantics, wallet
knows the ground truth and holds the spending transaction), the
classify/conclude flow, and that the legacy check_for_transaction
cannot surface Other and concludes optimistically on first sight.
@DanGould

Copy link
Copy Markdown
Member Author

Changed to be additive to target 1.1 on top of #1747

spacebear21 added a commit to DanGould/rust-payjoin that referenced this pull request Jul 20, 2026
The persisted SessionEvent format freezes at the payjoin-cli 1.0
database-backwards-compatibility commitment, so reshape the receiver's
terminal SessionOutcome while a format break is still free.

- Change `Success(Vec<(ScriptBuf, Witness)>)` to `Success(Txid)`. The
witness payload was recorded so the receiver could reconstruct the final
Payjoin on the non-segwit path, but payjoin#1218 removed that path entirely,
leaving the enum payload as dead code. The Receiver settlement
classifier (payjoin#1692) will restore non-segwit detection, which would make
the payload live again, but sufficient record of a settlement is the
txid of the transaction that settled it rather than the witness list.
The wallet that observed the spend already holds the full transaction.
Recording `Success(Txid)` also keeps the terminal variants consistent
where the outcome for the transaction we recognize carries the same
evidence as the one we do not.
- Add `Unrecognized(Txid)` for contested outpoints settled by an
unrecognized transaction. Nothing produces it yet but it is reserved now
because adding a variant after 1.0 would be a semver-breaking change.
Naming `Unrecognized` before the freeze makes the settlement classifier
additive in a 1.x follow-up while keeping the enum exhaustive.

Co-authored-by: DanGould <d@ngould.dev>
DanGould added a commit to DanGould/rust-payjoin that referenced this pull request Jul 21, 2026
The persisted SessionEvent format freezes at the payjoin-cli 1.0
database-backwards-compatibility commitment, so reshape the receiver's
terminal SessionOutcome while a format break is still free.

- Change `Success(Vec<(ScriptBuf, Witness)>)` to `Success(Txid)`. The
witness payload was recorded so the receiver could reconstruct the final
Payjoin on the non-segwit path, but payjoin#1218 removed that path entirely,
leaving the enum payload as dead code. The Receiver settlement
classifier (payjoin#1692) will restore non-segwit detection, which would make
the payload live again, but sufficient record of a settlement is the
txid of the transaction that settled it rather than the witness list.
The wallet that observed the spend already holds the full transaction.
Recording `Success(Txid)` also keeps the terminal variants consistent
where the outcome for the transaction we recognize carries the same
evidence as the one we do not.
- Add `Unrecognized(Txid)` for contested outpoints settled by an
unrecognized transaction. Nothing produces it yet but it is reserved now
because adding a variant after 1.0 would be a semver-breaking change.
Naming `Unrecognized` before the freeze makes the settlement classifier
additive in a 1.x follow-up while keeping the enum exhaustive.

Co-authored-by: DanGould <d@ngould.dev>
@DanGould

Copy link
Copy Markdown
Member Author

@xstoicunicornx I'd like to hand this off to you. This is the right shape but the FFI needs to change. Monitor::classify(Arc<dyn ChainBackend>) is a with_foreign trait that the library calls back into synchronously per outpoint.

This should follow #1446's shape, taking a uniffi record of outpoint spends instead and export the same accessors and conclude which are missing on the FFI side.

Bull Bitcoin's exchange hit the fallback monitoring problem which this fixes (SatoshiPortal/payjoin#6) and their monitor code seems not written yet. This saves them writing it on their own and rewriting it with PDK's API later.

xstoicunicornx added a commit to xstoicunicornx/rust-payjoin that referenced this pull request Sep 26, 2026
Introduce an implementation-agnostic interface for receiver
typestates that currently require callback-based validation to
advance. Previously, each validation step demanded a synchronous
closure, coupling the state machine to the caller's execution
model. This made integration difficult for wallets where signing,
broadcast checks, or ownership lookups are asynchronous or
handled by a separate process.

Each callback-based transition is now split into a two-phase
pattern: a method to extract the data that needs checking
(*_checklist, extract_tx_*, psbt_to_sign) and a corresponding
method to submit results and advance the state (apply_*_checklist,
apply_broadcast_suitability, finalize_signed_proposal). The
submitted ChecklistItem/MarkedChecklistItem list is validated
against the inputs or outputs of the receiver's own original PSBT,
so a list marked for a different proposal is rejected, and a PSBT
with no inputs or no outputs is handled as an empty list.

This applies across v1 and v2 receiver flows, including input
ownership, input-seen, output ownership, broadcast suitability and
proposal finalization. Transaction monitoring is left to the
settlement status work in payjoin#1692.

The original closure-based methods are preserved as convenience
wrappers over the new API, so this is backward-compatible for
existing integrators.

Co-authored-by: spacebear21 <git@spacebear.dev>
Co-authored-by: DanGould <d@ngould.dev>
@xstoicunicornx

Copy link
Copy Markdown
Collaborator

After reviewing this PR I had a couple questions on design and had pretty significant change I'd like to propose to the PR's current API surface. Looking for your opinion @DanGould .

Questions:

  • Should there be path for SessionOutcome::Aborted? (ie if caller wants to bail out of monitoring)
  • What is the intended purpose for proposal_outputs and proposal_substitutes_outputs?

I would like to remove ChainView from the API surface. Since the implementer will ultimately have to determine what their operational criteria is for calling conclude anyway there doesn't really seem to be much of a benefit of inserting the Payjoin API into that process. classify granularity would then get reduced down to transaction level rather than ChainView level. When I asked Opus 5.5 and Sol 5.6 their opinion they both recommended removing it. If you feel strongly about keeping it I will have bunch of follow up questions to dive into.

@DanGould

DanGould commented Oct 7, 2026

Copy link
Copy Markdown
Member Author
  • Should there be path for SessionOutcome::Aborted? (ie if caller wants to bail out of monitoring)

I don't believe we want to make it possible to represent a receiver who is monitoring as having bailed out. The only way to enter Monitor is by having processed a response that actually sends the Payjoin Proposal, in which case either the payjoin succeeds, fallback is detected, or the input is double spent (which is Other). Is there any other state? After handoff, an implementation may "abort" but that is not state we have any business representing. Before monitor, cancel() & close() can be used to record Aborted.

Thinking about it more, I don't want the library to persist a settlement outcome at all, which means conclude gets deleted too even though in September I said it would stay. The session log is append-only and Closed is terminal, so whatever we write there is permanent, and after a reorg it is wrong. That is the #1684 problem moved from Monitor into the log. I considered an extract/apply pair in the #1446 style, where the wallet hands in the spending transaction and the library classifies and closes, but the only thing the apply would do is persist that same verdict that a reorg could always change.

Posting the proposal is the receiver's last protocol action and PostedPayjoinProposal already marks the handoff. I'd close the session there, as PayjoinProposalSent, with that variant re-documented as "proposal sent, settlement is the wallet's to track" (the non-SegWit wording stops being true once classify handles non-SegWit). In 1.x that is a close() on Monitor that the wallet calls right after posting, with check_for_transaction marked deprecated beside it, since removing Monitor is a break. In 2.0 process_response can close the session directly. Settlement is then a question the wallet asks whenever it sees a transaction spend a contested outpoint, and it caches the answer the way it caches any transaction label in its UI. That means classify has to be reachable from SessionHistory, since the session is closed by the time anyone asks. A classifier type over the persisted PsbtContext that SessionHistory hands out would do it, rather than rebuilding a receiver from a closed log.

Closing at posting also settles an expiry problem that exists today. The session expiry is the mailbox expiry, and replay refuses any session that isn't closed once it has passed. A Monitor left open past that point can't be replayed, so a payjoin that confirms after the mailbox expired could never be recorded. That is true of check_for_transaction now too. Closed sessions are exempt from the expiry check, so once posting closes the session the wallet can classify a spend whenever it shows up. If you keep Monitor open for any reason, the PR needs to exempt it from expiry on purpose.

  • What is the intended purpose for proposal_outputs and proposal_substitutes_outputs?

Private, as we discussed. They exist for cut-through detection. It's possible for the receiver to create a proposal where they contribute no input so the classify function relies on the spending transaction's output set vs the proposal instead. The second one gates that check. If the proposal's outputs equal the original's, a matching output set could just be the fallback.

Regarding ChainView

Agree to drop it. It's vestigial from the June design of this PR to fix #1214 by probing outpoints, getting back the spending transaction, and then recording the probe result. That also replaces the uniffi record of outpoint spends I asked for in September, since there is nothing left to put in it. If the design changes from here, please edit the PR body to match.

The only thing payjoin actually knows that a wallet does not is which transaction represents fallback / payjoin / doublespend, so classify(&Transaction) -> Option<SettlementOutcome> is the API that makes sense (FFI API being transaction bytes in, enum out). A transaction spending any contested outpoint is enough to classify.

p.s. aborted sessions and the cli

While reading the exits I noticed the cli shows an Aborted session as the end of the story, even when it broadcast the fallback and the payment landed. I started to want a second exit that records FallbackBroadcasted, but that is conclude by another name. Aborted is right as the handoff, the protocol stopped and whatever fallback exists is the wallet's. Two things follow for the PR. The classifier from SessionHistory should work for any session that reached the fallback, not only posted ones, so the cli can classify aborted sessions the same way it classifies posted ones. And the doc on Aborted should say the protocol stopped, not that the payjoin "was not successful", which reads as a payment verdict. FallbackBroadcasted, Success and Other are then only ever written by the deprecated check_for_transaction. Once that is removed nothing writes them, and whether they stay in SessionOutcome for reading 1.x logs is a 2.0 question.

@xstoicunicornx

Copy link
Copy Markdown
Collaborator

This mostly sounds good to me. I was working on implementing this but thought of an issue with getting rid of Monitor typestate entirely. Once Monitor no longer exists it will be impossible for callers to discern which closed/completed sessions still need to be reconstructed to use the classifier. I am going to update the implementation to keep the Monitor typestate, keep the classify() method on this typestate, and just add a close() method which mimics the PendingFallback flow. close() will not attest to any specific settlement outcome, it only documents that the wallet has finished whatever monitoring it needs to do.

Just wanted to let you know about this change in direction. Let me know if you have objections or can find flaws in my reasoning.

@DanGould

Copy link
Copy Markdown
Member Author

Superseded by #1971

Feel free to close when you supersede next time @xstoicunicornx

@DanGould DanGould closed this Oct 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

receive receiving payjoin

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants