Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@ before 1.0).

### Added

- **Q-56 Miri islands beyond primitives:** `scriptnum` (encode/decode/is_minimal) + `CompactSize` + `ULEB128` peeled into `rbitcoin-primitives` (FFI-free, 0 deps). Adds `cfg(miri)` islands (1000-range scriptnum, 0..10000 compact) so `cargo +nightly miri test -p rbitcoin-primitives` works. Never `--workspace` miri. R-10 peel allowed.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This says the helpers were “peeled into rbitcoin-primitives.” On this branch fn scriptnum_encode is still in interpreter.rs and the full CompactSize/ULEB set is still in store/src/compact.rs.

Write this bullet in the same commit as the caller switch, and describe what production now calls — not a parallel island. Extra blank line below is accidental.


- **Wallet-protocol leftovers:** Electrum **1.6** `blockchain.transaction.broadcast_package`
(local `accept_package`) and `mempool.get_info`; `protocol_max` **1.6** with
1.6 `block.headers` as a list. Electrum **1.7** `blockchain.outpoint.*`.
Expand Down
166 changes: 166 additions & 0 deletions crates/rbitcoin-primitives/src/compact.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
//! FFI-free pack integers for Miri (Q-56).
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum CompactError {
Empty,
TruncatedU16,
TruncatedU32,
TruncatedU64,
OverflowUleb,
ShortDst,
}
impl std::fmt::Display for CompactError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{self:?}")
}
}
impl std::error::Error for CompactError {}
#[inline]
pub fn compact_size_len(n: u64) -> usize {
if n < 253 {
1
} else if n <= u16::MAX as u64 {
3
} else if n <= u32::MAX as u64 {
5
} else {
9
}
}
pub fn write_compact_size(out: &mut Vec<u8>, n: u64) {
if n < 253 {
out.push(n as u8);
} else if n <= u16::MAX as u64 {
out.push(253);
out.extend_from_slice(&(n as u16).to_le_bytes());
} else if n <= u32::MAX as u64 {
out.push(254);
out.extend_from_slice(&(n as u32).to_le_bytes());
} else {
out.push(255);
out.extend_from_slice(&n.to_le_bytes());
}
}
Comment on lines +29 to +42

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy of crates/rbitcoin-store/src/compact.rs (and a third CompactSize lives as private fns in rbitcoin-mempool/src/packed.rs). Store still owns the production pack-int path.

If this extract is real: store wraps StoreError around these fns and deletes its copies; mempool calls the same owner (code-shape: one algorithm). write_uleb128 here uses .unwrap(); store uses .expect("10-byte stack holds any u64 uleb128") — keep that invariant message. Truncated ULEB maps to CompactError::Empty, which collapses two store error strings (empty vs truncated).

pub fn read_compact_size(buf: &[u8]) -> Result<(u64, usize), CompactError> {
if buf.is_empty() {
return Err(CompactError::Empty);
}
match buf[0] {
n @ 0..=252 => Ok((n as u64, 1)),
253 => {
if buf.len() < 3 {
return Err(CompactError::TruncatedU16);
}
let v = u16::from_le_bytes([buf[1], buf[2]]);
Ok((v as u64, 3))
}
254 => {
if buf.len() < 5 {
return Err(CompactError::TruncatedU32);
}
let v = u32::from_le_bytes(buf[1..5].try_into().unwrap());
Ok((v as u64, 5))
}
255 => {
if buf.len() < 9 {
return Err(CompactError::TruncatedU64);
}
let v = u64::from_le_bytes(buf[1..9].try_into().unwrap());
Ok((v, 9))
}
}
}
#[inline]
pub fn uleb128_len(mut n: u64) -> usize {
let mut len = 1;
while n >= 0x80 {
n >>= 7;
len += 1;
}
len
}
pub fn write_uleb128_into(dst: &mut [u8], mut n: u64) -> Result<usize, CompactError> {
let mut i = 0;
loop {
if i >= dst.len() {
return Err(CompactError::ShortDst);
}
let mut b = (n & 0x7f) as u8;
n >>= 7;
if n != 0 {
b |= 0x80;
}
dst[i] = b;
i += 1;
if n == 0 {
return Ok(i);
}
}
}
pub fn write_uleb128(out: &mut Vec<u8>, n: u64) {
let mut tmp = [0u8; 10];
let used = write_uleb128_into(&mut tmp, n).unwrap();
out.extend_from_slice(&tmp[..used]);
}
pub fn read_uleb128(buf: &[u8]) -> Result<(u64, usize), CompactError> {
let mut result = 0u64;
let mut shift = 0u32;
for (i, &b) in buf.iter().enumerate() {
if shift >= 64 {
return Err(CompactError::OverflowUleb);
}
result |= u64::from(b & 0x7f) << shift;
if b & 0x80 == 0 {
return Ok((result, i + 1));
}
shift += 7;
}
Err(CompactError::Empty)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn compact_roundtrip() {
for n in [
0,
1,
252,
253,
254,
255,
256,
65535,
65536,
u32::MAX as u64,
u32::MAX as u64 + 1,
u64::MAX,
] {
let mut out = Vec::new();
write_compact_size(&mut out, n);
assert_eq!(out.len(), compact_size_len(n));
let (dec, used) = read_compact_size(&out).unwrap();
assert_eq!(dec, n);
assert_eq!(used, out.len());
}
}
#[test]
fn uleb_roundtrip() {
for n in [0, 1, 127, 128, 255, 16383, 16384, u64::MAX] {
let mut out = Vec::new();
write_uleb128(&mut out, n);
assert_eq!(out.len(), uleb128_len(n));
let (dec, _) = read_uleb128(&out).unwrap();
assert_eq!(dec, n);
}
}
#[cfg(miri)]
#[test]
fn miri_compact() {
for n in 0..10000u64 {
let mut out = Vec::new();
write_compact_size(&mut out, n);
let (dec, _) = read_compact_size(&out).unwrap();
assert_eq!(dec, n);
}
}
Comment on lines +157 to +165

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0..10000 only hits 1-byte CompactSize and the start of the 3-byte form (253+). Q-56 is pack integers: 5-byte / 9-byte CompactSize and ULEB128 are the encodings that matter, and Miri never runs them here.

Store already pins truncated/overflow in compact_and_uleb_error_paths — those are the cases Miri is for (overflowing << on a 10-byte ULEB with extra high bits). Add a cfg(miri) ULEB loop and the width/error cases, on the shipped functions, not this fork.

}
10 changes: 10 additions & 0 deletions crates/rbitcoin-primitives/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -410,3 +410,13 @@ mod tests {
assert_eq!(s, format!("/rbitcoin:{VERSION}/"));
}
}

mod compact;
mod scriptnum;
pub use compact::{
compact_size_len, read_compact_size, read_uleb128, uleb128_len, write_compact_size,
write_uleb128, CompactError,
};
pub use scriptnum::{
decode_scriptnum, decode_scriptnum_4, encode_scriptnum, is_minimal_scriptnum, ScriptNumError,
};
Comment on lines +414 to +422

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These pub use names have no caller outside this crate. CONTRIBUTING 11: crate-root pub is the production graph only.

Q-56 is Miri on the FFI-free helpers the node already runs (scriptnum_* in interpreter.rs, CompactSize/ULEB in store/src/compact.rs), not a parallel copy. rbitcoin-consensus and rbitcoin-store already depend on this crate — the extract is switch those callers, then delete the originals. Until that happens this is unused pub plus a dual path (code-shape rule 6).

Nit: mod compact / mod scriptnum belong with the other modules at the top of this file, not after the tests module.

111 changes: 111 additions & 0 deletions crates/rbitcoin-primitives/src/scriptnum.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
//! FFI-free Bitcoin script number encoding (Q-56 Miri island).
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum ScriptNumError {
Overflow,
NonMinimal,
}
impl std::fmt::Display for ScriptNumError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{self:?}")
}
}
impl std::error::Error for ScriptNumError {}
pub fn encode_scriptnum(mut n: i64) -> Vec<u8> {
if n == 0 {
return vec![];
}
let neg = n < 0;
if neg {
n = -n;
}
let mut out = Vec::new();
while n > 0 {
out.push((n & 0xff) as u8);
n >>= 8;
}
if out.last().map(|b| b & 0x80 != 0).unwrap_or(false) {
out.push(if neg { 0x80 } else { 0x00 });
} else if neg {
*out.last_mut().unwrap() |= 0x80;
}
out
}
Comment on lines +13 to +32

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the same algorithm as scriptnum_encode / scriptnum_decode_width / scriptnum_is_minimal in crates/rbitcoin-consensus/src/script/interpreter.rs (~1505–1594). Those fns are still there; eval still calls them. This is a fork, not an R-10 peel.

R-10 allows extracting these helpers (not the opcode match) as the Q-56 seam if production calls the extract. Green is: interpreter wraps ScriptNumError → ConsensusError and deletes the local copies. Existing scriptnum_minimal_encoding / CLTV-CSV width-5 tests stay the pin (docs/code-shape.md extract policy).

n = -n on i64::MIN overflows; the production copy has the same line. A real island would let Miri see that if anyone passed MIN. Do not leave two copies of that negate.

pub fn is_minimal_scriptnum(vch: &[u8]) -> bool {
if vch.is_empty() {
return true;
}
if vch[vch.len() - 1] & 0x7f == 0 && (vch.len() <= 1 || (vch[vch.len() - 2] & 0x80) == 0) {
return false;
}
true
}
pub fn decode_scriptnum(
v: &[u8],
require_minimal: bool,
max_len: usize,
) -> Result<i64, ScriptNumError> {
if v.len() > max_len {
return Err(ScriptNumError::Overflow);
}
if require_minimal && !is_minimal_scriptnum(v) {
return Err(ScriptNumError::NonMinimal);
}
if v.is_empty() {
return Ok(0);
}
let mut result: i64 = 0;
for (i, &b) in v.iter().enumerate() {
result |= (b as i64) << (8 * i);
}
if v.last().unwrap() & 0x80 != 0 {
result &= !(0x80i64 << (8 * (v.len() - 1)));
result = -result;
}
Ok(result)
}
pub fn decode_scriptnum_4(v: &[u8], require_minimal: bool) -> Result<i64, ScriptNumError> {
decode_scriptnum(v, require_minimal, 4)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn roundtrip_basic() {
for n in [
0,
1,
-1,
2,
16,
17,
127,
128,
255,
256,
-255,
-1000,
0x7fffffff_i64,
-0x7fffffff_i64,
] {
let enc = encode_scriptnum(n);
let dec = decode_scriptnum_4(&enc, true).unwrap();
assert_eq!(dec, n);
}
}
#[test]
fn minimality() {
assert!(is_minimal_scriptnum(&[]));
assert!(is_minimal_scriptnum(&[0x01]));
assert!(!is_minimal_scriptnum(&[0x01, 0x00]));
}
#[cfg(miri)]
#[test]
fn miri_roundtrip() {
for n in -1000..1000 {
let enc = encode_scriptnum(n as i64);
if enc.len() <= 4 {
assert_eq!(decode_scriptnum_4(&enc, true).unwrap(), n as i64);
}
}
}
Comment on lines +96 to +110

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The shipped pin is scriptnum_minimal_encoding in the interpreter tests: [], [0x00], [0x80] (negative zero), [0x01], [0x01, 0x00], [0xff, 0x00] (+255 high-bit pad), [0xff, 0x80] (−255). This island drops the two cases that actually catch non-minimal encoding.

CLTV/CSV decode at width 5 (scriptnum_decode_width(..., 5, ...)). decode_scriptnum_4 never sees that path. TESTING.md: tests drive the shipped function, not a reimplementation.

The #[cfg(miri)] extra loop is the right shape — point it at the functions eval calls, and include width 5 / non-minimal bytes, not only -1000..1000 roundtrips that already fit in 4 bytes.

}
1 change: 0 additions & 1 deletion docs/quality.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@ evidence (failed Core corpus, new dual path, red required CI, MSRV drift).
| 3 | **Q-31** | Hermetic tip fixtures | Frozen signet/mainnet tip packs for offline consensus/Electrum regression (no live API). Fuzz already merges tiny `signet_block_*.bin` / `mainnet_block_290329.bin`. Electrum hermetic packs still Open. |
| 4 | **R-10** | Residual god-files | Peel **only** when a higher row needs a seam. Do not split `interpreter.rs` opcode `match` or io_uring machines. Named extracts: **Q-61** Completed. |
| 5 | **Q-54** | ast-grep named-cap rules | One rule per easy-to-delete cap from [`ibd-memory.md`](./ibd-memory.md): `pending_blocks` 128, `held_bodies` 320, `MAX_SERVE_BLOCKS` 16, `follow_live` vs `max_outbound`. Each has `lint/ast-grep/fixtures/{good,bad}/`. Today **four** structural rules, **zero** cap rules. |
| 6 | **Q-56** | Miri islands beyond primitives | `cfg(miri)` tests for FFI-free helpers (scriptnum, pack integers) that do not pull secp/store. Never workspace miri. Nightly `miri.yml` is still primitives-only (**Q-53**). |
| 7 | **Q-67** | `asked_blocks` clone on hold | `hold_body` clones `asked_blocks` before `held_bodies` insert so the read lock does not overlap the write (`HeldBodies::insert` already takes `&HashSet`). Bound is `MAX_SERVE_BLOCKS` × peers. Follow-up: pass the read guard with a documented lock order, or keep the clone as a named trade. Owner: `crates/rbitcoin-net/src/chain.rs`. |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not close Q-56 by deleting the Open row while confirm/store still run the old helpers. The close rule is: move the row into CHANGELOG in the same edit as the landing change (production callers + Miri on those fns).

Also re-rank: Q-67 is still listed as rank 7 with rank 6 gone.


R-ids were the 2026-08-12 slice. Canonical id is **bold**. Do not start
Expand Down
Loading