Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- CI: format, clippy, and tests on Linux, Windows, and macOS.
- Tooling: `tools/changelog.py` generates reviewable release sections from
Conventional Commits; a dry-run workflow runs on demand and on release.
- Benchmark: `cargo run --release -p devkit-xdr --example mem_profile` reports
peak and current memory per decode input size (see `docs/MEMORY.md`).
- Fuzzing: `fuzz/` cargo-fuzz targets for XDR/SCVal decoding, exercised by a
weekly bounded scheduled workflow (see `docs/TESTING.md`).
- Governance: README, CONTRIBUTING, SECURITY, CODE_OF_CONDUCT, LICENSE,
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ Exit codes: `0` success, `2` usage error, `3` runtime/decode error.
- [Architecture](docs/ARCHITECTURE.md)
- [Getting started](docs/GETTING_STARTED.md)
- [CLI reference](docs/CLI.md)
- [Memory measurement](docs/MEMORY.md)
- [Testing guide](docs/TESTING.md)
- [Project status](docs/PROJECT_STATUS.md)
- [Roadmap](docs/ROADMAP.md)
Expand Down
168 changes: 168 additions & 0 deletions crates/devkit-xdr/examples/mem_profile.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
//! Measure peak memory while decoding large base64 XDR `ScVal` inputs.
//!
//! Run with:
//!
//! ```text
//! cargo run --release -p devkit-xdr --example mem_profile
//! ```
//!
//! Optionally pass comma-separated input sizes in bytes:
//!
//! ```text
//! cargo run --release -p devkit-xdr --example mem_profile -- 1024,65536,1048576
//! ```
//!
//! The example reports memory per input size as CSV. Peak/current memory is
//! read from platform APIs (Linux `/proc/self/status`, Windows
//! `GetProcessMemoryInfo`) with a documented fallback to `n/a` on other
//! platforms. See `docs/MEMORY.md` for methodology and limits.

use devkit_xdr::inspect_scval_base64;
use stellar_xdr::{BytesM, Limits, ScBytes, ScVal, WriteXdr};

const DEFAULT_SIZES: &[usize] = &[1024, 64 * 1024, 1024 * 1024];

#[derive(Clone, Copy)]
struct Memory {
current: Option<u64>,
peak: Option<u64>,
}

fn main() {
println!(
"size_bytes,base64_bytes,decoded_json_bytes,current_bytes,peak_bytes,delta_current_bytes"
);
let mut measured = false;
for size in sizes_from_args() {
let bytes = BytesM::try_from(vec![0xAB; size]).expect("sample within MAX");
let input = ScVal::Bytes(ScBytes::from(bytes))
.to_xdr_base64(Limits::none())
.expect("encode sample scval");
let before = platform::sample();
let json = inspect_scval_base64(&input).expect("decode sample scval");
let after = platform::sample();
let decoded_bytes = serde_json::to_string(&json).map(|s| s.len()).unwrap_or(0);
measured |= after.peak.is_some();
println!(
"{size},{},{decoded_bytes},{},{},{}",
input.len(),
opt(after.current),
opt(after.peak),
delta(after.current, before.current),
);
}
if !measured {
eprintln!("note: memory sampling is unsupported on this platform; see docs/MEMORY.md");
}
}

fn sizes_from_args() -> Vec<usize> {
let Some(spec) = std::env::args().nth(1) else {
return DEFAULT_SIZES.to_vec();
};
let sizes: Vec<usize> = spec
.split(',')
.filter_map(|part| part.trim().parse().ok())
.collect();
if sizes.is_empty() {
DEFAULT_SIZES.to_vec()
} else {
sizes
}
}

fn opt(value: Option<u64>) -> String {
value
.map(|v| v.to_string())
.unwrap_or_else(|| "n/a".to_string())
}

fn delta(after: Option<u64>, before: Option<u64>) -> String {
match (after, before) {
(Some(a), Some(b)) => a.saturating_sub(b).to_string(),
_ => "n/a".to_string(),
}
}

#[cfg(target_os = "linux")]
mod platform {
use super::Memory;

fn field(status: &str, key: &str) -> Option<u64> {
status
.lines()
.find(|line| line.starts_with(key))
.and_then(|line| line.split_whitespace().nth(1))
.and_then(|value| value.parse::<u64>().ok())
.map(|kib| kib * 1024)
}

pub fn sample() -> Memory {
let status = std::fs::read_to_string("/proc/self/status").unwrap_or_default();
Memory {
current: field(&status, "VmRSS:"),
peak: field(&status, "VmHWM:"),
}
}
}

#[cfg(windows)]
mod platform {
use super::Memory;
use std::ffi::c_void;
use std::mem;

#[repr(C)]
struct ProcessMemoryCounters {
cb: u32,
page_fault_count: u32,
peak_working_set_size: usize,
working_set_size: usize,
quota_peak_paged_pool_usage: usize,
quota_paged_pool_usage: usize,
quota_peak_non_paged_pool_usage: usize,
quota_non_paged_pool_usage: usize,
pagefile_usage: usize,
peak_pagefile_usage: usize,
}

#[link(name = "psapi")]
extern "system" {
fn GetCurrentProcess() -> *mut c_void;
fn GetProcessMemoryInfo(
process: *mut c_void,
counters: *mut ProcessMemoryCounters,
cb: u32,
) -> i32;
}

pub fn sample() -> Memory {
unsafe {
let mut counters: ProcessMemoryCounters = mem::zeroed();
counters.cb = mem::size_of::<ProcessMemoryCounters>() as u32;
if GetProcessMemoryInfo(GetCurrentProcess(), &mut counters, counters.cb) != 0 {
Memory {
current: Some(counters.working_set_size as u64),
peak: Some(counters.peak_working_set_size as u64),
}
} else {
Memory {
current: None,
peak: None,
}
}
}
}
}

#[cfg(not(any(target_os = "linux", windows)))]
mod platform {
use super::Memory;

pub fn sample() -> Memory {
Memory {
current: None,
peak: None,
}
}
}
68 changes: 68 additions & 0 deletions docs/MEMORY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# Memory measurement

Decode benchmarks measure time, not memory. CI containers are memory
constrained, so peak memory usage on large inputs is measured explicitly and
reported per input size.

## Running the measurement

```bash
cargo run --release -p devkit-xdr --example mem_profile
```

Pass comma-separated input sizes in bytes to override the defaults
(`1024,65536,1048576`):

```bash
cargo run --release -p devkit-xdr --example mem_profile -- 1024,262144,1048576
```

The example prints CSV:

```text
size_bytes,base64_bytes,decoded_json_bytes,current_bytes,peak_bytes,delta_current_bytes
1024,1376,2060,5595136,5595136,290816
262144,349536,524300,6590464,6848512,0
```

| Column | Meaning |
| ------ | ------- |
| `size_bytes` | Size of the `ScVal::Bytes` payload used to build the input. |
| `base64_bytes` | Length of the base64 XDR input that is decoded. |
| `decoded_json_bytes` | Length of the decoded JSON, proving the decode ran. |
| `current_bytes` | Resident memory after decoding (or `n/a`). |
| `peak_bytes` | Peak resident memory observed by the process (or `n/a`). |
| `delta_current_bytes` | Current-memory growth across the decode (or `n/a`). |

## Methodology

1. Build a `ScVal::Bytes` payload of the requested size and encode it to base64
XDR.
2. Sample memory before the call.
3. Decode with `devkit_xdr::inspect_scval_base64`.
4. Sample memory after the call and serialise the result to JSON.

Each size is measured in the same process. Peak memory is monotonic, so later
rows report the running maximum; use the current-memory column, or a fresh
process, for isolated per-input figures.

## Platform APIs and fallback

| Platform | Current | Peak | Source |
| -------- | ------- | ---- | ------ |
| Linux | `VmRSS` | `VmHWM` | `/proc/self/status` |
| Windows | `WorkingSetSize` | `PeakWorkingSetSize` | `GetProcessMemoryInfo` |
| Other | `n/a` | `n/a` | documented fallback |

On unsupported platforms the example still builds and runs and prints a note to
stderr; memory columns are reported as `n/a`.

## Limits

- Figures are process-wide: they include the allocator, the toolchain runtime,
and the decoded JSON value, not just XDR buffers.
- Decoding to JSON expands bytes substantially (each byte becomes a decimal
array element), so JSON size dominates the large-input measurements.
- Numbers are representative, not exact: run in `--release` and compare runs on
the same platform to detect regressions.
- Measurement is not cross-platform normalised; compare Linux to Linux.
Loading