Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 21 additions & 2 deletions .github/workflows/fuzz-smoke.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,21 @@ jobs:
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6

- name: Validate corpus layout
run: |
cd rust/fuzz
# Every [[bin]] target must have committed seeds in corpus/<target>/ —
# a target without them fuzzes cold from random bytes and the run is
# far shallower than its green badge suggests (LAB-1149). Pure bash,
# fails loudly on a missing/empty seed directory or a blown 10MB
# budget. Without this step the seed requirement is manual-only.
#
# Runs before the toolchain install deliberately: it needs only bash +
# coreutils and reads files present straight from checkout, so putting
# it here fails a missing seed directory in seconds instead of after
# ~12 min of rustup + cargo-fuzz install.
bash scripts/validate_corpus.sh

- name: Install Rust nightly
run: |
# Pin nightly: cargo-fuzz's rustix dependency uses
Expand Down Expand Up @@ -89,8 +104,12 @@ jobs:
# files in fuzz_targets/. So a new fuzz_targets/*.rs added without its
# stanza is never built and never run, and this job still goes green —
# the same silent-dark mode the old hardcoded array caused. Nothing in
# cargo enforces that the two agree, so assert it here.
SRC_COUNT=$(find fuzz_targets -maxdepth 1 -name '*.rs' | wc -l)
# cargo enforces that the two agree, so assert it here. Count only
# fuzz_target! sources: validate_corpus.sh deliberately tolerates a
# shared helper module in fuzz_targets/, and an unfiltered count would
# fail this job on the same tree that just passed validation — 12
# minutes later, with a misleading "add the missing stanza" error.
SRC_COUNT=$(grep -l 'fuzz_target!' fuzz_targets/*.rs | wc -l)
LIST_COUNT=$(printf '%s\n' "$TARGETS" | wc -l)
if [ "$SRC_COUNT" -ne "$LIST_COUNT" ]; then
echo "::error::fuzz_targets/ holds $SRC_COUNT sources but Cargo.toml declares $LIST_COUNT [[bin]] targets — add the missing [[bin]] stanza so the new target actually runs"
Expand Down
5 changes: 5 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -77,8 +77,13 @@ repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: 3e8a8703264a2f4a69428a0aa4dcb512790b2c8c # v6.0.0 # pragma: allowlist secret
hooks:
# Fuzz corpus seeds are opaque byte strings, not text — "fixing" their
# whitespace or final newline silently changes the seed (and breaks
# byte-exact StorageEnvelope seeds whose checksum must verify).
- id: trailing-whitespace
exclude: ^rust/fuzz/corpus/
- id: end-of-file-fixer
exclude: ^rust/fuzz/corpus/
- id: check-yaml
- id: check-added-large-files
args: [--maxkb=1000]
Expand Down
9 changes: 5 additions & 4 deletions rust/fuzz/.gitignore
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# Fuzz corpus and artifacts
# Corpus growth: allow initial seeds, ignore generated
corpus/*/
!corpus/*/.gitkeep
# corpus/ is deliberately NOT ignored: seeds are committed per target
# (corpus/<target>/, the layout cargo-fuzz loads by default). Local fuzz runs
# grow these directories with hash-named discoveries — triage them, then
# commit keepers (after minimize_corpus.sh) or discard with git clean.
# The old `corpus/*/` pattern here silently excluded every seed: LAB-1149.

# Artifacts: crashes, hangs, slow inputs
artifacts/
Expand Down
10 changes: 10 additions & 0 deletions rust/fuzz/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,14 @@ define warn_if_missing
@command -v $(1) >/dev/null 2>&1 || echo "$(YELLOW)⚠️ $(1) not found. $(2)$(RESET)"
endef

# Guard every recipe that loops over FUZZ_TARGETS: an empty list makes the loop
# body run zero times and the recipe exit 0, reporting green having fuzzed
# nothing. `target` does not need this — it lists FUZZ_TARGETS only on its
# missing-TARGET error path, which already exits 1.
define require_targets
@if [ -z "$(strip $(FUZZ_TARGETS))" ]; then echo "$(YELLOW)❌ no [[bin]] targets derived from Cargo.toml — a zero-target loop would report green having fuzzed nothing$(RESET)"; exit 1; fi
endef

help: ## Show fuzzing-specific commands
@echo "$(BLUE)cachekit/rust/fuzz - Fuzzing Commands$(RESET)"
@echo ""
Expand Down Expand Up @@ -68,6 +76,7 @@ quick: ## Run quick fuzzing smoke test (60s per target, ~14min total)
@echo "$(YELLOW)Logging to $(LOG_FUZZ_DIR)/quick_$(TIMESTAMP).log$(RESET)"
$(call require_binary,cargo,Install Rust: https://rustup.rs)
$(call warn_if_missing,cargo-fuzz,Install: cargo install cargo-fuzz)
$(call require_targets)
@{ \
for target in $(FUZZ_TARGETS); do \
echo "$(YELLOW)Fuzzing $$target (60s)...$(RESET)"; \
Expand All @@ -89,6 +98,7 @@ deep: ## Run deep fuzzing (8 hours per target, production-grade)
@echo "$(YELLOW)Logging to $(LOG_FUZZ_DIR)/deep_$(TIMESTAMP).log$(RESET)"
$(call require_binary,cargo,Install Rust: https://rustup.rs)
$(call warn_if_missing,cargo-fuzz,Install: cargo install cargo-fuzz)
$(call require_targets)
@{ \
for target in $(FUZZ_TARGETS); do \
echo "$(YELLOW)Deep fuzzing $$target (8 hours)...$(RESET)"; \
Expand Down
106 changes: 49 additions & 57 deletions rust/fuzz/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,24 +17,30 @@ This fuzzing suite provides **14 fuzz targets** covering:
```bash
# Install cargo-fuzz (libfuzzer-based)
cargo install cargo-fuzz

# Install AFL++ (optional, for mutation-based fuzzing)
cargo install cargo-afl
```

### Basic Fuzzing Workflow

Run from the repository root. The root Makefile's `fuzz-quick`, `fuzz-target`,
`fuzz-deep` and `fuzz-coverage` goals delegate to `rust/fuzz/Makefile`, whose
own goals drop the prefix — `cd rust/fuzz && make quick` is equivalent to
`make fuzz-quick` from the root:

```bash
# Quick smoke test (60s per target, ~14min total)
cd rust && make fuzz-quick
make fuzz-quick

# Fuzz single target for development
cd rust && make fuzz-target TARGET=byte_storage_corrupted_envelope
make fuzz-target TARGET=byte_storage_corrupted_envelope

# Deep fuzzing (8 hours per target, production validation)
cd rust && make fuzz-deep TARGET=encryption_key_derivation
# Deep fuzzing of one target (8 hours, production validation)
make fuzz-target TARGET=encryption_key_derivation TIME=28800

# Deep fuzzing of every target (8 hours each)
make fuzz-deep

# Generate coverage report
cd rust && make fuzz-coverage
make fuzz-coverage
```

## Fuzz Targets
Expand All @@ -52,7 +58,7 @@ cd rust && make fuzz-coverage
- Tests: u32::MAX, MAX_UNCOMPRESSED_SIZE ± 1, suspicious compression ratios

**byte_storage_checksum_collision.rs**
- Attack: Data corruption with manipulated Blake3 checksums
- Attack: Data corruption with manipulated xxHash3-64 checksums
- Validates: Integrity verification detects mismatches
- Tests: Bit flips, truncation, zero checksums, partial corruption

Expand Down Expand Up @@ -89,7 +95,7 @@ cd rust && make fuzz-coverage
- Tests: Null bytes, control characters, Unicode, 1-bit AAD modification

**encryption_large_payload.rs**
- Attack: Production-scale payloads (1MB, 10MB, 100MB)
- Attack: Production-scale payloads (seeded up to 256KB; harness accepts up to 100MB)
- Validates: Performance and correctness at scale
- Tests: Large allocations, memory efficiency, no artificial 4KB limits

Expand All @@ -103,54 +109,57 @@ cd rust && make fuzz-coverage
## Corpus Management

### Directory Structure
```
rust/fuzz/corpus/
├── byte_storage/
│ ├── valid_envelopes/ # Valid MessagePack envelopes
│ ├── corrupted_envelopes/ # Known corruption patterns
│ ├── size_edge_cases/ # MIN, MAX, boundary sizes
│ └── format_strings/ # Valid + malicious format identifiers
├── encryption/
│ ├── key_material/ # Valid 32-byte keys, edge cases
│ ├── tenant_ids/ # Realistic + malicious tenant IDs
│ ├── aad_patterns/ # Normal + injected AAD
│ └── ciphertext_samples/ # Valid + truncated ciphertext
└── integration/
└── layered_data/ # Compressed-then-encrypted samples
```

One committed seed directory per fuzz target — `corpus/<target>/`, named
exactly after the `[[bin]]` stanza in `Cargo.toml`, because that is the
layout `cargo fuzz run <target>` loads by default (locally and in CI, no
corpus argument). The full contract — seed provenance, growth, regression
seeds — lives in [`corpus/CORPUS_INFO.md`](corpus/CORPUS_INFO.md).

### Corpus Scripts

```bash
Comment thread
coderabbitai[bot] marked this conversation as resolved.
# Generate initial corpus from test fixtures
# (Re)generate the deterministic per-target seed set.
# Needs python3 with msgpack, lz4, xxhash at the versions pinned in the
# script — or via uv:
# cd rust/fuzz && uv run --no-project --with msgpack==1.2.1 --with lz4==4.4.5 --with xxhash==4.0.0 bash scripts/generate_corpus.sh
cd rust/fuzz && ./scripts/generate_corpus.sh
Comment thread
coderabbitai[bot] marked this conversation as resolved.

# Minimize corpus (deduplicate, reduce size)
# Minimize corpus after growth runs (cargo +nightly fuzz cmin per target)
cd rust/fuzz && ./scripts/minimize_corpus.sh

# Validate corpus integrity (< 10MB total)
# Validate: every Cargo.toml target has seeds, total < 10MB
cd rust/fuzz && ./scripts/validate_corpus.sh
```

**Corpus Size Limit**: Total corpus should remain under 10MB for fast CI smoke tests.

**Regression seeds**: when a crash is found and fixed, commit the minimized
reproducer into `corpus/<target>/` so it is re-tested on every future run.

## CI Integration

### Smoke Tests (PR Validation)

`.github/workflows/fuzz-smoke.yml` runs on every pull request:
- 60 seconds per target (~14 minutes total)
- Starts from the committed seeds in `corpus/<target>/` (cargo-fuzz's default
corpus path) instead of cold-starting from random bytes
- Catches fuzzing regressions before merge
- Uploads crash artifacts on failure

```bash
# Simulate CI smoke tests locally
cd rust && make fuzz-quick
# Simulate CI smoke tests locally (repo root)
make fuzz-quick
```

### Deep Fuzzing (Production Validation)

Run before releases or periodically:

```bash
# 8 hours per target (production-grade validation)
cd rust && make fuzz-deep TARGET=encryption_key_derivation
# 8 hours on one target (repo root)
make fuzz-target TARGET=encryption_key_derivation TIME=28800
```

## Crash Triage
Expand Down Expand Up @@ -178,23 +187,11 @@ cargo fuzz run byte_storage_corrupted_envelope artifacts/crash-xyz
cargo fuzz cmin byte_storage_corrupted_envelope
```

## AFL++ Fuzzing (Alternative Engine)

AFL++ provides mutation-based fuzzing complementary to libfuzzer's coverage-guided approach:

```bash
# Build AFL++ target
cd rust && cargo afl build --features afl

# Run AFL++ fuzzer
cd rust && make fuzz-afl TARGET=byte_storage_corrupted_envelope
```

## Coverage Reporting

```bash
# Generate LLVM coverage report
cd rust && make fuzz-coverage
# Generate LLVM coverage report (repo root)
make fuzz-coverage

# View HTML report
open rust/fuzz/coverage/html/index.html
Expand Down Expand Up @@ -251,25 +248,20 @@ All fuzz targets enforce fail-closed behavior:
- **Panic** = Test failure (fuzzer reports bug)
- **Undefined behavior** = Instant failure (sanitizers catch)

### Multi-Engine Strategy

- **Libfuzzer** (default): Coverage-guided, fast iteration, LLVM sanitizers
- **AFL++**: Mutation-based, finds different bug classes, mature tooling
- Both engines share corpus for cross-pollination

## Contributing

When adding new fuzz targets:
1. Follow naming convention: `<module>_<attack_vector>.rs`
2. Add target to `Cargo.toml` [[bin]] section
3. Create corpus subdirectory with `.gitkeep`
2. Add target to `Cargo.toml` [[bin]] section — the Makefile, CI, and corpus
scripts all derive the target list from these stanzas; there is no separate
list to update
3. Add seeds for the target to `scripts/generate_corpus.sh` and run it —
`validate_corpus.sh` fails until `corpus/<target>/` has seeds
4. Document attack vector in this README
5. Add target to `FUZZ_TARGETS` list in `rust/Makefile`
6. Verify with `make fuzz-target TARGET=your_new_target`
5. Verify with `make fuzz-target TARGET=your_new_target`

## References

- [Rust Fuzz Book](https://rust-fuzz.github.io/book/)
- [libfuzzer documentation](https://llvm.org/docs/LibFuzzer.html)
- [AFL++ documentation](https://aflplus.plus/)
- [cachekit security architecture](../../SECURITY.md)
Loading
Loading