Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ Develop inspectable, reproducible, **defensive** reliability and security resear
## Local testbed gate

- Never enable GitHub Pages, publish a project URL/dashboard, or add a hosted endpoint for this repository.
- Pin trusted host directory/file descriptors for any state-changing VM or evidence operation. Refuse group/other-writable ancestors, unexpected symlinks, and public evidence directories; no path-based revalidation after a slow process call.
- Evidence sealing and verification require a quiescent or externally snapshotted input; SHA-256 does not prove point-in-time consistency against malicious simultaneous writers.
- Keep guest networking disabled by default. No NIC, NAT, bridge, public graphics or host passthrough without separate explicit scope review.
- Never run scripts/testbed.py prepare or snapshot, virsh define/start, or any other state-changing host step without operator approval. GitHub CI must only test pure/mocked code.
- Do not represent a successful static XML audit as proof of real network containment or a completed RHEL experiment.
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ To inventory an evidence directory, write the manifest **outside** the evidence
uv run --no-project python scripts/evidence.py seal --input ./local/capture-001 --output ./local/capture-001.manifest.json
uv run --no-project python scripts/evidence.py verify --input ./local/capture-001 --manifest ./local/capture-001.manifest.json

The example requires the capture directory to exist. The manifest is deterministic (no generated timestamp), records relative paths, byte sizes and SHA-256 digests, rejects symlinks, and will not overwrite an existing manifest. For provenance, separately record target/kernel/package versions, test configuration, seed, exact source SHA, time, tool versions, and whether the test actually executed. A manifest establishes byte identity, not scientific validity.
The example requires the capture directory to exist. The manifest is deterministic (no generated timestamp), records relative paths, byte sizes and SHA-256 digests, rejects symlinks, and will not overwrite an existing manifest. Its output directory must be operator-owned and private (0700); it writes the manifest as 0600 using pinned directory descriptors. Seal only a quiescent or externally snapshotted evidence tree: per-file race checks cannot guarantee a global point-in-time view. For provenance, separately record target/kernel/package versions, test configuration, seed, exact source SHA, time, tool versions, and whether the test actually executed. A manifest establishes byte identity, not scientific validity.

## Virtual testbed (RHEL-LAB-002)

Expand Down
2 changes: 1 addition & 1 deletion docs/DAYBREAK_GATE.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Daybreak Defense Factory — staged security gate

**Status: NOT RUN.** RHEL-LAB-002 code should be reviewed through an ordinary GitHub PR and the non-privileged Python CI tests *before* requesting a Daybreak repository scan. Daybreak has not been authorized to run a scan by this milestone.
**Status: TWO SOURCE AUDITS COMPLETED, FINDINGS UNDER REMEDIATION.** Daybreak scanned RHEL-LAB at commit `889248b59e4e2cb7c06d8a635c0a07cb2cd39c1e` in runs `wfr_2ef0982c13d785cbdfbf9ecc157a3ccc863cb4085e09dcd70e9124f86bae68f2` and `wfr_f166c5a69c210c6a14bdb64c37311e11fffd8e8dd8a0e0e5e0bd0e0dfb0cac99`. Eight repository-source findings (three medium, five low) are being addressed. Neither a completed audit nor a green CI workflow proves a live RHEL guest is safe; **no RHEL experiments have been executed**.

## Controlled progression

Expand Down
6 changes: 3 additions & 3 deletions docs/VIRTUAL_TESTBED.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ This milestone supplies an offline-first *host-side implementation*, not a prein
- One x86_64 guest, 2 vCPUs, 4096 MiB RAM, standalone 48 GiB QCOW2 virtual disk. Configuration is in lab/testbed.json.
- **No guest NIC, bridge, NAT, hostdev, host share, USB redirection, graphics listener or public service** in the generated domain XML.
- Serial PTY console only. Guest ISO must be obtained separately from authorized Red Hat distribution; no proprietary image is committed.
- Images, private receipts and XML live in an operator-selected private 0700 directory **outside the Git repository**.
- Images, private receipts and XML live in a pinned, operator-owned 0700 directory under trusted ancestors **outside the Git repository**.
- Every host-changing command is explicit, opt-in, and fail-closed. No automatic boot, package installation, VM definition, snapshot restore, or host network configuration.
- A definition/active XML audit rejects unexpected disks, device passthrough, guest networking and external graphics. **This is a configuration audit, not proof of effective egress isolation.**
- No GitHub Pages, public dashboard or network endpoint.
Expand All @@ -17,7 +17,7 @@ This milestone supplies an offline-first *host-side implementation*, not a prein

Use a personally controlled Linux host with KVM support and local libvirt/QEMU tools installed by the operator according to their OS instructions. You need virsh, qemu-img, a working per-user qemu:///session connection, access to /dev/kvm and sufficient disk space. Do not install packages automatically as part of CI or this project.

Check the actual hardware, hypervisor, firmware support, host ownership, VM resource budget, filesystem permissions and guest media authenticity yourself. The scripted plan **does not** check that a physical host is isolated.
Check the actual hardware, hypervisor, firmware support, host ownership, VM resource budget, filesystem permissions and guest media authenticity yourself. The lab-root parent and ancestors must be root/operator-owned and not group/other writable (the root-owned sticky /tmp convention is recognized); a new lab root is created 0700. Avoid shared mountpoints. The scripted plan **does not** check that a physical host is isolated.

Download the correct RHEL installer ISO using your own authorized account, verify its official checksum/signature, and keep it outside Git. No vendor access credential belongs in a script, PR or public log.

Expand Down Expand Up @@ -76,7 +76,7 @@ The snapshot command:
4. Refuses existing destination/receipt names and uses a no-clobber hard-link for final publication.
5. Records SHA-256, exact source commit, byte count and UTC time in a local receipt.

This is an **offline disk copy**, not a libvirt RAM-state snapshot and not a guaranteed atomic capture against hostile/concurrent writers. Never use qemu-img against a running VM disk.
This is an **offline disk copy**, not a libvirt RAM-state snapshot and not a guaranteed atomic capture against hostile/concurrent writers. The operation now pins trusted directory and file descriptors, pre-creates output with exclusive owner-only permissions and invokes qemu-img through inherited Linux /proc/self/fd handles; `/proc` must be available. A non-cooperating writer with the operator's identity or a live guest can still invalidate a snapshot. Never use qemu-img against a running VM disk.

Verify the resulting evidence directory using scripts/evidence.py, storing its manifest outside the directory being sealed. Preserve guest kernel, release, package versions, original media checksum, VM settings, snapshot identifiers, command line, exit status and negative findings in a separate private baseline receipt.

Expand Down
198 changes: 125 additions & 73 deletions scripts/evidence.py
Original file line number Diff line number Diff line change
@@ -1,108 +1,160 @@
#!/usr/bin/env python3
"""Deterministic SHA-256 file inventory for an operator-controlled evidence directory.
"""Deterministic evidence manifests using descriptor-pinned, no-follow traversal.

The manifest establishes byte identity only. It is not a signature, chain of custody,
proof of origin, or proof that an experiment executed. No network or guest access.
A SHA-256 manifest describes observed bytes, not signed chain of custody.
Use a quiescent/external snapshot for a point-in-time inventory.
"""
from __future__ import annotations

import argparse
import hashlib
import json
import os
import stat
import sys
from pathlib import Path

try:
from . import safe_fs
except ImportError: # direct invocation: python scripts/evidence.py
import safe_fs

SCHEMA = "qsol-rhel-evidence/v1"


class EvidenceError(ValueError):
"""Invalid input, unsafe file type, or a mismatched evidence manifest."""
"""Evidence tree changed, file type unsupported, or manifest mismatch."""


def _identity(info: os.stat_result) -> tuple[int, int, int]:
return info.st_dev, info.st_ino, info.st_mode


def _unchanged(first: os.stat_result, last: os.stat_result) -> bool:
return (_identity(first) == _identity(last)
and first.st_size == last.st_size
and first.st_mtime_ns == last.st_mtime_ns
and first.st_ctime_ns == last.st_ctime_ns)


def _scan(fd: int, prefix: str, files: list[dict], depth: int = 0) -> None:
if depth > 256:
raise EvidenceError("evidence directory nesting exceeds safety limit")
before_dir = os.fstat(fd)
with os.scandir(fd) as iterator:
entries = sorted(iterator, key=lambda entry: entry.name)
for entry in entries:
relative = f"{prefix}/{entry.name}" if prefix else entry.name
observed = entry.stat(follow_symlinks=False)
if stat.S_ISLNK(observed.st_mode):
raise EvidenceError(f"symlink is not permitted: {relative!a}")
# Open relative to the pinned parent. O_NONBLOCK prevents raced FIFOs
# from blocking; fstat validates the object actually opened.
child_fd = os.open(entry.name, safe_fs.FILE_READ_FLAGS, dir_fd=fd)
try:
opened = os.fstat(child_fd)
if _identity(observed) != _identity(opened):
raise EvidenceError(f"evidence entry changed while opening: {relative!a}")
if stat.S_ISDIR(opened.st_mode):
_scan(child_fd, relative, files, depth + 1)
elif stat.S_ISREG(opened.st_mode):
digest = hashlib.sha256()
count = 0
while True:
chunk = os.read(child_fd, 1024 * 1024)
if not chunk:
break
digest.update(chunk)
count += len(chunk)
final = os.fstat(child_fd)
if not _unchanged(opened, final) or count != final.st_size:
raise EvidenceError(f"evidence file changed during hashing: {relative!a}")
files.append({"path": relative, "bytes": count,
"sha256": digest.hexdigest()})
else:
raise EvidenceError(f"non-regular file is not permitted: {relative!a}")
finally:
os.close(child_fd)
if not _unchanged(before_dir, os.fstat(fd)):
raise EvidenceError(f"evidence directory changed during traversal: {prefix!a}")


def _inventory_fd(root_fd: int) -> dict:
files: list[dict] = []
_scan(root_fd, "", files)
files.sort(key=lambda item: item["path"])
return {"schema": SCHEMA, "algorithm": "sha256", "files": files}


def _root(path: Path) -> Path:
if path.is_symlink() or not path.is_dir():
raise EvidenceError(f"input must be a real directory: {path}")
return path.resolve()
def inventory(input_dir: Path) -> dict:
"""Fail closed on symlink, special file, unreadable subtree, observed race."""
with safe_fs.directory(input_dir) as root_fd:
return _inventory_fd(root_fd)


def inventory(input_dir: Path) -> dict:
"""List relative regular files in stable order; reject symlinks and special files."""
root = _root(input_dir)
files = []
directories = [root]
while directories:
directory = directories.pop()
# pathlib.Path.rglob() may suppress scandir errors on some Python
# versions. Explicit scandir traversal makes incomplete evidence fatal.
with os.scandir(directory) as iterator:
children = sorted(iterator, key=lambda entry: entry.name)
for entry in children:
path = Path(entry.path)
relative = path.relative_to(root).as_posix()
if entry.is_symlink():
raise EvidenceError(f"symlink is not permitted: {relative}")
if entry.is_dir(follow_symlinks=False):
directories.append(path)
continue
if not entry.is_file(follow_symlinks=False):
raise EvidenceError(f"non-regular file is not permitted: {relative}")
digest = hashlib.sha256()
size = 0
with path.open("rb") as source:
for chunk in iter(lambda: source.read(1024 * 1024), b""):
digest.update(chunk)
size += len(chunk)
files.append({"path": relative, "bytes": size, "sha256": digest.hexdigest()})
files.sort(key=lambda item: item["path"])
return {"schema": SCHEMA, "algorithm": "sha256", "files": files}
def _manifest_location(manifest: Path) -> tuple[Path, str]:
value = safe_fs.absolute(manifest)
if value.name in ("", ".", ".."):
raise EvidenceError("manifest basename is invalid")
return value.parent, value.name


def seal(input_dir: Path, output: Path) -> dict:
"""Write a fresh, deterministic manifest outside the captured directory."""
root = _root(input_dir)
target = output.resolve()
if target == root or root in target.parents:
raise EvidenceError("manifest must be stored outside the input directory")
if output.is_symlink():
raise EvidenceError("manifest output may not be a symlink")
report = inventory(root)
# Exclusive creation refuses to replace an existing manifest or symlink.
with output.open("x", encoding="utf-8", newline="\n") as stream:
json.dump(report, stream, indent=2, sort_keys=True)
stream.write("\n")
return report
"""Write manifest with mode 0600 relative to a pinned private directory."""
parent, name = _manifest_location(output)
with safe_fs.directory(input_dir) as root_fd:
st = os.fstat(root_fd)
root_identity = (st.st_dev, st.st_ino)
with safe_fs.directory(parent, private=True, forbidden=root_identity) as dest_fd:
report = _inventory_fd(root_fd)
fd = os.open(name, safe_fs.FILE_WRITE_FLAGS, 0o600, dir_fd=dest_fd)
with os.fdopen(fd, "w", encoding="utf-8", newline="\n") as stream:
json.dump(report, stream, indent=2, sort_keys=True)
stream.write("\n")
stream.flush()
os.fsync(stream.fileno())
return report


def verify(input_dir: Path, manifest: Path) -> dict:
"""Recompute file inventory and fail on any missing, extra or modified file."""
if manifest.is_symlink() or not manifest.is_file():
raise EvidenceError("manifest must be an existing regular file (not a symlink)")
with manifest.open("r", encoding="utf-8") as stream:
expected = json.load(stream)
actual = inventory(input_dir)
if expected != actual:
raise EvidenceError("manifest mismatch: missing, extra, altered, or invalid files")
return actual
"""Verify manifest and evidence via pinned FDs, refusing observed changes."""
parent, name = _manifest_location(manifest)
with safe_fs.directory(input_dir) as root_fd:
st = os.fstat(root_fd)
with safe_fs.directory(parent, private=True,
forbidden=(st.st_dev, st.st_ino)) as manifest_fd:
fd = os.open(name, safe_fs.FILE_READ_FLAGS, dir_fd=manifest_fd)
try:
first = os.fstat(fd)
if not stat.S_ISREG(first.st_mode):
raise EvidenceError("manifest must be a regular file")
with os.fdopen(os.dup(fd), "r", encoding="utf-8") as stream:
expected = json.load(stream)
if not _unchanged(first, os.fstat(fd)):
raise EvidenceError("manifest changed while reading")
finally:
os.close(fd)
actual = _inventory_fd(root_fd)
if expected != actual:
raise EvidenceError("manifest mismatch: missing, extra, altered or invalid files")
return actual


def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
subparsers = parser.add_subparsers(dest="command", required=True)
seal_parser = subparsers.add_parser("seal", help="write a fresh SHA-256 manifest")
seal_parser.add_argument("--input", required=True, type=Path)
seal_parser.add_argument("--output", required=True, type=Path)
verify_parser = subparsers.add_parser("verify", help="verify an existing manifest")
verify_parser.add_argument("--input", required=True, type=Path)
verify_parser.add_argument("--manifest", required=True, type=Path)
subs = parser.add_subparsers(dest="command", required=True)
p_seal = subs.add_parser("seal", help="write a fresh SHA-256 manifest")
p_seal.add_argument("--input", required=True, type=Path)
p_seal.add_argument("--output", required=True, type=Path)
p_verify = subs.add_parser("verify", help="verify existing manifest")
p_verify.add_argument("--input", required=True, type=Path)
p_verify.add_argument("--manifest", required=True, type=Path)
args = parser.parse_args(argv)
try:
if args.command == "seal":
report = seal(args.input, args.output)
else:
report = verify(args.input, args.manifest)
except (EvidenceError, OSError, ValueError, TypeError) as exc:
print(f"FAIL: {exc}", file=sys.stderr)
report = seal(args.input, args.output) if args.command == "seal" else verify(args.input, args.manifest)
except (EvidenceError, safe_fs.UnsafePath, OSError, ValueError, TypeError) as exc:
# Escape attacker-provided filenames and terminal control characters.
print("FAIL: " + ascii(str(exc))[1:-1], file=sys.stderr)
return 1
print(f"PASS: {len(report['files'])} evidence files; {args.command}")
return 0
Expand Down
Loading
Loading