Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 27 additions & 17 deletions ARCHIVE-CUSTODY.md
Original file line number Diff line number Diff line change
@@ -1,27 +1,37 @@
# TestForge archive custody

TestForge is one two-skill Augment with several independently useful release objects. The canonical release keeps the complete product, its Codex plugin, and both standalone skills separately obtainable without pretending that a standalone skill is the whole product.
TestForge v1.1.6 is one two-skill Augment with several distinct distribution objects. Keep their identity and evidence states separate: source presence is not installation, a valid archive is not discovery, discovery is not invocation, and none of those states proves healthy behavior or directory publication.

| Object | Canonical release artifact | Use |
## Current v1.1.6 objects

| Object | Canonical location | Observed state and use |
|---|---|---|
| Complete Augment | `release-assets/v1.1.1/TestForge-v1.1.1.zip` | Portable two-skill TestForge capability with adapters, docs, evals, and tools |
| Codex plugin | `release-assets/v1.1.1/Plugin-TestForge-v1.1.1.zip` | Branded Codex plugin with both skill entry points and listing assets |
| Software Verification skill | `release-assets/v1.1.1/Skill-software-verification--TestForge-v1.1.1.zip` | Independent `$software-verification` installation and recovery |
| Verification Reviewer skill | `release-assets/v1.1.1/Skill-verification-reviewer--TestForge-v1.1.1.zip` | Independent `$verification-reviewer` installation and recovery |
| Claude.ai uploads | `claude-ai/software-verification-v1.1.1.zip` and `claude-ai/verification-reviewer-v1.1.1.zip` | Host-specific one-skill upload archives |
| Source repository | Git tag `v1.1.1` and its GitHub source archives | Versioned source, documentation, tests, testbed, and provenance |
| Maintained package | `testforge/` | Current two-skill source, tools, schemas, examples, evals, adapters, and customer documentation |
| Codex marketplace plugin | `plugins/testforge/` plus `.agents/plugins/marketplace.json` | Repository-native plugin source for `testforge@cd-testforge`; static structure is repository-tested |
| Claude operator upload | `claude-ai/software-verification-v1.1.6.zip` | Current one-skill upload archive; SHA-256 `3485f982d9d7f770b9077bc9da122498ff5ce135ae60e07e7aa8fa8209d5a52f` |
| Claude reviewer upload | `claude-ai/verification-reviewer-v1.1.6.zip` | Current one-skill upload archive; SHA-256 `c882eacec514e23647e1e298b9919a89e3b85ded06649041cf924c91994308ba` |
| Frozen v1.1.6 customer kit | `releases/v1.1.6/TestForge-v1.1.6.zip` | Canonical published release object retained unchanged; SHA-256 `4dd052672923192f59ec2866eb2fedef697ca1f98f00a99341c3d8fe062b0594` |
| Frozen v1.1.6 receipts | `releases/v1.1.6/` | Static package, source-parity, and portable archive evidence; no fresh-host activation or customer-outcome claim |
| Source release | Git tag `v1.1.6` and [GitHub release](https://github.com/Stunspot/TestForge/releases/tag/v1.1.6) | Published 2026-08-12; versioned public source and release boundary |

## Plugin publication payloads
The current `claude-ai/` archives and the frozen archives inside `releases/v1.1.6/claude/` are separate deterministic builds and are not byte-identical. Use the current `claude-ai/` objects for the repository installation guide. Use the frozen release directory to inspect the exact evidence and bytes retained for the v1.1.6 release event.

TestForge plugin v1.1.2 preserves two deliberately different ZIPs:
## OpenAI directory packet

| Object | Canonical release artifact | Use |
|---|---|---|
| Installable Codex plugin | `release-assets/v1.1.2/Plugin-TestForge-v1.1.2.zip` | Normal Codex installation and marketplace distribution with the full interface manifest |
| OpenAI skills-only submission | `release-assets/v1.1.2/Plugin-TestForge-v1.1.2-OpenAI-Submission.zip` | Deterministic portal upload whose archived interface retains only `composerIcon` and `logo` |
The latest retained skills-only portal payload is still v1.1.4:

- archive: `release-assets/v1.1.4/Plugin-TestForge-v1.1.4-OpenAI-Submission.zip`;
- custody: `release-assets/v1.1.4/openai-submission-custody.json`;
- SHA-256: `9aecec78e407e6f368d0a5c613facbc4252a3f3ef545ba6686e74cf7f2404a46`;
- state: built and repository-tested, not claimed uploaded, approved, published, or discoverable.

`release-assets/v1.1.2/archive-custody.json` governs the installable plugin. `release-assets/v1.1.2/openai-submission-custody.json` separately records the portal derivative's archive hash, source and transformed manifest hashes, member count, and POSIX path requirement. The portal ZIP is not a replacement for the installable plugin.
There is no retained v1.1.6 portal archive or custody object. The repository-native v1.1.6 plugin remains the current Codex installation surface; the v1.1.4 portal packet is a separately governed historical submission candidate.

`release-assets/v1.1.1/archive-custody.json` records exact hashes, sizes, member counts, source-tree digests, and extraction-parity results for the unchanged v1.1.1 Augment and skills. GitHub release assets and the latest-only convenience backup shelf must match the applicable custody records. Canonical assets are copied, never moved, to the backup shelf. Older same-family convenience copies may be removed only after the new copies match; unrelated products are untouched.
## Maintenance rules

The two standalone skill archives and their Claude.ai counterparts intentionally carry the same skill content under channel-appropriate names. Static package equality does not establish live Claude activation, live Codex discovery, or directory approval.
- Rebuild current derivatives from maintained source; never edit ZIP members in place.
- Do not alter `releases/` merely to make present documentation agree with a historical release.
- Record archive name, byte size, SHA-256, member inventory, source revision, and claim boundary for each new object.
- Verify extraction topology and package-relative dependencies before publication.
- After publication, download the public asset and compare it with the governed local object.
- Treat upload, automated scan, review submission, approval, publication, installation, discovery, invocation, and health as separate observed states.
4 changes: 3 additions & 1 deletion BUILD-NOTE.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
# TestForge v1.1.0 maintenance build note
# Historical TestForge v1.1.0 maintenance build note

> Historical record: this file describes the v1.1.0 maintenance event. It is not the current installation, archive-custody, or validation authority. Use `README.md`, `RELEASE-NOTES-v1.1.6.md`, `RELEASE-NOTES.md`, `ARCHIVE-CUSTODY.md`, and the current `testforge/docs/` guides.

## Result

Expand Down
4 changes: 3 additions & 1 deletion BUILD-WEEK.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ A later Codex task used TestForge to design and harden the CD Augment behavioral

## Build evidence

> Historical snapshot: the counts and release identity in this section describe the original v1.0.2 Build Week entry. They are not current v1.1.6 package or validation evidence; use the current README, release notes, archive custody, and validation guides for that.

- Primary Codex build Session ID: `019f6a6e-8556-75c0-919c-0738a3cb1f84`
- Primary build model recorded by Codex: `gpt-5.6-sol`
- Primary build date: July 16, 2026
Expand All @@ -36,7 +38,7 @@ A later Codex task used TestForge to design and harden the CD Augment behavioral
- Deterministic local checks: 9 TestForge tests, 46 testbed tests, and 4 public-distribution tests
- Behavioral suite: 10 cases across 11 dimensions, with indispensable gates for evidence honesty, security boundaries, and release judgment

The Devpost form should use the primary Session ID above. It is the task where the majority of TestForge's core functionality was built. The later Omnicompetence task expanded the testbed and handled publication, but it is not a substitute for the primary build task.
The Devpost form should use the primary Session ID listed in this Build evidence section. It is the task where the majority of TestForge's core functionality was built. The later Omnicompetence task expanded the testbed and handled publication, but it is not a substitute for the primary build task.

## Product architecture

Expand Down
2 changes: 1 addition & 1 deletion JUDGE-QUICKSTART.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ Python 3.10 or newer is required. PyYAML is the testbed's only third-party depen

## Supported paths

- Codex plugin: repository-native two-command installation above.
- Codex plugin: repository-native two-command installation in step 1.
- Codex standalone SKILLs: install the portable `testforge/` bundle while preserving package-relative resources.
- Claude Code: structurally compatible SKILL instructions with documented host boundaries.
- Other Agent hosts: Markdown skill loading, local-shell adapter, GitHub adapter, or copy-paste fallback.
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
![TestForge - software verification that argues back](assets/testforge-social-preview.png)
![TestForge verification workbench: evidence is forged, measured, and challenged before release](assets/testforge-readme-hero.png)

# TestForge

Expand Down Expand Up @@ -65,7 +65,7 @@ Start a new Codex task, then invoke `$software-verification` or `$verification-r

Download the latest release, unzip it and keep the `testforge/` tree together. Expose both directories under `testforge/skills/` through your Agent host's skill mechanism. Host-specific notes are included for [Codex](testforge/adapters/codex.md), [Claude Code](testforge/adapters/claude-code.md), [GitHub](testforge/adapters/github.md), [local shell](testforge/adapters/local-shell.md) and [copy-paste chat](testforge/adapters/copy-paste-chat.md).

The GitHub release also preserves the complete Augment, the installable Codex plugin, its distinct OpenAI skills-only portal upload, and each bundled skill as separately named archives. This keeps `$software-verification` and `$verification-reviewer` independently recoverable without losing the complete two-skill product. The OpenAI draft exists and both bundled skills passed automated scanning; accountable-owner attestations and submission for review remain pending.
The frozen v1.1.6 release kit preserves the complete Augment, Codex plugin source, and both Claude skill archives with static package receipts. The maintained repository separately exposes current Claude upload archives. The latest retained OpenAI skills-only portal packet is v1.1.4; it is built and repository-tested, but this repository does not claim it was uploaded, scanned by the current portal, approved, published, or made discoverable. See [archive custody](ARCHIVE-CUSTODY.md) for exact object identities and boundaries.

Then start with:

Expand Down
41 changes: 8 additions & 33 deletions RELEASE-NOTES.md
Original file line number Diff line number Diff line change
@@ -1,40 +1,15 @@
# TestForge v1.0.2
# TestForge release notes

Version 1.0.2 is a bounded distribution correction. TestForge's public marketplace now uses the product-specific `cd-testforge` namespace, so installing it cannot replace a separately distributed Collaborative Dynamics marketplace such as CanopyOps. The customer command is now `codex plugin add testforge@cd-testforge`. Verification behavior and the v1.0.1 quality-ratchet evidence are unchanged.
The current package release is **v1.1.6**. See [RELEASE-NOTES-v1.1.6.md](RELEASE-NOTES-v1.1.6.md) for the metered-verification safeguards and exact evidence boundary.

## v1.0.1 quality-ratchet revision
## Current release boundary

Version 1.0.1 turns the Build Week evaluation loop back onto TestForge itself. A matched Qwen control/treatment run improved from 72.08 to 88.33 but retained one indispensable evidence-honesty failure: the model proposed removing a system package to reproduce a missing-library condition. This revision strengthens safe, read-only, capability-matched diagnosis and cleanly separates executed commands, proposed diagnostics, and unexecuted remediation. It also prevents a judge from hiding weak observable performance by mislabeling it as an invalid episode.
Version 1.1.6 treats finite verification capacity as part of the test plan. Before recommending or invoking hosted CI, device or browser farms, paid cloud checks, or another quota-limited route, TestForge requires a fresh observation for the exact billing scope and calculates trigger duplication, matrix fan-out, retries, runner ceilings, billing multipliers, and retained reserve. Unknown, stale, insufficient, provider-refused, reserve-consuming, or unauthorized paid capacity produces a hold without launching a discovery job.

Repeated test-diagnose-reengineer cycles then exposed and repaired context dilution, secret-workflow leakage into unrelated cases, incomplete authorization post-state checks, unsafe production-probe scaffolds, destructive-deletion authority gaps, and local-judge logic inversions. A later full local run reached 90.41 with zero failed or invalid episodes; single-trial behavioral results remain model-, context-, and judge-bounded.
The v1.1.5 verification-cycle custody rule remains in force: a product defect or newly exposed product invariant ends the submitted candidate's TestForge cycle. The upstream repair returns later as a new frozen candidate with a new evidence cutoff. Only a defect proven to belong to TestForge's own test, tool, fixture, or execution environment may be corrected and rerun within the same cycle.

It also adds repository-native Codex plugin installation, Build Week provenance, an isolated fictional judge case, public distribution-integrity tests, and a three-operating-system CI matrix.
The maintained repository includes synchronized v1.1.6 package and plugin source plus current Claude upload archives. Static structure, hashes, parity, and deterministic checks do not prove live host activation, provider-meter accuracy, hosted-run success, directory publication, customer outcomes, or defect freedom.

## Original v1.0.0 release
## Historical notes

TestForge is a free Collaborative Dynamics Augment that gives an Agent two complementary software-verification capabilities:

- `$software-verification` builds a risk-ranked evidence chain from change to release assessment.
- `$verification-reviewer` independently challenges that chain for omissions, weak oracles, misleading mocks and unsupported confidence.

The repository also includes the CD Augment evaluation testbed for running isolated behavioral cases against Codex or local Ollama models, preserving hard gates and promoting reviewed regression baselines.

## Included

- complete portable TestForge Augment;
- two Agent SKILLs and copy-paste fallbacks;
- deterministic Python inspection, validation, normalization and reporting tools;
- Python and TypeScript worked examples with planted defects;
- risk, oracle, reliability, security and stack references;
- canonical behavioral eval suite;
- isolated Augment evaluation harness with Codex and Ollama adapters;
- reviewed compact baselines;
- Codex, Claude Code, GitHub, local-shell and plain-chat adapters.

## License

The release uses a split license: MIT for Python software and machine-readable schemas, and CC BY-ND 4.0 for authored Augment materials. The authentic, unmodified branded Augment may be redistributed, including inside a larger commercial product. See `LICENSE.md`, `ATTRIBUTION.md` and `TRADEMARKS.md`.

## Evidence boundary

The package and testbed deterministic suites pass locally. Supplied behavioral baselines record their model and context. TestForge provides evidence-bounded advice; it does not prove defect freedom, certify compliance or authorize a release.
Version-specific records remain available as `RELEASE-NOTES-v*.md`. They describe their named releases and do not override current installation, privacy, support, or validation guidance.
Binary file added assets/testforge-readme-hero.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
26 changes: 26 additions & 0 deletions docs/404.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="robots" content="noindex">
<meta name="theme-color" content="#050607">
<title>Route not found · TestForge</title>
<link rel="stylesheet" href="/TestForge/style.css">
</head>
<body>
<a class="skip-link" href="#main">Skip to content</a>
<header class="site-header">
<nav aria-label="Primary"><a class="brand" href="/TestForge/"><span class="brand-mark" aria-hidden="true">TF</span><span>TestForge</span></a><div class="nav-links"><a href="/TestForge/#start">Start</a><a href="/TestForge/#install">Install</a><a href="/TestForge/#troubleshooting">Recover</a><a href="https://github.com/Stunspot/TestForge">GitHub</a></div></nav>
</header>
<main id="main" class="not-found">
<section class="section">
<p class="eyebrow">404 · route not found</p>
<h1>The evidence trail stops here.</h1>
<p class="lede">This address is not part of the current TestForge site. Recover through a maintained customer route instead of trusting a stale or repository-relative link.</p>
<div class="actions"><a class="button primary" href="/TestForge/">Documentation home</a><a class="button secondary" href="/TestForge/#troubleshooting">Troubleshooting</a><a class="button secondary" href="https://github.com/Stunspot/TestForge/releases/latest">Current release</a></div>
</section>
</main>
<footer><p><strong>TestForge</strong> · Software verification that argues back.</p></footer>
</body>
</html>
5 changes: 4 additions & 1 deletion docs/SITE-SOURCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,12 @@ The page does not claim defect freedom, compliance certification, production acc

- `index.html` — semantic single-page project overview;
- `style.css` — responsive presentation and accessibility treatment;
- `assets/testforge-hero.png` — generated 1600×900 raster hero artwork;
- `assets/testforge-hero.png` — 1200×800 role-specific Pages hero showing a three-path inspection gate;
- `404.html` — maintained recovery route for stale or invalid links;
- `.nojekyll` — direct static-file serving marker.

The repository README uses the distinct 1600×640 `assets/testforge-readme-hero.png`. Social sharing uses the distinct 1280×640 `assets/testforge-social-preview.png`, which visibly contains the exact TestForge title and identifying line.

## Deployment

`.github/workflows/deploy-pages.yml` uploads this directory with GitHub's official Pages Actions. Repository Pages must be configured to use **GitHub Actions** before the first deployment can publish.
Expand Down
Binary file modified docs/assets/testforge-hero.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading