Deterministic, sub-millisecond AST Pull Request verification gate in the AI coding agent era. Catches test tampering, fake assertions, hallucinated phantom modules, and security vulnerabilities before human review.
Experience Tmy-Joy intercepting an AI coding agent (Cursor/Devin) attempting to sneak muted tests and hallucinated code past CI:
cargo run --release -p tb-cli -- demo================================================================================
π‘οΈ TOKENECTOMY-BOT (TMY-JOY) β LIVE ADVERSARIAL PR AUDIT DEMO
Scenario: AI Coding Agent (Cursor / Devin / Copilot) submits PR #42
================================================================================
π€ [AI CODING AGENT CLAIM]: "All tests passing! 100% CI green. Ready to merge! π"
β‘ AST Audit completed in 0.82 ms! (Throughput: ~1,210 files/sec)
π¨ [TMY-JOY PR QUALITY GATE VERDICT]: β BLOCKED (3 Errors, 1 Warning)
π [AUDIT FINDINGS & CAUGHT CHEATING TRICKS]:
1. π΄ BLOCKED `TB301` (phantom-symbol)
Trick : Phantom module detected (`./uncreated_gateway`). AI agent hallucinated uncreated file.
2. π΄ BLOCKED `TB002` (test-disabled)
Trick : Test was disabled using: `it.skip('handles network timeout')`
3. π΄ BLOCKED `TB003` (tautological-assertion)
Trick : Tautological assertion detected: `expect(true).toBe(true)` always evaluates to true.
4. π‘ WARNING `TB101` (silent-catch)
Trick : Empty catch block swallows fatal production exceptions.
Auto-Fix: β Available (1-click GitHub PR suggestion)
πΊοΈ [BLAST RADIUS & SENSITIVE PATH IMPACT]:
Risk Score : 43/100 (Medium) β Touched sensitive domain `src/billing/checkout.ts`
π [CRYPTOGRAPHIC AUDIT LEDGER SEAL]:
Hash : b57adbebfaa9980ed7b7576d5156a5c7a7d273e2d3288b3ab0dbbd7ad0d2cd07
Proof: Immutable record sealed. PR cannot be merged into main.
- β‘ Zero-LLM Core: Keputusan pass/fail 100% deterministik dan bebas halusinasi berbasis Tree-sitter AST traversal dalam hitungan milidetik (~0.8 ms per berkas).
- π‘οΈ Anti-Tampering Gate (TB0xx):
TB001: assertion-removed β assertion test berkurang bersih pada test suite yang masih aktif.TB002: test-disabled β pendeteksian.skip,xit,it.todo,test.fixme,#[ignore].TB003: tautological-assertion β assertion palsu sepertiexpect(true).toBe(true)atau variabel yang membandingkan dirinya sendiri.TB004βTB009: config-weakened, early-exit-injected, lazy-deletion (todo!()), domain-narrowing, fixture-snooping.
- π» Hallucination Buster (
TB301: phantom-symbol):- Cross-file AST symbol resolver yang memverifikasi apakah impor relatif (
./,../) benar-benar ada dan simbolnya benar-benar diekspor oleh berkas target.
- Cross-file AST symbol resolver yang memverifikasi apakah impor relatif (
- πΊοΈ Blast Radius & Architecture Visualizer:
- Pemetaan graph dependensi 2-tingkat (Direct Callers: depth 1, Indirect Callers: depth 2) melintasi repositori dalam waktu < 5 ms.
- Penilaian skor risiko kuantitatif (0β100) dan diagram alur visual Mermaid
graph TDotomatis dirender di PR Summary Sticky Comment.
- π οΈ Deterministic Auto-Fix Engine:
- Menerapkan perbaikan AST secara instan (
--fix) tanpa drift nomor baris. - Menghasilkan blok
suggestioninteraktif di GitHub Review sehingga reviewer/author dapat menerapkan perbaikan dengan 1 klik.
- Menerapkan perbaikan AST secara instan (
- π€ Autonomous M2M Bot Dialogue:
- Berinteraksi otonom dengan Dependabot dan Renovate: otomatis menerbitkan
@dependabot squash and mergejika bersih, atau@dependabot recreatejika rusak.
- Berinteraksi otonom dengan Dependabot dan Renovate: otomatis menerbitkan
- π Cryptographic SHA-256 Audit Ledger:
- Menjaga jejak audit rantai hash anti-pemalsuan untuk kebutuhan audit kepatuhan dan keamanan enterprise.
tokenectomy-bot initMembuat berkas tokenectomy.json default.
# Review terhadap base branch
tokenectomy-bot review --base origin/main --head HEAD
# Review berkas diff langsung
tokenectomy-bot review --diff-file my_change.diff --format text
# Integrasi CI dengan GitHub Annotations
tokenectomy-bot review --base origin/main --format github --fail-on errorRun as a background MCP stdio server to enable autonomous pre-PR audits in Claude Desktop, Cursor, Antigravity, and Cline:
tokenectomy-bot mcpWrite domain-specific rules as Tree-sitter .scm queries in .tokenectomy/rules/:
;; @id: TB_CUSTOM_001
;; @name: no-hardcoded-secret
;; @severity: error
;; @message: Potential hardcoded secret or token assignment detected in source code
;; @fix_hint: Move credentials to secure environment variables
;; @languages: typescript, javascript
(variable_declarator
name: (identifier) @id (#match? @id "(?i)(api_?key|secret|token|password)")
value: (string) @val) @matchValidate and test queries against test fixtures:
tokenectomy-bot rules validate .tokenectomy/rules/no_hardcoded_secrets.scm --fixture src/tests/fixture.tsMaintain immutable compliance proof for every PR audit:
# Record sealed entry upon review
tokenectomy-bot review --base origin/main --ledger .tokenectomy/audit-ledger.jsonl
# Verify cryptographic hash chain integrity (detects historical tampering)
tokenectomy-bot ledger verify .tokenectomy/audit-ledger.jsonl
# View executive metrics and compliance dashboard
tokenectomy-bot ledger metrics .tokenectomy/audit-ledger.jsonlEnforce uncompromisable rule baselines across repositories:
tokenectomy-bot review --org-policy /etc/tokenectomy/org-policy.jsonRepos cannot downgrade or disable mandated rules without explicit approved exception (--allow-policy-downgrade).
Run as an autonomous organization-level GitHub webhook listener without per-repo workflows:
tokenectomy-bot serve --port 8080 --secret "$GITHUB_WEBHOOK_SECRET" --ledger /var/log/audit.jsonlNotify engineering and security teams upon PR evaluation:
tokenectomy-bot review --base origin/main --webhook-url "https://hooks.slack.com/services/xxx"Supports Slack, Discord, and Microsoft Teams.
Tmy-Joy interacts directly in GitHub PR discussions, conversational threads, and machine-to-machine handshakes with other GitHub bots:
# Answer PR comments or questions locally or via GitHub Actions
tokenectomy-bot chat --author "dependabot[bot]" --comment "Bumps lodash" --gate-status passed
tokenectomy-bot chat --author "alice" --comment "/explain TB001"@tmy-joy statusor/status: Returns live deterministic gate status, error count, and ledger proof.@tmy-joy explain <RULE_ID>: Provides detailed rule rationale, security risks, good/bad code examples, and remediation steps.@tmy-joy rulesor/rules: Displays full active AST detection rules catalog.@tmy-joy ping: Health check, response latency, and Tree-sitter engine status.
- Dependabot (
@dependabot): Evaluates AST integrity. If passed, autonomously issues@dependabot squash and merge. If blocked by test tampering or security flaws, issues@dependabot recreate. - Renovate (
@renovate): Issues@renovate mergeon pass or@renovate rebaseon block. - CodeRabbit (
@coderabbitai): Correlates AI suggestions with Tree-sitter AST determinism (0% hallucination rate).
Gunakan komentar inline jika blok sengaja dilewati secara sah:
// tokenectomy-ignore: TB101 -- disengaja untuk fallback
try {
loadOptionalConfig();
} catch (e) {}Rigorously audited with a standalone 1,000-case stress test suite (stress_1k_benchmark):
- β‘ Speed: Processes 1,000 PR diff files in ~0.70 seconds (~1,428+ files/sec).
- π― Precision: 100.00% precision gate (0 false positives, 0 false negatives).
- π‘οΈ Stability: Zero panics or crashes across adversarial payloads, unicode emojis, binary markers, and extreme line numbers.
- π Languages: Native Tree-sitter parsing for TypeScript, JavaScript, Rust, Go, and Python.
For complete reproduction instructions and audit tables, see BENCHMARK.md.
tokenectomy-bot/
βββ crates/
β βββ tb-diff/ # Ekstraktor dan parser Unified Git Diff 2-sisi
β βββ tb-parse/ # Parser Tree-sitter & pemetaan semantic range overlap
β βββ tb-rules/ # Mesin aturan, model Finding, dan fingerprint stabil
β βββ tb-report/ # Reporter Text, JSON, SARIF 2.1.0, & GitHub Annotations
β βββ tb-cli/ # Binary executable CLI (tokenectomy-bot)
βββ action.yml # GitHub Composite Action
βββ ROADMAP.md # Rencana pengembangan v2
Lisensi: MIT