Skip to content

perf(detection): charge the scan budget per pass and cut monitor overhead - #173

Merged
rennf93 merged 4 commits into
masterfrom
perf/detection-cpu
Oct 16, 2026
Merged

rennf93 merged 4 commits into
masterfrom
perf/detection-cpu

Conversation

@rennf93

@rennf93 rennf93 commented Oct 16, 2026

Copy link
Copy Markdown
Member

Closes #172

The scan time budget now charges CPU once per pass instead of once per pattern: one thread CPU reading when a pass opens, a charge at each 2 ms slice boundary and around every await that can suspend (so other tasks' CPU is never charged), and a final charge when the pass closes. The 2 ms slice is request-scoped, so it spans values and the loop yields every 2 ms of scanning instead of almost never. The fallback scan also charges its inline CPU now. The performance monitor skips sanitizing anomalies when no callback is registered and computes the deviation in one C pass (math.dist), with results identical to 6 decimals, including the evt_pattern_anomaly_statistical case. Two pure moves (pattern source redaction and the scan window tables into their own modules) keep the maintainability rank at A, and the slice yield moved into a helper to keep the pattern loop at complexity rank B.

On the same request I measured about 130 ms of CPU per request, against about 148 ms on 4.3.1 and about 209 ms on 4.4.0. Fifty concurrent requests finish in 6.7 s (4.3.1: 7.5 s, 4.4.0: 13.0 s), and the worst event-loop stall is 26 ms (4.3.1: 7.5 s, 4.4.0: 86 ms). Every verdict, category and score over the 738 production corpus cases is identical to 4.4.0, sequential and concurrent runs agree, and with a 0.1 s budget 300 concurrent benign requests next to CPU-burning tasks are never flagged.

@github-actions

Copy link
Copy Markdown
Contributor

Detection gate report

All detection gate shards passed.

Shard arbiter-grammar

Gate green: no category regressed against its committed baseline and no benign string is newly flagged.

Total recall: n/a. Total false-positive rate: n/a.
Documented known gaps: 0. Documented known false positives: 0. Both stay in the denominators.

Full per-case output is in the job log.

Shard benchmarks-and-rest

Gate green: no category regressed against its committed baseline and no benign string is newly flagged.

Total recall: 378/386 (97.9%). Total false-positive rate: 23/370 (6.2%).
Documented known gaps: 8. Documented known false positives: 23. Both stay in the denominators.

Category Recall False positives
cmd_injection 38/46 (82.6%) 10/370 (2.7%)
cms_probing 36/36 (100.0%) 0/370 (0.0%)
code_injection 3/3 (100.0%) 0/370 (0.0%)
deserialization 18/18 (100.0%) 0/370 (0.0%)
dir_traversal 10/10 (100.0%) 0/370 (0.0%)
file_inclusion 16/16 (100.0%) 3/370 (0.8%)
file_upload 32/32 (100.0%) 0/370 (0.0%)
http_split 4/4 (100.0%) 0/370 (0.0%)
ldap 25/25 (100.0%) 0/370 (0.0%)
nosql 10/10 (100.0%) 0/370 (0.0%)
path_traversal 13/13 (100.0%) 0/370 (0.0%)
proto_pollution 8/8 (100.0%) 0/370 (0.0%)
recon 23/23 (100.0%) 0/370 (0.0%)
sensitive_file 16/16 (100.0%) 0/370 (0.0%)
sqli 38/38 (100.0%) 6/370 (1.6%)
ssrf 32/32 (100.0%) 4/370 (1.1%)
template 13/13 (100.0%) 1/370 (0.3%)
xml 4/4 (100.0%) 0/370 (0.0%)
xss 39/39 (100.0%) 0/370 (0.0%)

Full per-case output is in the job log.

Shard builtin-safety

Gate green: no category regressed against its committed baseline and no benign string is newly flagged.

Total recall: n/a. Total false-positive rate: n/a.
Documented known gaps: 0. Documented known false positives: 0. Both stay in the denominators.

Full per-case output is in the job log.

Shard cpu-load-arbiter-grammar

Gate green: no category regressed against its committed baseline and no benign string is newly flagged.

Total recall: n/a. Total false-positive rate: n/a.
Documented known gaps: 0. Documented known false positives: 0. Both stay in the denominators.

Full per-case output is in the job log.

Shard cpu-load-benchmarks-and-rest

Gate green: no category regressed against its committed baseline and no benign string is newly flagged.

Total recall: 378/386 (97.9%). Total false-positive rate: 23/370 (6.2%).
Documented known gaps: 8. Documented known false positives: 23. Both stay in the denominators.

Category Recall False positives
cmd_injection 38/46 (82.6%) 10/370 (2.7%)
cms_probing 36/36 (100.0%) 0/370 (0.0%)
code_injection 3/3 (100.0%) 0/370 (0.0%)
deserialization 18/18 (100.0%) 0/370 (0.0%)
dir_traversal 10/10 (100.0%) 0/370 (0.0%)
file_inclusion 16/16 (100.0%) 3/370 (0.8%)
file_upload 32/32 (100.0%) 0/370 (0.0%)
http_split 4/4 (100.0%) 0/370 (0.0%)
ldap 25/25 (100.0%) 0/370 (0.0%)
nosql 10/10 (100.0%) 0/370 (0.0%)
path_traversal 13/13 (100.0%) 0/370 (0.0%)
proto_pollution 8/8 (100.0%) 0/370 (0.0%)
recon 23/23 (100.0%) 0/370 (0.0%)
sensitive_file 16/16 (100.0%) 0/370 (0.0%)
sqli 38/38 (100.0%) 6/370 (1.6%)
ssrf 32/32 (100.0%) 4/370 (1.1%)
template 13/13 (100.0%) 1/370 (0.3%)
xml 4/4 (100.0%) 0/370 (0.0%)
xss 39/39 (100.0%) 0/370 (0.0%)

Full per-case output is in the job log.

Shard cpu-load-builtin-safety

Gate green: no category regressed against its committed baseline and no benign string is newly flagged.

Total recall: n/a. Total false-positive rate: n/a.
Documented known gaps: 0. Documented known false positives: 0. Both stay in the denominators.

Full per-case output is in the job log.

Shard hardening-canaries

Gate green: no category regressed against its committed baseline and no benign string is newly flagged.

Total recall: n/a. Total false-positive rate: n/a.
Documented known gaps: 0. Documented known false positives: 0. Both stay in the denominators.

Full per-case output is in the job log.

@rennf93
rennf93 merged commit 64e153f into master Oct 16, 2026
40 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: detection-engine Touches guard_core/detection_engine/ area: handlers Touches guard_core/handlers/ area: sync Touches guard_core/sync/ (sync mirror generated by unasync) tests Test suite changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Detection costs more CPU per request in 4.4.0 than in 4.3.x

1 participant