Repository navigation
perf(bluecell): precompile rule patterns and conditions once - #21
Open
PurpleCHOIms wants to merge 2 commits into
Open
PurpleCHOIms wants to merge 2 commits into
PurpleCHOIms wants to merge 2 commits into
Conversation
added 2 commits
October 27, 2026 16:39
RuleMatcher recompiled every regex via _compile_pattern and re-parsed + eval'd every condition on each tap event. Hoist both out of the per-event path: patterns compile once (memoized _compile_pattern) and conditions compile once into a reusable evaluator (_compile_condition), built at RuleMatcher construction. match() does no re.compile/eval work in its hot loop. Behavior preserved (public API and match semantics unchanged); add characterization tests covering single/multi selection, re: patterns, and/or/not/parentheses, empty + unknown-selection conditions, and that the hot loop performs zero compilation. ~2x on the 10-rule sample, scaling with rule count x patterns-per-rule.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
RuleMatcherrecompiled every regex (_compile_pattern) and re-parsed +eval'd every condition on each tap event. During active engagements the Blue Cell tap streams one event per sandbox/target log line, so this recompilation ran on the hot path at line rate.This hoists both out of the per-event path:
lru_cacheon_compile_pattern._compile_condition), built atRuleMatcherconstruction._evaluate_conditionnow delegates to it (single source of truth, no logic duplication).match()performs nore.compile/evalin its hot loop.Behavior
Unchanged. Public API (
RuleMatcher,load_rules,DetectionRule/DetectionEvent,score_mttd) and match semantics are preserved — includingre:regex vs. literal patterns,and/or/not/parentheses conditions, the empty-condition "all selections must match" rule, and the "unknown selection name → never fires" rule.Tests
New
tests/unit/bluecell/test_rule_match.pycharacterizes the matcher (single/multi selection,re:patterns, boolean conditions, empty + unknown-selection cases) and asserts the hot loop does zero compilation. Fullpackages/autobot/tests/unit/suite passes (259 tests).Performance
~2x on the 10-rule
sample_rules.jsonl(measured, same machine: 40.6 → 20.5 µs/event over 20k events). The gain scales with rule count × patterns-per-rule, so larger Sigma-style rule packs benefit more.Changelog
Recorded under Unreleased in
CHAGELOG.md. No package version or release implied.🤖 Generated with Claude Code