Skip to content

intent, naming-stability: reconcile the two pattern counts - #69

Merged
talwgx merged 1 commit into
mainfrom
docs/e99-pattern-counts
Oct 25, 2026
Merged

talwgx merged 1 commit into
mainfrom
docs/e99-pattern-counts

Conversation

@talwgx

@talwgx talwgx commented Oct 25, 2026

Copy link
Copy Markdown
Contributor

Both READMEs ended "the difference is not reconciled here" for the two 1.1.142 pattern counts: intent's 2,277 (otel215) and 406 (rec1155), counted over labelled events, against naming-stability's 2,687 and 582, counted over every event. This PR states the reconciliation and adds the script that computes it.

Result

capture events unlabelled patterns, labelled events patterns only in unlabelled events patterns, every event events named differently by the two runs
otel215 157,228 665 2,277 410 2,687 0
rec1155 41,573 240 406 176 582 0
  • Both benchmarks count the same unit: distinct message_pattern values, one per event.
  • One run with each benchmark's engine config (intent/bench/engine.sh adds symbolOriginField tenx_origin) cut each capture into the same events. Both pass the text-hash alignment against the labels, and they give every event the same name.
  • On otel215 the 665 unlabelled events take 411 patterns. One of them, Starting (2 metrics-server events), also names a labelled event (frontend:banner), so the unlabelled events add 410. On rec1155 no pattern is shared, so they add 176.
  • The largest contributor is frontend-proxy, which has no labelled events and gives each unlabelled event a pattern of its own: 114 on otel215 and 115 on rec1155. 103 of each come from Envoy's server.cc startup listing. Next come opensearch 83, jaeger 58 and prometheus-server 38 on otel215, and postgresql 23 and flagd 13 on rec1155.
  • intent/bench/score.py run on either new events file reproduces 2,277 and 406.

Changes

  • intent/bench/pattern_counts.py (new): checks both runs against the labelled capture and writes the counts. Its output is in intent/results/pattern-counts-1.1.142_{otel215,rec1155}.json.
  • intent/README.md line 50 and naming-stability/README.md lines 28 to 41: the "not reconciled" sentence is replaced with the reconciliation, in the same words in both. The intent Files list now names the script.
  • intent/METHOD.md lines 102 and 103: the unlabelled-by-container lists now match labels/labels_*.jsonl.gz. On rec1155, cart has 10 unlabelled events, not one; the old list summed to 231 of 240. On otel215 there are 11 smaller containers, not 13 (22 in all).

How the numbers were made

Image log10x/pipeline-10x:1.1.142@sha256:22ae6714…, with the hash-checked captures from naming-stability/run.sh. Each capture was run once with the arguments of naming-stability/bench/engine.sh and once with those of intent/bench/engine.sh, then:

python3 intent/bench/pattern_counts.py <capture> <intent run events.jsonl.gz> <naming-stability run events.jsonl.gz> --out intent/results/pattern-counts-1.1.142_<capture>.json

The engine outputs are not committed.

🤖 Generated with Claude Code

intent counts the patterns of labelled events (2,277 on otel215, 406 on
rec1155); naming-stability counts every event (2,687 and 582), both on the
1.1.142 image. bench/pattern_counts.py checks one run of each benchmark's
engine config against the labelled capture, event for event: the two runs
cut the same events and give every event the same name, and the difference
is the patterns only unlabelled events take (410 and 176). Both READMEs
state this. METHOD.md's unlabelled-by-container lines now match the labels
file (cart 10 on rec1155; 11 smaller containers on otel215).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@talwgx
talwgx merged commit ce7f971 into main Oct 25, 2026
15 checks passed
@talwgx
talwgx deleted the docs/e99-pattern-counts branch October 25, 2026 23:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant