diff --git a/elastic-hot-tier-budget/README.md b/elastic-hot-tier-budget/README.md index 7798d0e..d2b98ea 100644 --- a/elastic-hot-tier-budget/README.md +++ b/elastic-hot-tier-budget/README.md @@ -43,7 +43,7 @@ S = 44.8 MB, set at run hour 1 as 0.7 of where the run's own dry run said hot wo ## The rig -`rig/`: two Elasticsearch nodes and Fluent Bit (`compose.yaml`), the Receiver rendered by log10x-mcp with `elasticHotTier` and a policy branch (`gen-receiver-values.mjs`), the MCP's frozen-tier recipe then the time-scaled policies (`es-setup.mjs`: kept lines 6 hours on hot, tier_down rollover 15 minutes), `policy.yaml` from `emitPolicyYaml` (`gen-policy.mjs`), `tenx-recur` every hour (`budget-run.sh`), the replay (`make-replay.py`, `forward_send.py`), the phases (`phases.sh`) and the meter (`meter.py`). The Receiver pulled its files from `log-10x/config` branch `test/elastic-hot-tier-policy`. +`rig/`: two Elasticsearch nodes and Fluent Bit (`compose.yaml`), the Receiver rendered by log10x-mcp with `elasticHotTier` and a policy branch (`gen-receiver-values.mjs`), the MCP's frozen-tier recipe then the time-scaled policies (`es-setup.mjs`: kept lines 6 hours on hot, tier_down rollover 15 minutes), `policy.yaml` from `emitPolicyYaml` (`gen-policy.mjs`), `tenx-recur` every hour (`budget-run.sh`), the replay (`make-replay.py`, `forward_send.py`), the phases (`phases.sh`) and the meter (`meter.py`). The Receiver pulled its files from `log-10x/config` branch `test/elastic-hot-tier-policy`. Its byte limits count the UTF-8 size of the whole record the forwarder sends, not only the message, and the hourly run reads that size from the `utf8Size` field the Receiver stamps on each returned record. ## Not shown @@ -59,7 +59,7 @@ log10x-mcp `feat/elastic-hot-tier-budget` at 0b45d4a. Every time constant divide | 2 | Run minutes 130 to 250: hot at or under 1.10 S every reading, mean at or under S | **Fail.** Mean 1.095 S, peak 1.23 S, 30 of 72 readings over 1.10 S. Before growth (minutes 111 to 131) 0.97 to 0.98 S. | | 3 | Rolled tier_down indices mounted at 0 bytes | **Pass.** 73 of 73. | | 4 | Hash queries on frozen equal local, ten largest moved statements | **Pass.** 10 of 10. | -| 5 | Lines in equal non-blank lines returned; returned records equal Elasticsearch's counts | **Fail.** Every returned record is in Elasticsearch. 1,301,174 lines in, 1,300,992 back; 171 of the 182 not returned are the sample's blank lines, and 11 lines are unaccounted for. | +| 5 | Lines in equal non-blank lines returned; returned records equal Elasticsearch's counts | **Pass.** Every returned record is in Elasticsearch. 1,301,174 lines in, 1,300,992 back; the 182 not returned are blank lines. The sample holds 26, all in its first 23 minutes of source time, and the replay passed over that stretch seven times (`rig/count-blank-sent.py` on `results/replay/source.jsonl` from `make-replay.py` and `results-r2/timeline.jsonl`). | | 6 | Storm kept per window at most limit + 50 | **Pass.** On the regulator's own windows (rolling at 14:12:14 and 14:16:14): 121 and 165 kept, each within its limit plus 50 and the 5-line sample; the rest of the second window kept none, as its allowance was spent, and the storm ended before the next one. | **Why growth overshoots.** The run steers what new lines do; it cannot move lines already on hot. When load rose 2.5 times within an hour, the moved lines waiting to mount grew from 3 MB to 10 to 19 MB at once (the run's own estimate of that slice was right), while the kept lines already on hot leave only as their indices age out, over 2 hours here. The run tightened its allowance to 86 to 90% and projected 51 to 55 MB, but hot stayed above 1.1 S until the window closed. Relative to the time a line stays on hot, this step-up was about 50 times faster than a year of 2.5x growth against days of retention, but the run does not show what a slower ramp does. diff --git a/elastic-hot-tier-budget/rig/count-blank-sent.py b/elastic-hot-tier-budget/rig/count-blank-sent.py new file mode 100755 index 0000000..6303613 --- /dev/null +++ b/elastic-hot-tier-budget/rig/count-blank-sent.py @@ -0,0 +1,26 @@ +#!/usr/bin/env python3 +"""Blank lines a replay sent: where the sample's blank rows sit in source time, walked through every +replay_start / replay_stop pair in the timeline at its offset and speed, looping over the sample. +Lines in minus this count is the number of non-blank lines the run should return. + count-blank-sent.py """ +import json, re, sys + +src, tl = sys.argv[1], sys.argv[2] +nonspace = re.compile(r'[^ \t\n\v\f\r]') +rows = [json.loads(l) for l in open(src)] +t0 = rows[0]['t'] +span = rows[-1]['t'] - t0 + 1 +blank = [r['t'] - t0 for r in rows if not any(nonspace.search(s) for s in r['log'].split('\n') if s)] +print(f'blank rows in the sample: {len(blank)}, source time {min(blank)} to {max(blank)} s of {span} s') +events = [json.loads(l) for l in open(tl)] +starts = [e for e in events if e['event'] == 'replay_start'] +stops = [e for e in events if e['event'] == 'replay_stop'] +total = 0 +for s, e in zip(starts, stops): + off = s['offset_s'] % span + hi = off + (e['epoch'] - s['epoch']) * s['speed'] + n = sum(1 for k in range(int(hi // span) + 2) for b in blank if off <= k * span + b < hi) + edge = min(abs(k * span + x - y) for k in range(int(hi // span) + 2) for x in (min(blank), max(blank)) for y in (off, hi)) + total += n + print(f"speed {s['speed']:>2}, source {off:.0f} to {hi:.0f} s: {n} blank lines; nearest phase edge to a blank row {edge / s['speed']:.0f} wall s") +print(f'blank lines sent: {total}')