ClickHouse Cloud elasticity: compute against rows written, four pairs

README.md

@@ -32,6 +32,7 @@ rerun it and check every figure a post cites.
 | [`clickstack-e2e-gaps/`](clickstack-e2e-gaps/) | The six questions the end to end run left open, each over the whole capture: what a day predicate prunes across thirty days of objects, whether Vector can write the cold branch, whether the Retriever indexes and queries that layout, what a container killed mid-stream costs line by line, which dashboard answers change when rows leave the hot table, and how the per-type policy compares with a collector-only policy on insert and merge CPU. Plus a second pass on four of them: Vector 0.58.0 writing Parquet beside JSON with the Parquet reader's own counters, a `GROUP BY` and an `ORDER BY ... LIMIT` through the Merge table, the collector configured as its code owners prescribe, the `TTL TO VOLUME 'cold'` alternative costed on the same capture, and ClickHouse issue 116888 demonstrated with its guard. | Current. Not behind a post. |
 | [`clickstack-rows-gate/`](clickstack-rows-gate/) | Which ClickHouse system table counts the rows written into the logs table to the row, with the Receiver regulating in lines with a service total: `system.part_log` NewPart rows equals the hot count in both runs, `system.query_log` written_rows drifts with the exporter's async-insert entries; the totals held to the row, a policy rewrite mid-window reset nothing, and the events unit takes effect only from the config or an engine option. | Current. The meter for the ClickHouse rows run. Not behind a post. |
 | [`clickstack-rows/`](clickstack-rows/) | The ClickHouse rows run, metered: rows written into ClickStack held at a rate an hour while load grew 2.5x with a storm and a new service, read on ClickHouse's own `system.part_log`. FAIL as registered: the rate held at 0.99 to 1.04 of R in every complete 10-minute window but one, the floor has no allowance in the 10-minute test, one service's 1,422 new statements in a window passed their always-pass sample, and 33 rows were inserted twice. Rerun with the regulator's sample bound (config `baselineMaxOverTotal` 50): every 4-minute window under its bound, 10-minute windows 0.97 to 1.07 of R/6, every returned record stored once; fails as registered on the 10-minute sum alone. | Three runs. The first failed as registered (one 10-minute window at 1.36, 33 duplicates). With the sample bound, the rate held at 0.97 to 1.07 of R in every complete 10-minute window, none over, no duplicates. Beside a collector rule under the same 2.5x step, the hold read 0.91 to 1.13 (one window over the 1.10 bound, the storm's first) against the rule's 2.95 to 3.42; insert-plus-merge CPU did not separate the arms on this rig. |
+| [`clickhouse-cloud-elasticity/`](clickhouse-cloud-elasticity/) | How far does a ClickHouse Cloud service's compute fall, in units and in billed dollars, when the rows written into its logs table fall, above the smallest size, and how long does the autoscaler take? One service in AWS us-east-1, the ClickStack `otel_logs` schema under a fixed query load, two arms that differ only in the rows written (all of the replay, or the 65.3% the Receiver kept), four pairs, read on the replicas' own system tables and on `usageCost`. | Current. Not behind a post. From a 4-unit start the cut took the service from 4 units to 3, 0.75 in units and 0.749 in billed compute, in each of three pairs, 2.9 to 4.3 h after the cut; from a 3-unit start it read no change in 10.25 h. |
 | [`splunk-license/`](splunk-license/) | Splunk bills on what its licence meter counts, not on a file size. What does the 10x Receiver in front of Splunk do to that number? The same capture through the same Universal Forwarder into the same instance, twice, raw and compact, measured from `license_usage.log` and from the Monitoring Console's own searches, with every compact event read back through the 10x Splunk app and compared to the text that went in. | Current. Not behind a post. |
 | [`datadog-flex-accounting/`](datadog-flex-accounting/) | Datadog bills Standard indexing per million events. When the 10x Receiver moves events from Standard to Flex Logs under the plan the 10x MCP writes, how many events and GB move, and what do they cost at Datadog's list prices? The released engine and MCP on one capture at three targets, counted two independent ways, priced per million events moved, with the custom metrics the Datadog metric output creates. Computed from the engine's counts at list price; no Datadog account needed. | Current. Not behind a post. |
 | [`datadog-service-control/`](datadog-service-control/) | Inside one service a Datadog service filter cannot split the volume, so the rules a person writes are filters on the service's logger names. On the kafka container of two public captures of the same code, what does the per-pattern plan the 10x MCP writes move against the fewest `@logger.name` filters that move as many events, and what does each do on the second capture with nothing re-tuned? Computed from the engine's counts; no Datadog account needed. | Current. Not behind a post. |
@@ -62,6 +63,7 @@ each resolves its paths relative to itself, so a benchmark can be run from anywh
 | `clickhouse-clickstack/` | `./run.sh` | pinned engine image digest and a ClickHouse Docker image | `results/` | the `otel-sample-v2` release asset from `log-10x/config`, fetched into `data/` on first run |
 | `clickstack-e2e/` | `./run.sh` | pinned ClickStack, engine, collector and MinIO images | `results/` | the `otel-sample-v2` release asset from `log-10x/config`, fetched into `data/` on first run |
 | `clickstack-e2e-gaps/` | `./gap<N>_*.sh` | the same images by digest, plus Vector by digest; the receiver is the published `edge-10x` image | `results/` | the same `otel-sample-v2` asset, read from `../clickstack-e2e/data` |
+| `clickhouse-cloud-elasticity/` | `cloud/arms.py`, after the split, stage and meters in its README | the engine and collector images by digest (`results/images.txt`), Python 3.12+, a `clickhouse` client binary | `results/` | the `otel-sample-200mb.log` asset from `log-10x/config` tag `otel-sample-v1`, checked by sha256; a ClickHouse Cloud organization |
 | `splunk-license/` | `./run.sh` | pinned engine, Splunk and forwarder images, and the 10x Splunk app | `results/` | the `otel-sample-v2` release asset from `log-10x/config`, fetched into `data/` on first run |
 | `datadog-flex-accounting/` | `./run.sh` | pinned engine image digest, log10x-mcp from npm at a pinned version, and the shared scripts in `accounting-lib/` | `results/` | the `otel-sample-200mb.log` asset from `log-10x/config` tag `otel-sample-v1`, checked by sha256 |
 | `datadog-ceiling/` | `rig/run.sh` phases, `rig/budget-phase.sh` | the engine image by digest, log10x-mcp 1.30.71 from npm, the demo at tag 3.1.0 | `results/` | a Datadog org with Flex Logs; the demo's own logs |

clickhouse-cloud-elasticity/.gitignore

@@ -0,0 +1,5 @@
+data/
+build/
+split/build/
+split/build-*/
+__pycache__/

clickhouse-cloud-elasticity/README.md

@@ -0,0 +1,37 @@
+# ClickHouse Cloud elasticity
+
+One benchmark in the [log10x benchmarks](../) repository. It asks how far a ClickHouse Cloud service's compute falls, in units and in billed dollars, when the rows written into its logs table fall, above the smallest size, and how long the autoscaler takes to lower it.
+
+**Result: from a 4-unit start, writing 0.653 of the rows took the service from 4 units to 3, a ratio of 0.75, and billed compute to 0.749, in each of three pairs; the step down was done 2.9 to 4.3 hours after the cut. From a 3-unit start the same cut read no change in 10.25 hours.** At 3 units both arms sit inside the autoscaler's no-action band; at 4 units arm B sits below it. Every figure, with the file and field it was read from, is in [RESULT.md](RESULT.md).
+
+ClickHouse Cloud, AWS us-east-1, Scale tier, ClickHouse 26.6, 2 replicas, vertical autoscaling 8 to 32 GiB per replica, idle after 5 minutes. The ClickStack `otel_logs` schema under seven queries a minute. Arm A writes every record of the 215 MB OTel replay at 70,000 rows/s; arm B writes the 65.3% the Receiver (`log10x/edge-10x:1.1.153`) kept, at 45,716 rows/s. 2026-11-02 19:48 to 2026-11-05 03:47 UTC. The service was deleted at the end (`results/raw/teardown.jsonl`).
+
+| Pair | Arm A's size | Units, A to B | Ratio | usageCost computeCHC per 15 minutes, A to B | Cut to step down done |
+|---|---|---|---|---|---|
+| 1 | the autoscaler's | 4.0 to 2.997 | 0.749 | 0.298 to 0.224 | 4.33 h |
+| 2 | the autoscaler's, where pair 1 left it | 3.0 to 3.0 | 1.0 | 0.224 to 0.224 | no step in 10.25 h |
+| 3 | 16 GiB by config, released at the cut | 4.0 to 3.0 | 0.750 | 0.298 to 0.224 | 4.21 h |
+| 4 | 16 GiB by config, released at the cut | 4.0 to 3.0 | 0.750 | 0.298 to 0.224 | 2.91 h |
+
+## Rerun
+
+Needs a ClickHouse Cloud organization and API key, Docker, Python 3.12 or later and a `clickhouse` client binary (26.8 here). `cloud_api.py` reads the key from SSM or from `CHC_KEY_ID` and `CHC_KEY_SECRET` and never prints it. The scripts keep the service password and client config under `~/git/l1x-co/.claude-tmp/chce-secrets/` by default; `CHC_SECRET_FILE`, `CLIENT` and `CONF` move them.
+
+1. **Split.** `split/run_slices.sh` replays the `otel-sample-200mb.log` asset from `log-10x/config` tag `otel-sample-v1` (checked by sha256) through four Receivers side by side and joins what they returned into `split/build/wire/returned.json`, in about an hour.
+2. **Service.** Put the runner's address in `cloud/service.json`, then run `CHC_SECRET_FILE=<file> python3 cloud_api.py create cloud/service.json`. The service password goes to that file (mode 600), not to the terminal.
+3. **Tables.** Apply `cloud/schema.sql` with the client, then `cloud/stage.sh` and `cloud/stage_b.sh`.
+4. **Meters**, left running, each with `--service <id>` and an `--out` under `results/raw/`: `cloud/sample.py`, `cloud/meter.py --every 60 --lag 30` and `cloud/usage_poll.py`.
+5. **Feed and queries**, left running: `cloud/load.py` and `cloud/queries.py --overlap 3`. Both follow `results/raw/control.json`, which `cloud/phase.py` writes.
+6. **Probe.** `cloud/probe.py` with steps of 2,000 to 70,000 rows/s, 10 minutes each.
+7. **Arms.** `cloud/arms.py`. Its docstring gives both forms this run used: `--pairs` for the natural pairs, `--forced` for the pairs with arm A's size set by config, then the read calibration.
+8. **Report.** `cloud/arms_report.py --raw results/raw --lag-min 30` for every phase, segment and pair; `cloud/curve.py` for the probe. Both read the committed `.jsonl.gz` files as they are.
+
+## Files
+
+| Path | What |
+|---|---|
+| `RESULT.md` | Every figure, the file and field it came from, and what was not measured |
+| `split/` | The engine split |
+| `cloud/` | The service request, schema, stage, feed, queries, meters, driver and reports |
+| `results/` | The split and stage readings, the service creation response |
+| `results/raw/` | Every log the scripts wrote: `events.jsonl` (phases and size changes), `scaling-patches.jsonl`, `usage.jsonl`, `teardown.jsonl`, and the meter, feed, sampler and query logs gzipped |