Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .github/workflows/test-cli.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,3 +58,16 @@ jobs:

- name: Test
run: cargo test -p clickhousectl

# The live tests include the API library's harness by a path outside
# this crate, so the published crate must ship its sources only.
- name: Check packaged files
run: |
files=$(cargo package -p clickhousectl --list --locked)
for required in Cargo.toml LICENSE README.md src/main.rs; do
grep -qx "$required" <<<"$files" || { echo "missing from package: $required"; exit 1; }
done
if grep -v -e '^src/' -e '^Cargo\.' -e '^\.cargo_vcs_info\.json$' -e '^LICENSE$' -e '^README\.md$' <<<"$files"; then
echo "unexpected files in package (listed above)"
exit 1
fi
3 changes: 3 additions & 0 deletions crates/clickhousectl/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@ homepage.workspace = true
keywords = ["clickhouse", "cli", "database", "cloud"]
categories = ["command-line-utilities", "database"]
readme = "../../README.md"
# Ship the CLI sources only: the tests stay in the repository, and the live
# tests reach into the API library's test harness, outside this crate.
include = ["src/**", "LICENSE"]

[package.metadata.binstall]
pkg-url = "https://builds.clickhouse.com/clickhousectl/{ name }-{ target }-v{ version }{ archive-suffix }"
Expand Down
2 changes: 1 addition & 1 deletion docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -194,7 +194,7 @@ The suites load AWS credentials through the default provider chain: environment

All require `CLICKHOUSE_CLOUD_API_KEY`, `CLICKHOUSE_CLOUD_API_SECRET`, `CLICKHOUSE_CLOUD_TEST_ORG_ID`, `CLICKHOUSE_CLOUD_TEST_PROVIDER`, and `CLICKHOUSE_CLOUD_TEST_REGION` in the environment, and are wired into the scheduled `Cloud Integration` GitHub Actions workflow. The ClickPipes E2E suites additionally need AWS credentials and an `eu-west-1` region quota; `clickpipe_smoke_test` reads a pre-provisioned service ID from `CLICKHOUSE_CLOUD_TEST_CLICKPIPE_SERVICE_ID`.

Two live tests drive the real `clickhousectl` binary, so they live in the CLI crate as `crates/clickhousectl/tests/live_*_test.rs`. Cargo builds the binary for them, and they share the library's harness in `crates/clickhouse-cloud-api/tests/common/` through `#[path]`. They need the same Cloud environment variables, are `#[ignore]`d, and assert JSON error codes, exit codes and Cloud API state rather than CLI wording:
Two live tests drive the real `clickhousectl` binary, so they live in the CLI crate as `crates/clickhousectl/tests/live_*_test.rs`. Cargo builds the binary for them, and they share the library's harness in `crates/clickhouse-cloud-api/tests/common/` through `#[path]`. They need the same Cloud environment variables, are `#[ignore]`d, and assert JSON error codes, exit codes and Cloud API state rather than CLI wording. The published `clickhousectl` crate ships only `src/**` (plus `LICENSE` and the root README), because those `#[path]` includes point outside the crate. The CLI sources each test drives also select its live suite: `services/query.rs` and `services/query_key.rs` select `service`, and the `clickpipe create postgres` path (`clickpipes/create.rs`, `destination.rs`, `table_mapping.rs`, `sources/mod.rs`, `sources/postgres.rs`) selects `clickpipes`:

```bash
cargo test -p clickhousectl --test live_service_query_bind_test -- --ignored --nocapture # service query first-use bind
Expand Down
16 changes: 16 additions & 0 deletions scripts/classify-cloud-integration.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,22 @@
"crates/clickhousectl/src/cloud/query_api_endpoints.rs": NO_SUITES,
"crates/clickhousectl/src/cloud/saved_queries.rs": NO_SUITES,
"crates/clickhousectl/src/cloud/services/jwt_provider.rs": NO_SUITES,
# `live_clickpipe_postgres_cdc_test` drives `cloud clickpipe create
# postgres` live: the Postgres source, its shared auth helpers, and the
# create, destination and table-mapping builders that request uses.
"crates/clickhousectl/src/cloud/clickpipes/create.rs": frozenset({"clickpipes"}),
"crates/clickhousectl/src/cloud/clickpipes/destination.rs": frozenset(
{"clickpipes"}
),
"crates/clickhousectl/src/cloud/clickpipes/sources/mod.rs": frozenset(
{"clickpipes"}
),
"crates/clickhousectl/src/cloud/clickpipes/sources/postgres.rs": frozenset(
{"clickpipes"}
),
"crates/clickhousectl/src/cloud/clickpipes/table_mapping.rs": frozenset(
{"clickpipes"}
),
# `live_service_query_bind_test` drives `cloud service query` live.
"crates/clickhousectl/src/cloud/services/query.rs": frozenset({"service"}),
"crates/clickhousectl/src/cloud/services/query_key.rs": frozenset({"service"}),
Expand Down
21 changes: 21 additions & 0 deletions scripts/tests/test_classify_cloud_integration.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,13 @@ def test_path_mapping(self):
"crates/clickhouse-cloud-api/tests/service_query_key_cli_test.rs": frozenset(
{"service"}
),
"crates/clickhousectl/src/cloud/clickpipes/create.rs": frozenset(
{"clickpipes"}
),
"crates/clickhousectl/src/cloud/clickpipes/sources/postgres.rs": frozenset(
{"clickpipes"}
),
"crates/clickhousectl/src/cloud/clickpipes/manage.rs": classifier.NO_SUITES,
"Cargo.lock": classifier.ALL_SUITES,
"crates/clickhousectl/Cargo.toml": classifier.ALL_SUITES,
".cargo/config.toml": classifier.ALL_SUITES,
Expand Down Expand Up @@ -192,6 +199,20 @@ def test_udf_paths_select_service_lifecycle(self):
self.assertEqual(selection.suites, ("service",))
self.assertFalse(selection.failed_closed)

def test_live_cli_clickpipe_create_postgres_paths_select_clickpipes(self):
for path in (
"crates/clickhousectl/src/cloud/clickpipes/create.rs",
"crates/clickhousectl/src/cloud/clickpipes/destination.rs",
"crates/clickhousectl/src/cloud/clickpipes/sources/mod.rs",
"crates/clickhousectl/src/cloud/clickpipes/sources/postgres.rs",
"crates/clickhousectl/src/cloud/clickpipes/table_mapping.rs",
):
with self.subTest(path=path):
self.assertTrue((classifier.REPO_ROOT / path).is_file())
selection = classifier.select_records([("M", (path,))])
self.assertEqual(selection.suites, ("clickpipes",))
self.assertFalse(selection.failed_closed)

def test_every_current_cloud_api_test_file_has_an_explicit_mapping(self):
test_root = (
classifier.REPO_ROOT / "crates" / "clickhouse-cloud-api" / "tests"
Expand Down
Loading