chkit-py: full parity with TypeScript chkit + dual-language docs - #204
Open
Lucasgvdii wants to merge 50 commits into
Open
chkit-py: full parity with TypeScript chkit + dual-language docs#204Lucasgvdii wants to merge 50 commits into
Lucasgvdii wants to merge 50 commits into
Conversation
Under --json, whoami/logout/login/service-list/service-select printed a
JSON-encoded *string* (e.g. "Not logged in…") instead of an object: the
serializer JSON.stringify'd whatever it received, and those commands never
threaded jsonMode, so they passed plain strings. This broke the documented
agent contract — piping their --json output to jq failed because a bare
string isn't the {status,next}/{ok,error} envelope callers expect.
Fix in two deliberate layers:
- A catch-all in printOutput wraps any string in {schemaVersion, message}
while in --json mode. This closes the whole class of bug at its single
chokepoint, so no command can ever emit a bare string again — even ones
we don't special-case.
- Purpose-built envelopes for the two commands agents actually consume:
whoami (logged_in / not_logged_in / session_expired) and service list
(one object with a services[] array, instead of one JSON line per
service, which isn't valid single-JSON).
login/logout/alias are intentionally left to the catch-all: they aren't
part of the documented machine flow, so a dedicated envelope each would be
disproportionate plumbing for no chaining benefit.
BREAKING: whoami and service-list --json output changes shape from string
to object. This is the intended fix to the JSON contract.
…onnection With an ObsessionDB service selected, `chkit pull` printed "using service <name>" but then opened its own ClickHouse client from config.clickhouse (defaulting to localhost:8123) and failed with "connection refused". It discarded the executor the host already resolves and hands to every command — so pull could not introspect an ObsessionDB instance at all, and the "using service" line was misleading. Route pull through pluginContext.executor, the same executor that generate/migrate/status use. The host resolves it per command (the ObsessionDB remote executor when a service is selected, a direct ClickHouse executor otherwise), and the remote executor already implements exactly the methods pull needs (listSchemaObjects, listTableDetails, query). We only build a client from config.clickhouse when the host provides no executor. Chosen over "just fail clearly on an ObsessionDB target" because routing makes pull actually work against ObsessionDB — the headline onboarding target — rather than failing more politely. Custom introspectors still open their own raw-ClickHouse connection (and require the config block), and the genuine no-target case now errors with an actionable message instead of a silent localhost fallback.
An async data-load migration submits the query and polls queryStatus until a terminal state. The poll call had no error handling, so a single HTTP 524 (gateway timeout) on a *poll request* threw out of the loop and aborted the migration — even though the server-side INSERT kept running (we observed ~34.8M of 100M rows land after chkit had given up). A poll request timing out was wrongly conflated with the load itself failing. Wrap the poll in try/catch and treat a transient poll error as "keep polling", up to a bounded budget (MAX_TRANSIENT_POLL_ERRORS). Only a real ExceptionWhileProcessing status, or a submit-time rejection, is fatal. When the budget is exhausted we stop with an explicit message that the load may still be running and that re-running re-attaches via the deterministic query_id — never a silent abort and never an infinite loop. A bounded budget (rather than infinite tolerance) is deliberate: a genuinely dead endpoint must not hang forever, while normal gateway blips on a multi-minute load are absorbed. Note this path is only reached by migrations with an explicit `mode=async` operation (data loads); ordinary schema DDL generated by `generate` is synchronous and unaffected.
A single create-chkit run printed the "Next steps" block twice — once from create-chkit's own package-manager-aware printer, and once from the onboarding flow's printer, which additionally hardcoded `bunx`. So a user who passed --package-manager npm saw a duplicated block telling them to run `bunx chkit …`, the wrong runner. - Thread the resolved package manager into runOnboarding and derive the runner word (npx / pnpm dlx / yarn dlx / bunx) instead of hardcoding bunx. - runOnboarding already prints next-steps on every branch, so only call create-chkit's own printer when onboarding is skipped, removing the duplicate. Default to `npx` when the package manager is unknown: it is the most universal runner and matches the npm-first install instructions; an explicit --package-manager always overrides it.
The docs and the first-run hint tell users to install the agent skill with `npx skills add obsessiondb/chkit` — `skills` is a separate CLI, not a chkit subcommand. Users (and our own tutorial) naturally reached for `chkit skills add …`, which returned "Unknown command: skills". Add a thin `chkit skills` command that forwards its arguments to the external `skills` CLI (`npx skills <args>`) and passes through its exit code. It is intercepted early in dispatch, like `init`, because it needs no project config or executor. A proxy is chosen over a docs-only fix because it removes the foot-gun entirely — `chkit skills add …` now does the expected thing — while keeping `skills` as the real underlying tool. The spawner is injectable so the forwarding and exit-code passthrough are unit-tested without spawning a process.
… errors
In a non-TTY shell `chkit init` silently skipped onboarding and printed only
static next-steps, while `create-chkit` printed the full connect runbook —
an inconsistency that made it look like `init` had no connect step at all.
Separately, the dynamic import of the obsessiondb plugin was wrapped in a
bare `catch {}` that swallowed any failure, so an installed-but-broken plugin
degraded silently with no signal.
- Keep `--yes` as the silent path for CI/scripts, but otherwise always hand
off to runOnboarding, which self-gates on TTY: interactive menu when
attached, connect runbook when not — matching create-chkit.
- Only swallow a genuine "plugin not installed" (module-not-found of the
plugin package itself); any other import failure now propagates instead of
a false silent pass.
- Fix init's static next-steps to use `npx` rather than a hardcoded `bunx`.
The not-installed vs load-error distinction is the key decision: the plugin
is optional, so its absence must degrade gracefully — but a present plugin
that fails to load is a real problem the user needs to see.
`chkit obsessiondb logout` always printed "Logged out." even with no stored credentials, implying it had ended a session that never existed. Have clearCredentials return whether a credentials file actually existed, and let logout print "Logged out." vs "No active session." accordingly. Exit code stays 0 either way: logout is intentionally idempotent so scripts can call it unconditionally. The bug was the misleading message, not the exit behavior — so only the message changes.
One changeset per fix (mirrors the per-phase commits): structured --json envelopes, pull via the host executor, async-load poll resilience, create-chkit Next-steps dedup, the chkit skills proxy, init connect-runbook consistency, and the logout no-session message.
`runnerFor` (the package-manager → runner-word helper added with the create-chkit Next-steps fix) was exported but only used inside onboarding, which tooling flagged as an unused export. It has no external consumers, so drop the `export` rather than widen the public surface.
… client + expand top-level package re-exports httpx is required by chkit_plugin_obsessiondb's RFC 8628 device-code flow, OTP signup, and oRPC services/jobs/workbench clients (forthcoming commits). pytest-httpx mocks those calls in tests. The package re-export expansion surfaces SchemaLoaderError, ModuleLoadError, codec_raw, import_module_file, load_schema_definitions, is_synthesized_config_path, and SYNTHESIZED_CONFIG_PATH so user-facing config scripts can import them from the chkit root.
…odules - config_path: SYNTHESIZED_CONFIG_PATH sentinel + is_synthesized_config_path predicate for the obsessiondb credentials-only fallback config. - plugin_error: wrap_plugin_run shell mirroring TS wrapPluginRun; turns exceptions inside plugin command runs into JSON envelopes (--json) or text + exit code (2 for config error, 1 otherwise). - schema_loader: load_schema_definitions(globs, cwd) — glob-resolves user schema files, imports each via the ts_import loader, returns a canonicalized list of SchemaDefinition. - ts_import: import_module_file using compile() + exec() with a monotonic counter for the synthetic module name. Bypasses Python's mtime-keyed bytecode cache so consecutive in-test rewrites of the same schema file see the new content (Windows NTFS mtime granularity is ~10ms). Tests cover each module's surface + the bytecode-cache fix that motivated ts_import.
…, dependsOn camelCase) Exports added to chkit.core for the public surface used by plugin authors: key_clause helpers (split_top_level_comma, normalize_key_columns), SQL splitter helpers (split_sql_statements, extract_executable_statements), SQL normalizers (normalize_sql_fragment, normalize_engine). These were present internally but never re-exported. canonical.py: - canonicalize_definition now backfills empty primary_key from order_by, matching TS canonical.ts. Without this a snapshot written by the TS CLI (where the user omits primaryKey and TS substitutes orderBy) would appear to drift against a Python-written snapshot of the same schema. - _canonicalize_refresh emits the camelCase 'dependsOn' key (with by_alias=True on the inner TableRef dump) so cross-port snapshot JSON matches byte-for-byte.
…ion + DDL propagation polling ClickHouseClient (client.py): - execute / query / query_json / submit / query_status (system.processes + system.query_log polling for async migrations). - insert(table, rows, *, column_names, database) for bulk inserts. - list_databases / list_tables. - list_schema_objects() / list_table_details(databases) — exposed as methods that delegate to the standalone introspect helpers. - Module-level helpers: format_connection_error (auth vs network distinction via CH error codes 192/193/516), wrap_connection_error + ClickHouseConnectionError typed exception, is_unknown_database_error (CH code 81). create_table_parser.py: 8 parser functions (settings/ttl/engine/pk/orderBy/ partition/uniqueKey/projections) for system.tables.create_table_query. introspect.py: - list_schema_objects: enumerates non-system tables/views/MVs, skips _chkit_* journal tables. - list_table_details: joins system.tables + system.columns + system.data_skipping_indices into IntrospectedTable rows. - normalize_column_from_system_row / normalize_index_from_system_row with full skip-index variant coverage (minmax, set, bloom_filter, tokenbf_v1, ngrambf_v1). ddl_propagation.py: polls until DDL is visible across replicas. Operation- type-aware predicates: wait_for_table, wait_for_view, wait_for_column, wait_for_table_absent, wait_for_column_absent, wait_for_index, wait_for_index_absent, wait_for_projection, wait_for_projection_absent. 20 attempts × 500ms (~10s budget) matches TS p-retry defaults. Dispatcher routes alter_table_drop_column → column_absent, alter_table_add_index → index, alter_rename_table → table, etc. Tests cover the 8 parser clauses, all 5 skip-index variants, the introspection joins, the connection-error helpers, and every DDL propagation predicate + dispatcher route.
…t, get-context) Public plugin surface mirroring TS packages/cli/src/plugins.ts: Manifests + plugin: - ChxPluginManifest / ChxPluginManifestCompatibility / Cli - ChxPlugin (manifest + hooks + commands + options_schema + extend_commands) - LoadedPlugin (validated + options resolved) Hook contexts (10): - ChxOnInitContext, ChxOnCompleteContext - ChxOnConfigLoadedContext, ChxOnSchemaLoadedContext, ChxOnPlanCreatedContext - ChxOnBeforeApplyContext, ChxOnAfterApplyContext - ChxOnCheckContext, ChxOnCheckResult, ChxOnCheckReportContext, ChxCheckFinding - ChxOnBeforePluginCommandContext + ChxOnBeforePluginCommandHandled / Unhandled / Result (used by obsessiondb to route backfill status/cancel/ list through its jobs API before the local backfill plugin's stubs run). - ChxOnPullIntrospectContext: lets plugins inject SchemaDefinition lists, bypassing the SQL-based pull (obsessiondb metadata API use case). Commands + context: - ChxPluginCommand (name + run + description + flags) - ChxPluginCommandContext (config + flags + options + table_scope + plugin_runtime + plugin_context). - PluginContext (executor + has_executor) + ChxGetContextInput for the getContext hook (plugins return a custom executor; the obsessiondb plugin's remote executor uses this when a service is selected). PluginRuntimeProtocol: subset of the runtime exposed to plugin run methods (get_command + run_plugin_command, plus run_on_pull_introspect).
…al store, plumbing plugin_runtime.py: - PluginRuntime: load + validate (manifest + CLI compatibility) + dispatch all 11 lifecycle hooks (run_on_init, run_on_complete, run_on_config_loaded, run_on_schema_loaded, run_on_plan_created, run_on_before_apply, run_on_after_apply, run_on_check, run_on_check_report, run_on_before_plugin_command, run_on_pull_introspect). - run_plugin_command invokes on_before_plugin_command first; if any plugin returns Handled, short-circuits with its exit_code (lets obsessiondb's backfill routing intercept before the local backfill plugin runs). - resolve_context + dispose_context for the getContext hook (plugin-supplied executors, e.g. obsessiondb's remote-executor when a service is selected). - PluginExecutionError wraps third-party plugin exceptions with the failing stage name. table_scope.py + safety_markers.py: --table glob support across the 5 scope-aware commands (generate / migrate / status / check / drift), plus destructive-op detection (DROP, TRUNCATE, ALTER DROP COLUMN, DETACH, etc.) with both planner-marker + hand-written SQL scanning. migration_metadata.py: header parser for '-- log: ...' (KNOWN_KEYS set so unknown markers raise instead of being silently ignored). journal_store.py: the _chkit_migrations ClickHouse table abstraction. Stores migration name + applied_at + checksum + chkit_version + per-statement OperationState (query_id + status + timestamps + last_error) for async-apply resume support. UNKNOWN_DATABASE detection so status/drift gracefully report database-missing without crashing. json_output.py + logging_setup.py: small plumbing for --json envelope formatting and CHKIT_DEBUG=1 stdlib-logging configuration. main.py: Typer app wiring + register every core command. schema_loader.py: thin wrapper of chkit.core.schema_loader so CLI commands don't need to import the core sub-module directly.
… dispatch)
Mirrors TS init.ts:
- Writes clickhouse.config.py + src/db/schema/example.py.
- --yes silent mode.
- --connect / --email / --code / --org-name passthrough to the obsessiondb
onboarding wizard (Typer-validated enum for --connect).
- Dispatch via importlib.import_module('chkit_plugin_obsessiondb') with
ModuleNotFoundError graceful degrade (prints the static runbook + next
steps when the plugin isn't installed, mirroring TS's missing-package
fallback).
- --auto-deps intentionally not ported (Python convention: pip install
is explicit). Documented in DRIFT.md > init/auto-deps.
Tests cover every flag combination, the silent mode, the runbook output
shape when the plugin is absent, and the OBSESSIONDB_PLUGIN_MODULE
monkeypatch hook used by the cross-cutting test isolation.
…appings + codegen integration
generate.py: diffs current schema against the last snapshot, emits a
migration .sql + a snapshot .json.
Flags mirror TS exactly (--name / --migration-id / --rename-table /
--rename-column / --dryrun / --json / --config / --table).
plan-pipeline (generate_plan_pipeline.py):
- apply_explicit_table_renames: rewrites drop+create pairs as rename ops.
- apply_selected_rename_suggestions: collapses drop+create column pairs.
- build_explicit_column_rename_suggestions: turn CLI mappings into
suggestions for the renamer.
- assert_cli_column_mappings_resolvable: error if a CLI --rename-column
references a column not in the diff.
rename-mappings (generate_rename_mappings.py): parse_rename_*_mappings,
merge_*_mappings + conflict assertions, collect_schema_rename_mappings,
resolve_active_table_mappings, remap_old_definitions_for_table_renames.
Cross-DB renames emit CREATE DATABASE IF NOT EXISTS at the right
ordering rank.
Codegen integration: after writing the migration, if a 'codegen' plugin is
registered with run_on_generate != False, dispatch its codegen command.
Factory-supplied options are read from the plugin's hook closure (a
PluginConfig instance under .options) since load_plugin_runtime today
doesn't thread factory options through LoadedPlugin.options.
plan_diff is wrapped in try/except ChxValidationError — under --json the
error surfaces as {error: 'validation_failed', issues: [...]} instead of
a raw stack trace. scope is now included in every JSON output path
(apply / empty-plan / dryrun).
…ts + plugin hooks
migrate.py: --apply / --execute (alias) / --dryrun / --table / --json /
--config / --allow-destructive. Default behaviour is plan/preview.
Apply loop:
- Reads each pending file → runs run_on_before_apply (plugins may transform
the statement list) → executes each statement, dispatching to async-apply
for operations marked mode=async in their -- operation: header → writes
journal entry → runs run_on_after_apply.
- wait_for_ddl_propagation after each statement (operation-type-aware).
migrate_async_apply.py: deterministic query_id = chkit-{migration_checksum}-
{statement_index}. Submit + poll system.query_log until terminal. On
resume, attach to an existing query_id from the journal (checksum
validation refuses to continue if the migration file was edited mid-flight).
20 transient poll-error retries. Honors -- before-retry: SQL marker.
migrate_prompts.py: confirm_apply (TTY, CI auto-skip via CI=1 + isatty),
confirm_destructive_execution + print_destructive_operation_details.
is_background_or_ci() check matches TS exactly.
migrate_scope.py: filter_pending_by_scope splits pending into in-scope vs
undetermined (files whose operations don't carry a table key).
status.py: lists migration files, intersects with the _chkit_migrations
journal applied set (project-scoped — only counts THIS project's
migrations even when the journal table is shared across tenants).
Honors --table, --json, gracefully handles UNKNOWN_DATABASE (returns
{databaseMissing: true, database: ...} in JSON).
drift.py + drift_payload.py + drift_compare.py + drift_diff.py:
- build_drift_payload joins snapshot definitions against live
introspection (lists schema objects → fetch table details for any
database appearing in the snapshot).
- compare_schema_objects + compare_table_shape: per-kind + per-column
+ per-engine-arg + per-setting + per-skip-index + per-projection diff.
- summarize_drift_reasons: counts split into object-level vs table-level.
- --live flag (Python-only) opens a connection for full payload; default
is snapshot-only fast-path. Honors --table for partial scope.
Tests cover the drift_compare permutations.
Aggregates 4 categories:
- validation: validate_definitions(schema) issues.
- pending_migrations: pending count > 0 + fail_on_pending policy.
- checksum_mismatch: applied migrations whose checksum changed on disk.
- schema_drift: snapshot vs schema diff (drift_ops); --live extends with
live-DB drift via build_drift_payload (live_drifted).
Plugin on_check hooks contribute additional findings; each plugin's
result is collected into a ChxOnCheckResult and its plugin-name + ok
flag emitted as 'plugin:<name>' in failedChecks when failing.
JSON envelope (mirrors TS check/output.ts):
- top-level: strict, policy {failOnPending, failOnChecksumMismatch,
failOnDrift}, ok, failedChecks, pendingCount, pendingMigrations,
checksumMismatchCount, checksumMismatches, drifted, driftEvaluated
(true iff snapshot present), driftOperations, scope.
- when --live: liveDrifted, driftReasonCounts, driftReasonTotals
(object with total/object/table keys, not a single int).
- when plugins: 'plugins' object map keyed by plugin name (TS shape),
plus 'pluginCheckResults' array for back-compat.
- finding code for drift is 'schema_drift' (matches TS, not 'drift').
--strict overrides all policy fail_on_* to true. Exit 1 on any failure.
…ntrospect hook pull.py: introspects a live database into a Python schema file. --out-file / --database (repeatable) / --force/--overwrite / --dryrun / --json. Atomic write (temp + replace). pull_view_parser.py: parses CREATE VIEW / CREATE MATERIALIZED VIEW queries from system.tables.create_table_query: - parse_as_clause: extract the AS SELECT body. - parse_to_clause: handle backticked + dotted target table names. - parse_refresh_clause: EVERY / AFTER / OFFSET / RANDOMIZE / DEPENDS ON / SETTINGS / APPEND / EMPTY; strips DEFINER / SQL SECURITY for managed-CH compatibility. pull_render.py: renders SchemaDefinition list back to chkit Python DSL (round-trips through ts_import → canonicalize_definitions → match). on_pull_introspect hook: when any plugin's hook returns a list of SchemaDefinition, pull skips the SQL path entirely (used by obsessiondb to query its metadata API instead of running SQL). JSON output mirrors TS schemaEnvelope: command='schema' + outFile + definitionCount + tableCount + viewCount + materializedViewCount + databases + dryrun + skippedObjects (per-kind count of objects in the selected databases that didn't end up in the emitted schema). Skipped objects are computed via the new _summarize_skipped_objects helper.
…ON output) Single positional SQL arg (rejects multi-arg + empty). --json, --config. Text-table output: column width auto-sizing, header + separator, DEFAULT_SHOWN_ROW_LIMIT=25 with truncation indicator. JSON envelope = ClickHouseJsonQueryResult shape: data[] + meta[] (per column name + type) + rows + statistics + query_id. Matches TS exactly for downstream consumers. Error cleaning: strip the injected 'FORMAT JSON' artifact, truncate 'Expected one of [...]' lists past EXPECTED_TOKEN_CAP=8 tokens. Mirrors TS error-cleaner behaviour 1:1. Cell stringification matches TS: null → '', strings/numbers/bools as-is, complex values via JSON.
Three modes: - chkit plugin → list every registered plugin + its commands - chkit plugin <name> → list commands for one plugin - chkit plugin <name> <cmd> → dispatch the plugin command (with --args) Lazy ClickHouseClient.connect: not every plugin command needs a DB connection. When config.clickhouse is set we connect and pass through plugin_context.executor; otherwise null_plugin_context(). run_plugin_command (called via the runtime) invokes on_before_plugin_command first — this is what lets the obsessiondb plugin intercept backfill status/cancel/list and route them through the jobs API before the local backfill plugin's stubs run.
user_config.py: XDG-compliant user-config directory.
- get_user_config_dir() honors XDG_CONFIG_HOME, defaults to ~/.config,
always suffixed with /chkit.
- USER_PROFILE_CONFIG_FILE = 'config.py' (Python convention; TS uses
'config.ts').
- USER_CREDENTIALS_FILE = 'credentials.json' (same as TS).
- Composed-path sugar: get_user_profile_config_path /
get_user_credentials_path.
config_merge.py: merge_user_config(base, overlay) layers a user-profile
ChxUserConfig under a project ChxUserConfig with TS-matching semantics:
- scalar fields (schema / outDir / migrationsDir / metaDir): overlay
wins when set, else fall back to base.
- clickhouse / check / safety: shallow-merge (overlay wins per-key).
- plugins: merge by plugin name — overlay entries replace base entries
with the same name; entries present only in base or only in overlay
are preserved (preserved-from-base entries appear first, overlay
entries appended).
- plugin_name_of(registration) does best-effort name extraction for both
ChxPlugin objects and wrapped {plugin, name?} registration dicts.
…+ services + remote/backfill/onboarding)
Phase 1 — foundation:
- credentials.py: XDG-compliant 0600 ~/.config/chkit/credentials.json,
resolve_base_url honors OBSESSIONDB_API_URL env override.
- storage.py: SelectedService (only service_slug + service_name required —
the other org/service-id fields are optional Python additions, kept None
by default so a TS-written .chkit/obsessiondb.json deserializes here).
Project + user-global service-state files + alias map.
- engine.py: on_schema_loaded hook rewrites Shared* engines to standard
ClickHouse equivalents when targeting a non-obsessiondb host. Strips
cloud-only settings (storage_policy). --force-shared-engines /
--no-shared-engines overrides; auto-detect from URL.
- plugin.py: obsessiondb() factory + ChxPluginManifest + hooks +
5 commands (login / logout / whoami / signup / service).
Phase 2 — auth flows:
- api_client.py: RFC 8628 device-code (request + poll + slow_down +
expired_token), get_session, passwordless OTP (send + verify), org
create + set_active. SessionExpiredError on 401 surfaces from the
service_api/jobs_api/workbench_api layer; OtpRateLimitError on 429.
- auth_login.py: device-code flow + browser open + poll loop.
- auth_signup.py: 3 modes (interactive TTY, two-step CI via
--request-only then --code, scripted via --email + --code). Auto-create
personal org (derive_org_name strips +subaddress, slugify_org_name
appends 6-char random suffix).
- run_logout / run_whoami (with --json envelope).
Phase 3 — services:
- service_api.py: oRPC client (POST /rpc/{procedure_path} with
{input: ...} body, bearer auth). HTTP 401 → SessionExpiredError.
- service_select.py: render_service_organizations + interactive picker
+ save_selected_service.
- service_claim.py: eligibility check → claim → poll-until-running
(5min deadline, 3s poll) → save selection. Handles already_claimed,
none_available, provisioning_timeout with --json envelopes.
- service_commands.py: list / select / claim / alias subcommands.
* 'service list' honors --json (services array + selected flag).
* 'service alias set <name> <service-name>' accepts the service NAME
(matches TS); joins trailing args to allow multi-word names.
Validates: empty / whitespace / -- prefix; rejects collisions with
existing service names.
Phase 4 — remote + backfill + onboarding:
- workbench_api.py: workbench.query.execute oRPC client (returns
data/meta/rows/statistics/query_id/error envelope).
- jobs_api.py: jobs.get / list / cancel.
- remote_executor.py: RemoteClickHouseClient duck-typed to
ClickHouseClient surface (execute / query / query_json / submit /
query_status / insert / list_schema_objects / list_table_details /
database / __enter__ / __exit__). Lets drift/pull/migrate/query hit
obsessiondb cloud without code changes.
- backfill_handler.py: handle_backfill_command for the
on_before_plugin_command hook. Routes status / cancel / list to the
jobs API; --local + --plan-id bypass.
- onboarding.py: full wizard with ConnectChoice (claim/account/
clickhouse/later). ensure_obsessiondb_plugin_in_source text-rewrites
clickhouse.config.py to add obsessiondb() to plugins[]. Accepts
package_manager parameter (Python pkg-mgrs: uvx / pipx / poetry /
rye / pip) to prefix next-steps commands.
Tests cover credentials persistence, storage round-trip, engine
rewriting variants, every auth flow (device + OTP), services list /
select / claim / alias, the Phase-4 surface (workbench RPC + jobs RPC
+ remote executor + backfill_handler dispatch matrix + onboarding
wizard branches with httpx-mock).
Python equivalent of @chkit/plugin-codegen — the TS plugin emits TypeScript types + Zod schemas; Python emits Pydantic models (which cover both static typing and runtime validation in one shape). errors.py: CodegenError / CodegenConfigError / UnsupportedTypeError. options.py: Pydantic-validated PluginConfig + CodegenOptions: - out_file (default ./src/generated/chkit_models.py — Python convention, not chkit-types.ts). - table_name_style: 'pascal' | 'camel' | 'raw'. - bigint_mode: 'int' | 'str' (default 'int' — Python int is unbounded so it strictly supersedes TS bigint with no precision loss). Accepts TS aliases 'bigint' → 'int' and 'string' → 'str' so a TS-side config deserializes here. - include_views / run_on_generate / fail_on_unsupported_type. - CODEGEN_FLAGS + CODEGEN_FLAG_MAP for CLI plumbing. naming.py: Pascal / camel / raw class-name styles with collision suffix (_2, _3, ...). Non-identifier column names sanitized + aliased via Field(..., alias='original-name'). type_artifacts.py: CH-type → Python-type resolver. Handles Nullable / LowCardinality (unwrap) / Array / Map / Tuple / SimpleAggregateFunction / JSON / every CH scalar set (LARGE_INTEGER_TYPES, NUMBER_INT_TYPES, NUMBER_FLOAT_TYPES, STRING_TYPES, BOOLEAN_TYPES). Generates one BaseModel per table (or TypeAlias dict[str, Any] for views when include_views). plugin.py: codegen() factory + ChxPlugin. ChxPluginCommand 'codegen' with --check (returns 0 ok / 1 missing/stale) + write mode (atomic write via temp + os.replace). on_check + on_check_report hooks emit codegen_missing_output / codegen_stale_output / codegen_unsupported_type findings. Intentionally NOT ported (deferred per DRIFT.md): Zod schemas (Pydantic already covers validation), ingest helpers (clickhouse_connect.Client.insert + a Pydantic model is one line at the call site), migrations module (Python's importlib.resources is the equivalent for runtime-applied migrations).
…atus/cancel commands) Python port of @chkit/plugin-backfill — Phase 1 surface only. Shipped: - errors.py: BackfillConfigError. - options.py: Pydantic-validated PluginConfig + PlanOptions + RunOptions + ResumeOptions + StatusOptions + CheckOptions. Coercion helpers matching TS exactly: _normalize_timestamp (ISO + UTC + ms precision), _normalize_target (db.table regex), parse_byte_size (K/M/G/T suffix), _normalize_plan_id (16-char lowercase hex), _coerce_positive_int. CLI flag definitions + flag mappings. - types.py: Pydantic models for the persisted plan/run shapes (BackfillPlanState / BackfillRunState / BackfillStatusSummary / BackfillEnvironment / BackfillExecutionPlan / etc.). chunk_plan is intentionally an opaque dict in Phase 1 — Phase 2 will replace it with a typed ChunkPlan when the chunking engine is ported. - state.py: backfill_paths (plans/ + runs/ directories under metaDir), compute_environment_fingerprint (SHA256 of origin|database, first 16 chars), ensure_environment_match (refuses to apply a plan to a different host without --force-environment), read_plan / read_run / list_plan_ids / write_json. summarize_run_status counts chunks by status and rows_written; plan_status_for returns run.status verbatim (matches TS exactly — chunk-based status derivation is the engine's job). - plugin.py: backfill() factory with 2 functional commands: * status — reads run + summarizes via summarize_run_status. * cancel — marks run.status='cancelled' + persists. 4 stubs (plan / run / resume / doctor) print 'pending Phase 2' and exit 2. The remote path (chkit plugin backfill status --job-id / --service-slug) is fully functional today via obsessiondb's on_before_plugin_command hook. Deferred to Phase 2 (per DRIFT.md): the chunking engine (~1,400 LoC TS pure algorithm — strategies + services + boundary codec + SQL builders) and the async-backfill execution engine. Together they make 'plan / run / resume / doctor' work locally; their absence doesn't affect the remote-backfill path which already ships.
test_parity_fixes.py (20 tests, round-1 fixes): - #11 core public exports. - #18 ClickHouseClient.list_schema_objects / list_table_details as methods. - #12 ChxGetContextInput + resolve_context + dispose_context (defer chain + close-error swallowing). - #2 run_on_before_apply threads transformed statements. - #5 codegen accepts TS bigint aliases ('string' → 'str', 'bigint' → 'int'). - #6 SelectedService deserializes a TS-minimal {service_slug, service_name} record. - #1 plugin dispatch short-circuits on Handled. - #13 onboarding next-steps honors package_manager (uvx + bare-chkit cases). - #4+#10 _validate_alias rejects empty / whitespace-bounded / '--' prefix. - #17 plan_status_for returns run.status verbatim (no chunk-based override). - #3 RemoteClickHouseClient has insert / list_schema_objects / list_table_details bound methods. - #14+#15 generate --json on invalid schema exits 1 with structured envelope. - #7 check JSON envelope contains policy + driftEvaluated + scope + plugins map + schema_drift finding code (via source inspection). - #8 pull payload contains command + skippedObjects; _summarize_skipped_objects counts per-kind correctly. - #9 service list --json emits not_logged_in error envelope (not bare text). test_parity_round2_fixes.py (22 tests, round-2 fixes): - #R1 canonical primary_key falls back to order_by when empty. - #R2 MaterializedViewRefresh depends_on serializes as 'dependsOn' (camelCase) with no snake_case leak. - #R4 ddl_propagation new predicates (wait_for_column_absent / wait_for_index{,_absent} / wait_for_projection{,_absent}) + _parse_operation_key recognizes index: / projection: segments + dispatcher routes alter_table_drop_column to column_absent, etc. - #R5 validate.py issue codes (duplicate_column_name, primary_key_missing_column, order_by_missing_column, duplicate_object_name, refresh_every_after_mutually_exclusive, refresh_requires_every_or_after, refresh_depends_on_requires_every). - #R6 snapshot round-trip via canonicalize_definition + model_dump (by_alias=True). - #R7 service_claim envelope literals (already_claimed + provisioning_timeout) verified via source inspection. test_sql_render_parity.py (115 tests, push to_create_sql to 10/10): - 20 primitive types + 7 parameterized + 9 complex/nested. - Defaults (string literal / numeric / boolean lowercase / fn: prefix stripping + nested calls). - 5 codec variants + chain + nullable+codec + DEFAULT-before-CODEC ordering. - Column comments with escaped quotes, nullable wrapping. - 6 engine families, 3 PARTITION BY shapes, multi-column ORDER BY/PK. - TTL simple + DELETE. - SETTINGS numeric + multi-key (insertion-order preserved). - All 5 skip-index variants + expression-arg. - Projections simple + with ORDER BY. - Table comments (incl. escaped quotes). - Kitchen-sink table with every clause (22 assertions on one render). - 25-column table + reserved-word column names (select/from/table/index properly backticked) + deeply nested Array(Tuple(...)). - CREATE VIEW + CREATE MATERIALIZED VIEW (TO + REFRESH EVERY/AFTER + APPEND/OFFSET/RANDOMIZE/SETTINGS clause-order + DEPENDS ON + EMPTY). - ALTER MODIFY REFRESH (3 variants), ALTER ADD COLUMN (6 variants), ALTER MODIFY COLUMN with codec + REMOVE CODEC, DROP COLUMN, ADD/DROP INDEX, ADD/DROP PROJECTION, MODIFY/RESET SETTING, MODIFY/REMOVE TTL. Approach: structural assertions on the rendered SQL string (no live ClickHouse needed). Catches every byte-level rendering regression that would surface as drift against TS.
DRIFT.md (~700 lines): append-only ledger of every divergence from a 1:1 TS port + every audit finding's status: - Conventions established (load-bearing decisions like the obsessiondb package name, --auto-deps not being portable). - Items 'plumbed-pending-dependency' tracking. - Refactors that touched pre-existing Python code (the cli/schema_loader wrapper, the mtime cache bytecode-cache bug fix). - Bugs found while porting (status: project-scoped applied count). - TS-only modules not ported with rationale (skill-hint, cmd-skills, create-chkit, the cmd-dispatch/registry/global-flags suite, etc.). - Known limitations in ported features (backticked dotted names in pull). - ObsessionDB plugin port status (Phase 1 → 4 narratives). - create-chkit / rt-config / rt-exec-debug / ch/testkit explanations. - Cross-cutting polish (gen/codegen-integration, pull/introspect-custom, ch/exec/* helpers, rt/user-config + rt/config-merge). - Parity audit fixes (20-section sub-agent review) — every finding validated, status, fix approach. - Round-2 audit fixes (15-section deeper-dive) — same shape. - SQL render parity coverage push (R-l → 10/10) + post-fix score table. PARITY-CHECKLIST.html: master tracker with three classification sets: - PORTED_BY_DEFAULT (~150 items shipped + tested). - DECIDED_NA (~19 items — language/ecosystem convention difference; each entry's rationale lives in DRIFT.md). - DEFERRED_FUTURE_PHASE (~21 items — real work deliberately postponed). Outstanding-work view (items in none of the three sets) is empty: every checklist item has a decision recorded. MISSING.md: lightweight gap notes from prior porting passes (kept for context — most items are now resolved or tracked in DRIFT.md/HTML).
…very HTTP call
Mirrors TS `packages/plugin-obsessiondb/src/auth/api-client.ts` +
`client.ts`: replaces the bare 'chkit-cli' string with the package
version (`chkit/0.1.0`).
The ObsessionDB API forwards this header to ClickHouse, so chkit traffic
becomes attributable in `system.query_log.http_user_agent` — debugging
'what tool issued this query' goes from impossible to one filter.
Implementation:
- New module _version.py as the single source of truth for the package
version. Lives separately to avoid the circular import that would
happen if api_client.py imported __version__ from __init__.py (which
re-exports api_client symbols).
- api_client.USER_AGENT = f'chkit/{_version.__version__}'.
- jobs_api + workbench_api inherit the constant automatically: they
route HTTP through service_api._rpc_post which already sets the
User-Agent header from USER_AGENT.
Tests verify the constant shape and observe the header on both an
`/api/auth/get-session` call and an `/rpc/services/listAll` call.
…ted ObsessionDB service 1:1 port of TS `guardRemoteExecution` in `packages/plugin-obsessiondb/src/backfill/handler.ts` (commit bb62cd0). The problem: when a user is logged in AND has a service selected (the condition under which getContext hands out the remote executor), `chkit plugin backfill {plan,run,resume}` would silently fall through to the local backfill Phase-2 stub. The user thinks they're running against ObsessionDB but they aren't — at best confusing, at worst (if Phase 2 ships before the remote execution path) exfiltrating their intended-cloud queries to whatever `CLICKHOUSE_URL` happens to be set. Fix: add `_guard_remote_execution` to backfill_handler. It runs BEFORE the existing remote-subcommand routing — if the command is in {plan, run, resume} AND creds are present AND a service is selected (either via --service flag or a stored SelectedService for the project), refuse with a clear message that nudges the user toward --local or to unselect the service. Emits an {ok: false, command, error} envelope under json_mode, plain text otherwise. Honors the existing --local early-return. `doctor` is intentionally NOT guarded (it reads local state).
…oami envelope
1:1 port of TS `packages/plugin-obsessiondb/src/json-envelope.ts`:
centralises the shape of every JSON payload the plugin emits so
`jq` / CI consumers can rely on a stable schema across versions.
New module json_envelope.py exports:
- JSON_CONTRACT_VERSION = 1 (bump on incompatible envelope changes).
- error_envelope(command, code, message) → {command, schemaVersion,
ok: false, error: {code, message}}.
- whoami_envelope(*, email, name?) → {command, schemaVersion,
status: 'logged_in', email, next: null}. The TS envelope drops the
name field; we match that. `name` is accepted for forward-compat.
- service_list_envelope(services) → {command, schemaVersion,
status: 'ok', services}.
- Typed via TypedDicts so consumers get static-type guarantees.
Refactors:
- auth_login.run_whoami uses error_envelope + whoami_envelope instead
of the ad-hoc dicts it built before. Public --json shape now matches
TS exactly.
- service_commands._service_list migrated to the same helpers. The
audit-fix-#9 envelope had emitted {status: 'error', errorCode,
message} (non-TS-aligned); the helper aligns it to
{ok: false, error: {code, message}}.
Updated 2 existing tests + added 14 new tests in
test_main_sync_2026_06_29.py covering all three #M1/#M2/#M3 ports.
…tive
test_main_sync_2026_06_29.py groups tests by sync finding:
#M1 (User-Agent, 3 tests):
- Constant equals chkit/<plugin-version>.
- get_session HTTP call carries the header.
- service_api RPC POST carries the header (transitive coverage for
jobs_api + workbench_api which both route through _rpc_post).
#M2 (backfill guard, 7 tests):
- Unhandled when not authenticated.
- Unhandled when authed but no service selected (user hasn't opted in).
- Handled+exit-1 when authed + service stored on disk.
- Handled+exit-1 when --service flag is passed.
- json_mode emits {ok: false, command: 'backfill ...', error: ...}.
- --local flag bypasses the guard.
- doctor is NOT guarded (it reads local state).
#M3 (json_envelope helpers, 4 tests):
- error_envelope shape (matches TS errorEnvelope).
- whoami_envelope shape (status/email/next, no name).
- whoami_envelope ignores name for forward-compat.
- service_list_envelope shape.
DRIFT.md gets a new 'Main sync 2026-06-29' section detailing every TS
commit reviewed: 3 ports, 7 already-in-Python items, 3 decided-N/A
items. Includes the TS-reference rationale + the bonus
service_commands._service_list envelope realignment to TS shape.
Ports TS commit 6b87e6d (packages/core/src/on-cluster.ts + model changes) to Python. Enables self-managed multi-node ClickHouse clusters: setting `clickhouse.cluster` in the resolved config causes every DDL statement in the migration plan to be stamped with `ON CLUSTER <name>` as a final post-pass. - `chkit/core/on_cluster.py`: new module mirroring on-cluster.ts. Two anchor tables — after-object placement (CREATE/ALTER/DROP TABLE, CREATE VIEW/MV/DATABASE/DICTIONARY, DROP VIEW/DICTIONARY, plus a forward-compat safety net for CREATE FUNCTION, DROP DATABASE, ATTACH/DETACH/TRUNCATE/OPTIMIZE TABLE) and trailing-anchor placement (RENAME TABLE/DICTIONARY/DATABASE, EXCHANGE TABLES/DICTIONARIES). Trailing loop runs first so RENAME appends at end, not after first name. Skips optional IF [NOT] EXISTS guard so the clause lands after the object reference either way. Idempotency is checked positionally (against the slice after the object reference), so user-authored content like a column COMMENT containing "on cluster" cannot suppress injection. - `chkit/core/model.py`: adds `cluster: str | None` field to `ChxUserClickHouseConfig` and `ChxResolvedClickHouseConfig` (declared after `secure` for canonical serialization parity with TS). New private `_CLUSTER_NAME_PATTERN` accepts identifiers with dashes/dots (e.g. `prod-eu-1`, `eu.west.main`) and `{cluster}`-style macros — the characters legal in a `remote_servers` XML key, injection-safe inside single quotes. Uses `re.fullmatch` (not `re.match`) so a multi-line value like `"prod\nDROP TABLE x"` cannot slip past a start-only anchor. Error message matches the TS literal so existing test regexes continue to work. - Re-exports `apply_on_cluster_to_plan` and `on_cluster_clause` from `chkit.core` and top-level `chkit`. Tests: 19-case `test_on_cluster.py` mirroring on-cluster.test.ts case-by-case, plus 2 Python-specific regression guards (empty-string cluster short-circuit; multi-line name rejected by `fullmatch`).
…licated journal Ports the CLI-side of TS commit 6b87e6d — the journal-store rewrite and the 4 command call sites that pass `clickhouse.cluster` through. - `chkit/cli/journal_store.py`: `JournalStore.__init__` takes optional `cluster: str | None`. When set, the `_chkit_migrations` engine becomes `ReplicatedReplacingMergeTree('/clickhouse/tables/{uuid}/chkit_journal', '{shard}_{replica}', applied_at)` — no-`{shard}` Keeper path (one cluster-wide replication group), `{uuid}` for drop-recreate safety, `{shard}_{replica}` unique cluster-wide so multi-shard layouts don't collide. CREATE TABLE + both ALTER TABLE ADD COLUMN IF NOT EXISTS statements carry `ON CLUSTER '<name>'`. Non-cluster mode unchanged. - `chkit/cli/commands/generate.py`: calls `apply_on_cluster_to_plan` after `run_on_plan_created` (so plugin-injected SQL is also stamped) and before the empty-plan check. `migrate` never re-runs this — the clause is baked into the migration file at generate time and re-executed verbatim. - `chkit/cli/commands/migrate.py`, `status.py`, `check.py`: each pass `cluster=config.clickhouse.cluster if config.clickhouse else None` to `JournalStore(...)`. Python has 3 journal-store sites (TS has 4; TS `generate` uses the journal for a schema-existence probe that Python does not). Tests: - `test_journal_store_cluster.py` (5 tests): CREATE TABLE stamping and engine switch under cluster mode, both ALTERs stamped, plain engine when cluster absent, `{cluster}` macro form. - `test_on_cluster_generate_e2e.py` (3 tests): runs `chkit generate` (JSON dryrun + file write) with `clickhouse.cluster` set/unset; asserts every DDL line carries or omits the clause accordingly. Overall: 970 pytest passed, 1 skipped, 0 failed. Mypy clean on all new code; ruff clean on all new code (3 pre-existing PLR0917 warnings on run() signatures unchanged).
Records the ClickHouse ON CLUSTER port from TS commit 6b87e6d in the same shape as the 2026-06-29 sync entry: - #C1 apply_on_cluster_to_plan (new core module) - #C2 clickhouse.cluster config field + validation - #C3 journal store cluster mode (ReplicatedReplacingMergeTree) - #C4 generate command integration Plus: - What tests were added and where - What was deferred (docker-based live 2-node cluster e2e — behavior parity is covered by unit + integration tests) - Design divergences (dictionary op types in tests, no debug logging) - A follow-up parity fix surfaced by the Category C reviewer but not addressed here: pre-existing `generate.py` plan-transform ordering divergence — Python does `filter -> plugin` where TS does `plugin -> filter`. Predates this port (from commit ad94a16); tracked for a future parity pass. - The remaining commits from main-sync 2026-07-02 not touched in this pass (Category A bug-fix trio, Category B Dictionary primitive, Category D backfill submit, Phase-2 backfill fixes, TS-only refactors).
Ports the runtime-agnostic primitives from TS `packages/clickhouse/src/e2e-testkit.ts` + `packages/cli/src/test/e2e-testkit.ts` to a single Python module. - `LiveEnv` dataclass + `get_required_env` (hard-fail variant) + `resolve_live_env` (soft-default variant for dev docker). - `live_env_to_client_kwargs` — bridge to `clickhouse_connect.get_client`, parses URL into host/port/secure. - `quote_ident` — backtick-doubling identifier quoter. - `create_run_tag` / `create_prefix(label)` / `create_journal_table_name(label)` — unique-name builders for parallel-safe live tests; prefers `GITHUB_RUN_ID` for CI correlation, falls back to `<ms>_<rand>`. - `format_test_diagnostic` — structured CLI-failure message accepting any `exit_code`/`output` result (Protocol-typed so callers don't need to import typer). `tests/conftest.py` now uses the testkit for env resolution + client kwargs — dedupes the inline `_resolve_clickhouse_env` block that was partially reimplementing the TS helper. Deliberately NOT ported: TS `runCli` / `runCliWithRetry` / `waitForCliJson` — Python convention is in-process `typer.testing.CliRunner`, not subprocess spawn. Tests: 17 unit cases covering env resolution (hard-fail, soft-default, URL derivation, all-overrides), URL parsing (https default port, http default port, explicit port), ident quoting (backtick doubling), unique names (RNG-collision guard on 100 tags, GITHUB_RUN_ID preference, per-call divergence), and diagnostic formatting. `987 passed, 1 skipped, 0 failed`. Mypy + ruff clean on all new files (the 2 pre-existing conftest mypy warnings are unrelated).
Both files were derived from a 2026-06-05 pre-port audit. 8+ port passes have landed since then and nearly every "Critical" item they flag as missing is present: - `@chkit/core`: config_path, plugin_error, schema_loader, ts_import ✓ ported - `drift`: drift.py + drift_compare.py + drift_diff.py + drift_payload.py ✓ ported (the "big missing piece" claim is stale) - `pull`: pull.py + pull_render.py + pull_view_parser.py ✓ ported - `query`: query.py ✓ ported - `plugin`: plugin.py ✓ ported - Runtime: plugin_runtime, safety_markers, migration_metadata, migrate_async_apply, table_scope, config_merge, user_config, json_output ✓ all present - `@chkit/clickhouse`: create_table_parser, ddl_propagation, introspect ✓ all present - Plugins: chkit_plugin_codegen, chkit_plugin_backfill, chkit_plugin_obsessiondb ✓ all present; plugin_pull merged into cli/commands/pull.py + on_pull_introspect hook The remaining genuine non-parity items (`chkit skills`, `create-chkit`, `deps.ts` auto-install, `internal-plugins/skill-hint`) are explicitly documented as won't-port in DRIFT.md — `cmd-skills`, `create-chkit`, `--auto-deps`, `rt/skill-hint-*`. DRIFT.md is now the single source of truth for known divergences and non-parity decisions. PARITY-CHECKLIST.html's per-item localStorage state carries no meaning once items are actually ported, and the page actively misleads (everything renders as unchecked-missing). No incoming references outside these two files themselves.
Same disease as MISSING.md — the 2026-06-05 audit is stale enough that
90% of items it flags "Deferred" are actually ported now. The README's
"What is intentionally out of scope for this first base" bullet list
was equally stale (plugins ✓, --table ✓, --rename-table ✓, drift live
✓, per-op journal tracking ✓, ObsessionDB credentials ✓, etc.) and
existed only to anchor two PARITY.md links.
- Delete PARITY.md.
- Rewrite the README TypeScript-parity section:
- Drop the "271 ported tests" figure (out of date; the suite is 987).
- List the actual current coverage: all CLI commands, plugin runtime
+ all hooks by real name, first-party plugins, cluster mode.
- Replace the stale "out of scope" list with a "Not ported by
design" list of the 4 genuine won't-port items (skills / create-chkit
/ auto-deps / skill-hint), each cross-referenced in DRIFT.md.
- Point to DRIFT.md as the source of truth for divergences.
CHANGELOG.md references to PARITY.md are historical (they described what
0.1.4 added on 2026-06-05) and stay as history. DRIFT.md never
referenced PARITY.md.
Closes every remaining gap between chkit-py and the TS implementation: - Dictionary primitive (65c90d6): DSL, validation, CREATE/REPLACE/RENAME/ DROP planning with [HIDDEN]-password handling, create-dictionary parser, --rename-dictionary, safety markers, drift, pull, codegen models. - Category A trio: function expressions in primaryKey/orderBy (5a8d805), index-only projections (3f1db03), table-clause parsing past the column list + derived-PK drift fix (8296b8a). - Phase-2 backfill engine: chunking planner/SQL/strategies, async execution loop with checkpointing, plan/run/resume/doctor commands, managed submit to ObsessionDB jobs, on_check findings. Includes f85f568, 3f9a246, 9ad23f9. - CLI surface: top-level chkit codegen + chkit obsessiondb shortcuts, plugin dispatcher flag forwarding, function-style configs (ChxConfigEnv), check.failOnExtraObjects, per-table plugins field. - Fixes surfaced by parity reviewers: JS Number()/String() fidelity, URL.origin env fingerprints, atomic checkpoint writes, snapshot exclude_none for TS interop, wheel packaging of all plugin packages, ClickHouseClient.submit query_id crash. All ported 1:1 with tests (1185+ unit, e2e validated against live ClickHouse); mypy --strict and ruff clean. Full decision log appended to DRIFT.md.
Every page with code now documents TypeScript and Python side by side using Starlight synced tabs (syncKey) — pick a language once and the whole site follows: schema DSL reference, schema/config overviews, refreshable views, tutorial, all plugin pages, ObsessionDB pages, CI/CD guide (dual GitHub Actions/GitLab/shell pipelines), troubleshooting, and the CLI plugin page. New Python section (overview + core API); python/schema-dsl dissolved into the tabbed DSL reference. Positioning updated everywhere chkit was described as TypeScript-only: hero (removes the stale Python 'Coming soon' badge), footer, landing tagline, llms.txt/sitemap tagline, ai-agents runbook (adds the Python branch), CLI/guide prose, and both READMEs. Every Python example was verified by executing it against chkit-py. Also: raw-markdown integration fixed for Windows (URL.pathname drive prefix, backslash slugs) and dead 'coming soon' CSS removed.
Lucasgvdii
force-pushed
the
first-python-port
branch
from
August 10, 2026 16:37
c07c741 to
918a09f
Compare
…orkflow Bumps chkit-py to 0.2.0 (dictionary primitive, Phase-2 backfill engine, CLI shortcuts, dynamic configs, packaging fix — see CHANGELOG.md). Adds a tag-triggered publish workflow (chkit-py-v*) using PyPI trusted publishing (OIDC): build + twine check + clean-venv wheel smoke, then publish. Requires one-time trusted-publisher setup on pypi.org.
Lucasgvdii
changed the title
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Comment on lines
+14
to
+52
| runs-on: ubuntu-latest | ||
| defaults: | ||
| run: | ||
| working-directory: chkit_python | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: "3.12" | ||
|
|
||
| - name: Verify tag matches package version | ||
| run: | | ||
| version=$(python -c "import tomllib; print(tomllib.load(open('pyproject.toml','rb'))['project']['version'])") | ||
| tag="${GITHUB_REF_NAME#chkit-py-v}" | ||
| if [ "$version" != "$tag" ]; then | ||
| echo "Tag $GITHUB_REF_NAME does not match pyproject version $version" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| - name: Build sdist and wheel | ||
| run: | | ||
| python -m pip install build twine | ||
| python -m build | ||
| python -m twine check dist/* | ||
|
|
||
| - name: Smoke-test the wheel | ||
| run: | | ||
| python -m venv /tmp/wheeltest | ||
| /tmp/wheeltest/bin/pip install dist/*.whl | ||
| /tmp/wheeltest/bin/python -c "import chkit, chkit_plugin_backfill, chkit_plugin_codegen, chkit_plugin_obsessiondb; print(chkit.__version__)" | ||
| /tmp/wheeltest/bin/chkit --version | ||
|
|
||
| - uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: chkit-py-dist | ||
| path: chkit_python/dist/ | ||
|
|
||
| publish: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
chkit codegen/chkit obsessiondbCLI shortcuts, plugin flag forwarding, function-style configs, andcheck.failOnExtraObjects. Full decision log inchkit_python/DRIFT.md.Number()/String()fidelity for chunk boundaries, WHATWGURL.originenvironment fingerprints (TS-written plans run under Python), atomic checkpoint writes, snapshotexclude_nonefor cross-implementation interop, wheel packaging of all plugin packages, and aClickHouseClient.submit()crash.chkit-py-v*).Test plan
mypy --strict,ruff— all green; e2e suite validated 11/11 against live ClickHouse 24.8turbo run typecheck lint build— 28/28 tasks green (TS e2e runs in CI with secrets)twine checkpassed, clean-venv smoke: all four packages import,chkit --version→ 0.2.0