Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ work on a branch, with an associated issue and a PR.
(`.agents/skills/openapi-drift-remediation/SKILL.md`).
- Update the API library on its own; add CLI exposure separately. The CLI need not cover 100% of the library's
endpoints — be intentional.
- Project-local data lives in `.clickhouse/`; globally installed ClickHouse binaries in `~/.clickhouse/`. OAuth
- Project-local data lives in `.clickhouse/`; globally installed ClickHouse binaries and PostgreSQL bundles
(`postgres/`) in `~/.clickhouse/`. OAuth
tokens (`~/.clickhouse/tokens.json`) are the exception — global user identity, not project-scoped.
The project root is found git-style, once per run (`init::project_root`): the nearest ancestor with a `.clickhouse/`
directory owned by the user, stopping at home, whose `.clickhouse/` is global; `local init` uses the cwd.
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@

- **[ClickHouse](https://clickhouse.com/clickhouse)** is the open-source, column-oriented SQL database for analytics. Install and switch local versions, run isolated servers, and execute SQL.
- **[ClickHouse Cloud](https://clickhouse.com/cloud)** runs ClickHouse as a managed service on AWS, GCP, and Azure. Create, configure, scale, back up, and query services from the terminal.
- **[ClickHouse Managed Postgres](https://clickhouse.com/cloud/postgres)** runs PostgreSQL for transactional applications, with ClickHouse integration for analytics. Use Docker-backed Postgres locally, then create, monitor, configure, and restore managed services in Cloud.
- **[ClickHouse Managed Postgres](https://clickhouse.com/cloud/postgres)** runs PostgreSQL for transactional applications, with ClickHouse integration for analytics. Run Postgres with the pg_clickhouse extension locally, then create, monitor, configure, and restore managed services in Cloud.
- **[ClickPipes](https://clickhouse.com/cloud/clickpipes)** ingests data into ClickHouse Cloud from object storage, streams, and databases through change data capture. Create, inspect, update, scale, and troubleshoot pipelines.
- **[ClickStack](https://clickhouse.com/clickstack)** combines ClickHouse, OpenTelemetry, and the HyperDX UI for observability. Manage sources, roles, saved searches, dashboards, alerts, and webhooks for an existing [Managed ClickStack](https://clickhouse.com/cloud/clickstack) service.

Expand Down Expand Up @@ -85,7 +85,7 @@ clickhousectl local clickhouse stop dev

`local init` marks the directory as a project; later commands work from it or any subdirectory, as Git does. Each named server has isolated data under the project's `.clickhouse/`; stopping preserves it for the next start.

For local Postgres, start Docker first:
Local Postgres needs no Docker: the first start downloads a PostgreSQL build that includes the pg_clickhouse extension.

```bash
clickhousectl local postgres start dev
Expand Down
27 changes: 27 additions & 0 deletions crates/clickhousectl/src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -639,6 +639,33 @@ pub enum Error {
#[error("Postgres error: {0}")]
PostgresUsage(String),

/// No usable Postgres bundle for the native runtime: the manifest is not
/// readable by this version, or names nothing for this request and
/// platform. Self-composed, including its recovery guidance.
#[error("Postgres bundle unavailable: {0}")]
PostgresBundleUnavailable(String),

/// A downloaded Postgres bundle does not match the sha256 its manifest
/// names; nothing was extracted or installed.
#[error(
"Postgres bundle '{file}' failed verification: expected sha256 {expected}, got {actual}; nothing was installed"
)]
PostgresBundleChecksum {
file: String,
expected: String,
actual: String,
},

/// Run as root, native Postgres runs as an unprivileged user, and that
/// user cannot enter `dir`, which lies on the way to the PostgreSQL build
/// or to the instance. Nothing was started or created.
#[error(
"PostgreSQL cannot run as root, so it runs as user '{user}', who cannot enter '{dir}'. \
Use a project directory and HOME outside '{dir}', run `chmod o+x {dir}`, or pass `--docker`.",
dir = dir.display()
)]
PostgresServerUserAccess { user: String, dir: PathBuf },

/// A child process whose status must be returned unchanged. This is
/// intentionally not printed as a clickhousectl error by `run_parsed`.
#[error("child process exited with code {0}")]
Expand Down
64 changes: 44 additions & 20 deletions crates/clickhousectl/src/local/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,8 @@ const INSTALL_AFTER_HELP: &str = "\
CONTEXT FOR AGENTS:
The first ClickHouse version installed becomes the default; later installs do not change it.
`clickhousectl local use <version>` auto-installs a missing version and sets it as default.
`postgres@<tag>` pulls a Docker image instead (needs Docker running) and never sets a default.
`postgres@<version>` (17, 18 or <major>.<minor>) downloads a PostgreSQL build with pg_clickhouse
for `local postgres`; with --docker it pulls that image tag instead. Neither sets a default.
--binary copies the file (e.g. build/programs/clickhouse) and installs it as the version its
`--version` reports; rebuild and pass --force to replace that version.";

Expand Down Expand Up @@ -306,12 +307,13 @@ CONTEXT FOR AGENTS:
command: ServerCommands,
},

/// Manage local Postgres instances (Docker-backed)
/// Manage local Postgres instances
#[command(
visible_alias = "pg",
after_help = "\
CONTEXT FOR AGENTS:
Requires Docker installed and running.
Runs a downloaded PostgreSQL build with pg_clickhouse (`CREATE EXTENSION pg_clickhouse` can
query a `local clickhouse` server); `start --docker` runs a Postgres image instead.
Each instance is keyed on (name, major version); pass --version when one name has two majors.
Omitting NAME selects \"default\", else the only instance name; several without \"default\"
errors listing them. `remove` never guesses: it takes only \"default\" or an explicit name.
Expand Down Expand Up @@ -393,13 +395,13 @@ CONTEXT FOR AGENTS:
args: Vec<String>,
},

/// Install a ClickHouse version or Postgres image
/// Install a ClickHouse version or a Postgres build
#[command(
after_help = INSTALL_AFTER_HELP,
group(ArgGroup::new("source").args(["version", "binary"]).required(true))
)]
Install {
/// Version ("latest", "stable", "lts", 25.12, 25.12.9.61) or image selector (postgres@18)
/// Version ("latest", "stable", "lts", 25.12, 25.12.9.61) or Postgres selector (postgres@18)
version: Option<InstallVersionArg>,

/// Install a local ClickHouse build under the version it reports
Expand All @@ -409,6 +411,10 @@ CONTEXT FOR AGENTS:
/// Re-install even if already installed
#[arg(long)]
force: bool,

/// Pull the Postgres Docker image instead; only with postgres@<tag>
#[arg(long, conflicts_with = "binary")]
docker: bool,
},

/// List installed or available ClickHouse versions
Expand Down Expand Up @@ -684,13 +690,14 @@ pub enum PostgresCommands {
/// Start a Postgres instance
#[command(after_help = "\
CONTEXT FOR AGENTS:
An existing stopped instance for the same (name, major) is resumed with its stored settings, so
--port/--user/--password/--database/-e are ignored on a resume. A resume whose stored port another
process holds fails with port_in_use; free that port, then start again.
The first start of a version downloads its PostgreSQL build; later starts work offline.
An existing stopped instance for the same (name, major) is resumed in its own runtime with its
stored settings, so --port/--user/--password/--database/-e are ignored on a resume. A resume
whose stored port another process holds fails with port_in_use; free that port, then start again.
Without --version, an existing instance selects the major; two majors under one name error.
The generated password is printed once by start — re-read connection details later with
`postgres dotenv` or `postgres client`.
A failed fresh start rolls back the container and data it created; pre-existing data is kept.")]
A failed fresh start rolls back what it created; pre-existing data is kept.")]
Start {
/// Server name (default: "default", or random if default is already running)
#[arg(value_name = "NAME", conflicts_with = "name_flag", value_parser = parse_pg_server_name_arg, add = complete_postgres_servers())]
Expand All @@ -706,9 +713,9 @@ CONTEXT FOR AGENTS:
)]
name_flag: Option<String>,

/// Postgres image tag, major 17 or 18 (e.g. 17-alpine, 18.1). Default: 18
/// Postgres version: 17, 18 or <major>.<minor>. Default: 18
///
/// Pulls the image if it is not present locally.
/// With --docker, any 17 or 18 image tag (e.g. 17-alpine, 18-bookworm).
#[arg(long, short = 'v', value_parser = crate::local::postgres::parse_pg_tag_arg)]
version: Option<String>,

Expand All @@ -718,19 +725,19 @@ CONTEXT FOR AGENTS:
#[arg(long, value_parser = parse_port_arg)]
port: Option<u16>,

/// POSTGRES_USER (default: postgres)
/// Superuser name (default: postgres)
#[arg(long)]
user: Option<String>,

/// POSTGRES_PASSWORD (default: random 24-char alphanumeric)
/// Superuser password (default: random 24-char alphanumeric)
#[arg(long)]
password: Option<String>,

/// POSTGRES_DB (default: postgres)
/// Database to create (default: postgres)
#[arg(long)]
database: Option<String>,

/// Extra container env vars; repeatable, each key at most once
/// Container env vars, repeatable, one per key; only with --docker
///
/// POSTGRES_USER, POSTGRES_DB and PGDATA are managed and rejected here — use
/// --user/--database. POSTGRES_PASSWORD is accepted, but not together with --password.
Expand All @@ -742,6 +749,12 @@ CONTEXT FOR AGENTS:
)]
env: Vec<String>,

/// Run a Postgres Docker image instead (needs Docker running)
///
/// Applies to a new instance; an existing one keeps its runtime.
#[arg(long)]
docker: bool,

/// Seconds to wait for PostgreSQL readiness (maximum: 600)
#[arg(
long,
Expand Down Expand Up @@ -782,11 +795,11 @@ CONTEXT FOR AGENTS:
/// Stop all Postgres instances in this project
StopAll,

/// Remove a stopped Postgres instance and its data
/// Remove a Postgres instance and its data
#[command(after_help = "\
CONTEXT FOR AGENTS:
Irreversible: removes the container and deletes its data directory. Stop the instance first —
removing a running one errors.")]
Irreversible: deletes the instance's data directory, and its container if it has one. A
running native instance is stopped first; stop a Docker one before removing it.")]
Remove {
/// Name of the instance to remove (default: "default")
#[arg(value_name = "NAME", conflicts_with = "name_flag", value_parser = parse_pg_server_name_arg, add = complete_postgres_servers())]
Expand All @@ -810,8 +823,8 @@ CONTEXT FOR AGENTS:
/// Connect to a running Postgres instance with psql
#[command(after_help = "\
CONTEXT FOR AGENTS:
Managed mode (the default; NAME selects one) execs host `psql` when it is on PATH, else runs
psql inside the container via `docker exec`.
Managed mode (the default; NAME selects one) runs the `psql` that ships with the instance's
PostgreSQL build. For a Docker instance it uses host `psql` when on PATH, else the container's.
Direct mode (--host/--port) requires `psql` on PATH and connects as user/database \"postgres\"
with no password; it does not read managed credentials.
Put wrapper options before `--`; all arguments after it go to psql.
Expand Down Expand Up @@ -1302,6 +1315,7 @@ mod tests {
version: None,
binary: Some(binary),
force: true,
docker: false,
} = local_command(&[
"install",
"--binary",
Expand All @@ -1325,6 +1339,16 @@ mod tests {
"{version}"
);
}
// --docker selects how a Postgres selector installs, never a binary.
assert_eq!(
local_parse_error(&["install", "--binary", "./clickhouse", "--docker"]).kind(),
clap::error::ErrorKind::ArgumentConflict
);
let LocalCommands::Install { docker: true, .. } =
local_command(&["install", "postgres@18-bookworm", "--docker"])
else {
panic!("expected a Docker image install");
};
}

#[test]
Expand Down
46 changes: 41 additions & 5 deletions crates/clickhousectl/src/local/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ pub mod config;
pub mod discovery;
pub mod docker;
pub mod output;
pub(crate) mod pg_bundle;
pub(crate) mod pg_native;
pub mod postgres;
pub(crate) mod procinfo;
pub mod server;
Expand All @@ -28,9 +30,10 @@ pub async fn run(cmd: LocalCommands, json: bool) -> Result<()> {
version,
binary,
force,
docker,
} => match (version, binary) {
(None, Some(binary)) => install_binary(&binary, force, json).await,
(Some(version), None) => install(version, force, json).await,
(Some(version), None) => install(version, force, docker, json).await,
_ => unreachable!("clap requires exactly one of a version or --binary"),
},
LocalCommands::List { remote } => {
Expand Down Expand Up @@ -82,8 +85,32 @@ pub async fn run(cmd: LocalCommands, json: bool) -> Result<()> {
}
}

async fn install_postgres(tag: &str, force: bool, json: bool) -> Result<()> {
/// `local install postgres@<version>`: fetch the native bundle, or with
/// `--docker` pull the image.
async fn install_postgres(tag: &str, force: bool, docker: bool, json: bool) -> Result<()> {
postgres::validate_pg_tag(tag)?;
if docker {
return install_postgres_image(tag, force, json).await;
}
let Some(request) = pg_bundle::PgVersionRequest::parse(tag) else {
return Err(Error::usage(
clap::error::ErrorKind::ArgumentConflict,
format!(
"postgres@{tag} is a Docker image tag; add `--docker` to pull it, or use 17, 18 \
or <major>.<minor> for a native PostgreSQL build"
),
));
};
let bundle = pg_bundle::ensure_bundle(&request, force, json).await?;
let out = output::InstallOutput {
version: format!("postgres@{}", bundle.spec.postgres_version),
set_as_default: false,
};
output::print_output(&out, json);
Ok(())
}

async fn install_postgres_image(tag: &str, force: bool, json: bool) -> Result<()> {
let docker = docker::connect().await?;
if !force && docker::image_exists(&docker, tag).await? {
let out = output::InstallOutput {
Expand All @@ -107,11 +134,19 @@ async fn install_postgres(tag: &str, force: bool, json: bool) -> Result<()> {
Ok(())
}

async fn install(version: InstallVersionArg, force: bool, json: bool) -> Result<()> {
async fn install(version: InstallVersionArg, force: bool, docker: bool, json: bool) -> Result<()> {
let spec = match version {
InstallVersionArg::ClickHouse(spec) => spec,
InstallVersionArg::Postgres(tag) => return install_postgres(&tag, force, json).await,
InstallVersionArg::Postgres(tag) => {
return install_postgres(&tag, force, docker, json).await;
}
};
if docker {
return Err(Error::usage(
clap::error::ErrorKind::ArgumentConflict,
"--docker applies only to a Postgres selector such as postgres@18",
));
}
let platform = version_manager::platform::Platform::detect()?;

let version =
Expand Down Expand Up @@ -1306,7 +1341,8 @@ pub(crate) fn list_project_servers(list: output::ProjectList, json: bool) -> Res
match e.info {
Some(info) => {
let is_ch = info.engine == server::Engine::Clickhouse;
let pid = if is_ch && running {
let native_pg = info.pg_runtime() == Some(server::PgRuntime::Native);
let pid = if (is_ch || native_pg) && running {
Some(info.pid)
} else {
None
Expand Down
Loading
Loading