diff --git a/.claude/skills/issue-classification/SKILL.md b/.claude/skills/issue-classification/SKILL.md new file mode 100644 index 00000000..b7b7fdec --- /dev/null +++ b/.claude/skills/issue-classification/SKILL.md @@ -0,0 +1,175 @@ +--- +name: issue-classification +description: > + The classes of an issue in this ClickHouse client repository: the impacts + (Security, Data integrity, Stability, Performance) and the area (Connectivity, + Data types & formats, Queries, Configuration, API & diagnostics, Environment, + Docs & project, Other). Use this skill when the task is to "classify an + issue" or "give the classes of an issue", and when you triage, file, or + prioritize an issue: the issue-priority skill uses the impacts. +--- + +# Issue classification + +The classes name **what kind of problem an issue is about**, with the same words in every +ClickHouse client repo. They are an input to the priority, and they make issue counts +comparable across clients. + +An issue has: + +- **Impacts** — what goes wrong: none, one, or more of the four impact classes. +- **One area** — the part of the client whose behavior is wrong or missing. + +The classes do not replace the kind of request (bug, feature, question) or the reach tier of +a repo's priority rules. + +Every ClickHouse client repository has a copy of this file, so that any agent can classify an +issue in the repository. Keep the copies the same where you can, so that the classes mean the +same in every client. Put a change that applies only to this repository in the section "This +repository" at the end. + +## Rules + +1. Give each impact that applies, and one area. +2. Use only what the issue, a reproduction, or code inspection of the current code + **shows**. Do not give a class for an effect + that is only possible. Classes can raise the priority, so each class needs evidence. +3. Classify by the **observed effect**, not by the location of the code. Example: a + connection-string defect that makes the client drop rows with no error has the impact + Data integrity. +4. Data integrity and Stability are for defects only. Security and Performance are also for + feature requests: a new authentication method, a request to make an operation faster. + The priority rules rate such requests as feature requests. +5. An issue with no harm has no impact. This is the normal case. + +## Impact classes + +| Impact | Use when | +|---|---| +| **Security** | Two parts. A **vulnerability**: a defect makes the protection weaker: it exposes credentials or data, lets a value change the meaning of a query (injection), skips or weakens a TLS or identity check, corrupts memory (an out-of-bounds write, a use after free, also when only a hostile server or network causes it), or lets input that the application takes from its users stop the client; also a vulnerability in a dependency: a published one (CVE or GHSA) in code that the client calls, and any one that is not published yet. A hang, or a crash with no memory corruption, that only a hostile server or network causes is Stability, not Security. **Authentication**: an authentication method that does not work or is missing, or how the client takes credentials (for example a token provider or a credentials file). A secret that leaks (to a log, an error message, a URL, or another host) is a vulnerability, not authentication. Say which part applies: the priority rules of a repo rate the two parts differently. | +| **Data integrity** | Data that the caller wrote, or a result that the caller reads, is missing, incomplete, or different from the value given or stored, and the client reports no error. This includes a failed operation that the client reports as a success, a server error that the client ignores, overflow, lost precision, a time zone shift, a wrong decode, a stream that loses synchronization, a duplicate write, and a write to the wrong table. | +| **Stability** | The process stops in a way that the caller cannot handle as a normal error (abort, segmentation fault, unrecovered panic, stack overflow), a call does not return (deadlock, hang), or a resource leaks until none is left (connections, sockets, memory). Also a thread-safety defect. | +| **Performance** | The result is correct, but the operation is too slow, or uses too much memory or CPU. Also a request to make an operation faster. | + +Boundaries: + +- **Data integrity needs "no error".** When the client rejects a value with an error, the + issue has no Data integrity impact. When the caller gets an error, but the error is wrong + or not clear, the issue has no Data integrity impact either: it has the area API & + diagnostics. +- **Security and values that are not quoted.** A value or an identifier that the client + does not quote or escape gets Security only when the issue shows a path from untrusted + input. A string value that leaves its literal because the client does not escape it is + such a path. An identifier that the client does not quote, with no such path, is the area + Queries with no impact. +- **TLS.** A TLS check that the client skips or weakens is Security. A connection with TLS + that fails has no impact; its area is Connectivity. +- **Memory.** High memory use without a failure is Performance, not Stability. +- **An error is not Stability.** A normal exception or error value is not a crash. In Go + and Rust, a panic that the library does not recover is Stability. In Java, a + `StackOverflowError`, `OutOfMemoryError`, or `VerifyError` is Stability. + +## A vulnerability that is not public yet + +A security issue must not stay public with a description of how to reproduce it. An issue +is a **confirmed vulnerability** when a run or code inspection shows one of these: + +- credentials or data are exposed to a party that must not get them; +- a value changes the meaning of a query (injection); +- a TLS or identity check is skipped or weakened; +- memory corruption (an out-of-bounds write, a use after free), also when only a hostile + server or network causes it; +- a vulnerability in a dependency that is not published yet, also in code that the client + does not call (also report it to the maintainers of that dependency in private, through + their security policy, never in a public issue); +- input that the application takes from its users stops the client. + +A hang, or a crash with no memory corruption, that only a hostile server or network causes is +not a vulnerability: it is Stability, and it is triaged in public. A crash while the client +reads data from the server (for example an access violation or `SIGSEGV`) can be memory +corruption: rule that out by code inspection before you treat it as Stability. + +Never put a confirmed vulnerability into a public comment, issue, pull request, or +Classification line. Report it in a private GitHub vulnerability report for this repository +(**Security** → **Report a vulnerability**). The issue-priority skill of this repository says +how to handle the public issue and how to rate the vulnerability. When you can neither confirm +nor rule out a vulnerability, do not discuss it in public either: ask a maintainer in private, +or send a private report that says what is not confirmed. + +These are not affected: a vulnerability that is already public (a published CVE or GHSA, for +example in a dependency), and the part authentication (a method that does not work or is +missing). Classify them as usual. + +## Area classes + +| Area | Scope | +|---|---| +| **Connectivity** | Connect and handshake, a connection with TLS, proxies and redirects, the connection pool, keep-alive, reconnect and failover, retries, timeouts, cancellation, the transport (HTTP, native TCP), compression on the wire. | +| **Data types & formats** | Parsing of type names, encoding and decoding of values, the mapping to language types and to rows or objects (POJO, serde, struct), data formats (Native, RowBinary, JSON, Parquet, Arrow), support for a new type or format. | +| **Queries** | How the client builds a statement: parameters and placeholders, quoting and escaping, SQL parsing in the client, size limits of the statement and its parameters. How the client runs a statement: batches and streaming inserts, sessions and roles, external data, totals, asynchronous inserts, the queries that the client sends on its own (`DESCRIBE`, server information), caches of table structure. | +| **Configuration** | Reading and passing options: connection strings, DSNs, URLs, and option objects (parsing, validation, defaults); ClickHouse settings for a connection or a query; settings that the client sends on its own. | +| **API & diagnostics** | The shape of the public API (a missing method or overload, a wrong type signature), and conformance to a standard interface that the client implements (JDBC, ODBC, ADO.NET, `database/sql`, DB-API, SQLAlchemy, R2DBC), including metadata and catalog functions. Errors that are wrong, incomplete, or not clear. Logging, metrics, tracing, progress, the query ID. | +| **Environment** | A build from sources, support for a compiler, runtime, operating system, or architecture, package managers, released artifacts, installers, dependencies. Also a failure with one ClickHouse server version only, or with one third-party tool only (Power BI, Tableau, Spark, DBeaver). | +| **Docs & project** | Documentation, examples, or the changelog are wrong or missing. Tests, CI, the release process, refactoring. | +| **Other** | No area fits. | + +Boundaries: + +- **Configuration or the area that an option controls.** If the client reads an option + correctly but the behavior that the option controls is wrong, use the area of that + behavior. Example: a keep-alive option that has no effect is Connectivity. +- **Authentication.** An authentication issue has the impact Security and the area + Connectivity. +- **Environment or the area of the failure.** Use Environment when the same call works in + general, and fails only with one server version, one platform, or one third-party tool. + A failure with all server versions and without the tool has the area of the failure. +- **API & diagnostics or the area of the failure.** When the problem is the quality of an + error, use API & diagnostics. When the problem is that the operation fails at all, use + the area of the operation. + +Rules for **Other**: + +- Other is the last choice. Say why no area fits. +- Other does not raise the priority. +- A subject that repeats in Other is a sign that a class is missing: report it. + +## Examples + +| Issue | Impacts | Area | +|---|---|---| +| A bare `Enum(...)` type hint is written as `Enum8`, which corrupts the data | Data integrity | Data types & formats | +| A flush returns no error when the server rejects the INSERT | Data integrity | Queries | +| The connection pool leaks connections until none is left | Stability | Connectivity | +| String parameters are not escaped | Security (vulnerability) | Queries | +| `LowCardinality(Nullable(String))` cannot be read: the client returns an error | none | Data types & formats | +| A server error loses its code when the response arrives in two chunks | none | API & diagnostics | +| A request for Kerberos authentication | Security (authentication) | Connectivity | +| Each compressed read fails on one new server version | none | Environment | +| Parameter binding takes quadratic time | Performance | Queries | + +## How to write the classification + +One line, in this form. In a triage comment, the line has the name **Classification**: + +`impact: · area: ()` + +For Security, add the part in parentheses: `Security (vulnerability)` or +`Security (authentication)`. + +The **subject** names what the issue is about inside the area, in one to four words. Use the +names that ClickHouse or the client API uses: + +- **Data types & formats:** always the ClickHouse types or the data format, for example + `(Enum8, Enum16)`, `(Map)`, `(DateTime64)`, `(RowBinary)`, `(Parquet)`. "Data types & + formats" alone does not say which type. +- **Other areas:** the feature, for example `(cancellation)`, `(TLS)`, `(connection pool)`, + `(query parameters)`, `(sessions)`, `(mid-stream errors)`, `(logging)`, `(release process)`. +- With two areas, give a subject after each one. + +Examples: `impact: Data integrity · area: Data types & formats (Map)`, +`impact: none · area: Configuration (URL parameters)`, +`impact: Security (vulnerability), Stability · area: Connectivity (TLS)`. + +## This repository + +No changes to the shared text. diff --git a/.claude/skills/issue-priority/SKILL.md b/.claude/skills/issue-priority/SKILL.md new file mode 100644 index 00000000..07e3bab9 --- /dev/null +++ b/.claude/skills/issue-priority/SKILL.md @@ -0,0 +1,182 @@ +--- +name: issue-priority +description: > + Decide the priority of a `ClickHouse/clickhouse-cpp` issue: `Urgent`, + `High`, `Medium`, or `Low`, for the org issue field **Priority**. Use this + skill when the task is to "prioritize", "set the priority of", "rate", or + "triage" an issue, or to explain why an issue has a given priority. It + defines the reach tiers of the library, the severity rows, the base-level + matrix, the modifiers, and the confidence. +--- + +# Issue priority + +This skill is the single source of truth for the **Priority** of +`ClickHouse/clickhouse-cpp` issues. People and automated triage use the same +rules. + +The value goes into the org issue field **Priority** (`Urgent`, `High`, +`Medium`, `Low`). The field is visible to org members only. An automated tool +sets the value once, never changes a value that is already set, and keeps the +level and its reason out of public comments. + +## When to set no priority + +Set no priority when one of these is true: + +- A maintainer can close the issue with no further action: it is already fixed + on `master`, a duplicate, works as intended, already supported, answered, or + obsolete. "Works as intended" includes a failure that shows only when two + threads use one `Client` (README, "Thread-safety"). +- Only a deprecated public API reaches the defect: a `[[deprecated]]` + attribute, a header that prints a deprecation `#warning` (for example + `clickhouse/columns/utils.h`), or docs that call the API deprecated. Fixes to + deprecated code are not released. One exception: the Abseil-backed + `Int128`/`UInt128` (`CH_ABSEIL_BIGNUM_DEPRECATED`) is deprecated, but it is + the default of the 2.x build (`CH_USE_ABSEIL_FOR_BIGNUM=ON`). Rate its + defects normally (section 1). +- The issue is a question, or it is about CI, the release process, or a + discussion. + +## Vulnerabilities that are not public yet + +A security issue must not stay public with a description of how to reproduce +it. This section applies when a reproduction or code inspection confirms a +vulnerability that is not public yet. The classification skill of this +repository defines it +([`../issue-classification/SKILL.md`](../issue-classification/SKILL.md), "A +vulnerability that is not public yet"): memory corruption is one also when only +a hostile server or network causes it, and a hang, or a crash with no memory +corruption, that only such a server or network causes is not one. + +- Report it in a private GitHub vulnerability report for this repository + (**Security** → **Report a vulnerability**), never in a public issue, + comment, or pull request. If a public issue reports it, copy the title, the + text, and the comments of the issue into the report first. +- Set no Priority on the public issue. Close it with a comment that gives no + details, and ask a repository administrator to delete it: closing does not + hide the text of the issue. +- Rate the vulnerability with the rules below, and put the level and its reason + into the private report as the proposed severity. + +A vulnerability that is already public (a published CVE or GHSA, for example in +a dependency), and an authentication method that does not work or is missing, +get a Priority on the issue as usual. + +## 1. Reach + +The reach is the tier of the most-used public path that reaches the defect, +not the tier of the file that contains it. Every `Select()` parses the column +types that the server sends (`clickhouse/types/type_parser.cpp`) and creates +the columns (`clickhouse/columns/factory.cpp`). So a parser defect for a common +type is Core, and a parser defect that only an unusual type name reaches is +Narrow. + +| Tier | Public paths | +| --------- | ------------ | +| Core | `Client` connect, handshake, and authentication; `Execute()`, `Select()`, `Insert()`, `Ping()`, `ResetConnection()`; endpoint failover (`SetEndpoints`); timeouts and keep-alive in `ClientOptions`; TLS (`SetSSLOptions`, `WITH_OPENSSL`; ClickHouse Cloud needs it); error handling (`ServerException`, `ProtocolError`, and the connection state after an error); `Block` and the Native codec for `(U)Int8`–`(U)Int64`, `Float32`, `Float64`, `String`, `FixedString`, `Date`, `DateTime`, `DateTime64`, `Decimal32`–`Decimal128`, `UUID`, `Nullable(T)`, `Array(T)`, and `LowCardinality(String)`; the CMake build with the default options on a platform and compiler that CI tests | +| Common | compression (`SetCompressionMethod`: LZ4, ZSTD); multi-block insert (`BeginInsert`, `SendInsertBlock`, `EndInsert`); streaming and cancelable select (`BeginSelect`, `NextBlock`, `SelectCancelable`, `Cancel`); query settings and parameters (`Query::SetSetting`, `SetParam`); `Query` callbacks (progress, profile, server log, profile events, exception); external data (`SelectWithExternalData`); the types `Bool`, `Enum8`, `Enum16`, `Map`, `Tuple`, `IPv4`, `IPv6`, `Int128`, `UInt128`, and the other `LowCardinality` and nested wrappers; column methods (`Append`, `Slice`, `Reserve`, `Clear`, `Swap`, `GetItem`); the build with Bazel, MinGW, shared libraries, system dependencies (`WITH_SYSTEM_*`), `CH_USE_3X_API` and its options, and the `make install` layout | +| Narrow | rare types (`Point`, `Ring`, `Polygon`, `MultiPolygon`, `JSON`, `Time`, `Time64`, `Nothing`, `Nested`); unusual type names (escaped `Enum` labels, deep nesting); input that a correct server does not send (a malformed packet or type name); caller input that the type or the API does not allow, but that the library does not reject (an undeclared `Enum` value, a column of the wrong type); OpenTelemetry tracing context; a compiler, platform, or dependency version that CI does not test | +| Satellite | `examples/`, `bench/`, `ut/`, `tests/`, `docs/`, `README.md`; packages that other projects maintain (Conan Center, vcpkg, Homebrew, distribution packages) | + +The 2.x build uses the Abseil-backed `Int128`/`UInt128` by default. A defect +that only this implementation has is Common (`Int128`, `UInt128`). If the +self-contained implementation does not have the defect, the workaround is the +build flag `-DCH_USE_ABSEIL_FOR_BIGNUM=OFF` (severity "Workaround exists"). + +## 2. Severity + +First name the **impacts** of the issue with the classification skill of this +repository +([`../issue-classification/SKILL.md`](../issue-classification/SKILL.md)): none, +one, or more of Security (part vulnerability or part authentication), Data +integrity, Stability, and Performance. Most issues have none. + +Then use the first severity row that fits. With more than one impact, the +highest row applies: + +| Severity | Use when | +| ----------------------- | -------- | +| Wrong data or security | impact **Data integrity**: the client returns or stores wrong or incomplete data with no error (this includes a block stream that goes out of sync with no error), or reports success for a query that failed; or impact **Security** with a vulnerability: a security defect: a leak of credentials or data, a TLS verification that does not occur, SQL built from a caller value without escaping, memory corruption (an out-of-bounds write, a use after free), or a vulnerable dependency in code that the client calls. A hang, or a crash with no memory corruption, that only a hostile server or network causes does not use this row: rate it by the failure | +| Broken, no workaround | the feature fails (an exception, a hang, a crash, an abort, a leak that ends in a crash or in exhausted sockets), and the caller has no reasonable workaround; also a `Client` that stays unusable after an error, and a build that fails on a supported toolchain; also a wrong or incomplete error with no workaround: the failure is reported, but its type, code, or message is wrong or lost. The impact **Stability** (a hang, a crash, such a leak) uses this row when the caller has no reasonable workaround | +| Workaround exists | the feature fails, but the caller can avoid the failure with a small change in their own code or build flags (name the workaround in the reason); also impact **Performance**: a result that is correct but slow or uses too much memory | +| Cosmetic, docs, feature | wrong log or message text, a compiler warning, documentation, a feature request (this includes support for a new type or a new package manager) | + +An issue that has the impact Security with no vulnerability (an authentication +method that does not work or is missing) does not use the first row. Rate an +authentication method that does not work by its failure ("Broken, no +workaround" or "Workaround exists"), and a request for a missing method as a +feature request. + +A request to make an operation faster has the impact Performance, but it is a +feature request: use "Cosmetic, docs, feature". Only a defect with the impact +Performance (a correct result that is too slow) uses "Workaround exists". + +A vulnerability in a dependency uses "Wrong data or security" when the client +calls the vulnerable code. When it does not, use "Cosmetic, docs, feature": for +a published one (a CVE or GHSA), the issue asks for a dependency update. One +that is not published yet stays private also then (see "Vulnerabilities that +are not public yet"), and this level is the proposed severity of the private +report. + +## 3. Base level + +| Severity ↓ / Reach → | Core | Common | Narrow | Satellite | +| ----------------------- | ------ | ------ | ------ | --------- | +| Wrong data or security | Urgent | Urgent | High | Medium | +| Broken, no workaround | High | High | Medium | Low | +| Workaround exists | Medium | Medium | Low | Low | +| Cosmetic, docs, feature | Low | Low | Low | Low | + +## 4. Modifiers + +Apply them in this order: + +1. **Security with a shown path from untrusted input.** The issue or a + reproduction shows how a value that an application takes from its users + reaches the defect: `Urgent` at any reach. Skip the other modifiers. A + defect that puts a caller-supplied value into SQL without quoting or + escaping is a candidate (for example a table name, a setting, or a query + parameter): look for such a path. If none is shown, rate the defect by the + failure that it shows. A vulnerability that is not public yet goes to a + private vulnerability report, not to a public issue (see "Vulnerabilities + that are not public yet"). +2. **Regression in the latest published release**: up one level. This can give + `Urgent`. It is a regression only when the latest release shows the problem + and the release before it does not. The tags are `vX.Y.Z` (for example + `v2.6.2` and `v2.6.1`). A defect that only `master` has (after the latest + tag) is not a release regression: no modifier, but write "unreleased + regression" in the reason, so that it is fixed before the next release. +3. **Customer request in Linear.** The issue links (`linear.app`) a Linear + issue for a customer request: up one level, but never to `Urgent` (`High` + stays `High`). + +## 5. Confidence + +A missing reproduction lowers the confidence, not the level. Record the +confidence with the value when the tool that sets it supports it: + +| Confidence | Use when | +| ---------- | -------- | +| `HIGH` | a run shows the problem (or, for a feature, code inspection confirms its absence on `master`) | +| `MEDIUM` | code inspection shows a defect, but no reproduction was run (for example, it needs ClickHouse Cloud or a specific platform) | +| `LOW` | the report lacks what a reproduction needs, or a run did not show the problem | + +## 6. Reason + +Write one line with the value: + +` () · · · → .` + +Name the workaround when the severity is "Workaround exists". When the +confidence is `LOW`, say what is missing. + +## 7. Labels from earlier triage + +- `priority-high` and `priority-low` are older labels that a maintainer set. + When an issue has one, use its level (`High` or `Low`) instead of the table + value, and write "maintainer label `priority-…`" in the reason. Never add + these labels. +- `pg_clickhouse` marks a request from the pg_clickhouse PostgreSQL extension. + pg_clickhouse uses clickhouse-c instead of clickhouse-cpp from its v0.3.1 + release, so the label gives no modifier. diff --git a/AGENTS.md b/AGENTS.md index 9fe0f2f9..be71d59a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,3 +1,9 @@ # Communication style - Follow ASD-STE100 + +# When triaging issues + +- To set, check, or explain the Priority of an issue, follow [`.claude/skills/issue-priority/SKILL.md`](.claude/skills/issue-priority/SKILL.md). It is the single source of truth for the priority rules. +- To classify an issue (its impacts and its area), follow [`.claude/skills/issue-classification/SKILL.md`](.claude/skills/issue-classification/SKILL.md). +- A vulnerability that is not public yet goes into a private security report (**Security** → **Report a vulnerability**), never into an issue or a pull request. The classification skill defines such a vulnerability, and the priority skill says how to handle the public issue.