From 939f48f7a8424fdf28e1cd75c7abbf0c5040a85e Mon Sep 17 00:00:00 2001 From: Greg Harris Date: Mon, 24 Aug 2026 18:43:52 +0000 Subject: [PATCH 1/4] security_context_builder: propose .security/ context bundle --- .security/architecture.md | 55 +++++++++++++++++++++++ .security/secure-coding.md | 37 +++++++++++++++ .security/security.yaml | 23 ++++++++++ .security/threat-model.md | 52 ++++++++++++++++++++++ .security/trust-boundaries.md | 84 +++++++++++++++++++++++++++++++++++ 5 files changed, 251 insertions(+) create mode 100644 .security/architecture.md create mode 100644 .security/secure-coding.md create mode 100644 .security/security.yaml create mode 100644 .security/threat-model.md create mode 100644 .security/trust-boundaries.md diff --git a/.security/architecture.md b/.security/architecture.md new file mode 100644 index 0000000..0959c79 --- /dev/null +++ b/.security/architecture.md @@ -0,0 +1,55 @@ +> Derived-from: snowflakedb/snowflake-telemetry-python@8b2709d · generated 2026-04-08 +> Regenerate when: a new top-level package is added under `src/snowflake/telemetry/`, or a socket/HTTP/file-write call is added anywhere in `src/` + +## What this is + +`snowflake-telemetry-python` is a Python library, not a service: it has no listener, +no ingress, and no deployment topology of its own. It is installed into, and runs +inside, whatever Python process imports it — per `README.md`, that process is a +Python UDF, UDTF, or Stored Procedure executing inside Snowflake's Python runtime, or +any other Python program a developer chooses to add the dependency to. + +## Components + +- **`snowflake.telemetry`** (`src/snowflake/telemetry/__init__.py`) — the public API: + `add_event()` and `set_span_attribute()`, both thin wrappers around + `opentelemetry.trace.get_current_span()`. +- **`snowflake.telemetry.logs`** (`src/snowflake/telemetry/logs/__init__.py`) — + `SnowflakeLogFormatter`, a `logging.Formatter` that renders a Python `LogRecord` as + the JSON shape Snowflake's log ingestion expects. +- **`snowflake.telemetry.trace`** (`src/snowflake/telemetry/trace/__init__.py`) — + `SnowflakeTraceIdGenerator`, an OpenTelemetry `RandomIdGenerator` subclass. +- **`snowflake.telemetry._internal.serialize`** + (`src/snowflake/telemetry/_internal/serialize/__init__.py`) — a hand-written, + dependency-free protobuf wire-format encoder (`MessageMarshaler`, `Varint`) that + every generated marshaler class below is built on. This exists so the package does + not need to depend on the `protobuf` runtime package (`CHANGELOG.md`, 0.6.0 entry). +- **`snowflake.telemetry._internal.opentelemetry.proto.*`** — marshaler classes for + the OTLP trace/log/metric/resource/common protobuf messages, generated by + `scripts/proto_codegen.sh` from the `open-telemetry/opentelemetry-proto` schema + (see `threat-model.md` and `secure-coding.md` for how that generation is kept + verifiable). +- **`snowflake.telemetry._internal.opentelemetry.exporter.otlp.proto.common`** — + encoder functions (`encode_spans`, `encode_logs`, `encode_metrics`) vendored from + `open-telemetry/opentelemetry-python` by `scripts/vendor_otlp_proto_common.sh`, + converting OpenTelemetry SDK objects into the generated marshaler types above. +- **`snowflake.telemetry._internal.exporter.otlp.proto.{traces,metrics}`** — + `ProtoSpanExporter`/`ProtoMetricExporter` + (`src/snowflake/telemetry/_internal/exporter/otlp/proto/traces/__init__.py`, + `src/snowflake/telemetry/_internal/exporter/otlp/proto/metrics/__init__.py`), + OpenTelemetry SDK `SpanExporter`/`MetricExporter` + implementations that serialize to OTLP protobuf bytes and then call an + application-supplied `SpanWriter.write_span()` / `MetricWriter.write_metrics()` — + both declared `abc.abstractmethod` with no implementation shipped in this repo. + +## Nothing here is internet-facing + +There is no code in `src/` that opens a socket, makes an HTTP/gRPC call, spawns a +subprocess, or writes a file. A search of `src/` for the regex +`socket|urllib|requests|http\.client|grpc|subprocess|os\.system|open\(` (via both +`rg` and Python's `re` module) returns zero matches anywhere under `src/`. A +broader search (`import socket|import subprocess|import requests|urllib|grpc|os\.system|\bopen\(`) +likewise finds nothing under `src/`, corroborating the "no socket/HTTP/gRPC/subprocess/ +file-write code in `src/`" conclusion above. +See `trust-boundaries.md` for what this means for where data this library serializes +actually goes. diff --git a/.security/secure-coding.md b/.security/secure-coding.md new file mode 100644 index 0000000..956f860 --- /dev/null +++ b/.security/secure-coding.md @@ -0,0 +1,37 @@ +> Derived-from: snowflakedb/snowflake-telemetry-python@8b2709d · generated 2026-04-08 +> Regenerate when: a third vendoring/codegen script is added, or an existing +> `check-*.yml` workflow is removed without its script being removed too + +## Pin the upstream ref in the script, and add a `check-*.yml` workflow that regenerates and diffs + +This repository has two places where code is not hand-written but produced from an +upstream OpenTelemetry repository, and both follow the same pattern: + +- `scripts/proto_codegen.sh` pins the source repo/tag in a shell variable + (`PROTO_REPO_BRANCH_OR_COMMIT="v1.7.0"`, `scripts/proto_codegen.sh:15`), clones + `open-telemetry/opentelemetry-proto` at that ref, and regenerates the marshaler + code under `src/snowflake/telemetry/_internal/opentelemetry/proto/`. + `.github/workflows/check-codegen.yml` runs this script from a clean checkout on + every push/PR that touches `scripts/**` or that generated-code directory, and + fails the build if the regeneration differs from what is committed. +- `scripts/vendor_otlp_proto_common.sh` does the same for + `open-telemetry/opentelemetry-python` (`REPO_BRANCH_OR_COMMIT="v1.38.0"`, + `scripts/vendor_otlp_proto_common.sh:12`), regenerating + `src/snowflake/telemetry/_internal/opentelemetry/exporter/`. + `.github/workflows/check-vendor.yml` is the equivalent CI check. + +**When adding a third source of vendored or generated code, follow the same shape:** +pin the upstream ref in the script itself (not only in a comment), and add a +`check-*.yml` workflow, scoped by `paths:` to the script and its output directory, +that deletes the existing output, regenerates it, and fails on `git diff`. + +**What this convention proves, and what it does not** — see `trust-boundaries.md` +and `threat-model.md` (INV-2, INV-3): it proves the committed code matches what the +pinned ref currently produces. It does not prove the pinned ref's content is safe, +and none of the three refs pinned in this repository's scripts or workflows +(`v1.7.0`, `v1.38.0`, and the third-party GitHub Actions used in the workflow +files under `.github/workflows/`) are pinned to an immutable commit SHA — all are tags or +tag-like refs, which a repository owner could in principle move. Do not describe +this convention as verifying the safety of the upstream source; describe it as +verifying that this repository's copy is in sync with what that source currently +contains. diff --git a/.security/security.yaml b/.security/security.yaml new file mode 100644 index 0000000..c8809fd --- /dev/null +++ b/.security/security.yaml @@ -0,0 +1,23 @@ +apiVersion: snowflake.com/security/v1alpha1 +kind: SecurityContext +metadata: + name: snowflake-telemetry-python + annotations: + owner: "@snowflakedb/telemetry" +spec: + jira_area: "Data Platform: Data Pipelines" + jira_component: "DP - Telemetry" + product_owner: "joe.witt@snowflake.com" + documents: + - path: architecture.md + kind: context + description: "Package layout of this OTel-extension library and why nothing in it is internet-facing" + - path: trust-boundaries.md + kind: context + description: "Why this is a same-process library (no auth boundary), that it performs no network/file I/O itself, and the one caller-input guard it does enforce (log-formatter reserved attributes)" + - path: threat-model.md + kind: threat-model + description: "Systems this package's own code and build scripts talk to, and the invariants (log-attribute protection, vendored/generated-code sync, publish-token scoping) with their controls and status" + - path: secure-coding.md + kind: secure-coding + description: "The adopted convention for vendoring or regenerating code from an upstream OpenTelemetry repository" diff --git a/.security/threat-model.md b/.security/threat-model.md new file mode 100644 index 0000000..6de0030 --- /dev/null +++ b/.security/threat-model.md @@ -0,0 +1,52 @@ +> Derived-from: snowflakedb/snowflake-telemetry-python@8b2709d · generated 2026-04-08 +> Regenerate when: `setup.py`'s `opentelemetry-api`/`opentelemetry-sdk` pin changes, +> either upstream ref pinned in `scripts/proto_codegen.sh` or +> `scripts/vendor_otlp_proto_common.sh` changes, a CI workflow under +> `.github/workflows/` is added or removed, or an invariant's control changes + +## The system + +`snowflake-telemetry-python` is a **library** (see `architecture.md`), not a service. + +- **Driven by** whatever Python process imports it and calls its API. Per + `README.md`, that is a Python UDF, UDTF, or Stored Procedure running inside + Snowflake's Python execution environment, or any other Python program that adds + the dependency. +- **Changed by** anyone able to merge to this repository; `.github/CODEOWNERS:1` + designates `@snowflakedb/telemetry` as the owner of every path in the tree. +- **Trusted with** no credentials and no secrets — none are read, held, or + transmitted anywhere in `src/`. What it is trusted with is *correctness of + serialization*: every process that calls its API, or wires up + `SnowflakeTraceIdGenerator` / `ProtoSpanExporter` / `ProtoMetricExporter`, relies + on it to produce well-formed OTLP protobuf bytes and to hand them only to the + writer the embedding application supplied (`trust-boundaries.md`) — never to an + implicit destination of its own choosing. Because the package is published to the + public PyPI registry and, via a separate Jenkins-triggered build, to a private + Snowflake-operated conda channel (`trust-boundaries.md`, "Two independent + build/publish pipelines"), a change that broke either property would ship to every + process that installs the package from either channel. +- **Reads** nothing beyond its own Python-level function arguments and, at + maintainer-invoked dev time only, two upstream OpenTelemetry git repositories + cloned by `scripts/proto_codegen.sh` and `scripts/vendor_otlp_proto_common.sh` (see + below). It holds no runtime configuration, no credentials, and no per-tenant state. + +## Systems and trust boundaries + +| System | Role here | Crosses the boundary | Trusted for | +|---|---|---|---| +| `opentelemetry-api` / `opentelemetry-sdk` (PyPI, pinned `==1.38.0` in `setup.py`) | runtime dependency this package extends (`get_current_span`, `RandomIdGenerator`, `SpanExporter`, `MetricExporter`) | installed into, and imported by, every process using this package | that the pinned version behaves as documented — a defect or a compromised release at that exact version would run in every embedding process | +| `open-telemetry/opentelemetry-proto` (git, tag `v1.7.0` pinned at `scripts/proto_codegen.sh:15`) | schema source for the generated marshaler code under `_internal/opentelemetry/proto/` | cloned by a maintainer running `scripts/proto_codegen.sh`; never fetched at package-install or runtime | that the content at the pinned tag, when regenerated, is what CI (`.github/workflows/check-codegen.yml`) sees committed to this repo | +| `open-telemetry/opentelemetry-python` (git, tag `v1.38.0` pinned at `scripts/vendor_otlp_proto_common.sh:12`) | source of the vendored encoder code under `_internal/opentelemetry/exporter/otlp/proto/common/` | cloned by a maintainer running `scripts/vendor_otlp_proto_common.sh`; never fetched at package-install or runtime | that the content at the pinned tag, when regenerated, is what CI (`.github/workflows/check-vendor.yml`) sees committed to this repo | +| GitHub Actions (workflow files under `.github/workflows/`) + third-party actions (`actions/checkout@v3/v4`, `actions/setup-python@v3`, `pypa/gh-action-pypi-publish@release/v1`) | builds, tests, and (on `release: published`) publishes this package | source in, package artifact out; `PYPI_API_TOKEN` secret out to the publish step only | that only this repository's reviewed source is built, and that `PYPI_API_TOKEN` is used by, and visible to, no step other than `python-publish.yml`'s publish step | +| PyPI (public registry) | public distribution channel for the package built by `python-publish.yml` | release artifact out | serving exactly the artifact this repo's workflow uploaded, unaltered, to every `pip install snowflake-telemetry-python` | +| Jenkins job "SnowflakeTelemetryPythonPackageBuilder" + `repo.anaconda.com/pkgs/snowflake/` conda channel (`build.sh:4`, `build.sh:38`) | second, non-GitHub-Actions build/publish path, producing a conda package | source in (checked out by Jenkins, not visible from this repo), package artifact out to the private channel | building the published conda package only from this repository's own reviewed source, the same way `python-publish.yml` does for the PyPI artifact | +| The embedding application's `SpanWriter`/`MetricWriter` subclass | sole consumer of the OTLP protobuf bytes this repo's exporters produce | serialized bytes out, in-process | not silently discarding, corrupting, or misrouting the telemetry data this library hands it | + +## Invariants + +| ID | Invariant | Control | Status | +|---|---|---|---| +| INV-1 | A caller-supplied `logging` `extra=` value cannot overwrite the `code.lineno`, `code.function`, `code.filepath`, `exception.type`, `exception.message`, or `exception.stacktrace` attributes `SnowflakeLogFormatter` derives from the actual call site | `src/snowflake/telemetry/logs/__init__.py` (`_RESERVED_ATTRS`, `SnowflakeLogFormatter.format`) — mechanism detailed in `trust-boundaries.md` | Holding (`tests/test_snowflake_log_formatter.py#test_normal_log`) | +| INV-2 | The vendored encoder code under `_internal/opentelemetry/exporter/otlp/proto/common/` matches what a fresh regeneration from the pinned `opentelemetry-python` tag (`scripts/vendor_otlp_proto_common.sh:12`) produces | `.github/workflows/check-vendor.yml` (regenerates from scratch and fails on `git diff`) | Holding — covers only "the committed code matches the pinned tag's current content", not "that tag's content is safe"; see the systems-table row above and the gap this leaves open | +| INV-3 | The generated marshaler code under `_internal/opentelemetry/proto/` matches what a fresh regeneration from the pinned `opentelemetry-proto` tag (`scripts/proto_codegen.sh:15`) produces | `.github/workflows/check-codegen.yml` (regenerates from scratch and fails on `git diff`) | Holding — same partial-coverage caveat as INV-2 | +| INV-4 | `PYPI_API_TOKEN` is used only inside the publish step of `.github/workflows/python-publish.yml`, and that workflow runs only when a GitHub release is published | `.github/workflows/python-publish.yml` (trigger `release: types: [published]` at `:11-13`; token read only at `:39`, inside the `pypa/gh-action-pypi-publish` step at `:36`) | Holding | diff --git a/.security/trust-boundaries.md b/.security/trust-boundaries.md new file mode 100644 index 0000000..4890b43 --- /dev/null +++ b/.security/trust-boundaries.md @@ -0,0 +1,84 @@ +> Derived-from: snowflakedb/snowflake-telemetry-python@8b2709d · generated 2026-04-08 +> Regenerate when: `SpanWriter`/`MetricWriter` gain a default (non-abstract) +> implementation, `_RESERVED_ATTRS` changes, or any network/file I/O is added to `src/` + +## The caller is the trust boundary, not this code + +This is a library called in-process. Its only "input" is whatever arguments the +embedding Python process passes to `snowflake.telemetry.add_event()`, +`snowflake.telemetry.set_span_attribute()`, or a `logging` call that flows through a +`logging.Handler` configured with `SnowflakeLogFormatter` +(`src/snowflake/telemetry/logs/__init__.py`). The caller and the code executing this +library's functions run as the same principal, in the same process — there is no +network hop, no serialization boundary, and no authentication step inside this +repository between "whoever can execute Python here" and "whoever can call this +API." Any code that can import `snowflake.telemetry` can call every function in it +with arbitrary argument values; this repository enforces no authorization on top of +that. + +## This library never transmits or persists the data it serializes + +`ProtoSpanExporter._serialize_traces_data()` +(`src/snowflake/telemetry/_internal/exporter/otlp/proto/traces/__init__.py`) and +`ProtoMetricExporter._serialize_metrics_data()` +(`src/snowflake/telemetry/_internal/exporter/otlp/proto/metrics/__init__.py`) produce +`bytes` and immediately hand them to `self.span_writer.write_span(...)` / +`self.metric_writer.write_metrics(...)`. `SpanWriter.write_span` and +`MetricWriter.write_metrics` are declared with `@abc.abstractmethod` and have no +body in either class — the embedding application must subclass `SpanWriter` or +`MetricWriter` and implement the method itself before any serialized data goes +anywhere. This repository ships no such subclass (the closest thing, an +`InMemorySpanWriter`/`InMemoryMetricWriter`, exists only under `tests/`, per the +module docstrings in both files). + +Combined with the absence of any socket/HTTP/file-write call anywhere in `src/` +(see `architecture.md`), a finding that assumes this repository's code sends +telemetry data to a specific network destination, an S3 bucket, or a database is +describing the *embedding application's* writer implementation — which lives +outside this repository — not code that exists here. + +## The one caller-input guard this repo does enforce: reserved log attributes + +`SnowflakeLogFormatter.format()` (`src/snowflake/telemetry/logs/__init__.py`) builds +its output `attributes` dict in two steps: it first sets `code.lineno`, +`code.function`, `code.filepath` from the real `LogRecord` fields, and — when an +exception is attached — `exception.type`, `exception.message`, +`exception.stacktrace`. It then iterates `record.__dict__.items()` (which includes +whatever a caller passed via `logging`'s `extra=` argument) and skips any key that +appears in `_RESERVED_ATTRS`, a frozenset that includes exactly those six +`code.*`/`exception.*` keys plus the standard `logging.LogRecord` attribute names +(`src/snowflake/telemetry/logs/__init__.py:11-42`). The effect: a caller cannot use +`extra={"code.lineno": ...}` (or any of the other reserved keys) to overwrite the +values this formatter derives from the actual call site. This is exercised by +`tests/test_snowflake_log_formatter.py#test_normal_log`, which passes +`extra={"body": 123, "code.lineno": 35}` and asserts the emitted `code.lineno` is +the real line number (21 in that test), not the caller-supplied 35, while the +non-reserved `body` key is passed through unchanged. + +No other caller-supplied value (event names, span/log attribute values, metric +data points) is validated, size-capped, or filtered by this repository before being +handed to the OpenTelemetry SDK or serialized to protobuf bytes. + +## `SnowflakeTraceIdGenerator`'s trace-ID generation + +`SnowflakeTraceIdGenerator.generate_trace_id()` +(`src/snowflake/telemetry/trace/__init__.py`) builds a 16-byte trace ID from a +4-byte big-endian encoding of "minutes since the epoch" followed by 12 bytes from +`random.getrandbits(96)` — the stdlib's non-cryptographic PRNG, not the `secrets` +module. + +## Two independent build/publish pipelines exist for this package + +- `.github/workflows/python-publish.yml` runs on `release: types: [published]` + (`:11-13`) and publishes to the public PyPI registry using + `pypa/gh-action-pypi-publish@release/v1` (`:36`) authenticated with the + `PYPI_API_TOKEN` repository secret (`:39`) — a stored token, not PyPI's OIDC + "trusted publishing." +- `build.sh` and `pypi-build.sh` are, per their own header comments, "called from a + Jenkins job called SnowflakeTelemetryPythonPackageBuilder" (`build.sh:4`, + `pypi-build.sh:4`) rather than from any GitHub Actions workflow in this repo. + `build.sh` additionally adds a private conda channel, + `https://repo.anaconda.com/pkgs/snowflake/` (`build.sh:38`), before building the + conda-format package and removes it afterward (`build.sh:56`). Neither the + Jenkins job's configuration nor what consumes packages from that channel is + visible from this repository. From 75a72df0e5b7385e6510bcabb1a9b87dadb19772 Mon Sep 17 00:00:00 2001 From: Eugene Kim Date: Fri, 2 Oct 2026 17:12:36 +0000 Subject: [PATCH 2/4] security.yaml: route to PIE - Observability / Observability - External Observability Update Jira routing (area, component) and product_owner for this SecurityContext to the External Observability team. Co-authored-by: Cursor --- .security/security.yaml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.security/security.yaml b/.security/security.yaml index c8809fd..ff29178 100644 --- a/.security/security.yaml +++ b/.security/security.yaml @@ -5,9 +5,9 @@ metadata: annotations: owner: "@snowflakedb/telemetry" spec: - jira_area: "Data Platform: Data Pipelines" - jira_component: "DP - Telemetry" - product_owner: "joe.witt@snowflake.com" + jira_area: "PIE - Observability" + jira_component: "Observability - External Observability" + product_owner: "shangcheng.ying@snowflake.com" documents: - path: architecture.md kind: context From 1205ae2f1baedbf6349107b4188e73e90db0ace4 Mon Sep 17 00:00:00 2001 From: Eugene Kim Date: Fri, 2 Oct 2026 17:14:01 +0000 Subject: [PATCH 3/4] security.yaml: set owner annotation to triage-observability-external-dl Aligns the owner contact with the External Observability Jira routing. Co-authored-by: Cursor --- .security/security.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.security/security.yaml b/.security/security.yaml index ff29178..46e2280 100644 --- a/.security/security.yaml +++ b/.security/security.yaml @@ -3,7 +3,7 @@ kind: SecurityContext metadata: name: snowflake-telemetry-python annotations: - owner: "@snowflakedb/telemetry" + owner: "triage-observability-external-dl@snowflake.com" spec: jira_area: "PIE - Observability" jira_component: "Observability - External Observability" From 0863c1f6d8f8d2d344faa04b0004d3171d281384 Mon Sep 17 00:00:00 2001 From: Eugene Kim Date: Fri, 2 Oct 2026 21:06:59 +0000 Subject: [PATCH 4/4] security.yaml: strip internal routing metadata for public repo Remove jira_area, jira_component, and product_owner (internal Jira taxonomy and a personal email) and the internal DL from the owner annotation (restored to the public GitHub team). Internal routing lives in the internal mirror copy of this bundle. metadata.name and the documents list are unchanged. Co-authored-by: Cursor --- .security/security.yaml | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/.security/security.yaml b/.security/security.yaml index 46e2280..1783cb4 100644 --- a/.security/security.yaml +++ b/.security/security.yaml @@ -3,11 +3,8 @@ kind: SecurityContext metadata: name: snowflake-telemetry-python annotations: - owner: "triage-observability-external-dl@snowflake.com" + owner: "@snowflakedb/telemetry" spec: - jira_area: "PIE - Observability" - jira_component: "Observability - External Observability" - product_owner: "shangcheng.ying@snowflake.com" documents: - path: architecture.md kind: context