Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions .security/architecture.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
> Derived-from: snowflakedb/snowflake-telemetry-python@8b2709d · generated 2026-04-08
> Regenerate when: a new top-level package is added under `src/snowflake/telemetry/`, or a socket/HTTP/file-write call is added anywhere in `src/`

## What this is

`snowflake-telemetry-python` is a Python library, not a service: it has no listener,
no ingress, and no deployment topology of its own. It is installed into, and runs
inside, whatever Python process imports it — per `README.md`, that process is a
Python UDF, UDTF, or Stored Procedure executing inside Snowflake's Python runtime, or
any other Python program a developer chooses to add the dependency to.

## Components

- **`snowflake.telemetry`** (`src/snowflake/telemetry/__init__.py`) — the public API:
`add_event()` and `set_span_attribute()`, both thin wrappers around
`opentelemetry.trace.get_current_span()`.
- **`snowflake.telemetry.logs`** (`src/snowflake/telemetry/logs/__init__.py`) —
`SnowflakeLogFormatter`, a `logging.Formatter` that renders a Python `LogRecord` as
the JSON shape Snowflake's log ingestion expects.
- **`snowflake.telemetry.trace`** (`src/snowflake/telemetry/trace/__init__.py`) —
`SnowflakeTraceIdGenerator`, an OpenTelemetry `RandomIdGenerator` subclass.
- **`snowflake.telemetry._internal.serialize`**
(`src/snowflake/telemetry/_internal/serialize/__init__.py`) — a hand-written,
dependency-free protobuf wire-format encoder (`MessageMarshaler`, `Varint`) that
every generated marshaler class below is built on. This exists so the package does
not need to depend on the `protobuf` runtime package (`CHANGELOG.md`, 0.6.0 entry).
- **`snowflake.telemetry._internal.opentelemetry.proto.*`** — marshaler classes for
the OTLP trace/log/metric/resource/common protobuf messages, generated by
`scripts/proto_codegen.sh` from the `open-telemetry/opentelemetry-proto` schema
(see `threat-model.md` and `secure-coding.md` for how that generation is kept
verifiable).
- **`snowflake.telemetry._internal.opentelemetry.exporter.otlp.proto.common`** —
encoder functions (`encode_spans`, `encode_logs`, `encode_metrics`) vendored from
`open-telemetry/opentelemetry-python` by `scripts/vendor_otlp_proto_common.sh`,
converting OpenTelemetry SDK objects into the generated marshaler types above.
- **`snowflake.telemetry._internal.exporter.otlp.proto.{traces,metrics}`** —
`ProtoSpanExporter`/`ProtoMetricExporter`
(`src/snowflake/telemetry/_internal/exporter/otlp/proto/traces/__init__.py`,
`src/snowflake/telemetry/_internal/exporter/otlp/proto/metrics/__init__.py`),
OpenTelemetry SDK `SpanExporter`/`MetricExporter`
implementations that serialize to OTLP protobuf bytes and then call an
application-supplied `SpanWriter.write_span()` / `MetricWriter.write_metrics()` —
both declared `abc.abstractmethod` with no implementation shipped in this repo.

## Nothing here is internet-facing

There is no code in `src/` that opens a socket, makes an HTTP/gRPC call, spawns a
subprocess, or writes a file. A search of `src/` for the regex
`socket|urllib|requests|http\.client|grpc|subprocess|os\.system|open\(` (via both
`rg` and Python's `re` module) returns zero matches anywhere under `src/`. A
broader search (`import socket|import subprocess|import requests|urllib|grpc|os\.system|\bopen\(`)
likewise finds nothing under `src/`, corroborating the "no socket/HTTP/gRPC/subprocess/
file-write code in `src/`" conclusion above.
See `trust-boundaries.md` for what this means for where data this library serializes
actually goes.
37 changes: 37 additions & 0 deletions .security/secure-coding.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
> Derived-from: snowflakedb/snowflake-telemetry-python@8b2709d · generated 2026-04-08
> Regenerate when: a third vendoring/codegen script is added, or an existing
> `check-*.yml` workflow is removed without its script being removed too

## Pin the upstream ref in the script, and add a `check-*.yml` workflow that regenerates and diffs

This repository has two places where code is not hand-written but produced from an
upstream OpenTelemetry repository, and both follow the same pattern:

- `scripts/proto_codegen.sh` pins the source repo/tag in a shell variable
(`PROTO_REPO_BRANCH_OR_COMMIT="v1.7.0"`, `scripts/proto_codegen.sh:15`), clones
`open-telemetry/opentelemetry-proto` at that ref, and regenerates the marshaler
code under `src/snowflake/telemetry/_internal/opentelemetry/proto/`.
`.github/workflows/check-codegen.yml` runs this script from a clean checkout on
every push/PR that touches `scripts/**` or that generated-code directory, and
fails the build if the regeneration differs from what is committed.
- `scripts/vendor_otlp_proto_common.sh` does the same for
`open-telemetry/opentelemetry-python` (`REPO_BRANCH_OR_COMMIT="v1.38.0"`,
`scripts/vendor_otlp_proto_common.sh:12`), regenerating
`src/snowflake/telemetry/_internal/opentelemetry/exporter/`.
`.github/workflows/check-vendor.yml` is the equivalent CI check.

**When adding a third source of vendored or generated code, follow the same shape:**
pin the upstream ref in the script itself (not only in a comment), and add a
`check-*.yml` workflow, scoped by `paths:` to the script and its output directory,
that deletes the existing output, regenerates it, and fails on `git diff`.

**What this convention proves, and what it does not** — see `trust-boundaries.md`
and `threat-model.md` (INV-2, INV-3): it proves the committed code matches what the
pinned ref currently produces. It does not prove the pinned ref's content is safe,
and none of the three refs pinned in this repository's scripts or workflows
(`v1.7.0`, `v1.38.0`, and the third-party GitHub Actions used in the workflow
files under `.github/workflows/`) are pinned to an immutable commit SHA — all are tags or
tag-like refs, which a repository owner could in principle move. Do not describe
this convention as verifying the safety of the upstream source; describe it as
verifying that this repository's copy is in sync with what that source currently
contains.
20 changes: 20 additions & 0 deletions .security/security.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
apiVersion: snowflake.com/security/v1alpha1
kind: SecurityContext
metadata:
name: snowflake-telemetry-python
annotations:
owner: "@snowflakedb/telemetry"
spec:
documents:
- path: architecture.md
kind: context
description: "Package layout of this OTel-extension library and why nothing in it is internet-facing"
- path: trust-boundaries.md
kind: context
description: "Why this is a same-process library (no auth boundary), that it performs no network/file I/O itself, and the one caller-input guard it does enforce (log-formatter reserved attributes)"
- path: threat-model.md
kind: threat-model
description: "Systems this package's own code and build scripts talk to, and the invariants (log-attribute protection, vendored/generated-code sync, publish-token scoping) with their controls and status"
- path: secure-coding.md
kind: secure-coding
description: "The adopted convention for vendoring or regenerating code from an upstream OpenTelemetry repository"
52 changes: 52 additions & 0 deletions .security/threat-model.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
> Derived-from: snowflakedb/snowflake-telemetry-python@8b2709d · generated 2026-04-08
> Regenerate when: `setup.py`'s `opentelemetry-api`/`opentelemetry-sdk` pin changes,
> either upstream ref pinned in `scripts/proto_codegen.sh` or
> `scripts/vendor_otlp_proto_common.sh` changes, a CI workflow under
> `.github/workflows/` is added or removed, or an invariant's control changes

## The system

`snowflake-telemetry-python` is a **library** (see `architecture.md`), not a service.

- **Driven by** whatever Python process imports it and calls its API. Per
`README.md`, that is a Python UDF, UDTF, or Stored Procedure running inside
Snowflake's Python execution environment, or any other Python program that adds
the dependency.
- **Changed by** anyone able to merge to this repository; `.github/CODEOWNERS:1`
designates `@snowflakedb/telemetry` as the owner of every path in the tree.
- **Trusted with** no credentials and no secrets — none are read, held, or
transmitted anywhere in `src/`. What it is trusted with is *correctness of
serialization*: every process that calls its API, or wires up
`SnowflakeTraceIdGenerator` / `ProtoSpanExporter` / `ProtoMetricExporter`, relies
on it to produce well-formed OTLP protobuf bytes and to hand them only to the
writer the embedding application supplied (`trust-boundaries.md`) — never to an
implicit destination of its own choosing. Because the package is published to the
public PyPI registry and, via a separate Jenkins-triggered build, to a private
Snowflake-operated conda channel (`trust-boundaries.md`, "Two independent
build/publish pipelines"), a change that broke either property would ship to every
process that installs the package from either channel.
- **Reads** nothing beyond its own Python-level function arguments and, at
maintainer-invoked dev time only, two upstream OpenTelemetry git repositories
cloned by `scripts/proto_codegen.sh` and `scripts/vendor_otlp_proto_common.sh` (see
below). It holds no runtime configuration, no credentials, and no per-tenant state.

## Systems and trust boundaries

| System | Role here | Crosses the boundary | Trusted for |
|---|---|---|---|
| `opentelemetry-api` / `opentelemetry-sdk` (PyPI, pinned `==1.38.0` in `setup.py`) | runtime dependency this package extends (`get_current_span`, `RandomIdGenerator`, `SpanExporter`, `MetricExporter`) | installed into, and imported by, every process using this package | that the pinned version behaves as documented — a defect or a compromised release at that exact version would run in every embedding process |
| `open-telemetry/opentelemetry-proto` (git, tag `v1.7.0` pinned at `scripts/proto_codegen.sh:15`) | schema source for the generated marshaler code under `_internal/opentelemetry/proto/` | cloned by a maintainer running `scripts/proto_codegen.sh`; never fetched at package-install or runtime | that the content at the pinned tag, when regenerated, is what CI (`.github/workflows/check-codegen.yml`) sees committed to this repo |
| `open-telemetry/opentelemetry-python` (git, tag `v1.38.0` pinned at `scripts/vendor_otlp_proto_common.sh:12`) | source of the vendored encoder code under `_internal/opentelemetry/exporter/otlp/proto/common/` | cloned by a maintainer running `scripts/vendor_otlp_proto_common.sh`; never fetched at package-install or runtime | that the content at the pinned tag, when regenerated, is what CI (`.github/workflows/check-vendor.yml`) sees committed to this repo |
| GitHub Actions (workflow files under `.github/workflows/`) + third-party actions (`actions/checkout@v3/v4`, `actions/setup-python@v3`, `pypa/gh-action-pypi-publish@release/v1`) | builds, tests, and (on `release: published`) publishes this package | source in, package artifact out; `PYPI_API_TOKEN` secret out to the publish step only | that only this repository's reviewed source is built, and that `PYPI_API_TOKEN` is used by, and visible to, no step other than `python-publish.yml`'s publish step |
| PyPI (public registry) | public distribution channel for the package built by `python-publish.yml` | release artifact out | serving exactly the artifact this repo's workflow uploaded, unaltered, to every `pip install snowflake-telemetry-python` |
| Jenkins job "SnowflakeTelemetryPythonPackageBuilder" + `repo.anaconda.com/pkgs/snowflake/` conda channel (`build.sh:4`, `build.sh:38`) | second, non-GitHub-Actions build/publish path, producing a conda package | source in (checked out by Jenkins, not visible from this repo), package artifact out to the private channel | building the published conda package only from this repository's own reviewed source, the same way `python-publish.yml` does for the PyPI artifact |
| The embedding application's `SpanWriter`/`MetricWriter` subclass | sole consumer of the OTLP protobuf bytes this repo's exporters produce | serialized bytes out, in-process | not silently discarding, corrupting, or misrouting the telemetry data this library hands it |

## Invariants

| ID | Invariant | Control | Status |
|---|---|---|---|
| INV-1 | A caller-supplied `logging` `extra=` value cannot overwrite the `code.lineno`, `code.function`, `code.filepath`, `exception.type`, `exception.message`, or `exception.stacktrace` attributes `SnowflakeLogFormatter` derives from the actual call site | `src/snowflake/telemetry/logs/__init__.py` (`_RESERVED_ATTRS`, `SnowflakeLogFormatter.format`) — mechanism detailed in `trust-boundaries.md` | Holding (`tests/test_snowflake_log_formatter.py#test_normal_log`) |
| INV-2 | The vendored encoder code under `_internal/opentelemetry/exporter/otlp/proto/common/` matches what a fresh regeneration from the pinned `opentelemetry-python` tag (`scripts/vendor_otlp_proto_common.sh:12`) produces | `.github/workflows/check-vendor.yml` (regenerates from scratch and fails on `git diff`) | Holding — covers only "the committed code matches the pinned tag's current content", not "that tag's content is safe"; see the systems-table row above and the gap this leaves open |
| INV-3 | The generated marshaler code under `_internal/opentelemetry/proto/` matches what a fresh regeneration from the pinned `opentelemetry-proto` tag (`scripts/proto_codegen.sh:15`) produces | `.github/workflows/check-codegen.yml` (regenerates from scratch and fails on `git diff`) | Holding — same partial-coverage caveat as INV-2 |
| INV-4 | `PYPI_API_TOKEN` is used only inside the publish step of `.github/workflows/python-publish.yml`, and that workflow runs only when a GitHub release is published | `.github/workflows/python-publish.yml` (trigger `release: types: [published]` at `:11-13`; token read only at `:39`, inside the `pypa/gh-action-pypi-publish` step at `:36`) | Holding |
84 changes: 84 additions & 0 deletions .security/trust-boundaries.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
> Derived-from: snowflakedb/snowflake-telemetry-python@8b2709d · generated 2026-04-08
> Regenerate when: `SpanWriter`/`MetricWriter` gain a default (non-abstract)
> implementation, `_RESERVED_ATTRS` changes, or any network/file I/O is added to `src/`

## The caller is the trust boundary, not this code

This is a library called in-process. Its only "input" is whatever arguments the
embedding Python process passes to `snowflake.telemetry.add_event()`,
`snowflake.telemetry.set_span_attribute()`, or a `logging` call that flows through a
`logging.Handler` configured with `SnowflakeLogFormatter`
(`src/snowflake/telemetry/logs/__init__.py`). The caller and the code executing this
library's functions run as the same principal, in the same process — there is no
network hop, no serialization boundary, and no authentication step inside this
repository between "whoever can execute Python here" and "whoever can call this
API." Any code that can import `snowflake.telemetry` can call every function in it
with arbitrary argument values; this repository enforces no authorization on top of
that.

## This library never transmits or persists the data it serializes

`ProtoSpanExporter._serialize_traces_data()`
(`src/snowflake/telemetry/_internal/exporter/otlp/proto/traces/__init__.py`) and
`ProtoMetricExporter._serialize_metrics_data()`
(`src/snowflake/telemetry/_internal/exporter/otlp/proto/metrics/__init__.py`) produce
`bytes` and immediately hand them to `self.span_writer.write_span(...)` /
`self.metric_writer.write_metrics(...)`. `SpanWriter.write_span` and
`MetricWriter.write_metrics` are declared with `@abc.abstractmethod` and have no
body in either class — the embedding application must subclass `SpanWriter` or
`MetricWriter` and implement the method itself before any serialized data goes
anywhere. This repository ships no such subclass (the closest thing, an
`InMemorySpanWriter`/`InMemoryMetricWriter`, exists only under `tests/`, per the
module docstrings in both files).

Combined with the absence of any socket/HTTP/file-write call anywhere in `src/`
(see `architecture.md`), a finding that assumes this repository's code sends
telemetry data to a specific network destination, an S3 bucket, or a database is
describing the *embedding application's* writer implementation — which lives
outside this repository — not code that exists here.

## The one caller-input guard this repo does enforce: reserved log attributes

`SnowflakeLogFormatter.format()` (`src/snowflake/telemetry/logs/__init__.py`) builds
its output `attributes` dict in two steps: it first sets `code.lineno`,
`code.function`, `code.filepath` from the real `LogRecord` fields, and — when an
exception is attached — `exception.type`, `exception.message`,
`exception.stacktrace`. It then iterates `record.__dict__.items()` (which includes
whatever a caller passed via `logging`'s `extra=` argument) and skips any key that
appears in `_RESERVED_ATTRS`, a frozenset that includes exactly those six
`code.*`/`exception.*` keys plus the standard `logging.LogRecord` attribute names
(`src/snowflake/telemetry/logs/__init__.py:11-42`). The effect: a caller cannot use
`extra={"code.lineno": ...}` (or any of the other reserved keys) to overwrite the
values this formatter derives from the actual call site. This is exercised by
`tests/test_snowflake_log_formatter.py#test_normal_log`, which passes
`extra={"body": 123, "code.lineno": 35}` and asserts the emitted `code.lineno` is
the real line number (21 in that test), not the caller-supplied 35, while the
non-reserved `body` key is passed through unchanged.

No other caller-supplied value (event names, span/log attribute values, metric
data points) is validated, size-capped, or filtered by this repository before being
handed to the OpenTelemetry SDK or serialized to protobuf bytes.

## `SnowflakeTraceIdGenerator`'s trace-ID generation

`SnowflakeTraceIdGenerator.generate_trace_id()`
(`src/snowflake/telemetry/trace/__init__.py`) builds a 16-byte trace ID from a
4-byte big-endian encoding of "minutes since the epoch" followed by 12 bytes from
`random.getrandbits(96)` — the stdlib's non-cryptographic PRNG, not the `secrets`
module.

## Two independent build/publish pipelines exist for this package

- `.github/workflows/python-publish.yml` runs on `release: types: [published]`
(`:11-13`) and publishes to the public PyPI registry using
`pypa/gh-action-pypi-publish@release/v1` (`:36`) authenticated with the
`PYPI_API_TOKEN` repository secret (`:39`) — a stored token, not PyPI's OIDC
"trusted publishing."
- `build.sh` and `pypi-build.sh` are, per their own header comments, "called from a
Jenkins job called SnowflakeTelemetryPythonPackageBuilder" (`build.sh:4`,
`pypi-build.sh:4`) rather than from any GitHub Actions workflow in this repo.
`build.sh` additionally adds a private conda channel,
`https://repo.anaconda.com/pkgs/snowflake/` (`build.sh:38`), before building the
conda-format package and removes it afterward (`build.sh:56`). Neither the
Jenkins job's configuration nor what consumes packages from that channel is
visible from this repository.
Loading