Repository navigation
Initial public release of OpenTAMS - #1
Merged
Merged
Conversation
Akshay-Narayan-Pai
force-pushed
the
feat/opentams
branch
7 times, most recently
from
August 31, 2026 04:40
65ae933 to
bc0938e
Compare
OpenTAMS is a cloud-agnostic implementation of the BBC TAMS v8.0 API
(Time-addressable Media Store), pinned to spec commit b386c43.
This is the sanitized export of the internal development tree. Internal
material is excluded: the development-workflow tooling, the black-box
test-case catalogue, and the pre-implementation artefact set. The export
tooling that produced it is not carried over -- this repository is the
development home, so there is nothing left for it to sanitize, and it was
the only remaining file naming the internal directory layout.
References to internal planning vocabulary are removed throughout:
process-tier labels in CI comments and lint configuration, an internal
follow-up identifier in internal/server/validator.go and the server design
notes, and three design documents that cited an internal decision log
absent from this tree (the decisions themselves are stated inline instead).
The gitleaks AWS rationale keeps its technical justification but no longer
recounts an internal audit finding about history this repository does not
share.
Included:
- Go server (cmd/opentams), CLI (cmd/tamsctl), and supporting tools
- TAMS v8.0 OpenAPI contract, bundled spec, and generated types
- Postgres metastore with embedded migrations
- S3-compatible object store backend
- Helm chart, Terraform modules, and container build
- Governance set: LICENSE, CONTRIBUTING, GOVERNANCE, MAINTAINERS,
SECURITY, SUPPORT, CODE_OF_CONDUCT, NOTICE
- CI: build, lint, test, integration, release, scorecard, api-docs
Repository self-references are rewritten from the private amagimedia org
to amagioss, which is the canonical home and already the module path in
go.mod. References to github.com/amagimedia/go-commons are left intact:
that is a separate internal repository, not this one.
All CI runs on GitHub-hosted ubuntu-latest. A public repository must not
name self-hosted runner labels: no fork can schedule them, so an outside
contributor's run would queue forever, and `pull_request` workflows would
expose private infrastructure to untrusted fork code. Hosted minutes are
free for public repositories.
Supporting changes: job timeouts are raised to suit the smaller hosted
runners; the nightly integration job's timeout is raised to 60 minutes,
which also corrects a latent bug where the 30-minute job limit was below
the 40 minutes its two test steps could take; and api-docs now grants
`pages: write` / `id-token: write` to the deploy job alone rather than
workflow-wide, so the fork-reachable build job holds no write scope.
The `perf` budgets in the nightly job now run on shared hardware and will
see more variance than a dedicated machine; the workflow documents this
so a single breach is investigated rather than treated as proof of a
regression.
The BBC attribution is stated as independence rather than disclaimer. The
project describes itself as an independent implementation whose conformance is
self-assessed, replacing the emphatic non-endorsement wording, and acknowledges
the BBC's work in publishing TAMS as an open specification. No claim is made
about BBC certification: upstream governance lists deciding how any such
programme would operate as a future TSC responsibility, and is transferring the
TAMS name to a successor framework. The nominative-fair-use position is
unchanged.
CI is hardened for a public repository. Every third-party action is pinned
to a full commit SHA with the release recorded in a trailing comment, so a
retagged upstream action cannot change what runs. The release workflow no
longer accepts a caller-supplied ref -- one could name refs/pull/N/head and
run fork-authored code with publish scopes -- and its write scopes move from
the workflow to the release job, behind a `release` environment. Every
workflow now defaults to read-only permissions.
Four gates are added. Security runs gitleaks as a pinned, checksum-verified
binary rather than the gitleaks-action wrapper, which refuses to run for a
repository owned by an organisation without a paid licence; it also runs a
Trivy secret scan of the
tree, and a Trivy misconfiguration scan of the deployment manifests; Trivy's
dependency scanner is deliberately not enabled, because govulncheck already
owns Go vulnerability policy with reachability analysis and a justification
file, and Trivy has none, so it would re-report justified advisories as
failures. Contract lints the OpenAPI source and fails if the committed bundle
or generated code can be regenerated differently, lints the Helm chart against
each committed ci/*-values.yaml, and runs tofu fmt and validate across all
three modules with the committed lock verified read-only. CodeQL analyses Go.
Dependency review and CodeQL upload to GitHub features that need Advanced
Security on a private repository, so both are conditioned on the repository
being public. They are inert now and start running when it is published,
rather than sitting permanently red.
The load generator's in-flight accounting is corrected. Its worker released
the semaphore slot before decrementing the in-flight counter, leaving a window
in which the slot was free but the finished goroutine was still counted: the
dispatch loop could take that slot, increment, and latch a peak above the
configured maximum. Admission was never affected -- the semaphore bounds real
concurrency -- but the over-reported peak is published in scale test results.
Decrementing before releasing restores the invariant that counted workers are
a subset of slot holders. Reproduced under CPU contention at 1 failure in 150
runs before the change and 0 in 150 after.
Verified before commit: go build, go vet, go test -short, and
golangci-lint (0 issues) all pass; actionlint reports no findings on the
changed workflows; the export gate reports no secrets, no internal paths,
and no unresolved documentation links.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Akshay-Narayan-Pai
force-pushed
the
feat/opentams
branch
from
August 31, 2026 06:30
bc0938e to
f952365
Compare
Number the founding ADR set by design layer rather than by the order the
decisions happened to be written down. All four existing ADRs were dated
the same day, so chronology carried no information; design layer does.
0001 -> 0024 whole-batch reject on segment overlap
0002 -> 0023 RFC 9457 problem details for per-segment failures
0003 -> 0005 vendored TAMS schemas are modified
ADR-0005 is the parent of the other two -- it lists them as the
behavioural divergences it carries -- and was numbered after its own
children. All 19 cross-references across 12 files are updated, and every
ADR link in the repository resolves.
Record the exception in ADR-0000 and docs/adr/README.md: 0000-0037 is the
founding set, numbered by layer before the first public release, when
nothing outside the repository linked to those numbers. From 0038 onward
numbers are chronological and the never-renumber rule holds.
Add ADR-0001..0004 (foundations) and ADR-0006..0011 (API contract and
code generation).
Fix six documentation defects found while verifying the above against the
code:
- ADR-0005 described ADR-0023 as "proposed -- the server does not yet
implement it". The server does: internal/httpx/conversion/segment.go:146
emits the RFC 9457 {type, title, detail} subset.
- api/schemas/README.md repeated that claim.
- api/schemas/README.md said `make api-check` currently fails and that
`make api-bundle` breaks the build. `make api-check` passes, exit 0,
and regeneration is byte-identical. The Contract workflow has been
running it on every PR.
- BR-SVC-FLOW-08/09 described `store.DeleteFlow` returning zero-ref object
IDs and the service calling `obj.DeleteObjects`. The store returns only
an error, and no service code calls DeleteObjects. Object lifetime
belongs to the GC worker, which is not implemented.
- The handlers design-decision table had the union decode order backwards.
segment_params.go:27 tries the array variant first.
- internal/service/flow/flow.go's package comment claimed objectstore
cleanup on delete, contradicting New()'s own comment 53 lines below.
Comment only; no logic change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PZHJ9iWxVTq4EDYkzGcJYe
ADR-0012..0021 and ADR-0022, 0025..0027. ADR-0017 records a gap rather than claiming a guard that runs. metastore.VerifySchema implements the minimum-schema-version check and its three named failure modes (ErrSchemaNotMigrated, ErrSchemaDirty, ErrSchemaTooOld), but no non-test caller invokes it. The server starts against any schema version, and readiness only probes the database and the object store. A binary run against a schema older than it needs fails on the first query touching a missing column, not at boot. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PZHJ9iWxVTq4EDYkzGcJYe
…ility ADRs ADR-0028..0037, and rebuild docs/adr/README.md as a flat 38-row index. Runtime: Gin with split public/authenticated route groups, the auth Provider interface, environment-only configuration, liveness separated from readiness, and zap with a private Prometheus registry. Build and release: distroless static non-root image, image-only server release with cosign/SBOM/SLSA attestation, and CI on GitHub-hosted runners with all 50 action references pinned by commit SHA. Capabilities shipped unavailable: rate limiting (ADR-0036) and object garbage collection (ADR-0037). Both record a decision the code does implement -- configuration parsed and deliberately ignored, the error vocabulary defined, the schema support in place, and the gap documented where an operator meets it -- rather than a roadmap item. Verified: every ADR link in the repository resolves, go build, go vet, make lint (0 issues), and go test -short on the touched packages. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PZHJ9iWxVTq4EDYkzGcJYe
An ADR records a decision and its consequences, not the current state of the code against it. Unwired or unbuilt code is a defect or a backlog item, and it belongs in the issue tracker and the operator-facing docs, not in the record of why a choice was made. ADR-0017, 0018, and 0031 narrated that metastore.VerifySchema is never called at startup. That is a bug to fix, not a decision to record. The policy those ADRs exist to state -- expand-contract migrations, the required schema version treated as a minimum, migrations as an explicit step, liveness separate from readiness -- stands on its own. ADR-0020 and 0037 carried the GC worker's delivery state as a bold callout in Decision Outcome. Moved to Consequences, and ADR-0037 retitled from "Reclaim Object storage from a separate worker, and ship without it" to "Reclaim Object storage from a separate worker". The decision is the separate worker, the claim flag, and the partial indexes -- all of which the schema implements. Delivery state stays in conformance.md and configuration.md, where an operator already meets it. Index regenerated. Every ADR link in the repository resolves. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PZHJ9iWxVTq4EDYkzGcJYe
ADR-0037 records a design whose code has not landed. The schema supports it -- ref_count, the reaping claim flag, the two partial indexes, and the chunked idempotent DeleteObjects primitive -- but the sweep worker does not exist. The repository's own rule is that an ADR records a decision the code implements, so "accepted" claimed more than was true. "proposed" is MADR's own vocabulary for a design put forward but not settled by working code, and it puts the state in the index where a reader sees it. ADR-0036 stays accepted. Nothing is pending there: "rate limiting is the operator's responsibility" is in force today and stated in SECURITY.md. A limiter added later would be a new ADR superseding it, not a status change on this one. Close the workflow gap this exposes. docs/adr/README.md described only the happy path -- proposed in a pull request, accepted on merge -- which left a merged "proposed" ADR undefined, and never said what to do with a decision that is not taken. Two steps added: an agreed design whose code has not landed stays proposed after merge and moves to accepted in the pull request that implements it; a decision not taken becomes rejected and keeps its file and its number. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PZHJ9iWxVTq4EDYkzGcJYe
Apache License 2.0 section 4(d) requires a derivative work to carry the attribution notices of the Apache-2.0 works it redistributes. Eight dependencies compiled into the released binaries and container images ship a NOTICE file, and none of their text was reproduced: github.com/aws/aws-sdk-go-v2 (also carries a Stripe copyright) github.com/aws/smithy-go github.com/prometheus/client_golang (SoundCloud, beorn7/perks, protobuf) github.com/prometheus/client_model github.com/prometheus/common github.com/oasdiff/yaml3 (Canonical Ltd.) go.yaml.in/yaml/v2 (Canonical Ltd.) gopkg.in/yaml.v3 (Canonical Ltd.) All eight verified present in `go list -deps ./cmd/...`, so they are genuinely redistributed rather than build-only. Also in NOTICE: - The licence list claimed "Apache 2.0, MIT, BSD 3-Clause, and ISC". The measured spread across the 74 modules in the shipped binaries is 32 MIT, 29 Apache-2.0, 11 BSD-3-Clause, and 1 0BSD. ISC was claimed but absent; 0BSD was present but unclaimed. - It called docs/third-party-notices.md "the full list", while that file states it covers direct dependencies only. - Dropped the "Licensed under the Apache License, Version 2.0. See the LICENSE file" line, which restates LICENSE. docs/third-party-notices.md carried three errors, found by diffing it against go.mod: - gopkg.in/yaml.v3 was listed as MIT. Its LICENSE opens "This project is covered by two different licenses: MIT and Apache." The Apache half is what creates the section 4(d) duty above, so this mislabel is what hid it. - golang.org/x/sync: v0.20.0 listed, v0.21.0 in go.mod. - golang.org/x/tools: v0.42.0 listed, v0.47.0 in go.mod. Every direct dependency now matches go.mod on both version and licence, and the module set in the table matches the non-indirect block exactly. Verified: gitleaks over full history (no leaks), go build, go vet, and every link in the NOTICE-referencing documents resolves. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PZHJ9iWxVTq4EDYkzGcJYe
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ports the sanitized OpenTAMS tree into its canonical public home.
CI runners
Every workflow moves to GitHub-hosted
ubuntu-latest. Noruns-onin this repository names a self-hosted label any more.A public repository must not reference self-hosted runner labels. No fork can schedule them, so an outside contributor's run would queue forever; and
pull_requestworkflows would expose private infrastructure to untrusted fork code. Hosted minutes are free for public repositories.Both previously self-hosted workflows turned out fully portable — every tool arrives via an action (
setup-go,cosign-installer,sbom-action,goreleaser-action,buildx, QEMU), and registry auth uses the built-inGITHUB_TOKENrather than an org secret.Supporting changes:
go test -timeout20m→30m).integrationtimeout 30→60. This also fixes a latent bug: the 30-minute job cap sat below the 40 minutes its two test steps could legitimately take, so a slow run would report a truncation instead of the real failure.api-docsgrantspages: write/id-token: writeto the deploy job alone rather than workflow-wide, so the fork-reachable build job holds no write scope.build.ymlis unaffected by the smaller runner:build/Dockerfilecross-compiles via--platform=$BUILDPLATFORMwithGOARCH=${TARGETARCH}, so arm64 is compiled rather than emulated.Known consequence. The nightly
perfbudgets now run on shared hardware, which has more run-to-run variance than a dedicated machine. The workflow documents this so a single breach is investigated rather than treated as proof of a regression. Separately, and independent of the runner change, the perf suite is currently failing: roughly ten failures from a foreign-key ordering bug in three seed helpers, plus one genuine budget breach ininternal/objectstore. The first nightly will be red until those are fixed, and the budgets should then be recalibrated against observed hosted-runner numbers.Verification
go build ./...go vet ./...go test -short ./...make lint(golangci-lint v2.11.0)actionlint🤖 Generated with Claude Code