release-kbagent: publish-s3 fails OIDC AssumeRoleWithWebIdentity — apt/rpm/apk + brew/winget/choco stuck (v0.64.0)

[padak]

Summary

The release-kbagent native-distribution pipeline's signing steps were broken on every recent stable tag (v0.63.3, v0.63.4, v0.64.0) — that's why none of them produced a GitHub Release or native packages (only the PyPI wheel got out). Those signing failures are now fixed in #451 (merged). The one remaining blocker is publish-s3, which fails on AWS OIDC role assumption — that needs your AWS-side fix.

What was broken → fixed in #451

Verified on the re-tag run https://github.com/keboola/cli/actions/runs/27863857637 — all four freeze jobs are now green and the GitHub Release + binaries published.

1. macOS — stapler Error 73. build/package/macos/sign_notarize.sh ran xcrun stapler staple on the bare PyInstaller Mach-O binary. stapler only staples bundles (.app/.dmg/.pkg); a standalone executable cannot be stapled → deterministic Error 73 on every release. Notarization itself succeeds (notarytool → Accepted). Fix: drop the staple step — a Developer-ID-signed + notarized CLI binary is validated online by Gatekeeper (code-directory hash); stapling only matters for offline first-run of a bundle.
2. Windows — TSA timeout. build/package/windows/sign.sh timestamped against a single TSA (timestamp.digicert.com) with no retry/fallback → java.net.SocketTimeoutException: Connect timed out. Fix: jsign comma-separated --tsaurl failover (digicert,sectigo,ssl.com) + --tsretries 5 --tsretrywait 15.
3. PyPI — not idempotent. Added skip-existing: true to pypa/gh-action-pypi-publish so a re-tag/re-run after a downstream failure doesn't fail on an already-published version.

What still needs YOU (AWS infra)

publish-s3 fails:

Could not assume role with OIDC: Not authorized to perform sts:AssumeRoleWithWebIdentity

• Failed job: https://github.com/keboola/cli/actions/runs/27863857637/job/82464864215
• Effect: the S3 repo (cli-dist.keboola.com apt/rpm/apk index) + homebrew / winget / chocolatey (they depend on publish-s3) were skipped → those channels are NOT on 0.64.0.

Likely cause: the IAM role (secrets.AWS_ROLE_ARN) trust policy doesn't authorize the GitHub OIDC token for this repo/ref. The repo moved to keboola/cli; the trust policy's sub condition may still reference an old repo path, or may not allow tag refs. Please check:

• the IAM role trust policy allows the GitHub OIDC provider + repo:keboola/cli:* (incl. ref:refs/tags/*);
• secrets.AWS_ROLE_ARN and the release environment's AWS secrets are correct for keboola/cli.

How to finish 0.64.0 after the IAM fix

The signed binaries and the GitHub Release for v0.64.0 already exist; only the S3-backed channels are missing. After the trust-policy fix, re-run the failed + skipped jobs:

gh run rerun 27863857637 --failed

pypi is skip-existing now and freeze/github-release already succeeded, so the re-run only does publish-s3 → homebrew / winget / chocolatey.

Current state of 0.64.0

• ✅ PyPI keboola-cli==0.64.0
• ✅ GitHub Release v0.64.0 (Latest) — all native binaries + .deb/.rpm/.apk attached + release notes → CLI auto-update picks it up
• ❌ S3 apt/rpm/apk repos + Homebrew/WinGet/Chocolatey — blocked on the IAM fix above

cc @Matovidlo

[padak]

Correction / addition: publish-s3 has two serial blockers, not one

Re-checked the publish-s3 job in .github/workflows/release-kbagent.yml — the original write-up only captured the first blocker. There are two, and they fail in sequence inside the same job:

1. OIDC AssumeRole (this issue). configure-aws-credentials@v6 (role-to-assume: ${{ secrets.AWS_ROLE_ARN }}, ~line 292) fails first → the job dies before it ever runs the indexer. Still needs the AWS-side IAM trust-policy fix described above.
2. apk index tooling (fix(release): build apk repo index inside Alpine container (abuild not on Ubuntu) #449). Once OIDC is fixed and the job reaches bash build/package/linux/index.sh (~line 305), it would still fail on the apk step: apk/abuild-sign are Alpine-only and aren't installable via Ubuntu apt, so the APK_KEY_PRIVATE guard hit exit 1 on every real release. fix(release): build apk repo index inside Alpine container (abuild not on Ubuntu) #449 (approved, mergeable) fixes this by building + signing the apk index inside an alpine:3 container.

The apk bug has been masked by the OIDC failure — the job never reached index.sh, so it never surfaced in a real run. Fixing only the OIDC trust policy would push the failure one step down to the apk step and read like a fresh regression. Both fixes are needed; neither subsumes the other (OIDC is pure AWS-side IAM, apk is pipeline code).

Order of operations to finish the 0.64.0 native channels

1. Fix the IAM role trust policy for repo:keboola/cli:* (incl. ref:refs/tags/*) — this issue.
2. Merge fix(release): build apk repo index inside Alpine container (abuild not on Ubuntu) #449 (apk index in Alpine).
3. gh run rerun <run-id> --failed — pypi/freeze/github-release already succeeded (pypi is skip-existing), so the re-run only does publish-s3 → homebrew/winget/chocolatey.

cc @Matovidlo

[Matovidlo]

@padak it may be fixed now

[padak]

Confirming the AWS-side fix: publish-s3 (OIDC AssumeRoleWithWebIdentity) succeeded on two consecutive stable release runs — v0.71.0 and v0.72.0. The apt/rpm/apk repos and Homebrew legs published on both runs (Homebrew green end-to-end on v0.72.0). The signing failures tracked here were fixed earlier in #451. Remaining distribution gaps are tracked separately: Chocolatey 403 in #502, winget bootstrap in #484. Closing this one.