fix: Retry the test harness download and surface curl errors - #411
Conversation
Co-Authored-By: rlamb@launchdarkly.com <4955475+kinyoklion@users.noreply.github.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
|
@cursor review |
|
More evidence for this: I pushed six SDK PRs within a few seconds to validate v2.39.0 / v3.2.0-alpha.6 end-to-end, and nearly every contract-test job failed on The jobs that did get the binary confirm the new test works end-to-end, e.g. go with v3 counterpart of this change: #413. |
**Requirements** - [ ] I have added test coverage for new or changed functionality - [x] I have followed the repository's pull request submission guidelines - [x] I have validated my changes against all supported platform versions **Related issues** Same change as #411, on the v3 branch. **Describe the solution you've provided** The release-asset download is a single un-retried `curl --fail -s`, and `-s` hides the reason for a failure: ```diff -curl --fail -s -L -o "${TEMP_DIR}/archive.${EXTENSION}" "${DOWNLOAD_URL}" || (echo "Download failed" >&2; exit 1) +curl --fail -sS -L --retry 5 --retry-delay 2 \ + -o "${TEMP_DIR}/archive.${EXTENSION}" "${DOWNLOAD_URL}" \ + || { echo "Download failed" >&2; exit 1; } ``` **Describe alternatives you've considered** - `--retry-all-errors`: rejected, it requires curl 7.71+ and this script also runs on older macOS runners. - Sending the token on the asset download: the asset URL redirects to an unauthenticated object store, so an `Authorization` header there is at best useless and can break the redirect. **Additional context** Fresh evidence: six SDK PRs were pushed within a few seconds to validate v2.39.0 / v3.2.0-alpha.6 end-to-end, and nearly every contract-test job died on `Download failed` between 21:55 and 21:58 UTC (python-server-sdk, go-server-sdk, js-core, php-server-sdk, cpp-sdks), for both the `v2.39.0` and `v3.0.0-alpha.6` assets. Downloading the same URLs afterwards succeeds with HTTP 200, so the asset endpoint was throttling concurrent downloads. Link to Devin session: https://app.devin.ai/sessions/bfe54128e2804a96bb100e6120e9a3ef Requested by: @kinyoklion <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > Makes CI contract-test jobs more resilient when pulling **sdk-test-harness** release archives via `downloader/run.sh`. > > The release download `curl` now uses **`-sS`** so failures still print curl diagnostics, and **`--retry 5 --retry-delay 2`** to ride out brief GitHub/asset throttling (the motivation described in the PR). The failure path is unchanged in behavior: still exits with **Download failed**. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit c7e7f09. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
**Requirements** - [ ] I have added test coverage for new or changed functionality - [x] I have followed the repository's [pull request submission guidelines](../blob/master/CONTRIBUTING.md#submitting-pull-requests) - [x] I have validated my changes against all supported platform versions **Related issues** Follows up on the CI investigation in #411 (v2), where contract tests failed at the harness download step rather than at version resolution. **Describe the solution you've provided** Previously `GITHUB_TOKEN` was only used for the releases-listing request in `resolve_version`, so any run that passed a full version string (as CI usually does) made no authenticated requests at all, and the asset download was always anonymous. The `github.com/.../releases/download/...` path is not a REST endpoint: GitHub publishes no rate limit for it and returns no `x-ratelimit-*` headers, and passing a token there is useless because `curl -L` drops `Authorization` on the cross-host redirect to the signed asset host. This change routes the download through the documented release-assets API when a token is present: - `github_curl` centralizes auth (`Authorization: Bearer`) plus `-sS --retry 5 --retry-delay 2`, replacing the previous `eval`-built command string that interpolated the token into a shell command. - `resolve_asset_url` looks up the asset for the current OS/arch via `GET /releases/tags/<version>` and downloads it from `GET /releases/assets/<id>` with `Accept: application/octet-stream`. That lands in the documented authenticated rate limit bucket instead of an undocumented one. - Without a token — or if the asset lookup fails — it falls back to the previous anonymous `releases/download` URL, so external consumers are unaffected. - `curl -s` became `-sS` so a failed download prints the actual curl error/status instead of a bare "Download failed". Docs referenced: [rate limits for the REST API](https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api), [release assets endpoints](https://docs.github.com/en/rest/releases/assets). **Describe alternatives you've considered** - Sending the token to the existing `releases/download` URL — verified that this has no effect on the actual byte transfer, since the header is dropped at the redirect. - Retry-only (as merged for v2 in #411) — that surfaces the error and rides out transient failures, but leaves the download in the unauthenticated bucket. **How to test it** ```sh export VERSION=v3.0.0-alpha.6 PARAMS="-help" GITHUB_TOKEN=<token> sh downloader/run.sh # downloads from .../releases/assets/<id> sh downloader/run.sh # falls back to .../releases/download/... GITHUB_TOKEN=bogus sh downloader/run.sh # prints the 401, then falls back ``` Also exercised the partial-version path (`VERSION=v2`), the cached-binary path, and an unmatched version. **Additional context** No UI or screenshots apply; this is a shell script used by CI in the SDK repos. Note that the contract-tests GitHub Action currently fetches `run.sh` from the `v2` branch, so this only takes effect in CI once the follow-up ports land on `v2` (and `v3`), which will be opened after this is accepted. Link to Devin session: https://app.devin.ai/sessions/98e5024773d947509ad2575ae80bf31b Requested by: @kinyoklion <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > **`downloader/run.sh`** now uses **`GITHUB_TOKEN`** for **release asset downloads** as well as version resolution, so CI contract tests that pass a full version string still hit GitHub’s authenticated rate limits instead of anonymous download URLs. > > A shared **`github_curl`** helper sends **`Authorization: Bearer`**, adds **`--retry`**, and replaces the old **`eval`**-built curl for listing releases. **`resolve_version`** uses it and returns with **`return`** instead of **`exit`** when the version is already complete. > > When a token is set, **`resolve_asset_url`** resolves the OS/arch tarball via **`GET /releases/tags/<version>`** and downloads from **`GET /releases/assets/<id>`** with **`Accept: application/octet-stream`**, avoiding the **`releases/download`** redirect that strips **`Authorization`**. If there is no token or asset lookup fails, behavior falls back to the public **`releases/download`** URL (and clears auth on bogus-token fallback). Downloads use **`curl -sS`** so failures surface curl errors. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit f3a35c2. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
🤖 I have created a release *beep* *boop* --- ## [2.40.0](v2.39.0...v2.40.0) (2026-08-19) ### Features * add RETRY-conformance contract tests for FDv1 streaming and polling ([#404](#404)) ([ee4d85f](ee4d85f)) ### Bug Fixes * Retry the test harness download and surface curl errors ([#411](#411)) ([2a7533b](2a7533b)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > **Release 2.40.0** bumps the package version in `.release-please-manifest.json`, `main.go` (`versionString`), and documents the release in `CHANGELOG.md`. > > The changelog for this tag records two already-merged changes: **RETRY-conformance** contract tests for FDv1 **streaming** and **polling** (capability-gated, including long-running retry-regime checks), and a **downloader** fix that **retries** harness downloads and **surfaces curl errors** for easier CI debugging. > > There is no additional functional code in this PR diff beyond the version and changelog updates. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit b96a76c. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
**Requirements** - [ ] I have added test coverage for new or changed functionality - [x] I have followed the repository's [pull request submission guidelines](../blob/master/CONTRIBUTING.md#submitting-pull-requests) - [x] I have validated my changes against all supported platform versions **Related issues** Port of #415 (merged to `main`) and #416 (`v2`) to `v3`. The contract-tests action fetches `run.sh` from the `v2` branch today, so this port keeps `v3` from regressing when it becomes the branch CI uses. **Describe the solution you've provided** Previously `GITHUB_TOKEN` was only used for the releases-listing request in `resolve_version`, so any run that passed a full version string (as CI usually does) made no authenticated requests at all, and the asset download was always anonymous. The `github.com/.../releases/download/...` path is not a REST endpoint: GitHub publishes no rate limit for it and returns no `x-ratelimit-*` headers, and passing a token there is useless because `curl -L` drops `Authorization` on the cross-host redirect to the signed asset host. This change routes the download through the documented release-assets API when a token is present: - `github_curl` centralizes auth (`Authorization: Bearer`) plus `-sS --retry 5 --retry-delay 2`, replacing the previous `eval`-built command string that interpolated the token into a shell command. - `resolve_asset_url` looks up the asset for the current OS/arch via `GET /releases/tags/<version>` and downloads it from `GET /releases/assets/<id>` with `Accept: application/octet-stream`. That lands in the documented authenticated rate limit bucket instead of an undocumented one. - Without a token — or if the asset lookup fails — the token is cleared and it falls back to the previous anonymous `releases/download` URL, so external consumers are unaffected. Docs referenced: [rate limits for the REST API](https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api), [release assets endpoints](https://docs.github.com/en/rest/releases/assets). **Describe alternatives you've considered** - Sending the token to the existing `releases/download` URL — verified that this has no effect on the actual byte transfer, since the header is dropped at the redirect. - Retry-only (#411 on `v2`) — rides out transient failures, but leaves the download in the unauthenticated bucket. **How to test it** ```sh export VERSION=v3 PARAMS="-help" GITHUB_TOKEN=<token> sh downloader/run.sh # downloads from .../releases/assets/<id> sh downloader/run.sh # falls back to .../releases/download/... GITHUB_TOKEN=bogus sh downloader/run.sh # prints the 401, then falls back anonymously ``` Also exercised full and partial version strings, the cached-binary path, and an unmatched version. **Additional context** Shell-script-only change used by CI, so no UI screenshots apply. This also brings the retry and error-surfacing behavior of #411 to `v3`, which never received it. The Windows `zip` path is unchanged apart from going through `github_curl`. Link to Devin session: https://app.devin.ai/sessions/98e5024773d947509ad2575ae80bf31b Requested by: @kinyoklion <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > **CI downloader** (`downloader/run.sh`) now uses `GITHUB_TOKEN` for **release asset downloads**, not only when resolving partial version tags from the releases list. > > A shared **`github_curl`** helper sends **`Authorization: Bearer`** with retries and replaces the old `eval`-built curl that only authenticated the releases list—and did not help downloads when CI passed a full version string. > > When a token is set, **`resolve_asset_url`** fetches the OS/arch archive via the **releases assets REST API** (`Accept: application/octet-stream`), avoiding the public `releases/download` URL where **`curl -L` drops auth on redirect**. If asset lookup fails or no token is provided, behavior falls back to the anonymous download URL (token cleared on fallback so a bad token does not break public downloads). > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 7a13940. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Requirements
Related issues
Intermittent
Download failedfailures in SDK contract-test jobs, e.g. launchdarkly/python-server-sdk#484 where 4 of 5 matrix jobs died at the same second while the rest passed on identical code.Describe the solution you've provided
downloader/run.shresolves the release version through the GitHub API (authenticated whenGITHUB_TOKENis set), but then downloads the release asset with a single un-retriedcurl --fail -s, and-shides the reason for a failure:--retrycovers transient 5xx/408/429 responses (consistent with several matrix jobs failing at once while others succeed), and-sSmeans the next failure reports the actual curl error instead of a bareDownload failed.Describe alternatives you've considered
--retry-all-errors: rejected, it requires curl 7.71+ and this script also runs on older macOS runners.Authorizationheader there is at best useless and can break the redirect.Additional context
The
contract-testsGitHub action reads this script from thev2branch by default, so this also covers repos running the v3 harness through that action.mainhas the same line and should get the same change.Link to Devin session: https://app.devin.ai/sessions/bfe54128e2804a96bb100e6120e9a3ef
Requested by: @kinyoklion
Note
Overview
Contract-test jobs intermittently failed with a bare Download failed when fetching the
sdk-test-harnessrelease asset from GitHub.The download
curlindownloader/run.shnow uses--retry 5 --retry-delay 2for transient HTTP failures (aligned with matrix jobs failing together while others pass) and-sSinstead of-sso failures include the actual curl error message.Reviewed by Cursor Bugbot for commit 2c23020. Bugbot is set up for automated code reviews on this repo. Configure here.