feat(registry): Go module support - #126
Conversation
Adds `go` as a registry: version discovery through proxy.golang.org, plus the
recursive definition walk that Go module paths require.
Three things differ from the existing fetchers, each measured against the live
proxy rather than inferred:
- Uppercase letters must be escaped as "!" + lowercase.
github.com/BurntSushi/toml/@v/list returns 404; github.com/!burnt!sushi/toml
returns 200. Slashes are path separators and must survive, so
encodeURIComponent is the wrong tool.
- The "v" prefix is stripped before the shared helpers see it.
isPrerelease("v1.10.2") is true, so every Go version would be silently
discarded, and compareSemver("v1.10.2", "v1.9.1") is NaN, so sorting breaks.
Definitions restore the prefix with tag_pattern "v{version}", which keeps
both shared functions untouched.
- /@v/list carries no publish dates. Getting them costs one /@v/<version>.info
request per version, so --since filtering is unavailable for Go rather than
merely slow. publishedAt is left undefined.
listDefinitions only recursed into directories starting with "@", so
registry/go/github.com/spf13/cobra.yaml was never loaded — exit 0, no warning.
Every Go module path contains slashes, so this blocked the ecosystem rather
than one package. The general recursive walk that replaces the special case is
smaller than the case it removes, and scoped npm packages still resolve because
loadDefinition already derives expectedName from the path relative to the
manager directory.
registry/go/github.com/spf13/cobra.yaml is included as the first definition. It
was taken end to end before this PR: registry build produced 56 sections /
19631 tokens, context add installed it, and `context query
'github.com/spf13/cobra' 'persistent flags'` returned the right section.
Tests: four new ones. Three cover the fetcher (prefix stripping, case escaping,
a module with no tags) and one covers a definition nested several directories
deep. Each was mutation-checked — dropping the escaping, keeping the v prefix,
and removing the recursion each turn exactly the matching test red. 45/45 green,
biome clean.
One open question for a maintainer: `std` is not on the module proxy, so the Go
standard library would need an unversioned definition like python/python.yaml
rather than a `go` one. Not included here.
|
|
Thanks for this — I reviewed it closely and verified your central claims against the live proxy. The case-escaping is right ( Four things block merge. The first is serious. 1. A 4xx from the Go proxy kills the entire nightly run.
Reproduced with a bogus module plus a valid npm definition after it: The npm definition was never reached and no This is our fragility, not yours — 2. The nightly runs 3. 4. Go submodules resolve the wrong tag and skip forever. Default Docs need updating. Two smaller notes: deleting Good contribution — the mechanism is right and the verification in your description held up. Happy to review again once these are in. Generated by Claude Code |
) publish-all called discoverVersions outside the try that guards the rest of the loop, so anything it threw escaped and ended the process. Definitions are walked in sorted order, so one third-party registry API returning 404, 410 or 429 silently abandoned every package after it and printed no summary at all. version-check.ts aborts immediately below status 500, so there was no retry either. Discovery talks to an external API and fails for reasons unrelated to the package — a renamed module, a rate limit, an upstream blip. Those now record a failure for that definition and continue, exactly like a failed build. Found while reviewing #126, which adds a Go registry and makes this trivially reachable: go/github.com/spf13/cobra sorts second of 141, so one proxy hiccup would take down the other 139. The fragility is ours and predates it. The new tests run the real CLI against a scratch registry using a registry with no version fetcher, so discovery throws before any network call. Both were verified load-bearing: they fail with the guard reverted and pass with it.
moshest
left a comment
moshest
left a comment
There was a problem hiding this comment.
Still at cac38d9 with no new commits since my Aug 31 review, so items 2–4 are open. One update in your favour: blocker 1 landed separately as #128, so a 4xx from the proxy now records a per-package failure instead of abandoning the nightly run — that one is off your plate, and the branch still merges clean against current main. Answering your std question too: an unversioned definition in the python/python.yaml shape is what we'd want, not a go/ one.
What still blocks merge is that go/github.com/spf13/cobra.yaml ships the versions: + min_version: git shape while Go discovery silently no-ops --since and drops +incompatible releases, so it would go into the nightly for all ~140 packages with known-wrong version discovery. It's been three weeks — if I don't hear back by early October I'll close this as stale, and you're very welcome to reopen when you have time.
Generated by Claude Code
|
Thanks for the patience, and for landing #128 separately. I'm picking this back up now and will have items 2 to 4 pushed by Friday 3 October: dates resolved for the versions that survive dedup so --since works for Go, a warning plus a README note for +incompatible, and a worked tag_pattern example for submodules. The README updates and the two smaller notes (a test for the v filter and encoding the module path) will be in the same push. If you'd rather not wait, I'm also happy to drop cobra.yaml from this PR so the fetcher can land on its own, and add the definition back in a follow-up once discovery is solid. Your call. Thanks for the std answer too. I'll do that as a separate PR in the python/python.yaml shape. |
Resolve Go publish dates from /@v/<version>.info for versions that survive dedup, only under --since. Warn when a package has published versions but none match its ranges, which is how Go +incompatible releases showed up before: zero versions and exit 0. Percent-encode module path segments, and cover the v-prefix guard with a test. Document Go module paths, subdirectory tag patterns and the +incompatible limitation in the registry README, and list Go in the package README.
|
Pushed, with main merged in (no force-push, so your review comments still line up). Going through your list: 2. 3. The warning isn't Go-specific, and it doesn't fire when Heads up: running 4. Submodule tags. Documented under a new "Go modules" section in Docs. Smaller notes. There's a test for the Each new test fails when I remove the code it covers, and I'll send the |
|
Re-reviewed at 91cc529. I checked each fix myself. Install, lint, build and test all pass (context 227, registry 103). For cobra, One thing left: please delete Generated by Claude Code |
@neuledge/registry is private, so this changeset would only open a release PR that bumps it and publishes nothing.
|
Thanks for checking it all again. Removed |
) npm/angular asked npm about `angular` (AngularJS, latest 1.8.3) against min_version 17.0.0, and npm/strapi asked about `strapi` (v3, latest 3.6.11) against min_version 5.0.0, so neither ever discovered a version or published docs. Reported by @JayOfTheKeyboard on #126. - npm/@angular/core: from 20.3.0 with tag_pattern v{version}; older minors were tagged without the "v" prefix. - npm/@strapi/strapi: unversioned, built from strapi/documentation docusaurus/docs/cms. strapi/strapi's docs/ is contributor documentation. Built locally: @angular/core 22.2.0 1956 sections, 20.3.32 1493 sections; @strapi/strapi 1392 sections.
3 participants
What this adds
goas a registry: version discovery throughproxy.golang.org, and the recursive definition walk that Go module paths require.Three things Go does differently, each measured against the live proxy
Uppercase letters must be escaped as
!+ lowercase.Slashes are path separators and have to survive, so
encodeURIComponentis the wrong tool here.The
vprefix is stripped before the shared helpers see it.isPrerelease("v1.10.2")istrue— it reads the leading letter as a prerelease tag — so every Go version would be silently discarded.compareSemver("v1.10.2", "v1.9.1")isNaN, so sorting breaks too. Stripping it in the fetcher and restoring it withtag_pattern: "v{version}"keeps both shared functions untouched, which is why this PR does not go near them./@v/listcarries no publish dates. Getting them costs one/@v/<version>.inforequest per version, so--sincefiltering is unavailable for Go rather than merely slow.publishedAtis left undefined.The blocker underneath it
listDefinitionsonly recursed into directories starting with@, soregistry/go/github.com/spf13/cobra.yamlwas never loaded — exit code 0, no warning. Every Go module path contains slashes, so this blocked the whole ecosystem rather than one package.The general recursive walk that replaces the special case is smaller than the case it removes. Scoped npm packages still resolve, because
loadDefinitionalready derivesexpectedNamefrom the path relative to the manager directory.Taken end to end before opening this
registry build→ 56 sections / 19631 tokens →context add→context query 'github.com/spf13/cobra' 'persistent flags'returned the correct section.registry/go/github.com/spf13/cobra.yamlis included as the first Go definition.Tests
Four new. Three on the fetcher — prefix stripping, case escaping, a module with no tagged releases — and one on a definition nested several directories deep.
Each was mutation-checked rather than just watched to pass: dropping the escaping, keeping the
vprefix, and removing the recursion each turn exactly the matching test red. 45/45 green, biome clean.One question for a maintainer
stdis not on the module proxy, so the Go standard library would need an unversioned definition likepython/python.yamluses, not agoone. Not included here — it seemed like your call which shape you want.