Repository navigation
feat(ci): add a safe production link checker - #202
Conversation
scripts/check-links.ts renders every public route from the sitemap in Chromium and checks routes, internal links and #fragments, files, canonical and hreflang URLs, RSS feeds and approved external links. It runs against the build (served by scripts/preview-server.ts) or production with --base. External links must use a host from scripts/link-check/external-links.json and are requested through safe-request.ts: http(s) on ports 80 and 443 only, every resolved address must be public (IPv4-mapped, NAT64 and 6to4 forms are judged by the IPv4 address they carry), each hop connects to the validated address through a pinned lookup so DNS rebinding cannot redirect it, and redirects are re-validated. The browser is locked to the site under test. Requests run with bounded concurrency and retry transient failures. The 404 page gets a data-page marker so the checker can detect it.
- Register /press and add a real /terms page instead of linking to 404s.
- Localized links and hreflang alternates only point at routes that exist
under /es and /pt; og.ts no longer emits them for other routes, and the
SPA shell no longer carries the homepage canonical onto every fallback
route (React 19 hoists Helmet tags next to static ones).
- Stellar and Press render their own canonical URLs.
- The footer Compare link goes to /#compare, and pages scroll to hash
targets once lazy sections render.
- Status links use /status until the status host exists.
- Chain docs links use /sdk/chains/{evm,stellar,solana,ckb}, Aquarius moved
to aqua.network, the grants CTA points at the live Stellar Wave, and
footer docs links point at pages that exist.
- Audit, bounty and disclosure links go to anchored sections on /security,
Spectre goes to its roadmap milestone, and the placeholder PGP key is
hidden.
- Team GitHub profiles that return 404 are removed.
|
@aratass is attempting to deploy a commit to the truthixify's projects Team on Vercel. A member of the Team first needs to authorize it. |
|
@aratass Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
truthixify
left a comment
truthixify
left a comment
There was a problem hiding this comment.
The link checker pins validated DNS answers, revalidates redirects, blocks private address ranges, and passes all 96 focused tests. Approved.
…nch merges cleanly
|
Merged cleanly with develop now: the Stellar page head takes develop's seo values (#201), which already include the canonical and hreflang links this PR added. On the merge with develop: tsc -b, pnpm build, all 597 unit tests, format:check, lint:i18n, validate:content, validate:jsonld and test:link-check pass. |
truthixify
left a comment
truthixify
left a comment
There was a problem hiding this comment.
The rebased link checker is ready. The focused security and routing suite passes 96/96, the production build succeeds, and the rendered-site crawl checks 69 routes, 8 feeds, 188 fragments, and 6 files with no failures. The only red status is the external Vercel authorization check.
3 participants
Closes #188
Carries over #160 and addresses every point raised on #183.
Checker
pnpm check:links(scripts/check-links.ts) seeds from the sitemap,/and/feed.xml, renders each public route in Chromium and fails on:NotFoundnow carriesdata-page="not-found")#fragmentsscripts/link-check/external-links.json, or that failBy default it checks the build through
scripts/preview-server.ts.--base https://usewraith.xyzchecks production,--report <file>writes a JSON report and--no-externalskips external requests. Allowlist entries that accept a status (www.npmjs.comanswers bots with a Cloudflare 403) or skip the request (x.com,twitter.com) must give areason.SSRF and DNS rebinding
External requests go through
scripts/link-check/safe-request.ts:::ffff:127.0.0.1,::ffff:7f00:1), NAT64 and 6to4 addresses are judged by the IPv4 address they embedlookup, so a second DNS answer is never used. TLS still verifies the certificate for the hostnameConcurrency and retries
4 pages render at once and 6 external requests run at once (
--concurrency,--external-concurrency). External checks retry up to 2 times on network errors and on 408, 425, 429, 500, 502, 503 and 504, with exponential backoff (1 s base, 10 s cap) that honorsRetry-After. Refused destinations are never retried.Link fixes from the #183 review
/statusuntil the status host exists/sdk/chains/stellarand/sdk/chains/solana, Nervos to/sdk/chains/ckb, EVM chains and Horizen to/sdk/chains/evm/security(#audits,#bug-bounty,#disclosures)/roadmap#phase-2). Milestones have ids and the page scrolls to the hash/terms(in the sitemap), not Privacy./pressis routed/as canonical). Terms, Press and the Stellar locale pages declare their own, and hreflang es/pt is only emitted for routes that have localized pagesCI
ci.yml: newlink-checkjob on every pull request. It runs the checker's unit tests, builds, runspnpm check:linksagainst the build and uploads the JSON report on failureproduction-links.yml: daily at 06:23 UTC and on demand against https://usewraith.xyzVerification
pnpm test:link-check: 96 tests, including DNS rebinding (the resolver's second answer is private and is never used), a redirect to[::ffff:127.0.0.1], a redirect to a host that resolves to a private address, mixed public and private answers, NAT64 and 6to4, redirect limits, timeouts and retriespnpm build && pnpm check:links --no-external: 69 routes, 8 feeds, 188 fragments and 6 files, no failurespnpm format:check,lint:i18n,validate:content,test:a11y,validate:jsonldandtest:smokepasssrc/__tests__/seo.test.tsx(7) andsrc/__tests__/jsonld.test.ts(1) already fail on develop and are untouched here.