Skip to content

Draft /cite/ alias pages are indexed and listed in the sitemap #128

Description

@maehr

Problem

/cite/ alias pages for draft records are not marked noindex, and the sitemap includes them.

buildNoindexPredicate in src/lib/noindex.ts covers /id/… record paths and the /reg/work/{key}/ prefix. It does not cover /cite/. src/pages/cite/[...alias].astro writes one HTML redirect page for each alias. That page carries no robots meta tag. The sitemap filter in astro.config.mjs therefore keeps every alias page.

Against the real registry this publishes about 172,800 sitemap entries that point at draft references (86,397 qualified aliases and 86,397 bare aliases).

Why it matters

ADR-0003 (as amended by ADR-0004) requires draft record pages to be noindex, excluded from search, and excluded from the sitemap. The /cite/ gap defeats that rule. It also contradicts robots.txt, which tells crawlers to read /dump/ instead of crawling /id/ or /cite/.

Reproduce

  1. Run npm run build:fast.
  2. Open dist/sitemap-0.xml.
  3. Find /cite/fixture.work/fixture-alternate/1/. The fixture reference behind that alias has status: draft, and its /id/ref/… page is correctly excluded.

Suggested fix

Extend buildNoindexPredicate to return the /cite/ alias paths of draft records, and render the robots meta tag in src/pages/cite/[...alias].astro.

Found during review of #127. Not caused by that PR.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions