Skip to content

fix(context): only skip repo-meta filenames at the scan root - #125

Merged
moshest merged 3 commits into
neuledge:mainfrom
JayOfTheKeyboard:fix/ignored-files-root-only
Sep 24, 2026
Merged

moshest merged 3 commits into
neuledge:mainfrom
JayOfTheKeyboard:fix/ignored-files-root-only

Conversation

@JayOfTheKeyboard

Copy link
Copy Markdown
Contributor

The problem

IGNORED_FILES in packages/context/src/git.ts is matched by basename at every depth, not just at the scan root. The set is repo-root housekeeping — security, license, changelog, contributing, history, code_of_conduct, claude, and the two issue templates — but applied recursively it silently drops ordinary documentation pages that happen to share one of those names.

The build reports success either way. context add prints Found N markdown files and exits 0, so the only symptom is a later query returning nothing, which is indistinguishable from a query that simply has no answer.

Measured

Against @neuledge/context 1.2.3, on a fresh clone of codeberg.org/forgejo/docs:

138 markdown files under docs/
135 reach the builder

The two lost files are docs/admin/actions/security.md and docs/user/actions/security.md — the pages documenting how to secure Forgejo Actions. The first is the only source in the repository for container.valid_volumes, so context query forgejo valid_volumes returned nothing at all.

After this change, the same query returns:

Securing Forgejo Actions Deployments > Job Containers with Docker (part 2) — "The default value of valid_volumes is an empty array []. If an administrator changes this, they will allow a job container or service container to mount the listed volumes…"

Package: 130 → 132 documents, 728 → 742 sections.

Not just forgejo

Checked every repository I have a package for, via the GitHub trees API:

repo dropped by this rule
docker/docs content/manuals/extensions/extensions-sdk/architecture/security.md, content/manuals/billing/history.md, content/manuals/compose/intro/history.md, content/reference/api/dvp/changelog.md, content/reference/api/hub/changelog.md
excalidraw/excalidraw dev-docs/docs/introduction/contributing.mdx
reactjs/react.dev, spf13/cobra nothing outside .github/
golang/go only vendored subtrees

The change

Apply IGNORED_FILES only when basePath === "". That is the case the set was written for, and it leaves nested pages alone. Six lines in git.ts.

Two tests, both written before the change and both mutation-checked:

  • skips repo-meta files at the scan root — reddens if the filter is disabled entirely.
  • keeps a documentation page that merely shares a repo-meta filename — reddens if the basePath === "" guard is removed.

pnpm test is green (223/223) and biome is clean.

What this deliberately does not change

docs/license.md in the forgejo repo still gets skipped, because it sits at the scan root and genuinely is a licence. Link-only index.md pages are still dropped by isTableOfContents. Both are correct, and I checked them before assuming the whole gap was one bug.

One thing worth considering separately: a build that drops files could say so. A count of files found versus documents actually indexed, printed at the end, would have made this visible immediately rather than months later. Happy to open that as its own issue or PR if you'd like it.

`IGNORED_FILES` is matched by basename at every depth, so any documentation
page that happens to be named `security.md`, `license.md`, `changelog.md`,
`contributing.md` or `history.md` is dropped as if it were repo housekeeping.

Measured on codeberg.org/forgejo/docs with @neuledge/context 1.2.3: 138
markdown files under `docs/`, 135 reach the builder. The two lost files are
`docs/admin/actions/security.md` and `docs/user/actions/security.md` — real
documentation about securing Forgejo Actions, and the only source in the repo
for `container.valid_volumes`. `context add` prints "Found 135 markdown files"
and exits 0, so nothing signals the loss; a later query for `valid_volumes`
simply returns nothing.

Other repos hit by the same rule: docker/docs loses
`content/manuals/extensions/extensions-sdk/architecture/security.md`, two
`history.md` pages and two `changelog.md` API references;
excalidraw/excalidraw loses `dev-docs/docs/introduction/contributing.mdx`.

Restricting the check to `basePath === ""` keeps the original intent — those
names mean repo housekeeping at the top of a tree — while leaving nested pages
alone. Two tests cover both halves.
@changeset-bot

changeset-bot Bot commented Aug 29, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 263c096

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages
Name Type
@neuledge/context Patch
@neuledge/registry Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

moshest commented Aug 31, 2026

Copy link
Copy Markdown
Member

Good catch, and the diagnosis is right — basePath === "" is the correct root test (git.ts:530 seeds the walk with ""), and the forgejo case is real.

One thing to fix before I merge: the same bug survives one level up when docs_path is set.

searchPath = docsPath ? join(basePath, docsPath) : basePath, and the walk starts from searchPath with basePath = "". So when a definition sets docs_path: docs, the scan root is the docs directory — and a genuine page at docs/security.md is still dropped:

WITH docs_path='docs':          WITHOUT docs_path:
  docs/admin/actions/security.md   docs/admin/actions/security.md
  docs/guide.md                    docs/guide.md
                                   docs/security.md      <- kept here, dropped above

(Scratch repo with SECURITY.md at the repo root plus those three pages. SECURITY.md is correctly dropped in both.)

So the same file is kept or dropped depending on whether the definition happens to set docs_path — and 135 of our 140 definitions set it, so that's the common path, not the edge case.

The underlying reason is that IGNORED_FILES exists for repo roots — SECURITY.md, CONTRIBUTING.md, LICENSE.md are housekeeping there. Inside a docs tree they're just pages, which is exactly your argument, and it applies to the docs root too. Suggest gating on whether the scan root is the repo root rather than on depth:

// in readLocalDocsFiles
const atRepoRoot = !docsPath;
const markdownFiles = findMarkdownFiles(searchPath, ig, "", { lang, atRepoRoot });

and in findMarkdownFiles, if (matchingExt && basePath === "" && options.atRepoRoot).

Worth a third test alongside your two: docs/security.md with path: "docs" should be kept. Your existing pair is well chosen — the root case and the nested case are exactly right.

Everything else looks good: the changeset explains the user-visible impact well, and naming forgejo's container.valid_volumes as the concretely lost content is the kind of detail that makes a bug report verifiable.


Generated by Claude Code

@moshest moshest left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still at a24d887 with no new commits since my Aug 31 review. I re-checked against current main: the bug is still live (git.ts:479 is unchanged, nothing in #134/#135/#136 touched it) and your branch still merges clean, so this fix is still wanted. The one outstanding item is unchanged — gate on the scan root being the repo root rather than on depth, because 135 of our 140 definitions set docs_path and a genuine page at docs/security.md is still dropped under those.

Three weeks quiet now, so to be straight with you: if I don't hear back by early October I'll close this as stale. That's just housekeeping, not a judgement on the patch — reopen whenever you can pick it up.


Generated by Claude Code

`IGNORED_FILES` is housekeeping for a repository root, so the skip should
apply only when the walk starts there. The previous guard tested
`basePath === ""`, which is also true at the top of a `docs_path` folder,
because `readLocalDocsFiles` seeds the walk from `searchPath` with an empty
base. A genuine page at `docs/security.md` was therefore still dropped
whenever a definition set `docs_path`, which 135 of the 140 definitions do.

`readLocalDocsFiles` now passes `atRepoRoot: !docsPath` into
`findMarkdownFiles`, and the skip requires both that flag and the empty
base path. Same file is kept or dropped on what it is, not on how the scan
happened to be rooted.

Third test added: with `path: "docs"`, a root `SECURITY.md` is still skipped
and `docs/security.md` is kept. It fails without the flag, and the existing
root test fails if the flag is forced false, so both directions are covered.

224/224 green, biome clean.
@JayOfTheKeyboard

Copy link
Copy Markdown
Contributor Author

Sorry for the long delay, and thanks for holding it open. Pushed as 263c096.

You were right about docs_path. The scan root is the docs directory in that case, so basePath === "" matches at docs/ and a genuine docs/security.md was still dropped. With 135 of 140 definitions setting it, that is the common path, so the patch as it stood fixed the rarer half of the bug.

Done the way you suggested. FindMarkdownOptions gains atRepoRoot, readLocalDocsFiles passes atRepoRoot: !docsPath, and the skip now requires that flag as well as the empty base path. findMarkdownFiles has no other caller, so there is nothing else to thread it through.

Third test added as you asked: with path: "docs", a root SECURITY.md is still skipped while docs/security.md and docs/guide.md are both kept. Mutation-checked in both directions, since the pair is only worth having if each half can fail: the new test reddens against the old guard, and forcing atRepoRoot: false reddens your first case, skips repo-meta files at the scan root.

224/224 green, biome clean.

The separate point about a build reporting files found against documents actually indexed is still open on my side. Say the word and I will raise it as its own issue rather than growing this PR.

@moshest
moshest merged commit 8663d2f into neuledge:main Sep 24, 2026
3 checks passed

moshest commented Sep 24, 2026 •

Copy link
Copy Markdown
Member

Merged, thank you! The atRepoRoot change was exactly right, and the forgejo evidence made this easy to verify. It will go out in the next patch release.


Generated by Claude Code

@github-actions github-actions Bot mentioned this pull request Sep 24, 2026
moshest pushed a commit that referenced this pull request Sep 25, 2026
Releases @neuledge/context 1.2.5 -> 1.2.6 (patch).

Consumes one changeset, .changeset/lucky-pugs-repeat.md (patch on
@neuledge/context, from #125): repo-meta filenames are only skipped at the
repository root. @neuledge/registry 0.0.18 -> 0.0.19 is the automatic
dependent bump for the private workspace package.

Verified before merging: npm dist-tags.latest is 1.2.5 and 1.2.6 is not
published yet.
moshest added a commit that referenced this pull request Sep 25, 2026
findMarkdownFiles and readLocalDocsFiles built stored paths with
path.join, which uses backslashes on Windows, so docs added there were
stored as "docs\guide.md". The two repo-meta tests from #125 caught it
once #159 turned on Windows CI, and main's Windows test job was red.

Stored paths are package data, not filesystem paths, so they are now built
with posix.join. Reading files still uses the platform join. This also
means gitignore matching gets forward-slash paths on every platform.

Test (Windows) passes on this PR, along with Linux tests, lint and build.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants