fix(context): only skip repo-meta filenames at the scan root - #125
Conversation
`IGNORED_FILES` is matched by basename at every depth, so any documentation page that happens to be named `security.md`, `license.md`, `changelog.md`, `contributing.md` or `history.md` is dropped as if it were repo housekeeping. Measured on codeberg.org/forgejo/docs with @neuledge/context 1.2.3: 138 markdown files under `docs/`, 135 reach the builder. The two lost files are `docs/admin/actions/security.md` and `docs/user/actions/security.md` — real documentation about securing Forgejo Actions, and the only source in the repo for `container.valid_volumes`. `context add` prints "Found 135 markdown files" and exits 0, so nothing signals the loss; a later query for `valid_volumes` simply returns nothing. Other repos hit by the same rule: docker/docs loses `content/manuals/extensions/extensions-sdk/architecture/security.md`, two `history.md` pages and two `changelog.md` API references; excalidraw/excalidraw loses `dev-docs/docs/introduction/contributing.mdx`. Restricting the check to `basePath === ""` keeps the original intent — those names mean repo housekeeping at the top of a tree — while leaving nested pages alone. Two tests cover both halves.
🦋 Changeset detectedLatest commit: 263c096 The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
Good catch, and the diagnosis is right — One thing to fix before I merge: the same bug survives one level up when
(Scratch repo with So the same file is kept or dropped depending on whether the definition happens to set The underlying reason is that // in readLocalDocsFiles
const atRepoRoot = !docsPath;
const markdownFiles = findMarkdownFiles(searchPath, ig, "", { lang, atRepoRoot });and in Worth a third test alongside your two: Everything else looks good: the changeset explains the user-visible impact well, and naming forgejo's Generated by Claude Code |
moshest
left a comment
There was a problem hiding this comment.
Still at a24d887 with no new commits since my Aug 31 review. I re-checked against current main: the bug is still live (git.ts:479 is unchanged, nothing in #134/#135/#136 touched it) and your branch still merges clean, so this fix is still wanted. The one outstanding item is unchanged — gate on the scan root being the repo root rather than on depth, because 135 of our 140 definitions set docs_path and a genuine page at docs/security.md is still dropped under those.
Three weeks quiet now, so to be straight with you: if I don't hear back by early October I'll close this as stale. That's just housekeeping, not a judgement on the patch — reopen whenever you can pick it up.
Generated by Claude Code
`IGNORED_FILES` is housekeeping for a repository root, so the skip should apply only when the walk starts there. The previous guard tested `basePath === ""`, which is also true at the top of a `docs_path` folder, because `readLocalDocsFiles` seeds the walk from `searchPath` with an empty base. A genuine page at `docs/security.md` was therefore still dropped whenever a definition set `docs_path`, which 135 of the 140 definitions do. `readLocalDocsFiles` now passes `atRepoRoot: !docsPath` into `findMarkdownFiles`, and the skip requires both that flag and the empty base path. Same file is kept or dropped on what it is, not on how the scan happened to be rooted. Third test added: with `path: "docs"`, a root `SECURITY.md` is still skipped and `docs/security.md` is kept. It fails without the flag, and the existing root test fails if the flag is forced false, so both directions are covered. 224/224 green, biome clean.
|
Sorry for the long delay, and thanks for holding it open. Pushed as 263c096. You were right about Done the way you suggested. Third test added as you asked: with 224/224 green, biome clean. The separate point about a build reporting files found against documents actually indexed is still open on my side. Say the word and I will raise it as its own issue rather than growing this PR. |
|
Merged, thank you! The Generated by Claude Code |
Releases @neuledge/context 1.2.5 -> 1.2.6 (patch). Consumes one changeset, .changeset/lucky-pugs-repeat.md (patch on @neuledge/context, from #125): repo-meta filenames are only skipped at the repository root. @neuledge/registry 0.0.18 -> 0.0.19 is the automatic dependent bump for the private workspace package. Verified before merging: npm dist-tags.latest is 1.2.5 and 1.2.6 is not published yet.
findMarkdownFiles and readLocalDocsFiles built stored paths with path.join, which uses backslashes on Windows, so docs added there were stored as "docs\guide.md". The two repo-meta tests from #125 caught it once #159 turned on Windows CI, and main's Windows test job was red. Stored paths are package data, not filesystem paths, so they are now built with posix.join. Reading files still uses the platform join. This also means gitignore matching gets forward-slash paths on every platform. Test (Windows) passes on this PR, along with Linux tests, lint and build.
The problem
IGNORED_FILESinpackages/context/src/git.tsis matched by basename at every depth, not just at the scan root. The set is repo-root housekeeping —security,license,changelog,contributing,history,code_of_conduct,claude, and the two issue templates — but applied recursively it silently drops ordinary documentation pages that happen to share one of those names.The build reports success either way.
context addprintsFound N markdown filesand exits 0, so the only symptom is a later query returning nothing, which is indistinguishable from a query that simply has no answer.Measured
Against
@neuledge/context1.2.3, on a fresh clone ofcodeberg.org/forgejo/docs:The two lost files are
docs/admin/actions/security.mdanddocs/user/actions/security.md— the pages documenting how to secure Forgejo Actions. The first is the only source in the repository forcontainer.valid_volumes, socontext query forgejo valid_volumesreturned nothing at all.After this change, the same query returns:
Package: 130 → 132 documents, 728 → 742 sections.
Not just forgejo
Checked every repository I have a package for, via the GitHub trees API:
docker/docscontent/manuals/extensions/extensions-sdk/architecture/security.md,content/manuals/billing/history.md,content/manuals/compose/intro/history.md,content/reference/api/dvp/changelog.md,content/reference/api/hub/changelog.mdexcalidraw/excalidrawdev-docs/docs/introduction/contributing.mdxreactjs/react.dev,spf13/cobra.github/golang/goThe change
Apply
IGNORED_FILESonly whenbasePath === "". That is the case the set was written for, and it leaves nested pages alone. Six lines ingit.ts.Two tests, both written before the change and both mutation-checked:
skips repo-meta files at the scan root— reddens if the filter is disabled entirely.keeps a documentation page that merely shares a repo-meta filename— reddens if thebasePath === ""guard is removed.pnpm testis green (223/223) and biome is clean.What this deliberately does not change
docs/license.mdin the forgejo repo still gets skipped, because it sits at the scan root and genuinely is a licence. Link-onlyindex.mdpages are still dropped byisTableOfContents. Both are correct, and I checked them before assuming the whole gap was one bug.One thing worth considering separately: a build that drops files could say so. A count of files found versus documents actually indexed, printed at the end, would have made this visible immediately rather than months later. Happy to open that as its own issue or PR if you'd like it.