Repository navigation
feat: write artifacts to .codemetrics/ and stop measuring our own output - #117
Merged
Merged
Conversation
…ity-metrics-w3wn) Part 2 of the .codemetrics/ output-directory change: without this, a run following one whose output was ever committed would score its own prior local_commit_metrics.json and local_drift_report.html as production code and duplicated content. Added '**/.codemetrics/**' to both CONFIG.DUPLICATE_IGNORE_PATTERNS and CONFIG.ANALYSIS_IGNORE_PATTERNS in lib/config.js. This is a documented, narrow exception to ANALYSIS_IGNORE_PATTERNS' empty default: the directory did not exist in any repository's history before this change, so seeding it is provably retroactively inert. Updated the 43 include_in_derivation calibration observations' recorded DUPLICATE_IGNORE_PATTERNS to match (mechanical, following the existing '.terraform/' precedent), with a new calibration/observations.json config_change_log entry recording why no re-measurement is required. Updated CLAUDE.md's Configuration table row counts and the tests that asserted the prior empty/11-pattern defaults. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Part of code-quality-metrics-w3wn's .codemetrics/ output-directory change. CLI-only, no .codemetrics.json key, mirroring --max-commits' own reasoning: where a run writes its output is a property of the run, not a fact about the repository being measured. parseCliArgs only; collectLocalMetrics does not yet read options.outputDir. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…rget root Part 1 of code-quality-metrics-w3wn. Measured across the six repositories this toolkit evaluates: writing local_*.json/local_*.html directly into the analyzed repository's root left 235 untracked files behind, none protected by that repository's own .gitignore, each one a git add . away from being committed. .codemetrics/ pairs with the existing per-repo .codemetrics.json config file and is hidden by default. Created via fs.mkdirSync recursive when absent, left alone (and its prior contents overwritten in place, unchanged from the tool's existing behavior) when already present. --output-dir overrides the default location entirely. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Part of code-quality-metrics-w3wn's implementation hazard: local-code-metrics.js and generate-drift-report.js are a write/read boundary across two separate invocations, and a partial move of only the write side would make the later read (and PR #94's counted_additions/counted_deletions guard) fail with a confusing "stale file" message when the real fault is a path mismatch. generate-drift-report.js's CLI now defaults its no-argument target directory to <cwd>/.codemetrics, matching local-code-metrics.js's own new default. An explicit directory argument still overrides this entirely, unchanged, for a run whose local-code-metrics.js invocation used --output-dir. Decision recorded: when .codemetrics/ is absent but a legacy root-level local_metrics_summary.json or local_commit_metrics.json exists one directory up, readRequiredJson refuses rather than reading it, naming the legacy file's path and instructing a re-run of local-code-metrics.js. Given this project's repeated trouble with stale inputs producing plausible output, silently reading an old pre-move file would be exactly that failure mode. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…decision CLAUDE.md gains an Output Location section: the near-collision between the tracked .codemetrics.json config file and the ignored .codemetrics/ output directory, the --output-dir flag, the legacy-root-level-file refusal, and why the agent-invented backup-<timestamp>-<label>_<filename> convention is left an operator/agent concern rather than a tool-owned feature (it would be a materially larger feature than this issue's output-location move, and a backup kept inside .codemetrics/ already inherits the new exclusion). Also updates the Configuration table row counts' surrounding prose and the Architecture section's local-code-metrics.js description. README.md's tool descriptions, Drift Report section, new Output Location section, and File Structure diagram now show .codemetrics/ instead of the target repository's root. AGENTS.md's "Running the Report Against an External Repository" section now describes the .codemetrics/ default and points at CLAUDE.md's fuller treatment of the backup-file decision, rather than leaving process.cwd() as the stated (now-stale) output location. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds .codemetrics/ explicitly alongside the existing local_*.json/local_*.html patterns. Those patterns already matched every generated file by name regardless of directory, so this repository was never at risk, but the README and CLAUDE.md now tell every other target repository to add .codemetrics/ to their own .gitignore, and this project's own ignore file should show the canonical pattern rather than rely on a filename coincidence. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Output Location section sits above the Configuration table, not below it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
PR AnalysisSize: large (based on production code) Concerns
Strengths
Commit AnalysisTotal Commits: 7
Commit Details7b1ad51 Ken Judy (8/20/2026) 55c6284 Ken Judy (8/20/2026) aa48257 Ken Judy (8/20/2026) 11153c5 Ken Judy (8/20/2026) 6b8c655 Ken Judy (8/20/2026) 37493f2 Ken Judy (8/20/2026) 19d00d8 Ken Judy (8/20/2026) Test CoverageTest-to-Production Ratio: 0.37:1 (no calibrated target for this ratio; reported descriptively) DORA Capability AssessmentArchetype: mixed-signals
Duplicate CodeFound 37 duplicate block(s) across the changed files:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes code-quality-metrics-w3wn (P1) and GitHub #112.
Two halves, and the second is the one that gets forgotten
Running the tool used to leave six files in the analysed repository root. Measured across the six repositories this toolkit evaluates: 235 untracked files, and only code-quality-metrics ignores them via local_.json and local_.html in its own .gitignore. The five targets had no protection, so every file was one
git add .from being committed, and two of those are client repositories whose JSON carries author names, commit messages, prod_file_paths and model prose. Disclosure risk, not housekeeping.Output now goes to /.codemetrics/, created when absent. The name pairs with the existing per-repo .codemetrics.json config file: one convention rather than two, and hidden by default in a repository someone else opens. The deliberate near-collision between a tracked .codemetrics.json file and an ignored .codemetrics/ directory is stated in the documentation rather than left to be discovered.
The second half: .codemetrics/** is now in BOTH CONFIG.DUPLICATE_IGNORE_PATTERNS and CONFIG.ANALYSIS_IGNORE_PATTERNS. Without it the second run measures the first run output. local_commit_metrics.json is an array of records sharing a schema, which is precisely the shape a clone detector reads as massive duplication, and local_drift_report.html is around 140KB including six base64 font blobs. Verified by unit test and by a live two-run smoke test.
Why a global default here, when data files stayed per-repo
Hand-curated data files are repo-specific: which files those are is a fact about the repository, so they belong in a per-repo config. The tool own output directory has the same name and shape in every repository it is ever pointed at, so requiring each target to declare it would repeat the same discovery forever and fail silently wherever nobody had hit the problem yet.
CLAUDE.md states ANALYSIS_IGNORE_PATTERNS is deliberately empty because seeding it would change every existing measurement including the calibration observations. This addition cannot do that, and the reasoning is recorded beside the entry: .codemetrics/ is a directory this tool only began writing with this change, strictly after every observation was recorded, so a pattern excluding it cannot have matched a file in any past run.
The provenance gate fired, and was handled by precedent rather than by weakening it
Seeding DUPLICATE_IGNORE_PATTERNS broke tests/thresholdProvenance.test.js, which exact-matches the recorded pattern list on every observation. The 43 include_in_derivation records were updated to the new default set and a config_change_log entry records why, mirroring the existing .terraform/ entry from code-quality-metrics-53j. The justification here is stronger than that precedent: .terraform/ needed a live GitHub trees API check to confirm the directory happened not to exist in the six reference repos, whereas .codemetrics/ non-existence is true by construction. No metric value was edited; the diff touches pattern lists and the log entry only.
Two decisions recorded rather than assumed
A legacy root-level file is refused, not read. generate-drift-report.js detects one a directory up from an absent .codemetrics/ and throws naming its exact path with an instruction to re-run, rather than silently rendering stale data. Given this project repeated trouble with stale inputs producing plausible output, failing loudly is the right default.
The backup--_ convention is declared an operator concern, not tool-owned. No code ever implemented it, confirmed by grep; only AGENTS.md cautionary note referenced it, so building retention logic would be new scope. The documentation now states that the tool overwrites in place, and that a backup kept inside .codemetrics/ inherits the new exclusions automatically while one kept elsewhere does not.
--output-dir is CLI-only with no .codemetrics.json key, on the same reasoning that keeps --max-commits CLI-only: where a run writes its output is a property of the run, not a fact about the repository.
Verification
729 tests across 43 suites after rebasing onto the content-duplicate work, lint and typecheck clean. Ten cycles with quoted RED; one is disclosed as vacuously green before implementation and meaningful only after, rather than presented as a clean cycle. Six pre-existing tests broke as a direct predicted consequence of the new ANALYSIS_IGNORE_PATTERNS default and were updated as stale assumptions, each named in the report.
🤖 Generated with Claude Code