Skip to content

feat(CountBasedMetadata): add notes to example signal panels - #247

Open
saiflayouni wants to merge 2 commits into
jentic:mainfrom
saiflayouni:fix/235-example-signal-diagnostic-notes
Open

feat(CountBasedMetadata): add notes to example signal panels#247
saiflayouni wants to merge 2 commits into
jentic:mainfrom
saiflayouni:fix/235-example-signal-diagnostic-notes

Conversation

@saiflayouni

Copy link
Copy Markdown
Contributor

Summary

Closes #235

The HTML report shows present_examples and total_examples diagnostics pointing to the same locations, and the total_examples metadata count (e.g. 37) differs from the number of paths shown in the diagnostics (e.g. 27). This is correct engine behavior — diagnostics mark unique locations, while the metadata counts individual example objects — but was opaque to users.

  • Adds an optional note field to CountConfig in CountBasedMetadata.tsx
  • Renders the note in both the donut-chart branch (response_coverage) and the progress-bar branch (all other count signals), replacing the previous hardcoded string
  • For example_density: explains that expected_examples and present_examples diagnostics list overlapping location sets (they converge when all expected locations already have examples)
  • For example_validity: explains that the total-examples count in the summary can exceed the diagnostic location count when a single location holds multiple examples

Test plan

  • All 75 existing @jentic/api-scorecard-formatter-html unit tests pass (npm test -w @jentic/api-scorecard-formatter-html)
  • signals.test.tsx SSR tests for example_density and example_validity still pass (no undefined/NaN, expected numerator/denominator values)
  • response_coverage note text is unchanged (moved from hardcoded JSX to COUNT_SIGNAL_CONFIG)

Add a `note` field to `CountConfig` and render it in both the donut
and progress-bar branches, replacing the previous hardcoded string for
`response_coverage`.

For `example_density` and `example_validity`, the notes explain why
the diagnostic path count can differ from the summary metric total:
the engine emits one diagnostic path per unique location, while the
metadata counts individual example objects — so a location with
multiple examples appears once in diagnostics but is counted multiple
times in the total.

Refs jentic#235

Signed-off-by: saiflayouni <saiflayouni77@gmail.com>
The previous note implied expected_examples and present_examples overlap,
which is wrong: expected_examples is the superset (all elements that *can*
hold an example); present_examples is the subset that *do*. Reword to
reflect the element-vs-subset relationship confirmed in issue jentic#235.

Refs jentic#235

Signed-off-by: saiflayouni <saiflayouni77@gmail.com>
@saiflayouni

Copy link
Copy Markdown
Contributor Author

Thanks for the detailed explanation in #235, @char0n! The example_density note I had originally written implied expected_examples and present_examples overlap, which was wrong. Corrected in 6d3fda8 — it now describes them as superset/subset: expected_examples marks every element that can hold a value; present_examples is the subset that already does.

The example_validity note (total counts values, paths count elements) matches your explanation and is unchanged.

I saw #245 is open for the broader tooltip/help-text work — happy to help there too once #42 lands definitions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Diagnostics has duplicate metrics/items for present_examples , total_examples

1 participant