Skip to content

build(coverage): the ... exclude rule hides whole functions from the coverage report #1681

Closed

Description

@planetf1

What happens today

Add a function whose signature contains Callable[..., T], run the test suite, and look for it in the
coverage report. It is not there. Not listed as covered, not listed as missing, simply absent. The same
goes for any block whose opening line contains three dots.

So a reviewer who checks coverage on those functions sees a clean result for code that was never
measured, and the "new code has 100% coverage" item in our PR checklist cannot be verified by our own
tooling.

Why that is a problem

It is silent, and it hides exactly the kind of code we most want measured. In mellea/backends/huggingface.py
the affected functions include the composed local-file, embedded and prompt generation paths, which are
the three places adapter generation actually happens.

How it should behave

Three dots used as a stub body should be excluded, as intended. Three dots inside a type annotation
should not.

Cause

From pyproject.toml, [tool.coverage.report]:

exclude_lines = [
    ...
    "\\.\\.\\.",  # Ellipsis in function bodies
]

exclude_lines entries are regular expressions matched against each source line. When a matched line
opens a block, coverage excludes the whole block. generate_func: Callable[..., _T] matches, so the
function it heads disappears from the report.

Fix

Anchor the pattern to a stub body:

    "^\\s*\\.\\.\\.$",

Then check that the excluded-line count drops and that Protocol and overload stub bodies are still
excluded.

Detail: measured impact

From a full default-suite run (-m "not qualitative and not slow", all extras installed, 4546 passed):

  • 3015 excluded lines across mellea and cli
  • 14 excluded blocks headed by a line containing Callable[...]
  • four generation helpers in mellea/backends/huggingface.py entirely unmeasured

Related

Found while measuring branch coverage during the review of #1654, whose new weightless generation helper
is one of the functions this rule hides.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions