Keep the Loogle key count when a prediction does not parse

[SashaMIT]

get_exact_match counted a prediction that does not parse as one key. A four-key reference plus that prediction scored 4/5. The miss now keeps the reference key count, so the same pair is 4/8. get_partial_match already did this. An unparsable reference still counts as one key.

The scorer imports its optional eval packages inside the functions that use them, so this check can run without those packages installed.

No existing issue.

Drafted with Grok 4.7. The code has not yet been reviewed by a human.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[SimJeg]

Drafted with Grok 4.7. The code has not yet been reviewed by a human.

please review then @SashaMIT

[SashaMIT]

Reviewed. get_exact_match on {'a': 1, 'b': 2, 'c': 3, 'd': 4} with a prediction that does not parse returns (0, 4). The same reference matched against itself is still (4, 4), so that pair is 4/8. A reference that does not parse is still (0, 1). That matches get_partial_match, which already kept the reference key count. I'll own this change. The disclosure line stays.

[SimJeg]

Thanks! Could you only keep the updates in get_exact_match and revert other changes ? No tests are needed for this fix. Please also keep the imports at module scope, as moving them would cause evaluation to fail after inference.

[SimJeg]

Also, are you planning to use kvpress on the Loogle benchmark ? Or are you reporting a bug reported by a coding agent ?

[SashaMIT]

Thanks. The diff is only get_exact_match now. The imports are back at the top of the file, and the test file is gone.

I'm not running Loogle. I hit this while reading get_exact_match. An unparsed prediction was counted as 1 instead of the number of keys in the reference.

[SimJeg]

Thanks for your contribution @SashaMIT