Skip to content

Fix library_hold_ratio integer division in inventory report (PP-4552) - #3762

Merged
dbernstein merged 3 commits into
mainfrom
bugfix/inventory-report-shared-loan-count
Sep 28, 2026
Merged

dbernstein merged 3 commits into
mainfrom
bugfix/inventory-report-shared-loan-count

Conversation

@dbernstein

@dbernstein dbernstein commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Description

library_hold_ratio divided a bigint (from count()) by an integer column, so PostgreSQL truncated every ratio below 1.0 to zero. The numerator is now cast to numeric and the result rounded to two places.

A title with 31 holds against 61 owned copies reported 0; it now reports 0.51.

Note: this PR was originally larger and has been re-split. It is now the hold-ratio fix only; all loan-count changes moved to #3763. The branch name is left over from the original split and no longer describes the contents.

Motivation and Context

PP-4552. Library admins reported inventory activity report figures that disagreed with the Collection Manager.

How Has This Been Tested?

New test_inventory_activity_report_hold_ratio covers a fractional ratio (31/61), a whole-number ratio, zero holds, and the -1 no-owned-copies sentinel. Confirmed to fail against the pre-fix query, reporting 0.0 instead of 0.51.

The existing suite missed this because its only non-zero case was 3 holds / 1 license, where truncation is a no-op.

19 tests pass. mypy and pre-commit clean.

Checklist

  • I have updated the documentation accordingly.
  • All new and existing tests passed.

🤖 Generated with Claude Code

@dbernstein dbernstein added the bug Something isn't working label Sep 23, 2026
@claude

claude Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @dbernstein's task in 1m 23s —— View job


Summary

This looks good to merge. I found nothing that needs changing. The fix is small and correct: casting the numerator to Numeric makes PostgreSQL resolve the whole CASE to numeric, so the -1 sentinel still comes out as -1. That means the existing int(no_licenses_owned_row["library_hold_ratio"]) == -1 assertion in test_generate_report keeps passing. The values now reach the writers as Decimal. csv.QUOTE_NONNUMERIC treats Decimal as a number and leaves it unquoted, and openpyxl writes it as a number cell, so the column stays numeric in both the CSV and XLSX outputs. The only visible difference is formatting: whole ratios now print as 3.00 / 0.00 instead of 3 / 0. This division was the only one in the module, so no other column has the same bug. All the earlier review comments are addressed:

  • The docstring now says truncation happens at any magnitude.
  • There is now a fraction-below-1 case (31/61) and a fraction-above-1 case (7/2), plus the whole-number case.
  • The nested test helpers are annotated.

Because the test compares with pytest.approx at its default tolerance, it would also fail if the round(…, 2) were dropped (0.508… ≠ 0.51).

@greptile-apps

greptile-apps Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[Medium risk] Fixes integer division bug in hold ratio calculation.

The hold-ratio change appears safe to merge, with no outstanding findings in this PR.

Summary

This PR casts the hold count to a numeric value before calculating library_hold_ratio, preserving fractional results, and adds tests for fractional, whole-number, zero-hold, and no-owned-copy cases.

Reviews (8) · Last reviewed commit: "Correct hold ratio truncation wording an..."

Comment thread src/palace/manager/celery/tasks/generate_inventory_and_hold_reports.py Outdated
@codecov

codecov Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.71%. Comparing base (54de833) to head (028f078).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3762      +/-   ##
==========================================
- Coverage   93.71%   93.71%   -0.01%     
==========================================
  Files         510      510              
  Lines       46515    46518       +3     
  Branches     6314     6314              
==========================================
  Hits        43593    43593              
- Misses       1888     1889       +1     
- Partials     1034     1036       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@dbernstein
dbernstein force-pushed the bugfix/inventory-report-shared-loan-count branch from 72d622a to 256c289 Compare September 24, 2026 16:19
@dbernstein dbernstein changed the title Fix inventory activity report loan count and hold ratio (PP-4552) Fix library_hold_ratio integer division in inventory report (PP-4552) Sep 24, 2026
@dbernstein

Copy link
Copy Markdown
Contributor Author

Both review findings confirmed and addressed — this PR has been re-split.

I reproduced each finding against a live database rather than reasoning from the code:

  • ODL pools — a license with concurrency 1 and 1 checkout left goes to owned 0 / available 0 / reserved 0 the moment it is borrowed, so the subtraction reported 0 while a patron held the loan. With concurrency 5 and 3 checkouts left, 2 copies out reported 0 instead of 2. The formula is only correct while checkouts_left - k >= terms_concurrency.
  • Unlimited pools — 3 active loans reported 0, since all four counters are pinned to zero at import.

Both produced rows where library_active_loan_count exceeded shared_active_loan_count, which is self-evidently wrong.

Since the two findings share one root cause — deriving loans from availability counters — the fix is to count Loan rows instead, per @claude's suggestion. That is exact for all three pool types and guarantees library <= shared.

Re-split along the conceptual seam so each PR is focused:

  • This PR is now the library_hold_ratio integer-division fix only. No semantics, no open questions.
  • Count inventory report loans from Loan rows (PP-4552) #3763 carries all loan-counting work: both laterals folded into one helper with a shared definition of "active", the Loan-row count, and the expired-loan filter.

The branch name here is left over from the original split and no longer matches the contents; renaming would have meant closing this PR, so I left it.

New test asserts 31/61 = 0.51 and was confirmed to fail against the pre-fix query.

@dbernstein
dbernstein force-pushed the bugfix/inventory-report-shared-loan-count branch from 256c289 to 500887f Compare September 25, 2026 16:13
Comment thread tests/manager/celery/tasks/test_generate_inventory_and_hold_reports.py Outdated
@dbernstein
dbernstein requested a review from a team September 25, 2026 19:04
@dbernstein
dbernstein force-pushed the bugfix/inventory-report-shared-loan-count branch 2 times, most recently from d556243 to 52df195 Compare September 28, 2026 17:40

@tdilauro tdilauro left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks mostly good, but has some misleading test cases and comments. Approving now, but please resolve the comments before merging.

Comment thread src/palace/manager/celery/tasks/generate_inventory_and_hold_reports.py Outdated
Comment thread tests/manager/celery/tasks/test_generate_inventory_and_hold_reports.py Outdated
dbernstein and others added 3 commits September 28, 2026 16:20
library_hold_ratio divided a bigint count() by an integer column, so
PostgreSQL truncated every ratio below 1.0 to zero. A title with 31 holds
against 61 owned copies reported 0.

Cast the numerator to numeric and round to two places.

The existing test missed this because its only non-zero case was 3 holds /
1 license, where truncation is a no-op. The new test covers a fractional
ratio, a whole-number ratio, zero holds, and the no-owned-copies sentinel.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review feedback: work_with had no return annotation and ratio_for had no
annotation on its work parameter. CLAUDE.md requires type hints on new
functions, nested helpers included.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…4552)

Review feedback. Both the docstring and a test comment described integer
division as truncating ratios below 1.0 to zero. It actually discards the
fractional part at any magnitude: 7 holds against 2 copies reported 3
rather than 3.5.

The tests only exercised a fractional ratio below 1.0 (31/61) and a whole
ratio above it (6/2). The whole ratio is precisely the case truncation
leaves unchanged, so nothing above 1.0 was really covered. Add 7/2, which
fails against the unfixed query on its own.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@dbernstein
dbernstein force-pushed the bugfix/inventory-report-shared-loan-count branch from ae45336 to 028f078 Compare September 28, 2026 23:20
@dbernstein
dbernstein enabled auto-merge (squash) September 28, 2026 23:24
@dbernstein
dbernstein merged commit e9bfc35 into main Sep 28, 2026
25 checks passed
@dbernstein
dbernstein deleted the bugfix/inventory-report-shared-loan-count branch September 28, 2026 23:29
dbernstein added a commit that referenced this pull request Sep 29, 2026
> Stacked on #3762 — review and merge that one first. Only the last
commit belongs to this PR.

## Description

`shared_active_loan_count` reported `LicensePool.licenses_reserved` —
the copies held in reserve for patrons at the head of the hold queue,
not a loan count. The mapping came from the original raw-SQL report
(#1761) and was carried over verbatim into the SQLAlchemy rewrite.

Both loan counts now come from `Loan` rows, via a single lateral helper
that takes a library filter or not, so the two columns share one
definition of "active".

The change also excludes expired loans, which
`library_active_loan_count` was counting.

## Motivation and Context

PP-4552.

**Why not derive the count from the availability counters.** The obvious
fix is `licenses_owned - licenses_available - licenses_reserved`. That
was this PR's first approach, and review
([#3762](#3762 (comment)))
correctly rejected it — the counters cannot express a loan count for
every pool type:

- **`AGGREGATED` (ODL)** pools recompute `licenses_owned` from their
licenses as `min(checkouts_left, terms_concurrency)`, and
`License.checkout()` decrements `checkouts_left`. A loan therefore
shrinks the owned and available counts *together* and cancels itself
out. Verified against a live pool: a license with concurrency 1 and 1
checkout left goes to owned 0 / available 0 / reserved 0 the moment it
is borrowed, reporting **0** while a patron holds the loan. With
concurrency 5 and 3 checkouts left, 2 copies out reports **0** instead
of 2.
- **`UNLIMITED`** pools are imported with all four counters pinned to
zero
([`data_layer/circulation.py`](https://github.com/ThePalaceProject/circulation/blob/main/src/palace/manager/data_layer/circulation.py#L346-L353)),
so any derived count is always zero. Verified: a pool with 3 active
loans reported **0**.

Both produced a self-contradicting row, with the library's own loans
exceeding the supposedly-broader shared count:

```
library_active_loan_count = 3     shared_active_loan_count = 0
```

Counting `Loan` rows is exact for all three pool types and comes from
the same source as `library_active_loan_count`, so `library <= shared`
holds by construction. It also matches the column's own scope:
`is_shared_collection` means "more than one library on this CM's
integration", and the `loans` table covers exactly those libraries.

**Expired loans.** `_library_loans_lateral` had no filter on `Loan.end`,
unlike the holds lateral beside it. `loan_reaper` deletes expired loans,
but only for metered pools and only when it next runs, so the count
drifted above the Collection Manager between runs — and for `UNLIMITED`
and open-access pools was never corrected at all.

## How Has This Been Tested?

Three new tests:

- `test_inventory_activity_report_loan_counts` — library and shared
scoping on one shared collection, with expired loans in both scopes that
must not be counted.
- `test_inventory_activity_report_loan_counts_by_pool_type` —
parametrized over `AGGREGATED`, `UNLIMITED` and `METERED`, building real
`License` rows for the ODL case so the counter-cancellation path is
actually exercised.
- `test_inventory_activity_report_shared_loan_count_unshared_collection`
— the `-1` sentinel for unshared collections.

All three were confirmed to fail against the original
`licenses_reserved` query. The pool-type test was additionally run
against the rejected subtraction formula, where `odl` and `unlimited`
fail and `metered` passes — reproducing the review findings exactly.

24 tests pass. `mypy` and pre-commit clean.

## Checklist

- [x] I have updated the documentation accordingly.
- [x] All new and existing tests passed.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants