Skip to content

Add build_upgrade_snapshots for leadership upgrade-model training tables - #242

Merged
shivamlalakiya merged 1 commit into
mainfrom
e4-upgrade-snapshots
Sep 25, 2026
Merged

shivamlalakiya merged 1 commit into
mainfrom
e4-upgrade-snapshots

Conversation

@shivamlalakiya

Copy link
Copy Markdown
Contributor

Summary

Fundraising teams running an upgrade model, will a mid-level donor move into leadership giving next year, need a training table with one row per donor per fiscal year, built so next year's giving can't leak into this year's features. philanthropy.ingest.build_upgrade_snapshots(gifts, *, fiscal_years, threshold=1000, band=(100, 999), fiscal_year_start=7, activities=None, donors=None) does that.

For each fiscal year T in fiscal_years, the population is every donor whose FY T total giving falls inside band (inclusive) and stays below threshold even if band is wider than that; a donor already giving at leadership level in T is not an upgrade candidate. Every feature is built from gift (and, if given, activity) rows dated at or before the end of FY T: prior-year totals (T-1, T-2), trend, largest single gift, gift count, consecutive years given, and months since the last gift. activities_to_features is called with as_of set to the end of FY T when an activity log is supplied, and static donor attributes are joined in as given. target is 1 if the donor's FY T+1 total reaches threshold, computed from FY T+1 gift data only, including 0 for a donor who gave nothing at all that year. The output's fiscal_year and donor-id columns feed FiscalYearGroupedSplitter directly (covered by an end-to-end test).

Why philanthropy.ingest

It's the same shape as every other bridge already there: raw/normalised tables in, one donor-level feature table out, and it calls activities_to_features directly rather than duplicating its logic. philanthropy.preprocessing was the other candidate, but every class there is an sklearn TransformerMixin, and this function changes row count and manufactures a label column, neither of which a transform() can do. philanthropy.model_selection houses the splitter this feeds, but its declared scope is cross-validation, not building the table a splitter consumes.

Fiscal-year boundaries reuse FiscalYearTransformer rather than reimplementing the month arithmetic, matching its fiscal_year_start convention.

Note on branch base

This was originally planned to stack on e2-activities-to-features (PR #232) since it depends on activities_to_features, which wasn't merged yet when this work started. Both #232 and PR #233 merged to main while this was in progress, so this PR is based directly on current main instead; no stacking needed.

Test plan

  • python -c "from philanthropy.ingest import build_upgrade_snapshots; print('OK')"
  • tests/test_upgrade_snapshots.py (37 tests): band inclusion/exclusion at both bounds, the threshold carve-out on a wider band, fiscal_year set to T not T+1, multi-year stacking, target reading FY T+1 only (including "no gift at all" -> 0, and a T+2 gift never affecting it), feature values unchanged by the size or presence of FY T+1+ rows, idempotency on repeat calls, gift-derived feature correctness (totals, trend, largest gift, gift count, consecutive years given, months since last gift), the activities and donors joins (including the activity cutoff matching the end of FY T and a missing donor getting NaN rather than a dropped row), validation errors, and an end-to-end FiscalYearGroupedSplitter integration test.
  • make ci (flake8, mypy, doctest, full suite, coverage floor): 2177 passed, 8 skipped, overall coverage 98.62% against the 92% floor; _upgrade_snapshots.py itself at 100%.
  • make riskcov (risk-tier floor over preprocessing/, models/, ingest/, cli.py, utils/_persistence.py): 98% against the 93% floor.
  • sh scripts/install_hooks.sh pre-push hook ran the full suite on push: 2177 passed, 8 skipped.

Changelog / docs

  • Added an entry under ## [Unreleased] -> ### Added in CHANGELOG.md.
  • Added build_upgrade_snapshots to the Tier 2 (Beta) table in docs/reference/index.md: the gift-derived feature set is a minimal starting recipe and likely to be refined, same reasoning as the sibling ingest bridges.

Fundraising teams running an upgrade model (will a mid-level donor move
into leadership giving next year?) need a training table with one row per
donor per fiscal year, built without letting next year's giving leak into
this year's features. build_upgrade_snapshots(gifts, *, fiscal_years,
threshold=1000, band=(100, 999), fiscal_year_start=7, activities=None,
donors=None) does that: for each fiscal year T, it takes every donor whose
FY T total lands in the upgrade band (and stays below threshold even if the
band is wider), computes gift-derived features (prior-year totals, trend,
largest gift, gift count, consecutive years given, months since last gift)
from data through the end of T, optionally joins activities_to_features and
static donor attributes, and sets target to whether the donor crossed
threshold in FY T+1.

Lives in philanthropy.ingest: it is the same shape as the other bridges
there (raw/normalised tables in, one donor-level feature table out) and
calls activities_to_features directly. preprocessing was the other
candidate, but every class there is an sklearn TransformerMixin, and this
function changes row count and manufactures a label column, which a
transform() cannot do; model_selection's declared scope is cross-validation,
not building the table a splitter consumes.

Fiscal-year boundaries reuse FiscalYearTransformer rather than
reimplementing the month arithmetic. The output's fiscal_year and donor-id
columns feed FiscalYearGroupedSplitter directly, covered by an end-to-end
test.

Tested with 37 cases: band inclusion/exclusion (including the threshold
carve-out on a wider band), target reading only FY T+1, feature values
staying put when FY T+1+ rows are appended, idempotency, the
activities/donors joins, and the splitter integration. make ci and
make riskcov both pass with the new module at 100% coverage.
@shivamlalakiya
shivamlalakiya merged commit 2ab3186 into main Sep 25, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant