Add a Salesforce NPSP Opportunity export reader - #233
Merged
Merged
Conversation
Mirrors the existing Raiser's Edge bridge: read_npsp_opportunities loads an Opportunity export CSV (or a folder of them) and npsp_opportunities_to_features rolls it up into the donor-level feature table used by train/score, delegating the actual aggregation to the shared CiviCRM roll-up. NPSP's Recurring Donations feature creates one Opportunity per instalment, starting it in the Pledged stage and only moving it to a closed/won stage (Closed Won, or a site's own Posted) once the money is actually received. Depending on an org's instalment settings, the Pledged row and the Closed Won row recording its receipt can both exist for the same amount and close date, so summing every row's Amount double-counts that instalment. Pledged rows are dropped by default (DEFAULT_EXCLUDED_STAGES), same as Raiser's Edge drops its Pledge and Recurring Gift rows. Header aliases accept a Salesforce report export's column labels (Account Name, Close Date, Amount, Stage), the raw Opportunity API field names (AccountId, CloseDate, StageName), and NPSP's own Data Import template headers (Donation Date, Donation Amount, Donation Stage). Wired into the CLI as `philanthropy features --source npsp`. Added tests/test_npsp.py at the same depth as tests/test_raisers_edge.py plus two CLI integration tests, a CHANGELOG entry, a paragraph in docs/how-to/use_the_cli.md, and a Tier 2 (Beta) row in docs/reference/index.md.
shivamlalakiya
enabled auto-merge (squash)
September 25, 2026 17:38
This was referenced Sep 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a Salesforce Nonprofit Success Pack (NPSP) Opportunity export reader alongside the existing CiviCRM and Raiser's Edge bridges, and wires it into the CLI as
philanthropy features --source npsp.philanthropy.ingest.read_npsp_opportunities(path): reads an Opportunity export CSV, or a directory of them, delegating to the shared CiviCRM reader.philanthropy.ingest.npsp_opportunities_to_features(opportunities, *, reference_date=None, exclude_stages=DEFAULT_EXCLUDED_STAGES): rolls the export up into the donor-level feature table, droppingPledgedrows by default before handing the rest tocivicrm_contributions_to_features.philanthropy.ingest.DEFAULT_EXCLUDED_STAGES = ("Pledged",).Why the
Pledgedfilter exists: NPSP's Recurring Donations feature creates one Opportunity per instalment. The upcoming instalment is created in thePledgedstage and only moves to a closed/won stage (Closed Won, or a site's ownPosted) once the gift is actually received. Depending on an org's "Installment Opportunity Auto-Creation" setting, thePledgedrow and theClosed Wonrow recording the same instalment's receipt can both exist with the same amount and close date, so summing every Opportunity'sAmountnaively counts that instalment twice - the same commitment-vs-payment trap the Raiser's Edge bridge already guards against, expressed through Opportunity stage instead of a separate gift record.Header aliases accept a Salesforce report export's column labels (
Account Name/Account ID,Close Date,Amount,Stage,Record Type), the raw Opportunity API field names (AccountId,CloseDate,StageName,RecordType.Name), and NPSP's own Data Import template headers (Donation Date,Donation Amount,Donation Stage,Donation Record Type Name), plusPrimary Contactfor the Contact-keyed donor case.NPSP vocabulary was checked against Salesforce's own public documentation rather than assumed:
I deliberately did not model the
npe01__OppPayment__cPayment child object - this bridge reads an Opportunity-level export, matching the PR's scope, and I didn't want to assert Payment field names I couldn't verify against a primary source.Files changed
philanthropy/ingest/_npsp.py(new)philanthropy/ingest/__init__.py(export the three new names)philanthropy/cli.py(--source npsp)tests/test_npsp.py(new, same depth astests/test_raisers_edge.py)tests/test_cli.py(two CLI integration tests for the new source)docs/how-to/use_the_cli.md,docs/reference/index.md(Tier 2/Beta row)CHANGELOG.mdTest plan
python -c "from philanthropy.ingest import DEFAULT_EXCLUDED_STAGES, npsp_opportunities_to_features, read_npsp_opportunities"before writing tests, per CONTRIBUTING.md's order.make ci: 2117 passed, 8 skipped; coverage 98.52% (floor 92%);philanthropy/ingest/_npsp.pyat 100%.make riskcov: risk-tier coverage 98% (floor 93%).