Add map_columns for renaming CRM export headers - #231
Merged
Merged
Conversation
A no-code upload flow lets a user choose which of their own column headers means "donor ID" or "gift date" without ever showing the library the original names. map_columns applies that mapping and checks the result has every column the next ingest step needs, raising one error that lists every missing column instead of failing on the first one a caller happens to touch.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A no-code upload flow (column-mapping UI over an arbitrary CRM export) needs a way to rename whatever headers a user's file happens to have to the canonical names the ingest functions expect, and to fail loudly and completely if the mapping the user picked is missing something required.
philanthropy.ingest.map_columns(df, mapping, *, required=())renames columns per the mapping (unmapped columns pass through unchanged, mapping keys not present indf.columnsare ignored) and raises oneValueErrorlisting every still-missing required column, rather than surfacing aKeyErrorfor the first missing field a caller happens to touch.Test plan
python -c "from philanthropy.ingest import map_columns; print('OK')"tests/test_map_columns.py: rename + pass-through, unknown mapping keys ignored, required columns present passes, missing required columns raises one error naming all of them, no-required-columns never raises, input frame not mutatedmake ci(flake8, mypy, doctest, full suite, 92% coverage floor) - all greenmake riskcov(risk-tier floor overpreprocessing/,models/,ingest/,cli.py) - green