[Animal] Streamline Animal Culling Inputs - #3241
allisterakun wants to merge 12 commits into
Conversation
|
Current Coverage: 99% Mypy errors on streamline_animal_culling_inputs branch: 1133 |
|
🚨 Please update the changelog. This PR cannot be merged until |
|
Current Coverage: 99% Mypy errors on streamline_animal_culling_inputs branch: 1164 |
|
🚨 Please update the changelog. This PR cannot be merged until |
| if self.future_death_date == sys.maxsize: | ||
| if self.will_die_tomorrow(): | ||
| self.future_death_date = self.days_born | ||
| self._future_death_reason = animal_constants.DEATH_CULL | ||
|
|
||
| if self.future_cull_date == sys.maxsize: | ||
| if self.will_be_sold_tomorrow(): | ||
| self.future_cull_date = self.days_born | ||
| self.cull_reason = animal_constants.ACUTE_SALE_CULL |
There was a problem hiding this comment.
@gmg228 @jadamchick Please confirm this logic that if an animal is determined to be sold/dead, the we will mark her to be dead on the same day (at the EOD)
There was a problem hiding this comment.
Yes, thank you.
A quick clarification because I can't quite tell myself - once a cow is selected for removal/death, she is not still automatically evaluate for acute sale the same day, right?
Second, could we add in the change in risk based on 'day' (essentially stage of lactation being 2 categories, fresh or not). Below is the calculation that Julie put together!
Update daily risk according to DIM category (DIM=1-49 vs. DIM>=50)
Two equations with two unknowns:
Average daily removal probability = fresh_daily_removal_prob x % fresh + other_daily_removal_prob x % other
Use an example calving interval of 400d
So % fresh = 50/400 = 0.125
% other = 1 - 0.125 = 0.875
0.0001 = fresh_daily_removal_prob x 0.125 + other_daily_removal_prob x 0.875
Fresh_daily_removal_prob = 2.55 x other_daily_removal_prob
Solve for the unknowns: rearrange + substitute
… [rearrange and substitute] …
Other_daily_removal_prob = avg_daily_removal_prob / (2.55 x % fresh + % other)
Example:
Other_daily_removal_prob = 0.00011 / (2.55*0.125 + 0.875)
= 9.215 x 10^-5
Fresh_daily_removal_prob = 2.55 x other_daily_removal_prob
= 2.55 x 9.215 x 10^-5
= 0.00023
| """Return the 0-based index into a by-parity array, capping parity 4+ at the last entry.""" | ||
| return 3 if self.calves >= 4 else self.calves - 1 | ||
|
|
||
| def will_die_tomorrow(self) -> bool: |
There was a problem hiding this comment.
Help me choose a better function name plzzzz
There was a problem hiding this comment.
we can kick around some ideas with @jadamchick
will_be_removed_due_to_death
is_selected_for_mortality (or death)
will_exit_due_to_mortality (death)
will_have_mortality_event
will_have_death_event
| DISEASE_CULL = "culled for disease" | ||
| UDDER_CULL = "culled for udder" | ||
| UNKNOWN_CULL = "culled for unknown" | ||
| ACUTE_SALE_CULL = "culled for acute sale" |
There was a problem hiding this comment.
What should we put for the acute sale reason?
There was a problem hiding this comment.
Nope! We don't need that this time. Thanks for confirming!
|
Current Coverage: 99% Mypy errors on streamline_animal_culling_inputs branch: 1164 |
|
🚨 Please update the changelog. This PR cannot be merged until |
|
Current Coverage: 99% Mypy errors on streamline_animal_culling_inputs branch: 1164 |
|
🚨 Unauthorized changes detected in protected files. Please remove these changes if they are not intended. |
gmg228
left a comment
gmg228
left a comment
There was a problem hiding this comment.
I think we are really close!
I added a comment yesterday about adjusting to include risk of death/sale in early lactation vs. later (splitting at 50 days in lactation). I left some responses to your questions too that I hope help!
Thank you for getting this going so quickly.
Please let me know if we need to discuss anything further!
| if self.future_death_date == sys.maxsize: | ||
| if self.will_die_tomorrow(): | ||
| self.future_death_date = self.days_born | ||
| self._future_death_reason = animal_constants.DEATH_CULL | ||
|
|
||
| if self.future_cull_date == sys.maxsize: | ||
| if self.will_be_sold_tomorrow(): | ||
| self.future_cull_date = self.days_born | ||
| self.cull_reason = animal_constants.ACUTE_SALE_CULL |
There was a problem hiding this comment.
Yes, thank you.
A quick clarification because I can't quite tell myself - once a cow is selected for removal/death, she is not still automatically evaluate for acute sale the same day, right?
Second, could we add in the change in risk based on 'day' (essentially stage of lactation being 2 categories, fresh or not). Below is the calculation that Julie put together!
Update daily risk according to DIM category (DIM=1-49 vs. DIM>=50)
Two equations with two unknowns:
Average daily removal probability = fresh_daily_removal_prob x % fresh + other_daily_removal_prob x % other
Use an example calving interval of 400d
So % fresh = 50/400 = 0.125
% other = 1 - 0.125 = 0.875
0.0001 = fresh_daily_removal_prob x 0.125 + other_daily_removal_prob x 0.875
Fresh_daily_removal_prob = 2.55 x other_daily_removal_prob
Solve for the unknowns: rearrange + substitute
… [rearrange and substitute] …
Other_daily_removal_prob = avg_daily_removal_prob / (2.55 x % fresh + % other)
Example:
Other_daily_removal_prob = 0.00011 / (2.55*0.125 + 0.875)
= 9.215 x 10^-5
Fresh_daily_removal_prob = 2.55 x other_daily_removal_prob
= 2.55 x 9.215 x 10^-5
= 0.00023
| """Return the 0-based index into a by-parity array, capping parity 4+ at the last entry.""" | ||
| return 3 if self.calves >= 4 else self.calves - 1 | ||
|
|
||
| def will_die_tomorrow(self) -> bool: |
There was a problem hiding this comment.
we can kick around some ideas with @jadamchick
will_be_removed_due_to_death
is_selected_for_mortality (or death)
will_exit_due_to_mortality (death)
will_have_mortality_event
will_have_death_event
| DISEASE_CULL = "culled for disease" | ||
| UDDER_CULL = "culled for udder" | ||
| UNKNOWN_CULL = "culled for unknown" | ||
| ACUTE_SALE_CULL = "culled for acute sale" |
There was a problem hiding this comment.
Nope! We don't need that this time. Thanks for confirming!
2 participants
Streamlines the animal culling and death input system by replacing the complex multi-stage culling pipeline (reason selection → day-in-lactation interpolation → future-date scheduling) with a simple daily probability roll for mortality and acute sale, evaluated every day rather than once per lactation.
Context
Issue(s) closed by this pull request: closes #2694.
What
determine_future_death_date()withwill_die_tomorrow()— converts the parity-indexed annualparity_death_probabilityto a daily rate (÷365) and rolls once per day, returning a boolean.determine_future_cull_date()withwill_be_sold_tomorrow()— converts the parity-indexed annualparity_acute_sale_probabilityto a daily rate (÷365) and rolls once per day, returning a boolean. Renamed from "cull" to "acute sale" for clarity._assess_removal_risk()method onAnimal— called daily for every cow (and once when a heifer first calves), independently rolls mortality and acute sale risk. Only schedules a removal if no pending event of that type already exists, preventing later rolls from overwriting earlier scheduled removals. Removal is scheduled for the current day (self.days_born)._parity_index()helper — returns a 0-based index into by-parity arrays, capping parity 4+ at index 3._assess_removal_risk()call from insidedaily_reproduction_update(where it ran only at calving) todaily_routines(where it runs every day for all cows).AnimalConfig:death_day_probability,parity_cull_probability,cull_day_count,feet_leg_cull_probability,feet_leg_cull_day_probability,injury_cull_probability,injury_cull_day_probability,mastitis_cull_probability,mastitis_cull_day_probability,disease_cull_probability,disease_cull_day_probability,udder_cull_probability,udder_cull_day_probability,unknown_cull_probability,unknown_cull_day_probability. These entire reason-specific CDF and interpolation datasets are no longer needed.parity_cull_probabilitytoparity_acute_sale_probabilityinAnimalConfig, updated its docstring.LAMENESS_CULL,INJURY_CULL,MASTITIS_CULL,DISEASE_CULL,UDDER_CULL,UNKNOWN_CULLconstants fromanimal_constants.py. AddedACUTE_SALE_CULL.cull_reasonfromHerdStatisticsand its reporting inAnimalModuleReporter.default.jsonproperties file to remove all removed culling parameters and rename the remaining ones.example_freestall_animal.jsonandexample_open_lot_animal.jsonto use the new streamlined parameters.animal_cross_validation.jsonwith cross-validation rules for the new culling inputs.test_animal.py,test_animal_config.py,test_animal_module_reporter.py,test_herd_statistics.py,pytest_fixtures.py, andtest_herd_manager_herd_statistics.py.Why
The previous culling system was overly complex: it evaluated removal risk only once per lactation (at calving), selected a specific cull reason via nested probability cascades, then interpolated a future removal date from reason-specific cumulative day-probability distributions. This required ~20 input parameters (reason probabilities, per-reason CDFs, day count arrays) that were difficult for users to configure. The new approach evaluates mortality and acute sale risk daily using just two parity-indexed annual probability arrays, which are converted to daily rates. This is simpler to understand, configure, and maintain, eliminates ~800 lines of code, and distributes removal risk over time rather than concentrating it at calving events.
How
Each day in
daily_routines,_assess_removal_risk()is called for every cow. It checks whether a death or acute sale event is already pending (if so, that risk type is skipped). For each non-pending risk, it callswill_die_tomorrow()orwill_be_sold_tomorrow(), which divides the annual parity-indexed probability by 365 to get a daily rate and compares a random draw against it. If the draw succeeds, the removal is scheduled for the current day._parity_index()maps the cow's parity to a 0-based index (capping parity ≥4 at index 3). The specific cull reason selection and day-in-lactation interpolation are entirely removed.Test plan
will_die_tomorrow,will_be_sold_tomorrow,_assess_removal_risk, and_parity_index.test_animal_config.pyto reflect the removed parameters.test_animal_module_reporter.pyandtest_herd_statistics.pyto removecull_reasonreferences.pytest_fixtures.pyandtest_herd_manager_herd_statistics.py.Input Changes
death_day_probability,parity_cull_probability,cull_day_count,feet_leg_cull_probability,feet_leg_cull_day_probability,injury_cull_probability,injury_cull_day_probability,mastitis_cull_probability,mastitis_cull_day_probability,disease_cull_probability,disease_cull_day_probability,udder_cull_probability,udder_cull_day_probability,unknown_cull_probability,unknown_cull_day_probabilityfromdefault.jsonand animal input JSON files.parity_acute_sale_probability(renamed fromparity_cull_probability) indefault.jsonand animal input JSON files.input/metadata/cross_validation/animal_cross_validation.json.Output Changes
HerdStatistics.cull_reason: Removed.AnimalModuleReporter.report_herd_statistics_data.cull_reason: Removed.Filter