Repository navigation
fix(initializer): preserve model default ignore patterns - #3595
immanuwell wants to merge 1 commit into
Conversation
|
🎉 Welcome to the Kubeflow Trainer! 🎉 Thanks for opening your first PR! We're happy to have you as part of our community 🚀 Here's what happens next:
Join the community:
Feel free to ask questions in the comments if you need any help or clarification! |
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Updates environment-driven config loading so ignore_patterns defaults aren’t unintentionally overwritten when the env var isn’t set, and adjusts/extends tests accordingly.
Changes:
- Update
get_config_from_envto preserve dataclass defaults forignore_patternswhen the env var is absent. - Relax expectations in existing utils tests that previously asserted
ignore_patterns=None. - Add an S3 test to ensure default
ignore_patternsremain intact afterload_config().
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| pkg/initializers/utils/utils.py | Adjusts ignore_patterns env parsing to avoid overriding defaults when unset. |
| pkg/initializers/utils/utils_test.py | Updates test expectations to align with preserved defaults behavior. |
| pkg/initializers/model/s3_test.py | Adds regression test ensuring S3 default ignore_patterns are preserved. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
4495bd3 to
857b59d
Compare
|
@akshaychitneni @jinchihe please take a look |
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
|
This pull request has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it. |
|
Sorry for the delay @immanuwell, could you rebase this PR, so we can merge this bug fix in 2.4 release? |
|
@andreyvelich: Reopened this PR. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
Signed-off-by: immanuwell <pchpr.00@list.ru>
857b59d to
b2e4937
Compare
|
@andreyvelich done, rebased onto current master, python unit and integration tests pass locally |
|
/lgtm |
| def test_get_config_from_env_preserves_default_value(mock_env_vars): | ||
| mock_env_vars(IGNORE_PATTERNS=None) | ||
|
|
||
| result = utils.get_config_from_env(IgnorePatternsWithDefault) | ||
|
|
||
| assert result == {} | ||
| assert IgnorePatternsWithDefault(**result).ignore_patterns == ("*.ckpt",) | ||
|
|
||
|
|
||
| def test_get_config_from_env_empty_ignore_patterns_clears_default(mock_env_vars): |
There was a problem hiding this comment.
Could you add more test cases to the test_get_config_from_env as described here: https://github.com/immanuwell/trainer/blob/b2e49370d02aa93622a227251e236c23bb5a554e/AGENTS.md#python-testing-patterns
| def test_get_config_from_env_preserves_default_value(mock_env_vars): | |
| mock_env_vars(IGNORE_PATTERNS=None) | |
| result = utils.get_config_from_env(IgnorePatternsWithDefault) | |
| assert result == {} | |
| assert IgnorePatternsWithDefault(**result).ignore_patterns == ("*.ckpt",) | |
| def test_get_config_from_env_empty_ignore_patterns_clears_default(mock_env_vars): |
| from pkg.initializers.model.s3 import S3 | ||
|
|
||
|
|
||
| def test_load_config_preserves_default_ignore_patterns(mock_env_vars): |
There was a problem hiding this comment.
same suggestion
| def test_load_config_preserves_default_ignore_patterns(mock_env_vars): |
Small fix, but real.
Bug:
get_config_from_env()setsignore_patterns=NonewhenIGNORE_PATTERNSis unset.That wipes the dataclass defaults for model initializers.
Why this matters:
S3().load_config()then leavesignore_patternsempty, so S3 model downloads can pull.binand.ptfiles too. Kinda sneaky, and pretty easy to hit with mixed checkpoint dirs.Fix:
Keep the default-factory value for
ignore_patternswhen the env var is missing.Added a regression test on the real
S3().load_config()path.Repro before this patch:
Before:
NoneAfter:
['*.msgpack', '*.h5', '*.bin', '*.pt', '*.pth']Checks:
PYTHONPATH=$(pwd) pytest -q pkg/initializers/utils/utils_test.py pkg/initializers/model/s3_test.py pkg/initializers/model/huggingface_test.py