fix(tests): put pipelines/utils on sys.path so utils resolves - #258
Open
abhishekKokadwar wants to merge 1 commit into
Open
abhishekKokadwar wants to merge 1 commit into
abhishekKokadwar wants to merge 1 commit into
Conversation
Three test modules abort at collection on main:
ImportError: cannot import name 'clean_content' from 'utils'
(unknown location)
`pytest` exits 2, so the whole suite stops before running and both
workflows that invoke it are red.
kubeflow#249 moved `utils.py` into `pipelines/utils/`. These modules put only
`pipelines/` on `sys.path`, where the sibling directory of the same name
now shadows the module as a namespace package — `utils.__file__` is None
and `utils.__path__` is the directory, so no attribute resolves.
The pipelines themselves already handle this with a `_UTILS_DIR` entry
pointing at `pipelines/utils`; this applies the same thing to the tests
that import `utils` directly.
Recovers 72 tests across the four affected modules. The 7 failures and 63
errors that remain are pre-existing on dedbc84 and are not touched here:
`test_mcp_server.py` references `server.client`, removed when kubeflow#249 split
the MCP server, and `test_docs_pipeline.py` exercises the `store_milvus`
component that kubeflow#249 replaced with `utils/milvus_store.py`.
Signed-off-by: Abhishek <abhikokadwar2@gmail.com>
google-oss-prow
Bot
requested review from
franciscojavierarceo and
tarekabouzeid
September 30, 2026 05:24
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
This was referenced Sep 30, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Since #249,
pytestonmainaborts during collection before running a single test:Exit code 2, reproduced on a clean checkout of
dedbc84. Because collection is interrupted, no test in the suite runs, including the ones unrelated to these three files.Root cause
#249 moved
utils.pyintopipelines/utils/utils.py. These three modules put onlypipelines/onsys.path, where the newutils/directory (no__init__.py) shadows the module as a namespace package:So
from utils import clean_contentfinds a directory, not the module.The pipelines already handle this with
_UTILS_DIR = Path(__file__).resolve().parent / "utils"onsys.path. This PR applies the same idea to the three test modules: one extrasys.path.insert, pointing atpipelines/utils.What this does and does not fix
Being explicit, because CI will still be red after this merges:
dedbc84The three modules fixed here go from 0 to 69 passing. The remaining 7 failures and 63 errors are pre-existing on
dedbc84(verified by running those files directly on a clean checkout, which gives the same counts) and are outside this change:test_mcp_server.py: 63 errors,AttributeError: module 'docs_agent_mcp_server' has no attribute 'client'.clientwas removed when feat(rag): hybrid docs retrieval and readable ingest path #249 split the server into router/search/citations modules.test_docs_pipeline.py: 4 failures. It exercises thestore_milvuscomponent that feat(rag): hybrid docs retrieval and readable ingest path #249 replaced withutils/milvus_store.py.test_widget_markdown.py: 3 failures, from the widget changes in the same PR.Those need someone who knows the intended new interfaces, so I've left them alone. The value of this PR is that the suite runs again and those failures become visible, instead of being hidden behind a collection abort.
Do the recovered tests still check intended behaviour?
A path-only fix should not just make old tests pass. They should still describe what production does, so I checked that.
No assertion is changed. The diff is three
sys.pathlines plus comments. No expected value, test or skip marker is touched.The mirrors match what production runs. Per the
issues-pipeline.pyheader, KFP components are self-contained andutils/*holds mirrors of their logic for unit testing. The tests only mean something if those mirrors match the inline copies the components actually execute. I extracted the inline functions from the component source withastand ran both versions on the same inputs:resolve_github_token(inline in all 4 pipelines)issues_utils: metadata parsing,resolve_issue_metadata, prefix, content segmentscode_utils: YAML multi-doc, Python AST,chunk_code_file(YAML/Python/JSON/large text)The tests load the real modules and catch regressions. I deliberately broke each module and re-ran:
truncate_for_teistops truncatingGithub_Patfallback removed fromresolve_github_tokenState:kindlookup incode_utilsbrokenOne caveat: 19 of the 26 tests in
test_pipeline_utils.pycoverclean_content, which has no caller onmain(nor before #249). The v4 docs pipeline cleans content inhugo_ingest/canonical_rag_ingestinstead. Those 19 tests pass, but they don't exercise production code. I've left them in place, since removing tests is a maintainer call, but they shouldn't be read as coverage of the v4 path.Verification
requirements-test.txtplus the KFP SDK). Without KFP, as inPR Safety, the KFP-dependent component tests skip, so absolute counts are lower there. The collection abort happens either way.sys.path.pyflakesclean on all three files.pycodestylereports onlyE402, which already appears onmainand is allow-listed for exactly these three files inpyproject.toml. I couldn't runruffitself: a Windows Application Control policy blocks the binary on my machine, so CI is the authoritative lint check.