Repository navigation
Conversation
Add WindowSpec, DocumentWindowFrame, RangeWindowFrame, and WindowBound types in pipeline_types, the AddWindowFields stage and add_window_fields() pipeline builder method, and the Rank() window function and Expression.over() frame override method. Include unit tests and pipeline_e2e acceptance tests in window.yaml.
There was a problem hiding this comment.
Code Review
This pull request introduces window functions and window framing capabilities to the Google Cloud Firestore Python client library, enabling advanced analytical operations such as running totals and rankings. It adds support for the add_window_fields pipeline stage, along with new classes like DocumentWindowFrame, RangeWindowFrame, WindowSpec, and Rank. Feedback on the changes highlights a potential bug in WindowSpec initialization where passing a string to the sort parameter would silently convert it into a list of characters, leading to serialization errors. A validation check has been suggested to prevent this issue.
| if sort is None: | ||
| self.sort: list[Ordering] = [] | ||
| else: | ||
| self.sort = [sort] if isinstance(sort, Ordering) else list(sort) |
There was a problem hiding this comment.
If sort is a string (e.g., "rating"), isinstance(sort, Ordering) is False, so list(sort) is called. Since strings are iterable in Python, this silently converts "rating" into a list of its characters: ['r', 'a', 't', 'i', 'n', 'g']. This will later cause an AttributeError during serialization because str objects do not have a _to_pb() method.
To prevent this silent bug and provide a clear error message, we should validate that all elements in the sort sequence are instances of Ordering.
| if sort is None: | |
| self.sort: list[Ordering] = [] | |
| else: | |
| self.sort = [sort] if isinstance(sort, Ordering) else list(sort) | |
| if sort is None: | |
| self.sort: list[Ordering] = [] | |
| else: | |
| sort_list = [sort] if isinstance(sort, Ordering) else list(sort) | |
| for s in sort_list: | |
| if not isinstance(s, Ordering): | |
| raise TypeError(f"Expected Ordering, got {type(s).__name__}") | |
| self.sort = sort_list |
Context & Problem
Firestore Pipelines supports evaluating window functions and aggregations over partitioned, sorted, and framed subsets of documents without collapsing rows (
add_window_fields). This PR adds support for pipeline window functions to the Python Firestore SDK.Summary of Changes
WindowBound,DocumentWindowFrame,RangeWindowFrame, andWindowSpecingoogle/cloud/firestore_v1/pipeline_types.pyand re-export them fromgoogle.cloud.firestoreandgoogle.cloud.firestore_v1(including.librarian/generator-input/client-post-processing/firestore-integration.yaml).AddWindowFieldsstage ingoogle/cloud/firestore_v1/pipeline_stages.pyandadd_window_fields()builder method on_BasePipelineingoogle/cloud/firestore_v1/base_pipeline.py.WindowFunction,Rank(), andExpression.over()ingoogle/cloud/firestore_v1/pipeline_expressions.py.test_pipeline_stages.py,test_pipeline_expressions.py,test_pipeline.py, andtest_async_pipeline.py, along with end-to-end acceptance tests intests/system/pipeline_e2e/window.yaml.Verification
pytest tests/unit/v1/test_pipeline_stages.py tests/unit/v1/test_pipeline_expressions.py tests/unit/v1/test_pipeline.py tests/unit/v1/test_async_pipeline.py) with 100% line and branch coverage on modified lines.tests/system/test_pipeline_acceptance.py -k window.yaml(sync and async) against the Firestore nightly sandbox backend (test-firestore.sandbox.googleapis.com:443).