Skip to content

WIP - feat(firestore): add pipeline window functions support - #18961

Draft
ohmayr wants to merge 1 commit into
mainfrom
feat/firestore-pipeline-window-functions
Draft

ohmayr wants to merge 1 commit into
mainfrom
feat/firestore-pipeline-window-functions

Conversation

@ohmayr

@ohmayr ohmayr commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Context & Problem

Firestore Pipelines supports evaluating window functions and aggregations over partitioned, sorted, and framed subsets of documents without collapsing rows (add_window_fields). This PR adds support for pipeline window functions to the Python Firestore SDK.

Summary of Changes

  • Add WindowBound, DocumentWindowFrame, RangeWindowFrame, and WindowSpec in google/cloud/firestore_v1/pipeline_types.py and re-export them from google.cloud.firestore and google.cloud.firestore_v1 (including .librarian/generator-input/client-post-processing/firestore-integration.yaml).
  • Add the AddWindowFields stage in google/cloud/firestore_v1/pipeline_stages.py and add_window_fields() builder method on _BasePipeline in google/cloud/firestore_v1/base_pipeline.py.
  • Add WindowFunction, Rank(), and Expression.over() in google/cloud/firestore_v1/pipeline_expressions.py.
  • Add unit tests in test_pipeline_stages.py, test_pipeline_expressions.py, test_pipeline.py, and test_async_pipeline.py, along with end-to-end acceptance tests in tests/system/pipeline_e2e/window.yaml.

Verification

  • Ran unit test suites (pytest tests/unit/v1/test_pipeline_stages.py tests/unit/v1/test_pipeline_expressions.py tests/unit/v1/test_pipeline.py tests/unit/v1/test_async_pipeline.py) with 100% line and branch coverage on modified lines.
  • Ran tests/system/test_pipeline_acceptance.py -k window.yaml (sync and async) against the Firestore nightly sandbox backend (test-firestore.sandbox.googleapis.com:443).

Add WindowSpec, DocumentWindowFrame, RangeWindowFrame, and WindowBound
types in pipeline_types, the AddWindowFields stage and
add_window_fields() pipeline builder method, and the Rank() window
function and Expression.over() frame override method. Include unit tests
and pipeline_e2e acceptance tests in window.yaml.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces window functions and window framing capabilities to the Google Cloud Firestore Python client library, enabling advanced analytical operations such as running totals and rankings. It adds support for the add_window_fields pipeline stage, along with new classes like DocumentWindowFrame, RangeWindowFrame, WindowSpec, and Rank. Feedback on the changes highlights a potential bug in WindowSpec initialization where passing a string to the sort parameter would silently convert it into a list of characters, leading to serialization errors. A validation check has been suggested to prevent this issue.

Comment on lines +496 to +499
if sort is None:
self.sort: list[Ordering] = []
else:
self.sort = [sort] if isinstance(sort, Ordering) else list(sort)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

If sort is a string (e.g., "rating"), isinstance(sort, Ordering) is False, so list(sort) is called. Since strings are iterable in Python, this silently converts "rating" into a list of its characters: ['r', 'a', 't', 'i', 'n', 'g']. This will later cause an AttributeError during serialization because str objects do not have a _to_pb() method.

To prevent this silent bug and provide a clear error message, we should validate that all elements in the sort sequence are instances of Ordering.

Suggested change
if sort is None:
self.sort: list[Ordering] = []
else:
self.sort = [sort] if isinstance(sort, Ordering) else list(sort)
if sort is None:
self.sort: list[Ordering] = []
else:
sort_list = [sort] if isinstance(sort, Ordering) else list(sort)
for s in sort_list:
if not isinstance(s, Ordering):
raise TypeError(f"Expected Ordering, got {type(s).__name__}")
self.sort = sort_list

@ohmayr ohmayr changed the title feat(firestore): add pipeline window functions support WIP - feat(firestore): add pipeline window functions support Nov 2, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant