Skip to content

feat(python): support message partitioning strategies - #3927

Open
jiengup wants to merge 2 commits into
apache:masterfrom
jiengup:python-partition-strategies
Open

feat(python): support message partitioning strategies#3927
jiengup wants to merge 2 commits into
apache:masterfrom
jiengup:python-partition-strategies

Conversation

@jiengup

@jiengup jiengup commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR address?

Closes #3896

Rationale

The Python SDK only supported sending messages to an explicit partition, while the Rust SDK also supports balanced and message-key partitioning.

What changed?

IggyClient.send_messages() now accepts a Python Partitioning object supporting fixed, balanced, and message-key routing. Existing integer partition IDs remain fully compatible, while string message keys are encoded as UTF-8 and validated against the 1–255 byte limit.

Type stubs, tests, and a runnable Python example covering all three strategies are included.

Local Execution

  • Passed all 87 tests in foreign/python/tests/test_message_operations.py
  • Passed Rust formatting and Clippy checks
  • Passed Ruff and Pyrefly checks for the Python SDK and examples
  • Successfully ran the partitioning example against a local Iggy server
  • Pre-commit hooks ran and passed

AI Usage

  1. Tool: OpenAI Codex
  2. Scope: Repository analysis, PyO3 implementation, tests, type stubs, documentation, and the Python example
  3. Verification: Reviewed the generated changes, ran the complete message operations test suite, static checks, pre-commit hooks, and the example against a local server
  4. Yes, I can explain every line of the changes

@github-actions

Copy link
Copy Markdown

Thanks for the PR. It is labeled S-waiting-on-review and queued for review.

Slash commands (own line, regular comment) move it around the queue:

  • /ready - back to S-waiting-on-review after addressing feedback
  • /author - flip to S-waiting-on-author while you finish changes
  • /request-review @user-or-team - request a reviewer

See CONTRIBUTING.md for details.

@github-actions github-actions Bot added the S-waiting-on-review PR is waiting on a reviewer label Aug 20, 2026
@jiengup jiengup changed the title Python SDK: support message partitioning strategies feat(python): support message partitioning strategies Aug 20, 2026
@codecov

codecov Bot commented Aug 20, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.19%. Comparing base (ffb186d) to head (7b4ec8f).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff            @@
##             master    #3927   +/-   ##
=========================================
  Coverage     84.18%   84.19%           
  Complexity     1398     1398           
=========================================
  Files          1219     1220    +1     
  Lines        173564   173590   +26     
  Branches     140662   140662           
=========================================
+ Hits         146122   146148   +26     
  Misses        23472    23472           
  Partials       3970     3970           
Components Coverage Δ
Rust Core 84.99% <ø> (ø)
Java SDK 67.23% <ø> (ø)
C# SDK 75.39% <ø> (ø)
Python SDK 90.18% <100.00%> (+0.11%) ⬆️
PHP SDK 85.65% <ø> (ø)
Node SDK 96.07% <ø> (ø)
Go SDK 69.09% <ø> (ø)
Files with missing lines Coverage Δ
foreign/python/src/client.rs 99.85% <100.00%> (+<0.01%) ⬆️
foreign/python/src/lib.rs 100.00% <100.00%> (ø)
foreign/python/src/partitioning.rs 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ethanlin01x ethanlin01x left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some suggestions. The stub one I think should be fixed before merge.

Comment thread foreign/python/apache_iggy.pyi
Comment thread foreign/python/src/partitioning.rs Outdated
Comment thread foreign/python/tests/test_message_operations.py Outdated
Comment thread examples/python/partitioning/producer.py Outdated
Comment thread examples/python/README.md
Comment thread foreign/python/src/client.rs Outdated
Comment thread foreign/python/src/partitioning.rs Outdated
Comment thread foreign/python/src/partitioning.rs
Comment thread foreign/python/src/client.rs Outdated
Comment thread foreign/python/tests/test_message_operations.py
Comment thread foreign/python/tests/test_message_operations.py
Comment thread foreign/python/tests/test_message_operations.py
Comment thread foreign/python/tests/test_message_operations.py
@github-actions github-actions Bot added S-waiting-on-author PR is waiting on author response and removed S-waiting-on-review PR is waiting on a reviewer labels Aug 23, 2026
@jiengup
jiengup force-pushed the python-partition-strategies branch from f715f3c to 7b4ec8f Compare August 27, 2026 12:38
@jiengup

jiengup commented Aug 27, 2026

Copy link
Copy Markdown
Contributor Author

apache_iggy.pyi is re-generated through stub_gen binary and formatted by ruff, some changes are unrelated to this PR.

@jiengup

jiengup commented Aug 27, 2026

Copy link
Copy Markdown
Contributor Author

/ready

@jiengup
jiengup requested a review from slbotbm August 27, 2026 16:32
@github-actions github-actions Bot added S-waiting-on-review PR is waiting on a reviewer and removed S-waiting-on-author PR is waiting on author response labels Aug 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

S-waiting-on-review PR is waiting on a reviewer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python SDK: expose partitioning strategies for send_messages

3 participants