Skip to content

Add configurable grouping token limit - #874

Open
RamiNoodle733 wants to merge 3 commits into
andialbrecht:masterfrom
RamiNoodle733:agent/configure-grouping-token-limit
Open

Add configurable grouping token limit#874
RamiNoodle733 wants to merge 3 commits into
andialbrecht:masterfrom
RamiNoodle733:agent/configure-grouping-token-limit

Conversation

@RamiNoodle733

Copy link
Copy Markdown

Fixes #841

Summary

  • add sqlparse.engine.set_max_grouping_tokens(limit) so trusted offline workloads can raise or disable the grouping token cap without mutating an internal module constant directly
  • validate that configured limits are positive integers or None
  • add regression tests covering enforcement, disabling the cap, and invalid values

Safety

The default remains unchanged at 10,000 tokens. Passing None is explicitly documented as appropriate only for trusted input, so the DoS protection remains enabled unless a caller intentionally changes it.

Validation

  • setter validation smoke test passed locally
  • Python syntax compilation passed for the new configuration helper
  • full project pytest/flake8 could not be run in this automation runtime because it does not provide a repository checkout or network access for cloning dependencies

Copilot AI lite review requested due to automatic review settings August 11, 2026 10:54

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a public configuration hook to adjust sqlparse’s DoS-prevention token cap for grouping, addressing the need to raise/disable the limit for trusted offline workloads without directly mutating internal module constants.

Changes:

  • Introduces sqlparse.engine.set_max_grouping_tokens(limit) with validation (int >= 1 or None).
  • Exports the setter from sqlparse.engine for public consumption.
  • Adds tests for enforcement, disabling, and invalid input handling.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
sqlparse/engine/config.py Adds the public setter that updates grouping.MAX_GROUPING_TOKENS with input validation.
sqlparse/engine/__init__.py Re-exports set_max_grouping_tokens from the engine package namespace.
tests/test_grouping_config.py Adds regression tests covering enforcement, disabling, and invalid values.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread sqlparse/engine/config.py
# This module is part of python-sqlparse and is released under
# the BSD License: https://opensource.org/licenses/BSD-3-Clause

from sqlparse.engine import grouping
Comment on lines +25 to +30
def test_set_max_grouping_tokens_can_disable_limit(restore_grouping_token_limit):
set_max_grouping_tokens(None)

statements = sqlparse.parse("SELECT 1")

assert len(statements) == 1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make MAX_GROUPING_TOKENS can be set outside of grouping.py

2 participants