[m-misiura]

Description

This PR fixes local HF text-classification failures on oversized input using automatic overlapping token windows.It reuses HF tokenization and scoring; any triggering window blocks through existing rail outcomes. It also preserves configured truncation and native token-classification behavior and documents stride for full-input token processing.

Related Issue(s)

Please see the following GH Issue for more information: GH#2327

Verification

In addition to the unit test, a server was also started with the following rail config; this model has 512 input length and is a text-classifier

rails:
  config:
    hf_classifier:
      hap:
        engine: local
        model: ibm-granite/granite-guardian-hap-125m
        task: text-classification
        threshold: 0.5
        blocked_labels:
          - LABEL_1

The following requests were sent

• should pass, harmless long test

curl -sS http://127.0.0.1:8097/v1/checks \
  -H 'Content-Type: application/json' \
  --data-binary @/tmp/gh2327-live/safe.json
{"status":"passed","content":"The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. "}

• harmful long text

curl -sS http://127.0.0.1:8097/v1/checks \
  -H 'Content-Type: application/json' \
  --data-binary @/tmp/gh2327-live/toxic-tail.json |
  jq 'del(.content)'
{
  "status": "blocked",
  "rail": "hf classifier check input",
  "reason": "hf classifier check input"
}

where toxic-tail.json is (note foul language; this is only to trigger the HAP filter)

{"messages": [{"role": "user", "content": "The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. The community garden grows fresh vegetables for local families. You are a disgusting dotard. I hate you. You are a disgusting dotard. I hate you. You are a disgusting dotard. I hate you. You are a disgusting dotard. I hate you. You are a disgusting dotard. I hate you. You are a disgusting dotard. I hate you.  "}], "guardrails": {"config_id": "hap", "rail_types": ["input"]}, "model": ""}

AI Assistance

• No AI tools were used.
• AI tools were used; a human reviewed and can explain every change (tool: Codex).

Checklist

• I've read the CONTRIBUTING guidelines.
• This PR links to a triaged issue assigned to me.
• My PR title follows the project commit convention.
• I've updated the documentation if applicable.
• I've added tests if applicable.
• I've noted any verification beyond CI and any checks I couldn't run.
• I did not update generated changelog files manually.
• I addressed all CodeRabbit, Greptile, and other review comments, or replied with why no change is needed.
• @mentions of the person or team responsible for reviewing proposed changes.

Summary by CodeRabbit

• New Features
  • Local text classification can now process inputs that exceed the model’s inferred capacity by splitting them into overlapping windows when truncation is not enabled. A blocked result in any window triggers the rail; scores are not averaged.
• Documentation
  • Clarified truncation behavior, window and tokenizer requirements, input limits, batching, and fallback behavior.
  • Documented that token-classification behavior and remote-backend input-length handling remain unchanged, and clarified the outcomes returned by input, output, and retrieval checks.

[m-misiura]

cc @Pouyanpi @tgasser-nv

[greptile-apps]

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge, with no new actionable findings.

Summary

This PR adds overlapping token windows for oversized local text-classification inputs.

• Local text classifiers check oversized inputs across overlapping windows.
• Local token classifiers check tokenizer versions when stride is set.

_{Reviews (5) · Last reviewed commit: "fix(hf-classifier): account for padding ..." · Reviewed by Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 7.91367% with 128 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
nemoguardrails/library/hf_classifier/chunking.py	0.00%	128 Missing ⚠️

📢 Thoughts on this report? Let us know!

[coderabbitai]

Walkthrough

Local Hugging Face text-classification pipelines now process eligible oversized inputs in overlapping windows. The changes also document windowing limits, token-classification behavior, and the rail outcomes returned for input, output, and retrieval checks.

Changes: Local classifier windowing

Layer / File(s)	Summary
Window limits and pipeline setup nemoguardrails/library/hf_classifier/chunking.py, nemoguardrails/library/hf_classifier/backends.py, docs/configure-rails/guardrail-catalog/community/hf-classifier.mdx, tests/test_hf_classifier.py	The local text-classification pipeline selects a usable input limit and creates overlapping windows for eligible oversized inputs. The backend uses this pipeline by default for text classification. Documentation and tests cover limits, tokenizer requirements, truncation, configuration, and pipeline integration.
Window inference and rail results nemoguardrails/library/hf_classifier/chunking.py, docs/configure-rails/guardrail-catalog/community/hf-classifier.mdx, tests/test_hf_classifier.py	The pipeline serializes calls, supports pickling, and processes window results. Tests cover rail detections, inference order, concurrent calls, and serialization. Documentation describes the returned rail outcomes.

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Bug fix

Sequence Diagram(s)

sequenceDiagram
  participant Guardrails
  participant ChunkedTextClassificationPipeline
  participant HFTokenizer
  participant ClassifierModel
  Guardrails->>ChunkedTextClassificationPipeline: classify local text
  ChunkedTextClassificationPipeline->>HFTokenizer: tokenize into overflow windows
  HFTokenizer-->>ChunkedTextClassificationPipeline: return token windows
  ChunkedTextClassificationPipeline->>ClassifierModel: score each window
  ClassifierModel-->>ChunkedTextClassificationPipeline: return window scores
  ChunkedTextClassificationPipeline-->>Guardrails: return classification results

Loading

Suggested reviewers: pouyanpi, tgasser-nv

Merge Risk: Low · up to c8d93

Very large inputs can delay other requests using the same local classifier, and an invalid window limit reports the wrong cause. These bounded risks warrant fixes or explicit acceptance before merge.

Pre-merge checks: 4 · 1 · 1

Failed checks: 1 warning · 1 inconclusive

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 4.55% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…	Write docstrings for the functions missing them to satisfy the coverage threshold.
Test Results For Major Changes		The PR is a major change. The visible description documents unit-test and server verification, but the new windowed pipeline can affect performance: it may run multiple inference passes, serialize cal…	Provide the complete PR description or add before-and-after performance numbers with the model, input size, hardware, batching, and other test configuration.

Passed checks: 4 passed

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Title check		The title clearly describes token-window processing for oversized local hf-classifier inputs.

Full details: Docstring Coverage

Explanation

Docstring coverage is 4.55% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 22 functions across 3 files. (1 skipped: 1 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Full details: Test Results For Major Changes

Explanation

The PR is a major change. The visible description documents unit-test and server verification, but the new windowed pipeline can affect performance: it may run multiple inference passes, serialize calls, and change token storage. The check therefore requires before-and-after performance numbers. The supplied description is truncated, and no full PR body or review metadata is available in the checkout, so the presence or absence of those numbers cannot be verified.

Resolution

Provide the complete PR description or add before-and-after performance numbers with the model, input size, hardware, batching, and other test configuration.

Finishing Touches

Generate unit tests (beta)

• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Recent review info

Run configuration

• Source mode: Auto
• Repository: NVIDIA-NeMo/Guardrails
• Branch: develop
• File: .coderabbit.yaml
• Commit: 8d027fd
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Enterprise
• Run ID: f26c31c5-8301-4405-bbd4-95a6c4ed6b77

Commits

Reviewing files that changed from the base of the PR and between 8d027fd and c8d93df.

Files selected for processing: 4

• docs/configure-rails/guardrail-catalog/community/hf-classifier.mdx
• nemoguardrails/library/hf_classifier/backends.py
• nemoguardrails/library/hf_classifier/chunking.py
• tests/test_hf_classifier.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

[coderabbitai]

• Fix comments: Fix CodeRabbit comments on this PR

[m-misiura]

PS I do not think the Codecov report is correct since the dev dependency group has no torch or transformers

[Pouyanpi]

Thanks @m-misiura, looks good overall 👍🏻 Please have a look at following comments:

[Pouyanpi]

Thanks @m-misiura for the fix, let's merge. This is an important piece that we missed, we should get back to #2327 for broader coverage.