[Pouyanpi]

Superseded by the smaller draft #2517.

Description

Accept straight and curly message delimiters in streaming and non-streaming replies. Preserve inner quotes and multiline content, strip closing delimiters before trailing whitespace, and flush buffered replies when generation ends without a prefix match.

Related Issue(s)

Resolves NGR-1047

Summary by CodeRabbit

• Bug Fixes
  • Improved handling of generated messages containing straight or curly quotation marks, including multiline text and embedded quotes.
  • Streaming responses now stop more reliably at dialogue boundaries and quote endings, including when these boundaries span multiple chunks or have trailing whitespace.
  • Fixed cases where subsequent dialogue could appear in a generated message or buffered output could fail to flush at handoff.
  • Improved consistency across single-call and separate-call modes, as well as streaming and non-streaming responses.

[greptile-apps]

Confidence Score: 3/5

[Medium impact] The PR changes how generated replies are parsed and streamed across Guardrails entry points.

Fix the message boundaries before merging so replies neither lose valid text nor include extra bot turns.

Findings

1.  Inner quotes cut replies short ▶
2.  Extra bot turns appear ▶

Fix with agent prompt

### Issue 1
nemoguardrails/llm/completion_parsing.py:62-65
The new stops can cut off valid content at an inner curly quote. For example, `"The phrase is “helpful bot”\nbot software can assist."` now becomes `The phrase is “helpful bot`, losing the rest of the reply and its inner closing quote. The pre-scan ignores quote depth, and streaming uses the same stops. Check whether the quote closes the message before treating the following line as another turn, and add a test for this case.

### Issue 2
nemoguardrails/llm/completion_parsing.py:30-31
Replacing the single-call stop `"\n` with exact header matches misses dialogue separated by a blank line. If generation continues after handoff with `  "Hello!"\n\nbot unwanted\n  "Wrong reply!"`, no stop matches. Streaming emits the first closing quote and the unwanted turn, stripping only the final quote. Non-streaming still stops after `Hello!`. Allow whitespace between the closing delimiter and the next header, and test this with chunks arriving after handoff.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Adds curly quote support to generated replies and updates streaming completion handling.

• Bot messages keep their inner quotes and multiline text.
• Bot-message streams handle both quote styles and stop at dialogue boundaries.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    A[Generated text] --> B{Streaming?}
    B -->|No| C[Scan message boundaries]
    C --> D[Check lines and strip quotes]
    B -->|Yes| E[Buffer and remove prefix]
    E --> F[Match stops and strip suffix]
    D --> G[Bot reply]
    F --> G

Loading

_{Reviews (1) · Last reviewed commit: "chore: remove documentation additions fr..." · Reviewed by Greptile}

[greptile-apps]

Inner quotes cut replies short

The new stops can cut off valid content at an inner curly quote. For example, "The phrase is “helpful bot”\nbot software can assist." now becomes The phrase is “helpful bot, losing the rest of the reply and its inner closing quote. The pre-scan ignores quote depth, and streaming uses the same stops. Check whether the quote closes the message before treating the following line as another turn, and add a test for this case.

Prompt To Fix With AI

This is a comment left during a code review.
Path: nemoguardrails/llm/completion_parsing.py
Line: 62-65

Comment:
**Inner quotes cut replies short**

The new stops can cut off valid content at an inner curly quote. For example, `"The phrase is “helpful bot”\nbot software can assist."` now becomes `The phrase is “helpful bot`, losing the rest of the reply and its inner closing quote. The pre-scan ignores quote depth, and streaming uses the same stops. Check whether the quote closes the message before treating the following line as another turn, and add a test for this case.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[greptile-apps]

Extra bot turns appear

Replacing the single-call stop "\n with exact header matches misses dialogue separated by a blank line. If generation continues after handoff with "Hello!"\n\nbot unwanted\n "Wrong reply!", no stop matches. Streaming emits the first closing quote and the unwanted turn, stripping only the final quote. Non-streaming still stops after Hello!. Allow whitespace between the closing delimiter and the next header, and test this with chunks arriving after handoff.

Prompt To Fix With AI

This is a comment left during a code review.
Path: nemoguardrails/llm/completion_parsing.py
Line: 30-31

Comment:
**Extra bot turns appear**

Replacing the single-call stop `"\n` with exact header matches misses dialogue separated by a blank line. If generation continues after handoff with `  "Hello!"\n\nbot unwanted\n  "Wrong reply!"`, no stop matches. Streaming emits the first closing quote and the unwanted turn, stripping only the final quote. Non-streaming still stops after `Hello!`. Allow whitespace between the closing delimiter and the next header, and test this with chunks arriving after handoff.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[coderabbitai]

No actionable comments were generated in the recent review.

Recent review info

Run configuration

• Source mode: Auto
• Repository: NVIDIA-NeMo/Guardrails
• Branch: develop
• File: .coderabbit.yaml
• Commit: e6b1c2b
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Enterprise
• Run ID: 79834398-f4c4-4701-8a43-570e0ee66274

Commits

Reviewing files that changed from the base of the PR and between e6b1c2b and a7655f9.

Files selected for processing: 7

• nemoguardrails/actions/llm/generation.py
• nemoguardrails/llm/completion_parsing.py
• nemoguardrails/streaming.py
• tests/llm/test_completion_parsing.py
• tests/test_generation_equivalence.py
• tests/test_llmrails_singlecall.py
• tests/test_streaming_handler.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

Walkthrough

The change updates completion parsing for straight and curly quotes, extends streaming prefix, suffix, and stop handling, and configures generation paths to use these behaviors. Added tests cover parsing boundaries, streaming behavior, and single-call handoff.

Changes: Quoted message streaming

Layer / File(s)	Summary
Quote-aware completion parsing nemoguardrails/llm/completion_parsing.py, tests/llm/test_completion_parsing.py	Parsing now recognizes straight and curly quote delimiters and quote-newline dialogue boundaries. Tests cover quote stripping, multiline extraction, and following dialogue.
Streaming pattern and stop handling nemoguardrails/streaming.py, tests/test_streaming_handler.py	StreamingHandler now supports alternative prefixes and suffixes, pattern-specific stops, partial matches, and trailing whitespace after suffixes. Tests cover stop precedence, buffering, metadata, and quote variants.
Generation integration and handoff nemoguardrails/actions/llm/generation.py, tests/test_generation_equivalence.py, tests/test_llmrails_singlecall.py	Generation configures streaming patterns and stop sequences. Single-call output is parsed before a registered handoff marker is appended. Tests cover generation modes and streaming handoff.

Priority: Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Bug fix

Sequence Diagram(s)

sequenceDiagram
  participant LLM
  participant StreamingHandler
  participant generate_bot_message
  participant get_multiline_response
  LLM->>StreamingHandler: Emit completion chunks
  StreamingHandler->>generate_bot_message: Emit text through configured stop
  generate_bot_message->>get_multiline_response: Parse single-call output
  get_multiline_response->>generate_bot_message: Return extracted response
  generate_bot_message->>StreamingHandler: Append registered handoff marker

Loading

Possibly related PRs

• NVIDIA-NeMo/Guardrails#2168: Introduces single-call generation helpers and streaming handoff paths that this change updates.
• NVIDIA-NeMo/Guardrails#2188: Refactors the single-call generation and streaming handoff paths used by this change.

Suggested labels: bug

Suggested reviewers: christinaexyou, tgasser-nv

Merge Risk: Minimal · up to a7655

No actionable issue was established in the quoted-message parsing or streaming changes. The PR is mergeable after normal checks.

Pre-merge checks: 4 · 2

Failed checks: 2 warnings

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 38.24% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documenta…	Write docstrings for the functions missing them to satisfy the coverage threshold.
Test Results For Major Changes		The PR contains major changes: it modifies streaming stop handling and the public StreamingHandler.set_pattern signature, changes completion parsing, and adds 642 lines across seven files. The autho…	Update the PR description with the testing performed and results, including the relevant test commands and whether they passed. Include performance or regression evidence only if applicable.

Passed checks: 4 passed

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly describes the primary change: support for curly quotes in generated bot messages during streaming. It is concise and directly related to the broader delimiter-handling fixes.
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 38.24% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 34 functions across 7 files.

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Full details: Test Results For Major Changes

Explanation

The PR contains major changes: it modifies streaming stop handling and the public StreamingHandler.set_pattern signature, changes completion parsing, and adds 642 lines across seven files. The authored description only states the feature behavior and related issue; it contains no test command, test results, or testing information. Added test files in the diff are not documented test results.

Resolution

Update the PR description with the testing performed and results, including the relevant test commands and whether they passed. Include performance or regression evidence only if applicable.

Finishing Touches

🧪 Generate unit tests (beta)

• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}