Skip to content

refactor: consolidate extra_body merge logic into ModelOption.merge_extra_body - #1695

Merged
jakelorocco merged 1 commit into
generative-computing:mainfrom
cptnm3:consolidate_extra_body/chat_template_kwargs
Sep 30, 2026
Merged

jakelorocco merged 1 commit into
generative-computing:mainfrom
cptnm3:consolidate_extra_body/chat_template_kwargs

Conversation

@cptnm3

@cptnm3 cptnm3 commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Pull Request

Issue

Fixes #1615

Description

Promote _merge_extra_body to the public merge_extra_body static method and make it the single canonical implementation used by all three merge paths. Malformed chat_template_kwargs now raises TypeError instead of the old silent fallback.

Changes:

  • model_options.py: rename _merge_extra_body → merge_extra_body; raise TypeError (naming the offending value and side) when chat_template_kwargs is not a dict; omit chat_template_kwargs from the output when the merged mapping is empty
  • openai.py: rewrite _merge_user_extra_body as two chained merge_extra_body calls (default < base < user); update init call-site to public name
  • litellm.py: replace inline hand-rolled merge block with a single merge_extra_body call; update seed call-site to public name
  • test_model_options.py: replace silent-fallback tests with TypeError tests (base and overwrite side); add empty-both-sides-drops-key test; update no-mutation test to public name
  • test_options.py: add cross-path delegation tests for Scenarios A–D across all four paths (_via_helper, _via_merge_model_options, _via_openai, _via_litellm); convert Scenarios A–C to async to include the LiteLLM path

Assisted-by: IBM Bob

Testing

  • Tests added to the respective file if code was changed
  • New code has 100% coverage if code was added
  • Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

Attribution

  • AI coding assistants used

Adding a new component, requirement, sampling strategy, or tool?

If your PR adds or modifies one of the types below, check the matching box. A checklist of type-specific review items will be posted as a comment.

  • Component
  • Requirement
  • Sampling Strategy
  • Tool

NOTE: Please ensure you have an issue that has been acknowledged by a core contributor and routed you to open a pull request against this repository. Otherwise, please open an issue before continuing with this pull request.

…xtra_body

Promote `_merge_extra_body` to the public `merge_extra_body` static method
and make it the single canonical implementation used by all three merge paths.

Changes:
- model_options.py: rename _merge_extra_body → merge_extra_body; raise
  TypeError (naming the offending value and side) when chat_template_kwargs is
  not a dict; omit chat_template_kwargs from the output when the merged mapping
  is empty
- openai.py: rewrite _merge_user_extra_body as two chained merge_extra_body
  calls (default < base < user); update __init__ call-site to public name
- litellm.py: replace inline hand-rolled merge block with a single
  merge_extra_body call; update seed call-site to public name
- test_model_options.py: replace silent-fallback tests with TypeError tests
  (base and overwrite side); add empty-both-sides-drops-key test; update
  no-mutation test to public name
- test_options.py: add cross-path delegation tests for Scenarios A–D across
  all four paths (_via_helper, _via_merge_model_options, _via_openai,
  _via_litellm); convert Scenarios A–C to async to include the LiteLLM path

Assisted-by: IBM Bob
Signed-off-by: Vishal V <VishalV@ibm.com>
@cptnm3
cptnm3 requested a review from a team as a code owner September 29, 2026 12:23
@github-actions github-actions Bot added the enhancement New feature or request label Sep 29, 2026

@planetf1 planetf1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Consolidation does what it says — merge_extra_body is now the single implementation behind merge_model_options, LiteLLMBackend, and OpenAIBackend._merge_user_extra_body, and the new cross-path tests exercise all three call sites (not just the helper in isolation). Ran the test suite on this head: passing.

One non-blocking note: malformed chat_template_kwargs now raises TypeError instead of the old silent fallback — a real behaviour change on public ModelOption, worth a line in the PR description so it shows up in release notes, but no current caller depends on the old behaviour and the new failure mode matches the existing pattern in formatters/granite/base/util.py.

@cptnm3

cptnm3 commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor Author

Hi @planetf1, Thanks for the review! I would like to bring to your attention that docs/versioned_docs/version-0.8.0/api/mellea/backends/model_options.mdx still references old private method _merge_extra_body. I have left it untouched hoping it will be reconstructed by the CI. I hope that's ok.
I'll add the line in PR noting the behaviour change.

@planetf1

Copy link
Copy Markdown
Contributor

Hi @planetf1, Thanks for the review! I would like to bring to your attention that docs/versioned_docs/version-0.8.0/api/mellea/backends/model_options.mdx still references old private method _merge_extra_body. I have left it untouched hoping it will be reconstructed by the CI. I hope that's ok. I'll add the line in PR noting the behaviour change.

The version-0.8.0 docs won’t get updated by CI as they are intended specifically for the 0.8.0 version of the code. There we did have ModelOption._merge_extra_body

The 0.9.0 docs will get created by the release process, and they should pick up changes you make - the api docs are constantly rebuild during dev too so https://docs.mellea.ai/next will give you a build of docs from main

@jakelorocco
jakelorocco added this pull request to the merge queue Sep 30, 2026
Merged via the queue into generative-computing:main with commit 3f31b36 Sep 30, 2026
20 of 21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

refactor(backends): consolidate the three extra_body/chat_template_kwargs deep-merge implementations

3 participants