[mtrefilek]

Perplexity and OpenRouter decision models read images, but LiteLLM answered 400 to any image, or dropped it under drop_params. This PR sends them as image_url items inside state on /v1/decisions and /v1/systemone

Breaking change: Perplexity and OpenRouter decision callers with drop_params now send images that main dropped. Before upgrading, check rate limit room for a second send, and that nothing needs a router fallback for a dropped image

/stupid: What the PR adds

/stupid: What breaks

Original

TLDR

Problem this solves:

• Perplexity and OpenRouter decision models answer 400 to images, or drop them under drop_params

How it solves it:

• Images go out as image_url items inside state, inlined from remote URLs and in standard base64
• Images past the upstream model's type, count or size limit drop under drop_params, else answer 400
• Under drop_params on Perplexity and OpenRouter, a request whose images are refused or time out is sent again without them
• OpenRouter Jev 1.14 and jev-latest take box questions

User Flow

Before: a developer asks pplx-decider about a product photo

1. They send POST https://litellm-domain/v1/decisions to perplexity/pplx-decider-v1.2 with text and an input_image data URL
2. LiteLLM answers 400 perplexity does not support parameters: ['input_image'], or ignores the image under drop_params

After: the same request gets answers about the photo

1. They send the same request
2. LiteLLM answers 200 with "is there a red square?" at 0.98, as do /v1/systemone and OpenRouter Jev and Clef flash
3. An image past a limit answers 400 naming it, or drops under drop_params
4. A box question to OpenRouter Jev answers 200 with a box

Decisions

Choices made and the alternatives

• Images go inside state as {"type": "image_url", "image_url": {"url": "data:..."}} items
  • Both providers answer 400 to a top-level images field and name this shape
  • Shared by both through one config flag, so a third provider with the same shape sets the flag
• Text items are plain strings
  • {"type": "text"} parts also work live, plain strings keep the body smaller
• A JSON object state sent with images stays one object item ahead of the images
  • Both providers accept object items in state, so the model still reads the caller's fields
  • Turning it into a JSON string was the alternative, which hides that structure from the model
• A state that is already a list keeps its items, followed by the image items
  • Wrapping the whole list as one string item was the alternative, which loses the caller's items
• Remote https image URLs are downloaded and inlined as data URLs
  • Both providers reject remote URLs, so passing them through only moves the 400 upstream
• Data URLs are sent with the same normalized media type as the images field
  • data:IMAGE/PNG;name=x.png;base64,... passes our type check, so it goes out as data:image/png;base64,...
• Inside state, URL-safe base64 (- and _) is converted to standard base64, line breaks are removed, missing = padding is added, and image/jpg goes out as image/jpeg
  • Perplexity and OpenRouter answer 400 to each of these, and the native TypeSafe path already converts URL-safe base64
  • The images field that Cloudflare, vLLM and LM Studio get keeps the caller's base64 as on main, since nothing here proves those providers want it changed
• Unsupported image types answer 400, or are dropped under drop_params, per the usual convention
• On OpenRouter, image rules follow the upstream model's own provider
  • TypeSafe models (typesafe/..., ~typesafe/...) take only PNG and JPEG, as TypeSafe does natively
  • Cloudflare models take at most 4 images of 4 MiB each and 8 MiB in total, as Clef does natively
  • Perplexity models (perplexity/...) take at most 8 images, as pplx-decider does natively
  • Other OpenRouter decision models are text-only, so images to them answer 400 before the call, or drop under drop_params, as on main
  • One list for every OpenRouter model was the alternative, which sends requests the upstream model refuses and turns main's 200 under drop_params into an upstream 400
• Perplexity takes at most 8 images, checked before the call
  • Perplexity answers 400 above 8, so the error names the limit before any upload
• Images past a count or size limit are dropped under drop_params and answer 400 without it, the usual convention
  • Each image over the per-image size limit drops on its own, and the rest are kept in order while each still fits the count and total size limits, so an oversized image never takes a count slot and a small image after one past the total is still sent
  • Without drop_params, more images than the count limit answer 400 before any download
  • Under drop_params, remote URLs past the count are dropped before any download, so a long list costs at most the count in fetches, as main fetched none past it
  • Cutting at the first oversized image was the alternative, which drops images the provider accepts
  • Refusing with or without drop_params was the alternative, which turns a 200 into a 400 for callers whose images main dropped
  • Dropping every image, as main did, was the other, which throws away images the model can read
  • Cloudflare clef answered 400 over its limits on main even under drop_params (feat(decisions)!: send images to Cloudflare Clef through its images field #45734). It now follows the convention, so under drop_params it answers 200 with the images that fit, and without it still answers 400 before any download
• A remote image that fails to download (unreachable, an error status, or blocked by the URL policy) is dropped under drop_params, and answers 400 without it or when the request has nothing left to decide on
  • Failing the call was the alternative, which turns main's 200 under drop_params into a 400
• Image bytes whose leading bytes are none of the model's image types are dropped under drop_params, and answer 400 when nothing is left to decide on
  • On providers that took images on main, no text left (blank, null or empty-valued state or input_text) with every image dropped and no audio or video clips counts as nothing left, since those drops were errors on main. An image dropped for its declared type, downloaded or not, keeps main's answer, so that request still reaches the model. A downloaded image dropped for its type after remote images past the count were dropped does not, since main refused that request for the count
  • A state nested past DEFAULT_MAX_RECURSE_DEPTH levels counts as no text, so the check never walks past the stack. Counting it as text was the alternative, which answered garbage bytes with a billed 200 from no input where main answered an error
  • Every string in state counts as text, since state is the caller's own JSON, except a type value in an object that also has a text key, where the part's text is in text. So {"type": "refund"} keeps main's 200 and [{"type": "text", "text": ""}] keeps main's error. Counting every type was the alternative, which answered a blank text part with a billed 200 from no input, and skipping every type was the other, which turned main's 200 on {"type": "refund"} into a 400
  • Perplexity and OpenRouter keep main's 200 on an empty state, which main sent them with no images
  • Without drop_params they still go upstream and get the provider's 400, as on main
  • Checking only the declared media type was the alternative, which sends a mislabeled GIF or plain text upstream and turns main's 200 under drop_params into a 400
• Under drop_params on Perplexity and OpenRouter, when the provider answers 400, 408, 413 or 422 to a request that carried images, or the request times out, LiteLLM sends it once more without the images, which is the request main sent
  • A file with a good header and a broken body, like a cut-off PNG, passes the leading-bytes check, and Perplexity answers it with image decode failed
  • Checking that each image fully decodes before the call was the alternative, which needs an image library LiteLLM does not ship and still misses files a provider's decoder refuses
  • Matching the provider's error text was the other, which breaks when a provider rewords it
  • A 401, 429 or 5xx, a request with no input left without its images, or any refusal without drop_params reaches the caller after one call
  • Whether to ship the second send waits on product signoff, since near the rate limit it answers fewer 200s than main. The options are to ship as is (the recommendation), remove the second send so every refused image answers 400 at any traffic, not ship, or put image sending under drop_params behind a setting that is off by default
  • Models that took images on main (native Cloudflare, TypeSafe, OpenAI) are not sent again, so their refusal reaches the caller and router fallbacks as on main. Sending again there was the alternative, which answers before a configured fallback runs
• Bodies with no images are unchanged, so state stays an object there
• Logging callbacks get the body that was sent, with the caller's URL in place of each remote image LiteLLM downloaded, matched by position so two URLs with the same bytes each log their own URL. On the success event, a dropped image is not logged and neither is downloaded base64
• Box support for OpenRouter Jev comes from two cost map rows, openrouter/typesafe/jev-1.14 and openrouter/~typesafe/jev-latest, with supports_box_questions: true
  • Box is decided from the cost map for every provider, and OpenRouter had no Jev rows
  • The rows carry no prices, so billing still uses the usage.cost OpenRouter returns. A live call reported $2.79e-05 with the rows and $2.77e-05 without
  • Reusing the TypeSafe rows was the alternative, but their ids (jev-1.14.0) differ from OpenRouter's and typesafe/jev-latest lacks the flag
  • typesafe/jev-1.13 gets no row, since it rejects box questions

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/unit/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• Every automated review on my PR's latest commit has finished, and I have addressed or replied to each of its findings before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Both legs run litellm --config config.yaml --port $PORT --num_workers 2 with no database and real provider keys. Before is the merge base 4ce724e, and e8f572b, c4b01b2 or f4a2e2b for the reruns

Last updated: 9c005c9. QA ran on 1c82abd. Changes since QA: no-text checks and merges of main. Proof remains valid because requests with text or a kept image take the same path. Verification: decisions unit tests, and the headline and changed cases live at 9c005c9, all matching

Proxy config and request variables

model_list:
  - model_name: pplx-decider
    litellm_params:
      model: perplexity/pplx-decider-v1.2
      api_key: os.environ/PERPLEXITY_API_KEY
  - model_name: pplx-decider-drop
    litellm_params:
      model: perplexity/pplx-decider-v1.2
      api_key: os.environ/PERPLEXITY_API_KEY
      drop_params: true
  - model_name: jev
    litellm_params:
      model: openrouter/typesafe/jev-1.14
      api_key: os.environ/OPENROUTER_API_KEY
  - model_name: jev-latest
    litellm_params:
      model: openrouter/~typesafe/jev-latest
      api_key: os.environ/OPENROUTER_API_KEY
  - model_name: jev-drop
    litellm_params:
      model: openrouter/typesafe/jev-1.14
      api_key: os.environ/OPENROUTER_API_KEY
      drop_params: true
  - model_name: clef-flash
    litellm_params:
      model: openrouter/cloudflare/clef-flash
      api_key: os.environ/OPENROUTER_API_KEY
  - model_name: clef-flash-drop
    litellm_params:
      model: openrouter/cloudflare/clef-flash
      api_key: os.environ/OPENROUTER_API_KEY
      drop_params: true
  - model_name: or-pplx
    litellm_params:
      model: openrouter/perplexity/pplx-decider-v1.2
      api_key: os.environ/OPENROUTER_API_KEY
  - model_name: or-pplx-drop
    litellm_params:
      model: openrouter/perplexity/pplx-decider-v1.2
      api_key: os.environ/OPENROUTER_API_KEY
      drop_params: true
  - model_name: cf-clef
    litellm_params:
      model: cloudflare/clef
      api_key: os.environ/CLOUDFLARE_API_KEY
  - model_name: cf-clef-drop
    litellm_params:
      model: cloudflare/clef
      api_key: os.environ/CLOUDFLARE_API_KEY
      drop_params: true
  - model_name: solar-flash
    litellm_params:
      model: openrouter/upstage/solar-decide-flash
      api_key: os.environ/OPENROUTER_API_KEY
  - model_name: solar-flash-drop
    litellm_params:
      model: openrouter/upstage/solar-decide-flash
      api_key: os.environ/OPENROUTER_API_KEY
      drop_params: true
litellm_settings:
  callbacks: image_log.recorder
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY

# image_log.py
import json
from pathlib import Path

import litellm
from litellm.integrations.custom_logger import CustomLogger

LOG = Path("/tmp") / f"decision-images-{Path(litellm.__file__).parents[1].name}.log"


def _state_images(state):
    return [item["image_url"]["url"] for item in state if isinstance(item, dict) and item.get("type") == "image_url"]


def _logged_images(body):
    if "images" in body:
        return body["images"]
    if isinstance(body.get("state"), (list, tuple)):
        return _state_images(body["state"])
    return None


class ImageLog(CustomLogger):
    async def async_log_success_event(self, kwargs, response_obj, start_time, end_time):
        body = kwargs.get("input")
        images = _logged_images(body) if isinstance(body, dict) else None
        if images is not None:
            with LOG.open("a") as f:
                f.write(json.dumps([image if image.startswith("http") else image[:60] for image in images]) + "\n")


recorder = ImageLog()

LITELLM_LOCAL_MODEL_COST_MAP=True is set on both legs. KEY is the master key. red.png is a 400x300 white PNG with one red square, and the other variables come from it

B=$(base64 -w0 red.png)
W=$(base64 -w0 red.webp)
U=$(base64 -w0 red.png | tr '+/' '-_')
V=$(base64 -w0 red.png | tr '+/' '-_' | tr -d '=')
J=$(base64 -w0 red.jpg)
P="\"data:image/png;base64,$B\""
BIG=$(base64 -w0 big.png)
G=R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7
URL=https://raw.githubusercontent.com/BerriAI/litellm/cfc651dcd5c4daa9f3e12a97b56e7da066680fd0/.github/screenshots/before_403_error.png
MISSING=https://raw.githubusercontent.com/BerriAI/litellm/cfc651dcd5c4daa9f3e12a97b56e7da066680fd0/.github/screenshots/no_such_image.png
WRAPPED=$(base64 -w0 red.png | fold -w 76 | awk '{printf "%s\\n", $0}')
NOTIMG=$(printf 'not an image' | base64 -w0)
CUT=$(head -c 1068 red.png | base64 -w0)
MID=$(base64 -w0 mid.png)
DEEP=$(printf '[%.0s' {1..101})'""'$(printf ']%.0s' {1..101})

red.webp and red.jpg are red.png saved as WebP and JPEG, $U is the same PNG in URL-safe base64, $V is $U without its = padding, $P is one quoted PNG data URL, big.png is a 5 MB noise PNG, $G is a 1x1 GIF, $URL is a dashboard screenshot with an error popup, $MISSING answers 404, $WRAPPED is the PNG's base64 in 76-character lines joined by JSON \n escapes, $NOTIMG is base64 bytes that are no image, $CUT is the first half of red.png, mid.png is a 3 MB noise PNG, and $LOGF is /tmp/decision-images-<worktree>.log, where the logging callback writes the images it was given

Case	Before	After
Perplexity decisions input_image	400 unsupported	red 0.98, blue 0.04
Jev decisions input_image	400 unsupported	red 0.98, blue 0.02

Perplexity /v1/decisions with input_image

curl -s -m 180 -o /tmp/decision-$PORT.json -w 'HTTP %{http_code}\n' localhost:$PORT/v1/decisions -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" -d @- <<EOF
{"model":"pplx-decider","input":[{"role":"user","content":[{"type":"input_text","text":"Product photo"},{"type":"input_image","image_url":"data:image/png;base64,$B"}]}],"questions":[{"type":"predicate","name":"red","instructions":"Is there a red square in the photo?"},{"type":"predicate","name":"blue","instructions":"Is there a blue circle in the photo?"}]}
EOF
jq -c '{model, answers: [.answers[]?|{name, probability}], error: .error.message}' /tmp/decision-$PORT.json

Before (4ce724e)

HTTP 400
{"model":null,"answers":[],"error":"litellm.UnsupportedParamsError: perplexity does not support parameters: ['input_image'], for model=perplexity/pplx-decider-v1.2. To drop these, set `litellm.drop_params=True` or for proxy:\n\n`litellm_settings:\n drop_params: true`\n\n\nLiteLLM: model group 'pplx-decider' failed with the error above. No fallback was attempted."}

After (1c82abd)

HTTP 200
{"model":"pplx-decider-v1.2","answers":[{"name":"red","probability":0.9786596426949823},{"name":"blue","probability":0.03782685712897145}],"error":null}

73 more rows

Case	Before	After
Perplexity systemone images	400 unsupported	red 0.98, blue 0.04
Perplexity drop_params input_image	red 0.18, blue 0.21	red 0.98, blue 0.04
Perplexity systemone mixed-case data URL	400 unsupported	red 0.98, blue 0.04
Perplexity systemone GIF	400 unsupported	400 names JPEG/JPG/PNG/WEBP
Perplexity systemone JSON state, no image	bug 1.00	bug 1.00
Jev decisions WebP	400 unsupported	400 names JPEG/JPG/PNG
Perplexity systemone URL-safe base64	400 unsupported	red 0.98, blue 0.04
Jev systemone JSON state, image	400 unsupported	red 0.98, blue 0.01
Perplexity systemone 9 images	400 unsupported	400 at most 8 images
Clef flash systemone WebP	400 unsupported	red 0.91, blue 0.01
Clef flash drop_params 5 images	red 0.08, blue 0.06	red 0.95, blue 0.01
OpenRouter Perplexity systemone 9 images	400 unsupported	400 at most 8 images
Perplexity systemone data:image/jpg	400 unsupported	red 0.96, blue 0.02
Perplexity unpadded URL-safe base64	400 unsupported	red 0.98, blue 0.04
Cloudflare Clef 5 images	400 at most 4 images	400 at most 4 images
Cloudflare Clef drop_params 5 images	400 at most 4 images	red 0.98, blue 0.01
Cloudflare Clef URL-safe base64	422 from Cloudflare	422 from Cloudflare
Solar Decide flash drop_params image	red 0.08, blue 0.04	red 0.20, blue 0.05
Solar Decide flash image	400 unsupported	400 unsupported
Cloudflare Clef drop_params 5 MB then small	400 over 4 MiB	red 0.97, blue 0.01
Jev systemone box question	400 unsupported	square box
Jev decisions box question	400 unsupported	square box
Jev latest decisions box question	400 unsupported	square box
Clef drop_params 5 MB then remote, logged	400 over 4 MiB	red 0.44, blue 0.30
Clef drop_params four 5 MB then small	400 at most 4 images	red 0.97, blue 0.01
Perplexity line-wrapped base64	400 unsupported	red 0.98, blue 0.04
OpenRouter Perplexity drop_params not-an-image, PNG	red 0.15, blue 0.17	red 0.99, blue 0.17
Jev drop_params not-an-image, PNG	red 0.20, blue 0.20	red 0.98, blue 0.01
Clef drop_params not-an-image, PNG	422 from Cloudflare	red 0.97, blue 0.01
Perplexity drop_params only not-an-image	400 unsupported	400 not an image
OpenRouter Perplexity drop_params cut-off PNG, PNG	red 0.15, blue 0.17	red 0.15, blue 0.17
Perplexity Claude Code agent, cut-off PNG	red 0.18, blue 0.21	red 0.18, blue 0.21
Perplexity drop_params non-ASCII base64, PNG	red 0.15, blue 0.17	red 0.99, blue 0.17
Perplexity drop_params empty state, not-an-image	red 0.24, blue 0.21	red 0.24, blue 0.21
Clef flash systemone images	400 unsupported	red 0.90, blue 0.01
Perplexity decisions https URL	400 unsupported	error_popup 1.00, dog 0.02
Perplexity drop_params 9 images	red 0.15, blue 0.17	red 0.99, blue 0.90
Clef one image under two URLs, logged	red 0.22, blue 0.13, logs URL, URL?copy=1	red 0.22, blue 0.13
Perplexity two URLs, logged	400 unsupported	red 0.03, blue 0.70
Perplexity drop_params 404 URL, image	red 0.15, blue 0.17	red 0.99, blue 0.17
Perplexity drop_params not-an-image, PNG	red 0.15, blue 0.17	red 0.99, blue 0.17
Jev drop_params WebP and PNG	red 0.21, blue 0.25	red 0.99, blue 0.02
OpenRouter Perplexity systemone image	400 unsupported	red 0.98, blue 0.04
Perplexity systemone not-an-image	400 unsupported	400 from Perplexity
Perplexity drop_params cut-off PNG, PNG	red 0.15, blue 0.17	red 0.15, blue 0.17
Clef drop_params three 3 MB then small	400 over 8 MiB total	red 0.96, blue 0.01
Clef omni drop_params empty state, audio, not-an-image	422 from Cloudflare	tone 0.96
Perplexity drop_params ten remote URLs	red 0.15, blue 0.17	red 0.03, blue 0.74
Perplexity drop_params eight 5 MB, 2 s timeout	red 0.15, blue 0.17	red 0.15, blue 0.17
Clef drop_params empty state, not-an-image	422 from Cloudflare	400 not an image
Clef decisions drop_params text, not-an-image	422 from Cloudflare	red 0.05, blue 0.04
Cloudflare drop_params blank state, not-an-image	422 from Cloudflare	400 not an image
Cloudflare drop_params empty values, not-an-image	422 from Cloudflare	400 not an image
Cloudflare drop_params blank state, GIF	red 0.10, blue 0.13	red 0.10, blue 0.13
Cloudflare drop_params empty state, GIF	red 0.12, blue 0.17	red 0.12, blue 0.17
Clef decisions drop_params no text, GIF	red 0.12, blue 0.17	red 0.12, blue 0.17
Cloudflare drop_params state of null, not-an-image	422 from Cloudflare	400 not an image
Cloudflare drop_params empty text part, not-an-image	422 from Cloudflare	400 not an image
Clef decisions drop_params no text, not-an-image	422 from Cloudflare	400 not an image
Cloudflare drop_params blank state, remote GIF	red 0.10, blue 0.13	red 0.10, blue 0.13
Clef decisions drop_params no text, remote GIF	red 0.12, blue 0.17	red 0.12, blue 0.17
Cloudflare drop_params state 101 deep, not-an-image	422 from Cloudflare	400 not an image
Cloudflare drop_params type-only state, not-an-image	422 from Cloudflare	red 0.08, blue 0.09
Perplexity drop_params type-only state, not-an-image	red 0.04, blue 0.13	red 0.04, blue 0.13
Cloudflare drop_params blank state, five remote GIFs	400 at most 4 images	400 names JPEG/JPG/PNG/WEBP
Clef decisions drop_params empty text, five remote GIFs	400 at most 4 images	400 names JPEG/JPG/PNG/WEBP
Cloudflare drop_params blank state, four GIFs then a WEBP	400 at most 4 images	400 names JPEG/JPG/PNG/WEBP
Cloudflare drop_params blank state, four GIFs then a 404	400 at most 4 images	400 names JPEG/JPG/PNG/WEBP
Cloudflare drop_params blank state, four remote GIFs	red 0.10, blue 0.13	red 0.10, blue 0.13
Cloudflare drop_params text, five remote GIFs	400 at most 4 images	red 0.05, blue 0.04
Cloudflare drop_params blank text part, not-an-image	422 from Cloudflare	400 not an image
Cloudflare drop_params two empty text parts, not-an-image	422 from Cloudflare	400 not an image
Cloudflare drop_params text part with text, not-an-image	422 from Cloudflare	red 0.03, blue 0.04

The other 74 cases' commands and full output on both legs are in this gist

QA observations

• pplx-decider says blue 0.90 for eight copies, direct too

Type

🆕 New Feature

Caveats (if any)

Severe

• Under drop_params on Perplexity and OpenRouter, a refused image is sent again without it, so near the rate limit fewer requests answer 200, 92 of 120 on main and 48 here. Without the second send, main's 200 becomes a 400. Options are in Decisions

Low

• The signoff on the severe caveat also covers these changes a caller can see
• OpenRouter Jev bills usage.cost, or $0 without it, and has no image count limit. Its two new rows turn /health from 503 to 200, add two /v1/models ids, and let box questions reach Jev
• image_rules_for returns a provider config for four limits
• Without drop_params, native Cloudflare still answers 422 to URL-safe base64, as on main
• On Perplexity and OpenRouter, one refused image drops every image, since finding it needs a decoder or a call per image
• The second send added 5 s to 9 s where measured, at most twice the timeout, and repeats a 400 or 422 the image did not cause. LiteLLM limits count one request
• A resent call logs log_pre_api_call twice, the first with the refused image, and its spend row prices only the second send
• A local image drop never reaches a router fallback, like every drop_params drop
• The proxy now fetches caller URLs for Perplexity and OpenRouter, so a hanging URL holds the call until timeout (100 s, default 6000 s), eight 30 MiB URLs used 1.6 GB, and a failed download is not replaced
• Cloudflare gets a local 400 where main got its 422 once nothing readable is left, and its own 422 for wrapped base64 near its size limit
• A blank request losing its only image to its declared type still reaches the model with no input, as on main
• A state nested past DEFAULT_MAX_RECURSE_DEPTH counts as no text, so lowering that setting refuses more of what main refused

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[codspeed]

Merging this PR will not alter performance

1 warning

⚠️ 1 benchmark is dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

✅ 43 untouched benchmarks

---

_{Comparing mtrefilek:openrouter-decisions-images (9c005c9) with main (f4a2e2b)}

[codecov]

Codecov Report

❌ Patch coverage is 98.26990% with 5 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/llms/base_llm/decisions/transformation.py	95.45%	3 Missing ⚠️
litellm/decisions/call.py	98.94%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

[mateo-berri]

@greptileai

[greptile-apps]

Confidence Score: 4/5

[Medium impact] The changes affect image-bearing requests in a bounded set of decision-model workflows.

Explicit review of the retry’s rate-limit and timeout costs is required before merging

Findings

1.  Retries reduce successful requests ▶

Summary

Adds image support for Perplexity and supported OpenRouter decision models through state

• Perplexity and supported OpenRouter models receive images alongside decision text.
• Some image refusals now get one more decision request without the images.
• OpenRouter Jev 1.14 and Jev latest accept box questions.

mateo-berri accepted returning existing provider configs for image rules and removing all images after a refusal. mateo-berri accepted price-free Jev rows to avoid stale copied prices, and mtrefilek retained that choice

_{Reviews (11) · Last reviewed commit: "Merge branch 'openrouter-decisions-image..." · Reviewed by Greptile}

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[veria-ai]

Low: Image download limit is enforced after buffering

Enabling remote inlining here and in OpenRouterDecisionsConfig lets an authenticated caller submit a public image URL serving an oversized body and exhaust proxy-worker memory. Both download helpers use non-streaming HTTP handlers, so _process_image_response() checks MAX_IMAGE_URL_DOWNLOAD_SIZE_MB only after HTTPX buffers the entire response—enforce the cap during retrieval using max_response_bytes for the async handler and bounded streaming for sync downloads.

[veria-ai]

PR overview

The PR adds support for sending images in the state array to Perplexity and OpenRouter decision models, including inlining images from remote URLs.

One issue remains open, with none addressed so far. An authenticated caller can provide a public image URL with an oversized response that is fully buffered before the download-size limit is checked, potentially exhausting proxy-worker memory. Enforcing the limit during retrieval would address this resource-exhaustion risk.

Open issues (1)

• Low: Image download limit is enforced after buffering — litellm/llms/perplexity/decisions/transformation.py:11

Fixed/addressed: 0 · PR risk: 6/10

[mateo-berri]

@greptileai

[greptile-apps]

Want your agent to iterate on Greptile's feedback? Try greploops.

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[greptile-apps]

Retries reduce successful requests

Near provider limits, image retries reduce successes and can double timeout. Repository rules require explicit review of these costs before merging

Rule Used: What: do not treat the caveats mentioned in the PR description as acceptable tradeoffs that some human has read and signed off on. Why: almost every time, the PR description is AI-generated, and in fact, many times, the PR description is very long. ... (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!