feat(api-service,framework): agents vision and file support fixes NV-7810 - #12423
Merged
Conversation
Contributor
|
Preview deployment for your docs. Learn more about Mintlify Previews.
|
✅ Deploy Preview for dashboard-v2-novu-staging canceled.
|
|
TesterArmy is running exploration tests on this Pull Request.
🚧 Blocked by environmentThe test could not produce a product verdict because of an environment or setup problem. This is not a product failure. It is not counted as a test failure. The target environment is protected by Vercel authentication, which prevents access to the application. SummaryThe target environment is protected by Vercel authentication, which prevents access to the application. Planned Steps (1/8 completed)
Failed step details (1)Step 1: Log in as the test user
|
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

What changed? Why was the change needed?
Agent vision and file support.
Now Managed agents and custom code agent are able to see image files, read the PDF, txt file contents.
Screenshots
Image vision:

PDF file:

Custom code agent LangChain:

Greptile Summary
This PR adds image, PDF, and plain-text attachment support across managed and custom-code agent runtimes.
Confidence Score: 4/5
The PR does not yet appear safe to merge because accepted local attachments can still produce an oversized custom-code model request.
Local hydration accepts files up to 25 MiB, LangChain base64-expands them, and both framework mappers hydrate attachments independently without bounding the complete invocation payload, so a near-limit or multi-attachment turn can still fail before model execution.
Files Needing Attention: packages/framework/src/resources/agent/attachment-url.ts, packages/framework/src/langchain/history-mapper/hydrate-attachment-urls.ts, packages/framework/src/ai-sdk/history-mapper/hydrate-attachment-urls.ts
Important Files Changed
Flowchart
%%{init: {'theme': 'neutral'}}%% flowchart LR Inbound[Inbound attachment] --> Store[Store and sign attachment] Store --> Runtime{Agent runtime} Runtime -->|Managed| Managed[Read bytes and build model content] Runtime -->|Custom code| History[Map attachment history] History --> Hydrate[Hydrate unreachable local URLs] Managed --> Model[Model invocation] Hydrate --> ModelReviews (3): Last reviewed commit: "chore(framework): suggestions from the p..." | Re-trigger Greptile