Skip to content

feat(api-service,framework): agents vision and file support fixes NV-7810 - #12423

Merged
LetItRock merged 4 commits into
nextfrom
nv-7810-agents-file-vision
Aug 24, 2026
Merged

feat(api-service,framework): agents vision and file support fixes NV-7810#12423
LetItRock merged 4 commits into
nextfrom
nv-7810-agents-file-vision

Conversation

@LetItRock

@LetItRock LetItRock commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

What changed? Why was the change needed?

Agent vision and file support.
Now Managed agents and custom code agent are able to see image files, read the PDF, txt file contents.

Screenshots

Image vision:
Screenshot 2026-08-21 at 17 29 12

PDF file:
Screenshot 2026-08-21 at 18 00 59

Custom code agent LangChain:
Screenshot 2026-08-21 at 18 32 16

Greptile Summary

This PR adds image, PDF, and plain-text attachment support across managed and custom-code agent runtimes.

  • Stores and rehydrates inbound attachment data for managed agents.
  • Maps attachment history into AI SDK and LangChain model messages.
  • Adds human-interaction tooling, documentation, templates, and focused attachment tests.

Confidence Score: 4/5

The PR does not yet appear safe to merge because accepted local attachments can still produce an oversized custom-code model request.

Local hydration accepts files up to 25 MiB, LangChain base64-expands them, and both framework mappers hydrate attachments independently without bounding the complete invocation payload, so a near-limit or multi-attachment turn can still fail before model execution.

Files Needing Attention: packages/framework/src/resources/agent/attachment-url.ts, packages/framework/src/langchain/history-mapper/hydrate-attachment-urls.ts, packages/framework/src/ai-sdk/history-mapper/hydrate-attachment-urls.ts

Important Files Changed

Filename Overview
apps/api/src/app/agents/managed-runtime/build-user-message-content.ts Builds bounded model content for images, PDFs, and plain-text attachments while preserving text on both managed-session paths.
packages/framework/src/resources/agent/attachment-url.ts Restricts local hydration and aligns its per-file cap with ingress, but still permits hydrated payloads that can exceed the downstream request budget.
packages/framework/src/langchain/history-mapper/hydrate-attachment-urls.ts Replaces unreachable attachment URLs with inline base64 independently per part, leaving the outstanding aggregate request-size failure reachable.
packages/framework/src/ai-sdk/history-mapper/hydrate-attachment-urls.ts Hydrates local file URLs into inline bytes without applying an aggregate model-request budget.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Inbound[Inbound attachment] --> Store[Store and sign attachment]
  Store --> Runtime{Agent runtime}
  Runtime -->|Managed| Managed[Read bytes and build model content]
  Runtime -->|Custom code| History[Map attachment history]
  History --> Hydrate[Hydrate unreachable local URLs]
  Managed --> Model[Model invocation]
  Hydrate --> Model
Loading

Reviews (3): Last reviewed commit: "chore(framework): suggestions from the p..." | Re-trigger Greptile

@LetItRock
LetItRock requested review from ChmaraX and scopsy August 21, 2026 17:26
@LetItRock LetItRock self-assigned this Aug 21, 2026
@mintlify

mintlify Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
novu 🟢 Ready View Preview Aug 21, 2026, 5:28 PM

@linear-code

linear-code Bot commented Aug 21, 2026

Copy link
Copy Markdown

NV-7810

@netlify

netlify Bot commented Aug 21, 2026

Copy link
Copy Markdown

Deploy Preview for dashboard-v2-novu-staging canceled.

Name Link
🔨 Latest commit bec501c
🔍 Latest deploy log https://app.netlify.com/projects/dashboard-v2-novu-staging/deploys/6a8c2b4a31e9e00008ab1e0d

@github-actions github-actions Bot changed the title feat(api-service,framework): agents vision and file support feat(api-service,framework): agents vision and file support fixes NV-7810 Aug 21, 2026
@testerarmy

testerarmy Bot commented Aug 21, 2026

Copy link
Copy Markdown

TesterArmy is running exploration tests on this Pull Request.

Result Dashboard Duration Commit Updated (UTC)
🚧 Open 20s bec501c Aug 24, 2026 11:35am

🚧 Blocked by environment

The test could not produce a product verdict because of an environment or setup problem. This is not a product failure. It is not counted as a test failure.

The target environment is protected by Vercel authentication, which prevents access to the application.

Summary

The target environment is protected by Vercel authentication, which prevents access to the application.

Planned Steps (1/8 completed)

# Step Result
1 Log in as the test user
2 Navigate to the Agents section via the sidebar or /env/development/agents
3 Verify the agents list loads and displays agents without errors
4 Open an agent's detail page by clicking on an agent
5 Verify the agent detail page renders correctly with its tabs and configuration
6 Screenshot the agent detail page
7 Navigate to the Activity > Conversations section from the sidebar
8 Verify the conversations/activity feed loads without errors
Failed step details (1)

Step 1: Log in as the test user

The target environment is protected by Vercel authentication, which prevents access to the application.

Recording (1)

Test Recording

Comment thread packages/framework/src/resources/agent/attachment-url.ts

@hacktron-app hacktron-app Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 2 files

Severity Count
HIGH 2

View full scan results

Comment thread packages/framework/src/resources/agent/attachment-url.ts Outdated
@vercel

vercel Bot commented Aug 24, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
human-md Ready Ready Preview Aug 24, 2026 11:30am

Request Review

@hacktron-app hacktron-app Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 1 file

Severity Count
HIGH 1

View full scan results

Comment thread packages/framework/src/resources/agent/attachment-url.ts
Comment thread packages/framework/src/resources/agent/attachment-url.ts
@LetItRock
LetItRock merged commit 5e6b20f into next Aug 24, 2026
41 checks passed
@LetItRock
LetItRock deleted the nv-7810-agents-file-vision branch August 24, 2026 11:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant