fix(eval): reject an ARN for config-bundle / gateway / online-eval ids

[jariy17]

Problem

For agentcore eval config-bundle list (and the other eval commands that take --id/--gateway/--online-eval), if you pass in an ARN instead of the bare id, the API throws a misleading AccessDenied error. (The ARN's slashes make the service misparse the request path as an unknown operation, which surfaces as a permissions error rather than a validation one.)

Fix

I decided to validate that the id is not an ARN and throw an error — an actionable InputValidationError instead of letting the request go through and hit the misleading AccessDenied:

--id must be a bare resource id, not an ARN (pass the id, e.g. the segment after the last '/').

Guarded at the core-method boundary for config-bundle (get/update/delete/listVersions), online-eval (get/update/setExecutionStatus/delete), and the createABTest gateway lookup (both A/B variants).

Tests

Config-bundle core test: a full ARN rejects with the bare-id message before any SDK call; a bare id still works. Full eval suite green; typecheck clean.

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)

[agentcore-devx-automation]

AgentCore Harness Review

Verdict: Looks good

The fix is well-targeted: normalizing IDs at the core-method boundary with the existing serviceIdFromArn (which passes non-ARN inputs through unchanged) is the right lever, and it's applied consistently to every touched entry point. The added test mocks only at the SDK client boundary and asserts the bare id lands in every command variant (get / list versions / delete / update-with-read).

One adjacent gap worth considering (optional, not a blocker for this PR since it's out of the stated scope):

• src/core/eval.tsx lines 1161–1163 (setOnlineInsightExecutionStatus) and 1175–1177 (deleteOnlineInsight) do a pre-check GetOnlineEvaluationConfigCommand({ onlineEvaluationConfigId: id }) with the raw id before delegating to the now-normalized setOnlineEvaluationExecutionStatus / deleteOnlineEvaluationConfig. Because online-insight configs are the same underlying resource as online-eval configs, an ARN passed to eval online-insight pause/resume/delete would still hit the same misleading AccessDenied on the pre-check. Fix would be a one-line id = serviceIdFromArn(id); at the top of each method (or normalize once before the GetOnlineEvaluationConfigCommand call). Fine to defer to a follow-up if that's out of scope here.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.26%. Comparing base (a23e472) to head (17eae89).
⚠️ Report is 1 commits behind head on refactor.

Additional details and impacted files

@@            Coverage Diff            @@
##           refactor    #2352   +/-   ##
=========================================
  Coverage     97.26%   97.26%           
=========================================
  Files           610      610           
  Lines         40590    40609   +19     
=========================================
+ Hits          39480    39499   +19     
  Misses         1110     1110           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)