Skip to content

chore(e2e): add logs to determine source of failure - #2394

Open
Hweinstock wants to merge 2 commits into
aws:refactorfrom
Hweinstock:chore/add-logs
Open

Hweinstock wants to merge 2 commits into
aws:refactorfrom
Hweinstock:chore/add-logs

Conversation

@Hweinstock

@Hweinstock Hweinstock commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

Problem

The strands py container test is failing in CI only (passes locally) and we are missing sufficient logs to determine the source:

 FAIL  e2eTest/project/templates.test.ts > add, dev, deploy, invoke for runtime templates > local invocation > 'py_strands_container' runs locally
Error: Test timed out in 180000ms.
If this is a long-running test, pass a timeout value as the last argument or configure it globally with "testTimeout".
 ❯ e2eTest/project/templates.test.ts:236:35
    234|       });
    235|
    236|       test.each(RUNTIME_TEMPLATES)(
       |                                   ^
    237|         "$name runs locally",

    238|         { concurrent: true, timeout: TIMEOUT_MS.PROJECT_INVOKE },

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/1]⎯

 Test Files  1 failed (1)
      Tests  1 failed | 37 passed (38)
   Start at  20:43:12
   Duration  1084.32s (transform 58ms, setup 27.28s, import 94ms, tests 1056.80s, environment 0ms)

Solution

Add some error details to be able to root cause the issue.

@github-actions github-actions Bot added the size/xs PR size: XS label Sep 22, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added agentcore-harness-reviewing AgentCore Harness review in progress claude-security-reviewing Claude Code /security-review in progress labels Sep 22, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Sep 22, 2026

@agentcore-devx-automation agentcore-devx-automation Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AgentCore Harness Review

Verdict: Looks good

Small, focused diagnostic PR that adds error logs before each throw inside the retry block in e2eTest/project/templates.test.ts, plus a lightweight TestLogger helper. Purpose (root-causing the py_strands_container CI-only failure) is clear and the change is scoped to e2e tests only. No production code, no telemetry surface, and no new mocks.

One non-blocking observation the author may want to consider as a follow-up: the logger.error(pendingOutput) calls sit inside the retry(...) callback, which retries every 250ms up to TIMEOUT_MS.PROJECT_INVOKE. In particular the !port branch is the expected state while the dev server warms up, so on a slow CI run this will emit the full pending output on every attempt — potentially a lot of duplicated log noise. Logging only the final failure (e.g. by having retry surface the last error to the caller, or by moving the diagnostic dump into a catch around the retry call) would give the same signal with less noise. Fine to land as-is for the immediate debugging need.

@agentcore-devx-automation agentcore-devx-automation Bot removed the agentcore-harness-reviewing AgentCore Harness review in progress label Sep 22, 2026
@codecov-commenter

codecov-commenter commented Sep 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.25%. Comparing base (bafd09e) to head (84d7c2b).

Additional details and impacted files
@@            Coverage Diff            @@
##           refactor    #2394   +/-   ##
=========================================
  Coverage     97.25%   97.25%           
=========================================
  Files           613      613           
  Lines         41028    41028           
=========================================
  Hits          39902    39902           
  Misses         1126     1126           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions github-actions Bot added size/xs PR size: XS and removed size/xs PR size: XS labels Sep 22, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Sep 22, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Sep 22, 2026
@github-actions github-actions Bot added size/xs PR size: XS and removed size/xs PR size: XS labels Sep 22, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Sep 22, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Sep 22, 2026
@Hweinstock
Hweinstock marked this pull request as ready for review September 22, 2026 22:48

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/xs PR size: XS

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants