From a0e9edbe01660e0aecb744edca3de27776cebd38 Mon Sep 17 00:00:00 2001 From: Bharath Krishna <118433150+bharathk-testsigma@users.noreply.github.com> Date: Wed, 7 Oct 2026 18:27:00 +0530 Subject: [PATCH] updated docs as per new results UI --- src/pages/docs/ai-agents/analyzer.md | 53 +++++---- .../auto-healing/auto-healing-insights.md | 90 +++++++------- .../docs/reports/runs/drill-down-reports.md | 93 +++++++++------ .../docs/runs/debug-test-case-failures.md | 110 +++++++++++------- src/templates/page.scss | 21 ++++ 5 files changed, 219 insertions(+), 148 deletions(-) diff --git a/src/pages/docs/ai-agents/analyzer.md b/src/pages/docs/ai-agents/analyzer.md index 2c730b4e..57d9ba8e 100644 --- a/src/pages/docs/ai-agents/analyzer.md +++ b/src/pages/docs/ai-agents/analyzer.md @@ -43,44 +43,47 @@ The Analyzer Agent analyzes a failed test step and returns the error type, the r 2. Select the run that contains failed test steps. -3. Open the test case with failures and select the failed test step. +3. Click the test case with failures. - The step header shows the error message. Click **Read more** to see the full message. The **Analysis** tab shows the error code, such as **#NO_SUCH_ELEMENT**, along with the **Visual Evidence** section. +4. On the **Test Case Results** page, select the failed step. -4. Click **Analyze with Agent** in the action bar at the bottom of the page. -  + The **ANALYZER VERDICT** section on the step already shows a short analysis of the failure. Use the Analyzer Agent for suggested fixes you can apply, follow-up questions, and bug reports. For details on the verdict, refer to the [documentation on debugging test case failures](https://testsigma.com/docs/runs/debug-test-case-failures/). -5. Review the analysis, which displays: - - - **Error Type**: The category of failure identified during execution, such as ELEMENT_NOT_FOUND. - - **Root Cause**: An explanation of why the step failed, based on the error message and the captured evidence. - - **Visual Evidence**: The screenshot captured at the time of execution. Click **Expand Images** to view it at full size, or the download icon to save it. - - **Suggestions**: A numbered list of ways to resolve the failure. +5. Click **Analyze with Agent** in the action bar at the bottom of the page. +  + + The **Analyzer with Atto** panel opens and reads "Analyzing step failure..." while it runs. + +6. Review the analysis. +  + + | **Section** | **What it shows** | + |---|---| + | **Error Type** | The failure type identified during execution | + | **Root Cause** | Why the step failed | + | **Visual Evidence** | Screenshots **At the time of Recording** and **At the time of Execution**. Only the execution screenshot appears when no recording screenshot exists. | + | **Suggestions** | Numbered fixes you can apply to the step | [[info | **NOTE**:]] | To ask a follow-up question about the failure, enter it in **Debug with Atto** and click **Analyze**. -**Additional Information**: Responses are generated by Atto AI. Review the analysis before acting on it. Use the feedback icons or **Give Feedback** to tell us how useful it was. - --- ## **Apply Fix Using Analyzer Agent** -1. In the **Suggestions** list, select the suggestion you want to apply. +1. In the **Suggestions** list, select one suggestion. **Apply Fix** stays disabled until you select one. - [[info | **NOTE**:]] - | You can select only one suggestion at a time. +2. Click **Apply Fix**. Atto returns an **Add/Update Step** card. -2. Click **Apply Fix**. -  +3. Click **Add/Update Step**. +  + In the step list, the updated step appears as an extra row below the original steps, marked with the Analyzer icon. The current result still shows the original step as failed, because the change applies from the next run. - Atto generates the fix and returns an **Update Test Step** card explaining what it found, followed by the current version of the step and the proposed version. +4. Re-run the test to confirm the fix. + - For a test plan run, click the run number in the breadcrumb to return to **Run Results**, then click **Rerun**. + - For a dry run, click **Re-Run** in the page header. -3. Compare the two versions of the step. - -4. Click **Update Step** to apply the change. - -5. Rerun the test to validate the fix. + The next run uses the updated step. --- @@ -98,6 +101,8 @@ The Analyzer Agent analyzes a failed test step and returns the error type, the r [[info | **NOTE**:]] | The fields displayed vary based on the selected bug tracking tool. -**Additional Information**: If the tool returns no projects, check that the credentials configured for it have access to the projects you expect. For Jira, the panel reads "For given JIRA Credentials, the projects list is empty. Please make sure it has the right access to required projects on JIRA". +>
Additional Information
+> +> If the tool returns no projects, check that the credentials configured for it have access to the projects you expect. For Jira, the panel reads "For given JIRA Credentials, the projects list is empty. Please make sure it has the right access to required projects on JIRA". --- diff --git a/src/pages/docs/auto-healing/auto-healing-insights.md b/src/pages/docs/auto-healing/auto-healing-insights.md index 91355acc..b400b89c 100644 --- a/src/pages/docs/auto-healing/auto-healing-insights.md +++ b/src/pages/docs/auto-healing/auto-healing-insights.md @@ -13,17 +13,11 @@ contextual_links: name: "Prerequisites" url: "#prerequisites" - type: link - name: "Identify Healed Steps in a Run" - url: "#identify-healed-steps-in-a-run" + name: "Review a Healed Step in Run Results" + url: "#review-a-healed-step-in-run-results" - type: link - name: "Review the Heal on the Analysis Tab" - url: "#review-the-heal-on-the-analysis-tab" -- type: link - name: "View Autoheal Details" - url: "#view-autoheal-details" -- type: link - name: "Read the Locator Trace" - url: "#read-the-locator-trace" + name: "Locators Tried" + url: "#locators-tried" - type: link name: "When a Heal Fails" url: "#when-a-heal-fails" @@ -46,65 +40,69 @@ Review how auto-healing behaved during a run to see which locator failed, what r --- -## **Identify Healed Steps in a Run** +## **Review a Healed Step in Run Results** -1. Open the test case results for the run. +When a saved locator fails during execution, Testsigma searches for the element and continues the run with a new locator. The step's status reads **Healed**, and the new locator is kept as a pending heal until you approve it. -2. Select the healed step. -  +1. On the **Test Case Results** page, select a step that shows the auto-heal icon. +  -[[info | **NOTE**:]] -| A healed step reports **Passed**, because the step completed. The run itself carries a **Healed** status in the run list, which distinguishes it from a run that passed without intervention. +2. Review the **Healed this step** card below the step header. +  ---- + The **Locator change** section shows two values: + - **Previous · recorded**: The saved locator that failed, shown struck through. + - **Current · applied**: The locator Testsigma used to continue the run. -## **Review the Heal on the Analysis Tab** + The count at the bottom of the card shows how many linked artifacts use this element. + -The **Analysis** tab shows a **Healed this step** card explaining what happened: "Existing locator failed during execution, and we've auto-healed it using AI. The element locator was updated with a new XPath to ensure your test continues to run smoothly." +3. Click **Locators tried** to see how Testsigma found the element. For details, refer to [Locators Tried](#locators-tried). -Choose one of the following: - -- Click **Approve as Primary** to make the healed locator the locator the test uses from now on. -- Click **Ignore** to leave the original locator in place. The heal still applied to this run only. +4. Click **Approve as Primary**. This will open **Autoheal Details** panel, showing the element name and the locator type used for the heal, such as **Autohealed using CSS Selector**. The failed locator appears struck through above the new one. -[[info | **NOTE**:]] -| In the **Visual Evidence** section, the current run panel carries a **Healed** badge, and the **Element** field has an edit icon for correcting the locator directly. +5. Click **Update**. The **Update Element** dialog opens. +  ---- +6. Review the test cases listed under **Linked to Test Cases**. Click the open icon next to a test case to view it in a new tab. -## **View Autoheal Details** +7. Click **Update** to update the element in all linked test cases. +  + The **Healed this step** card now shows **Approved**. -Open the **Autoheal Details** panel to see exactly which locator changed. -The panel shows: - -- The element that was healed, named at the top. -- The prompt "Do you want to update the autohealed element in all the linked test cases?" with **Update** and **Ignore** buttons. -- The healing method and how long it took, such as "Autohealed using CSS Selector" and "32s 912ms". -- The locator that failed, struck through, followed by the locator that replaced it. +--- -Click **Update** to apply the healed locator across every test case linked to the element, or **Ignore** to leave those test cases unchanged. +## **Locators Tried** -**Additional Information**: Use the feedback icons under "Is this autoheal helpful?" to tell us whether the heal was correct. The same feedback icons appear on the auto-healed banner above the step list. + ---- +The **Locators tried** panel lists each stage of the search in the order it runs. The panel header shows the step number, the number of stages, and how the element was found. -## **Read the Locator Trace** +| **Stage** | **What it does** | +|---|---| +| **Primary** | Tries the primary locator saved on the element. Every step starts here. | +| **Last Working** | Tries the locator that found the element in the previous run. It is kept on the element as a pending heal and is not part of the backup pool. | +| **Backup Pool** | Makes a single pass over the live screen to see which stored backup locators still resolve. No action is performed on the page at this stage. | +| **Relearn Screen** | Reads the screen as the app rendered it and writes new locators for the element. The result is kept as a pending heal and is tried first on the next run. | -Every auto-heal event records a locator trace: the full sequence the engine worked through to arrive at a heal, and whether that heal succeeded or failed. + -Use the trace to understand why a particular locator was chosen, or why no replacement could be found. +Each stage shows its outcome, such as **Did not match**, **Skipped**, or **Located**. Each locator in a stage shows: +- Its result, such as **NOT FOUND**, **AGREED**, or **NEW**. +- Its source and type, such as **smart-recorder · csspath**. +- Its role, such as **PRIMARY**, **LAST WORKING**, **ANCHOR**, or **CANDIDATE**. +- Its quality, **Good** or **Fragile**. -**Additional Information**: The locator trace is provided by the Auto Heal V2 architecture. +>Additional Information
+> +> A pending heal is not the element's primary locator. Testsigma tries it before searching again on later runs, but the primary locator changes only when you click **Approve as Primary**. --- ## **When a Heal Fails** -A heal attempt that fails gets the same root cause treatment as a failed step. Open the **Root cause** block on the **Analysis** tab and click **Explain this failure** to see why the engine could not resolve the element. - -[[info | **NOTE**:]] -| This requires Analyzer V2. Without it, a failed heal reports that it did not succeed, but no explanation is generated. +When auto-healing cannot find the element, the step fails. Select the failed step and review the **ERROR** and **ANALYZER VERDICT** sections to see why the element could not be resolved. For details, refer to the [documentation on debugging test case failures](https://testsigma.com/docs/runs/debug-test-case-failures/). --- @@ -112,7 +110,7 @@ A heal attempt that fails gets the same root cause treatment as a failed step. O If you would rather not accept the healed locator, you have two alternatives: -- **Update element** corrects the locator directly. In the **Visual Evidence** section, the **Element** field on the current run side carries an edit icon for this. +- **Update element** corrects the locator directly. - **Relearn step** re-captures the step so a fresh locator is recorded. --- diff --git a/src/pages/docs/reports/runs/drill-down-reports.md b/src/pages/docs/reports/runs/drill-down-reports.md index 8fa072e6..38668081 100644 --- a/src/pages/docs/reports/runs/drill-down-reports.md +++ b/src/pages/docs/reports/runs/drill-down-reports.md @@ -1,6 +1,6 @@ --- title: "Test Plan Run Results in Testsigma" -metadesc: "View and download reports from the Run Results page in Testsigma | Drill into results at the test case, suite, and machine level, read step analysis, and compare runs." +metadesc: "View and download reports from the Run Results page in Testsigma | Drill into results at the test case, suite, and machine level, read step details, and compare runs." noindex: false order: 14.23 page_id: "Test Plan Run Results" @@ -17,12 +17,15 @@ contextual_links: - type: link name: "Steps to View Test Case Results" url: "#steps-to-view-test-case-results" -- type: link - name: "Step Details Tabs" - url: "#step-details-tabs" - type: link name: "Steps to View Test Machine Results" url: "#steps-to-view-test-machine-results" +- type: link + name: "View a Test Case Run in Progress" + url: "#view-a-test-case-run-in-progress" +- type: link + name: "Step Details Panels" + url: "#step-details-panels" - type: link name: "Test Runs in Run Results" url: "#test-runs-in-run-results" @@ -48,7 +51,7 @@ contextual_links: --- -View and download reports from the **Run Results** page. The page presents results at the test case, test suite, and test machine level, and shows the count of passed and failed tests along with the reason for each failure. Drill into a single step to read its analysis, check its logs, and compare it against an earlier run. +View and download reports from the **Run Results** page. The page presents results at the test case, test suite, and test machine level, and shows the count of passed and failed tests along with the reason for each failure. Drill into a single step to read its details and errors, check its logs, and compare it against an earlier run. --- @@ -65,7 +68,7 @@ View and download reports from the **Run Results** page. The page presents resul 1. From the left navigation bar, go to **Run Results** and click the test plan for which you want to check the results. -2. Results open at test suite level by default. +2. Results open at the test suite level by default.  3. Expand a test suite to check the test case results inside it. @@ -89,38 +92,36 @@ View and download reports from the **Run Results** page. The page presents resul 4. Click a test case to view its detailed results. -5. On the **Test Case Results** page, the step list appears on the left, headed by the step count and the number of failed steps. Select a step to open its details on the right. Drag the divider between the two panes to resize either side. +5. On the **Test Case Results** page, the step list appears on the left, headed by the step count. Select a step to open its details on the right.  + Each step in the list shows its status icon. A healed step also shows the auto-heal icon at the end of its row. -6. Review the step header, which shows the step number, its duration, its status, and the result message. On a failed step, the header shows the error message with a **Read more** link. - -7. Click **Affected tests** to see what else depends on the element this step uses. The **Affected Instances** panel opens with counts across **Test Cases**, **Step Groups**, **Test Suites**, and **Test Plans**. - -8. Click **More details** to open **Test Case Overview**, which shows the step breakdown across **Passed**, **Failed**, **Not Executed**, and **Stopped**, along with the run message and **Machine Details**. - ---- + [[info | **NOTE**:]] + | When a run contains healed steps, a banner above the step list shows how many steps were auto-healed. Use the thumbs up and thumbs down icons on the banner to rate the heal. -## **Step Details Tabs** +6. Review the step details on the right. -Each step presents its details across five tabs. + The header above the step list shows the total run duration, the browser version, and the operating system version. The step header shows the step status, start time, duration, step number, and step action. -| **Tab** | **What it contains** | -|---|---| -| **Analysis** | The step result summary, the **Root cause** block on a failed step, and the **Visual Evidence** section comparing the authoring screenshot against this run | -| **Locators** | The locators used for the step | -| **Logs** | Selenium, console, and network logs for the step or for the whole run | -| **Step Settings** | Maximum wait time, prerequisite, whether the step result is ignored in the test case result, and whether visual testing is enabled for the step | -| **Metadata** | Test data and its type, along with the step ID and action | + The **THIS RUN** section shows the screenshot captured for the step. Click the expand icon to view it full size, or the download icon to save it. Below the screenshot, the step details show: - + | **Field** | **What it shows** | + |---|---| + | **Element** | The element the step targets. Steps without an element read "This step targets no element". | + | **Locator** | The locator type and value used for the element | + | **Run Type** | The type of run that produced this result | + | **Test data type** and **Test data** | The data type and the value passed to the step | + | **Start time** and **Duration** | When the step started and how long it ran | + | **Step level timeout** and **Plan level timeout** | The wait limits applied to the step | + | **Error code** and **Error message** | The error raised during execution, if any | -Below the two panels, the **Analysis** tab lists the element, element name, run type, action, start time, duration, step level timeout, plan level timeout, error code, error message, test data type, and test data for each side of the comparison. The **Page source** section at the bottom holds the captured HTML for each side as a downloadable file. + [[info | **NOTE**:]] + | For a failed step, the **ERROR** and **ANALYZER VERDICT** sections appear above **THIS RUN**. For details, refer to the [documentation on debugging test case failures](https://testsigma.com/docs/runs/debug-test-case-failures/). -On a failed step, the tab opens with the error code for the failure, such as **#NO\_SUCH\_ELEMENT**. +7. Click **Affected tests** to see what else depends on the element this step uses. The **Affected Instances** panel opens with tabs for **Test Cases**, **Step Groups**, **Test Suites**, and **Test Plans**, each with its count. -[[info | **NOTE**:]] -| For details on reading the logs and investigating a failure, refer to the [documentation on debugging test case failures](https://testsigma.com/docs/runs/debug-test-case-failures/). For a root cause explanation and suggested fixes, refer to the [documentation on the Analyzer Agent](https://testsigma.com/docs/ai-agents/analyzer/). +8. Click **More details** to open **Test Case Overview**, which shows the step breakdown across **Passed**, **Failed**, **Not Executed**, and **Stopped**, the run message, and **Machine Details**. --- @@ -136,6 +137,24 @@ On a failed step, the tab opens with the error code for the failure, such as **# [[info | **NOTE**:]] | To view the test cases that are part of a test machine, go to the **Test Suites** view and apply a **Test Machine** filter. This shows all test cases in the selected test suite and test machine. + +--- + +## **Step Details Panels** + +The icons in the step header open side panels with more information about the selected step. + +| **Panel** | **What it contains** | +|---|---| +| **1. Locators** | The locator search for a healed step. Appears only on healed steps. For details, refer to the [documentation on auto-healing insights](https://testsigma.com/docs/auto-healing/auto-healing-insights/). | +| **2. Logs** | Console and network logs, for the step or a wider scope, with a download option | +| **3. Step Settings** | Maximum wait time, prerequisite, and whether the step stops the test case on failure, is ignored in the test case result, or uses visual testing, Smart Execution, or accessibility testing | +| **4. Metadata** | The step's test data and type, its ID and action, whether it uses a password, whether it was migrated, any additional data, and its step type and priority | + + + +Click the close icon to return to the step details. + --- ## **Test Runs in Run Results** @@ -171,9 +190,8 @@ Compare two runs of the same test case side by side. 2. Select a run from the list at the top of either panel.  -Each panel shows the run's duration, date, step count, and failed count, followed by every step with its own duration. -Comparison also spans the steps on either side of the failure, because the step that reports an error is often not the step that caused it. + Each panel shows the run's duration, date, step count, and failed count, followed by every step with its own duration. [[info | **NOTE**:]] | The run list includes **Authoring Run**, which holds the values captured when the test case was authored, along with each earlier run identified by its run ID. Every entry shows its run type and its status as **Passed**, **Failed**, or **Healed**, so you can pick the last run that behaved as expected without opening runs one by one. @@ -184,10 +202,11 @@ Comparison also spans the steps on either side of the failure, because the step Compare one step against the same step in another run. -1. On the **Analysis** tab, in the **Visual Evidence** section, click **Compare Steps**. The step comparison overlay opens. The step under comparison appears in the breadcrumb. - +1. In the **THIS RUN** section of the step, click **Compare Steps**. The step comparison overlay opens, with the step under comparison in the breadcrumb.  + The left panel defaults to **Authoring Time**, and the right panel shows the current run with its date and time and its status, such as **Passed** or **Healed**. To change the comparison, select a different run from the list at the top of either panel. + 2. From the **View** list, select a comparison mode.  @@ -199,7 +218,7 @@ Compare one step against the same step in another run. | **Current Step Only** | The current run's screenshot on its own | | **Element Source** | A diff of the captured page source across the two runs, alongside the locator value for each | - In **Overlay Wipe**, the capture recorded at authoring time and the capture taken during execution load into a single frame separated by a vertical divider. Drag the handle on the divider to wipe between them. Stacking the two makes a shifted control, a modal that did not close, or a reflowed layout visible at once, rather than something to find by comparing two panels. + In **Overlay Wipe**, the capture recorded at authoring time and the capture taken during execution load into a single frame separated by a vertical divider. Drag the handle on the divider to wipe between them.  @@ -216,10 +235,10 @@ Compare one step against the same step in another run. Each entry shows the step number, the step name, its duration, and its status, including **Not Executed** for steps the run never reached. -5. Click the download icon to download the comparison artifacts. +5. Click the download icon on a panel to save that screenshot, or the download icon at the top right to save the full comparison. [[info | **NOTE**:]] -| The baseline panel defaults to **Authoring Time** and is marked **Recorded**. When a step targets no element, the panel reads "No recorded evidence captured for this step". +| When a step targets no element, the left panel reads "No recorded evidence captured for this step". --- @@ -247,7 +266,7 @@ Compare one step against the same step in another run. 3. For **JUnit** format, the report is downloaded instantly as an XML file. -4. For **XLSX** format, a downloadable link of the report is sent to your email. +4. For **XLSX** format, a download link to the report is sent to your email. --- @@ -278,4 +297,4 @@ The **Run Results** page shows the values as they were when the run executed. Re [[info | **NOTE**:]] | Runs that executed before this release fall back to the current name of each item, and show a hyphen (-) where the item has since been deleted. ---- \ No newline at end of file +--- diff --git a/src/pages/docs/runs/debug-test-case-failures.md b/src/pages/docs/runs/debug-test-case-failures.md index c645bebb..fc7bb413 100644 --- a/src/pages/docs/runs/debug-test-case-failures.md +++ b/src/pages/docs/runs/debug-test-case-failures.md @@ -1,6 +1,6 @@ --- title: "Debug Test Case Failures" -metadesc: "How to debug a failed test case in Testsigma | Open the failing step, read its analysis, check its logs, and compare it against an earlier run." +metadesc: "How to debug a failed test case in Testsigma | Open the failing step, read the error and the analyzer verdict, check its logs, and compare it against an earlier run." noindex: false order: 9.71 page_id: "debug-test-case-failures" @@ -15,11 +15,11 @@ contextual_links: name: "Open the Failing Step" url: "#open-the-failing-step" - type: link - name: "Read the Analysis" - url: "#read-the-analysis" + name: "Read the Error and the Analyzer Verdict" + url: "#read-the-error-and-the-analyzer-verdict" - type: link - name: "Explain the Failure" - url: "#explain-the-failure" + name: "When the Failure Started at an Earlier Step" + url: "#when-the-failure-started-at-an-earlier-step" - type: link name: "Check the Logs" url: "#check-the-logs" @@ -42,7 +42,7 @@ contextual_links: --- -Investigate a failed test case from the **Run Results** page. Open the failing step, read its analysis, check its logs, and compare it against an earlier run. Testsigma captures a screenshot, the element details, and the logs for the step. +Investigate a failed test case from the **Run Results** page. Open the failing step, read the error and the analyzer verdict, check its logs, and compare it against an earlier run. Testsigma captures a screenshot, the element details, and the logs for the step. --- @@ -62,79 +62,110 @@ Investigate a failed test case from the **Run Results** page. Open the failing s 3. Select the failed step from the step list on the left. -  +  The list is headed by the step count and the number of failed steps. Each step carries an icon showing whether it passed, failed, or was not executed. Drag the divider between the step list and the details pane to resize either side. - The step header shows the step number, its duration, its status, and the error message. Click **Read more** to see the full message. - - The details pane on the right carries the **Analysis**, **Locators**, **Logs**, **Step Settings**, and **Metadata** tabs. + The step header shows the step status, start time, duration, step number, and step action. [[info | **NOTE**:]] -| The step that reports a failure is not always the step that caused it. Check the steps immediately before it as well. +| The step that reports a failure is not always the step that caused it. For details, refer to [When the Failure Started at an Earlier Step](#when-the-failure-started-at-an-earlier-step). --- -## **Read the Analysis** +## **Read the Error and the Analyzer Verdict** -The **Analysis** tab opens with the error code for the failure, such as **#NO\_SUCH\_ELEMENT**. Below it, the **Visual Evidence** section places the screenshot captured at authoring time next to the one captured in this run. +For a failed step, the **ERROR** and **ANALYZER VERDICT** sections appear below the step header. + - +1. Review the **ERROR** section, which shows the error code and the error message. Click **Read more** for the full message. -Below the two panels, the tab lists the element, element name, run type, action, start time, duration, step level timeout, plan level timeout, error code, error message, test data type, and test data for each side. Use the icons on a screenshot to expand it or download it. + When the page contains iframes or shadow DOM hosts that were not searched, the error message says so. If the element is inside one of them, add a step to switch into it before the failed step. -At the bottom of the tab, the **Page source** section holds the captured HTML for each side. Click the file name to open it, or the download icon to save it. +2. Review the **ANALYZER VERDICT** section below it. Testsigma analyzes the failure automatically when you open the step. -[[info | **NOTE**:]] -| The authoring panel is marked **Recorded**. When no screenshot exists for a side, the panel reads "No screenshot captured". When a step targets no element, it reads "This step targets no element, so nothing was captured when it was authored". + The verdict contains: + + | **Part** | **What it shows** | + |---|---| + | Category | The type of failure, shown as a badge, such as **Flow divergence** | + | Summary | One line naming the failed step and the cause, followed by a short explanation | + | **REASON** | Why the step failed, compared against the passing baseline. Click **Read more** to expand it. | + | Recommendation | What to change, and how to confirm the fix | + | Evidence | What the screenshots show for this run and the baseline | + | Fix | A named fix with a one-line description. When Testsigma can make the change for you, the fix lists options to select and an **Apply fix** button. | + +3. To compare the failed step against the baseline, click **Compare Steps** in the **THIS RUN** section. + +4. To fix the step, do one of the following: + - If the verdict lists fix options, select one and click **Apply fix**. + - Click **Analyze with Agent** at the bottom of the page to open the **Analyzer with Atto** panel. The panel lists suggestions you can apply, lets you ask Atto follow-up questions, and lets you report the failure as a bug. For details, refer to the [documentation on the Analyzer Agent](https://testsigma.com/docs/ai-agents/analyzer/). -**Additional Information**: For a root cause explanation and suggested fixes, click **Analyze with Agent** in the action bar at the bottom of the page. Refer to the [documentation on the Analyzer Agent](https://testsigma.com/docs/ai-agents/analyzer/). +[[info | **NOTE**:]] +| Steps after the failed step are not run when the step is set to stop the test case on failure. These steps show the not executed icon. --- -## **Explain the Failure** +## **When the Failure Started at an Earlier Step** + +A step can fail because the run left the recorded path before it. An earlier step opens a different page from the one recorded, or does not finish loading it, so the element the failed step needs never appears. The failed step reports the error, but the cause lies in the steps before it. + +The verdict category shows which of these happened. -Testsigma can read the step's error, screenshot, page source, and heal attempts, and explain what broke. +| **Category** | **What happened** | **Fix** | +|---|---|---| +| **Flow divergence** | The run reached a different page from the one recorded, so the element the step targets does not exist on the current page. | Re-record the steps from the point where the run left the recorded path, or add the missing navigation before the failed step. | +| **Prerequisite not completed** | An earlier step did not finish as recorded, such as a page that had not finished loading, so the next step could not find its element. | Select the fix option in the verdict, such as **Add a wait for the page to settle**, and click **Apply fix**. | -1. On the **Analysis** tab, find the **Root cause** card. +1. On the **Test Case Results** page, select the failed step. -2. Click **Explain this failure**. +2. Review the **ANALYZER VERDICT** section. +  + - The category badge reads **Flow divergence** or **Prerequisite not completed**. + - The summary names the failed step and what was on the page instead, such as a product page where a search page was expected. + - **REASON** explains how the run left the recorded path, and whether earlier failures caused it or it is an independent deviation. Click **Read more** to expand it. + - The recommendation names the step from which to re-record, or what to add before the failed step. -  +3. Select the steps before the failed step in the step list, and check their screenshots in the **THIS RUN** section to find where the run left the recorded path. - The explanation opens with a one-line summary of what failed, such as "Step failed: no Shop Back to School link on Google homepage", followed by the reasoning and a recommended change. +4. Fix the test case: + - For **Flow divergence**, edit the test case and re-record the steps from the point where the run diverged, or add the navigation step that is missing. + - For **Prerequisite not completed**, select the fix option in the verdict and click **Apply fix**. -3. Click **Read less** to collapse the explanation, or **Read more** to expand it again. +5. Re-run the test to confirm the fix. [[info | **NOTE**:]] -| The **Root cause** card requires Analyzer V2. Without it, the page works as described everywhere else, but the card does not appear and failure analysis is available through **Analyze with Agent** in the action bar instead. +| Not every verdict has an **Apply fix** button. When the fix requires re-recording steps, the verdict names the fix and describes it, but you make the change in the test case. --- ## **Check the Logs** -1. Open the **Logs** tab. +1. In the step header, click the **Logs** icon. The **Logs** panel opens on the right. -2. Select **Selenium**, **Console**, or **Network**. +2. Click **Console** or **Network**. +  -3. Use the scope list to switch between **This step** and the whole run. +3. From the scope list, select **This step** to see only the logs captured during the step. -4. Click **Jump to error** to move to the first error in the log. + The **Network** tab shows the number of failed requests, the time window the requests started in, and a table of requests with their status, method, URL, size, and time. The total request count appears below the table. -5. Click the download icon to save the log. +4. Click the download icon to save the logs. [[info | **NOTE**:]] | Some logs have no step markers. When that happens, the log shows every entry for the run and displays the message "Not scoped to this step: this log has no step markers, so the whole run is shown". -**Additional Information**: Network logs are captured only when they are switched on for the test machine, and they arrive once the lab finishes uploading them. Until then, the **Network** tab reads "No network log was recorded for this run". +>Additional Information
+> +> Network logs are captured only when they are switched on for the test machine, and they appear once the upload from the test machine finishes. Until then, the **Network** tab reads "No network log was recorded for this run". --- ## **Check Whether a Step Was Healed** -When auto-healing resolves a locator mid-run, a banner above the step list reports how many steps were healed, and the **Analysis** tab shows a **Healed this step** card with **Approve as Primary** and **Ignore**. +When auto-healing resolves a locator during the run, the step's status reads **Healed**, and the step shows the auto-heal icon in the step list. A banner above the step list shows how many steps were healed. -A healed step reports **Passed**, because the step completed. For the full workflow, refer to the [documentation on auto-healing insights](https://testsigma.com/docs/auto-healing/auto-healing-insights/). +Select the healed step to see the **Healed this step** card with the locator change and the **Approve as Primary** button. For the full workflow, refer to the [documentation on auto-healing insights](https://testsigma.com/docs/auto-healing/auto-healing-insights/). --- @@ -143,9 +174,7 @@ A healed step reports **Passed**, because the step completed. For the full workf Comparing a failing step against an earlier run shows what changed between them. There are two entry points: - Click **Compare Runs** in the page header to compare the whole test case across two runs, step by step. -- Click **Compare Steps** in the **Visual Evidence** section to compare the current step alone, using the screenshot and element source comparison modes. - -Comparison also spans the steps on either side of the failure, because the step that reports an error is often not the step that caused it. +- Click **Compare Steps** in the **THIS RUN** section to compare the current step alone. For both workflows, refer to the [documentation on test plan run results](https://testsigma.com/docs/reports/runs/drill-down-reports/). @@ -173,11 +202,10 @@ When an element causes a failure, check which other tests depend on it before yo Resolve the failure without leaving the results page. +- If the **ANALYZER VERDICT** section lists fix options, select one and click **Apply fix**. When the run left the recorded path, re-record the affected steps instead. - Click **Analyze with Agent** in the action bar to get the error type, the root cause, and a set of suggestions you can apply as a step update. - Click **Run using Copilot** in the action bar to rerun the test case interactively. -- Use **Update element** to correct a locator yourself. On the current run side of **Visual Evidence**, the **Element** field carries an edit icon for this. -- Use **Relearn step** to re-capture the step. -- Where auto-healing fired, approve or reject the healed element directly on the first screen. +- For a healed step, approve the new locator from the **Healed this step** card. For details, refer to the [documentation on auto-healing insights](https://testsigma.com/docs/auto-healing/auto-healing-insights/). --- diff --git a/src/templates/page.scss b/src/templates/page.scss index 3c3eca3c..99979c7c 100644 --- a/src/templates/page.scss +++ b/src/templates/page.scss @@ -435,6 +435,27 @@ blockquote:has(#prerequisites){ font-weight: 700; padding-left: 2rem; } + +blockquote:has(.additional-info){ + border: 1px solid #C1C7D0; + background: #FFFFFF; + color: #172B4D; + padding: 22px 20px 18px; + margin-top: 30px; + margin-bottom: 30px; + border-radius: 8px; + background-image: url("data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIyMDAiIGhlaWdodD0iMjAwIiB2aWV3Qm94PSIwIDAgMjQgMjQiIGZpbGw9Im5vbmUiPjxjaXJjbGUgY3g9IjEyIiBjeT0iMTIiIHI9IjkiIHN0cm9rZT0iIzQyNTI2RSIgc3Ryb2tlLXdpZHRoPSIyIi8+PHBhdGggZmlsbD0iIzQyNTI2RSIgZD0iTTExIDE3aDJ2LTZoLTJ2NlptMS04cS40MjUgMCAuNzEzLS4yODhUMTMgOHEwLS40MjUtLjI4OC0uNzEzVDEyIDdxLS40MjUgMC0uNzEzLjI4OFQxMSA4cTAgLjQyNS4yODguNzEzVDEyIDlaIi8+PC9zdmc+"); + background-size: 1.5rem; + background-position: 1.5rem 2.1rem; + background-repeat: no-repeat; + > * { + padding-left: 2rem; + } +} +.additional-info{ + font-size: 22px; + font-weight: 700; +} .edit-button { padding-top: 40px; @media (min-width:992px) {