Bump tracelens-analysis-orchestrator to a2575ae

skills/tracelens-analysis-orchestrator/.federated.json

@@ -2,9 +2,9 @@
   "source": "amd-agi-tracelens",
   "repo": "AMD-AGI/TraceLens",
   "ref": "main",
-  "commit": "c5da90ee7d59e49bf75e8602f0211da1f6dcd8b3",
+  "commit": "a2575ae57960f7c88cd7c5bbbfb51647b1492d18",
   "path": "TraceLens/Agent/Analysis/skills/analysis-orchestrator",
   "license": "MIT",
-  "content_hash": "0e829f9bf572f9dba30436dea555397bca6f60e3501b4be2867ee8803e631850",
-  "imported_at": "2026-10-08T06:52:06Z"
+  "content_hash": "bad41c1959d80227a8c3b54e330ae54e99de498b9e667bbbb9a335e5995c6f4d",
+  "imported_at": "2026-10-28T06:35:54Z"
 }

skills/tracelens-analysis-orchestrator/SKILL.md

@@ -5,8 +5,8 @@ description: >-
   reports, prepares category data, runs system-level and compute-kernel subagents in
   parallel, validates outputs, and writes a prioritized stakeholder report (analysis.md).
   Use when the user asks to follow the analysis orchestrator, run the agentic analysis
-  workflow, analyze a trace, compare two traces, or mentions standalone or comparative
-  TraceLens analysis.
+  workflow, run it from config <run.yaml>, analyze a trace, compare two traces, or
+  mentions standalone or comparative TraceLens analysis.
 license: MIT
 ---
 
@@ -22,13 +22,13 @@ Coordinate **system-level** analysis (CPU/idle, kernel fusion, multi-kernel / co
 
 ## Full procedure
 
-Follow **[reference.md](reference.md)** for every step (user prompts, `<prefix>` / `{CMD}` usage, CLI commands, subagent launch text, validation, report `tee` order, plot embedding, and trace diagnostics).
+Follow **[reference.md](reference.md)** for every step (user prompts, `<prefix>` / `{CMD}` usage, CLI commands, subagent launch text, validation, report `tee` order, and plot embedding).
 
 ## Workflow index
 
 ```
-0. Query User Inputs (Platform, Trace Path(s), Analysis Mode, Environment Setup)
-1. Generate Performance Report (branches on analysis mode: training vs inference then, comparison scope)
+0. Query User Inputs (interactive answers or a run YAML, Environment Setup) → cache/run_config.yaml
+1. Generate Performance Report (one wrapper call on cache/run_config.yaml; semantic pre-processing first)
 2. Trace-Quality Gate → Deterministic fallback analysis
 3-6. Prepare Category Data (GPU Util, Top Ops, Tree Data, Multi-Kernel Data, Category Filtering)
 7. System-Level Analysis (PARALLEL) → system_findings/

skills/tracelens-analysis-orchestrator/agents/cpu-idle-analyzer.md

@@ -38,7 +38,7 @@ When invoked by the orchestrator, you will receive the following context:
 ## Error Handling
 
 **If category data files are missing:**
-1. Read gpu_utilization directly from category_data/category_manifest.json
+1. Read gpu_utilization from metadata/cpu_idle_metadata.json (focus job only on multi-trace)
 2. Provide analysis based on available data
 3. Note limitations in findings
 

skills/tracelens-analysis-orchestrator/agents/model-identification-agent.md

@@ -22,7 +22,6 @@ When invoked by the orchestrator, you will receive the following context:
 
 **Required context provided by orchestrator:**
 - `output_dir`: Base analysis output directory
-- `comparison_scope`: `standalone` or `comparative`
 - `prefix`: Command prefix from `<output_dir>/cache/cmd_prefix.txt` — contains a template with `{CMD}` placeholder; substitute `{CMD}` with the actual command
 
 **Input (produced by script in Step 1):**
@@ -57,7 +56,7 @@ Execute the Python script to extract the **name**, **Input type**, and **Input D
 <prefix> python3 -c "
 import sys
 from TraceLens.Agent.Analysis.utils.report_utils import prepare_model_identification_data
-if not prepare_model_identification_data('<output_dir>', '<comparison_scope>'):
+if not prepare_model_identification_data('<output_dir>'):
     sys.exit(1)
 "
 ```

skills/tracelens-analysis-orchestrator/agents/multi-kernel-analyzer.md

@@ -29,10 +29,8 @@ When invoked by the orchestrator, you will receive the following context:
 - `output_dir`: Base analysis output directory
 - `prefix`: Command prefix from `<output_dir>/cache/cmd_prefix.txt` — contains a template with `{CMD}` placeholder; substitute `{CMD}` with the actual command
 
-**Input files (pre-computed by orchestrator):**
-1. `<output_dir>/category_data/multi_kernel_data.json` - Pre-computed memcpy/communication/overlap data
-2. `<output_dir>/metadata/multi_kernel_metadata.json` - Platform specs and GPU utilization
-3. `<output_dir>/category_data/category_manifest.json` - Contains gpu_utilization metrics
+**Input file (pre-computed by orchestrator):**
+- `<output_dir>/category_data/multi_kernel_data.json` - Pre-computed memcpy/communication/overlap data
 
 **Output file you must write:**
 - `<output_dir>/system_findings/multi_kernel_findings.md`
@@ -41,11 +39,6 @@ When invoked by the orchestrator, you will receive the following context:
 
 ## Error Handling
 
-**If multi_kernel_data.json is missing:**
-1. Read gpu_utilization from category_data/category_manifest.json
-2. Report based on exposed_memcpy_time_percent and exposed_comm_time_percent
-3. Note limitations in findings
-
 **If analysis script fails:**
 1. Write a findings file with Status: ERROR
 2. Include the error message and traceback

skills/tracelens-analysis-orchestrator/evals/EVAL_RUBRICS.md

@@ -327,14 +327,14 @@ For each P-item (`### ...P{N}:`) under `## Compute Kernel Optimizations`:
 
 ### marker\_eval\_3: Detail Estimate Markers
 
-**Sub-indices:** `marker_eval_3_P{N}` (one per compute P-item in Detailed Analysis)
+**Sub-indices:** `marker_eval_3_P{N}` (one per compute P-item in Detailed Analysis) and `marker_eval_3_fusion_P{N}` (one per P-item under `### Kernel Fusion Insights`)
 
-For each compute P-item section under `## Detailed Analysis`:
+For each compute P-item section under `## Detailed Analysis`, and each fusion P-item under `### Kernel Fusion Insights`:
 
 | Check | Pass Criteria |
 |-------|---------------|
 | `kind=detail_estimate` marker or sentinel | Either a `<!-- impact-begin kind=detail_estimate ... -->` / `<!-- impact-end -->` pair exists, or the text "not quantifiable from trace data" is present |
-| Required attributes | If marker present, it contains `low` and `high` attributes |
+| Required attributes | If marker present, it contains `low`, `mid` and `high` attributes (compute and fusion) |
 | Pairing | Every `impact-begin` has a matching `impact-end` |
 
 **Pass:** Marker with correct attributes present, or not-quantifiable sentinel present.

skills/tracelens-analysis-orchestrator/evals/eval_utils/workflow_scripted_evals.py

@@ -290,7 +290,7 @@ def _check_plot(output_dir: str) -> tuple[str, str]:
         "Unified perf. report exists",
         _check_unified_perf_report,
         "pipeline",
-        "Re-run TraceLens_generate_perf_report_pytorch (Step 1)",
+        "Re-run TraceLens_generate_perf_report_from_config (Step 1)",
         True,
     ),
     (
@@ -1067,6 +1067,13 @@ def _check_marker_p_items(output_dir, comparison_scope=None):
     return rows
 
 
+def _p_blocks(tier, subsection, matches):
+    """(tier, P number, block text) for each P-item header in a Detailed Analysis subsection."""
+    for i, match in enumerate(matches):
+        end = matches[i + 1].start() if i + 1 < len(matches) else len(subsection)
+        yield tier, int(match.group(1)), subsection[match.start() : end]
+
+
 def _check_marker_detail_estimates(output_dir, comparison_scope=None):
     """Marker eval 3: kind=detail_estimate markers in Detailed Analysis."""
     content = _read_report(output_dir)
@@ -1089,7 +1096,7 @@ def _check_marker_detail_estimates(output_dir, comparison_scope=None):
         ]
 
     m = re.search(
-        r"^### Compute Kernel Insights[^\n]*\n(.*?)(?=^### (?!.*P\d)|^## |\Z)",
+        r"^### Compute Kernel Insights[^\n]*\n(.*?)(?=^### (?![^\n]*P\d)|^## |\Z)",
         detailed_section,
         re.MULTILINE | re.DOTALL,
     )
@@ -1099,7 +1106,15 @@ def _check_marker_detail_estimates(output_dir, comparison_scope=None):
     if not matches:
         matches = list(_DETAIL_P_HEADER_FALLBACK_RE.finditer(compute_subsection))
 
-    if not matches:
+    blocks = list(_p_blocks("compute", compute_subsection, matches))
+    fusion_subsection = _extract_detailed_analysis_subsection(
+        content, "### Kernel Fusion Insights"
+    )
+    if fusion_subsection:
+        fusion_matches = list(_DETAIL_P_HEADER_RE.finditer(fusion_subsection))
+        blocks += _p_blocks("fusion", fusion_subsection, fusion_matches)
+
+    if not blocks:
         intentionally_empty = (
             "No compute kernel optimization opportunities identified"
             in compute_subsection
@@ -1124,13 +1139,8 @@ def _check_marker_detail_estimates(output_dir, comparison_scope=None):
         ]
 
     rows = []
-    for i, match in enumerate(matches):
-        pnum = int(match.group(1))
-        start = match.start()
-        end = (
-            matches[i + 1].start() if i + 1 < len(matches) else len(compute_subsection)
-        )
-        block = compute_subsection[start:end]
+    for tier, pnum, block in blocks:
+        pname = f"P{pnum}" if tier == "compute" else f"{tier} P{pnum}"
 
         errors = []
         detail_markers = []
@@ -1150,8 +1160,7 @@ def _check_marker_detail_estimates(output_dir, comparison_scope=None):
         elif detail_markers:
             for inner in detail_markers:
                 attrs = _marker_attrs_from_inner(inner)
-                required = ("low", "high")
-                missing = [a for a in required if a not in attrs]
+                missing = [a for a in ("low", "mid", "high") if a not in attrs]
                 if missing:
                     errors.append(
                         f"kind=detail_estimate marker missing attributes: "
@@ -1167,13 +1176,13 @@ def _check_marker_detail_estimates(output_dir, comparison_scope=None):
         details = "; ".join(errors) if errors else ""
         rows.append(
             _make_marker_row(
-                f"marker_eval_3_P{pnum}",
-                f"Detailed Analysis P{pnum} estimate marker (kind=detail_estimate)",
+                f"marker_eval_3_{pname.replace(' ', '_')}",
+                f"Detailed Analysis {pname} estimate marker (kind=detail_estimate)",
                 result,
                 details,
                 "template" if result == "FAIL" else "",
                 (
-                    f"Fix kind=detail_estimate marker for P{pnum} in Detailed Analysis"
+                    f"Fix kind=detail_estimate marker for {pname} in Detailed Analysis"
                     if result == "FAIL"
                     else ""
                 ),

skills/tracelens-analysis-orchestrator/evals/partial_tests/README.md

@@ -44,7 +44,7 @@ For each model (DeepSeek-R1, Qwen3-30B-A3B) the fixture ships:
 The runner invokes the **semantic-comparison workflow** (the no-capture,
 name-first + coherence bucketing method) on the DECODE pair, running only
 through its "Generate TraceDiff Output" step — equivalent to the main
-orchestrator through Step 1.S. That produces
+orchestrator through Step 1 pre-processing. That produces
 `analysis_output/tracediff_output/diff_stats.csv`, whose LCA partition is then
 compared to gold.
 

skills/tracelens-analysis-orchestrator/evals/partial_tests/run_partial_tests.sh

@@ -15,7 +15,7 @@ set -uo pipefail
 # far enough to produce the artifact a scripted eval scores against a
 # pre-baked reference. The first such test is semantic_purity, which runs the
 # semantic-comparison workflow through its "Generate TraceDiff Output" step
-# (== main orchestrator through Step 1.S) to produce
+# (== main orchestrator through Step 1 pre-processing) to produce
 #   <OUTPUT_DIR>/tracediff_output/diff_stats.csv
 # and scores its LCA-partition purity against the committed gold.
 #