Skip to content

feat(evalboard): show conversation transcript for simulation tasks#23

Open
CarlesUIPath wants to merge 1 commit into
mainfrom
evalboard/simulation-conversation-log
Open

feat(evalboard): show conversation transcript for simulation tasks#23
CarlesUIPath wants to merge 1 commit into
mainfrom
evalboard/simulation-conversation-log

Conversation

@CarlesUIPath

Copy link
Copy Markdown

Summary

Simulation runs already write a clean user<->agent transcript to conversation.log (orchestrator._log_conversation), but the evalboard never surfaced it. Add a Conversation section to the task detail page that renders the dialog as chat bubbles, below the Success criteria section. Gated on parsed turns, so single-shot tasks (which never write conversation.log) are unaffected.

Changes

  • lib/runs.ts: readConversationLog (mirror of readLogTail) + a conversation parser returning ConversationTurn[].
  • _sections.tsx: ConversationSection renders turns as bubbles (user right/gray, agent left/blue).
  • page.tsx: fetch + parse, render below Success criteria. It only renders the conversation if "conversation.log" exists in the runs/ directory.

Tests

Tests mirror repo conventions: unit tests for parseConversation (alongside parseMessages) and a render test for ConversationSection (alongside message-timeline). readConversationLog is left untested, matching its untested twin readLogTail.

Ui Example

When running a simulation task, a "conversation.log" text is generated. This is rendered in the run page after the Success criteria" as follows.
image

Simulation runs already write a clean user<->agent transcript to
conversation.log (orchestrator._log_conversation), but the evalboard
never surfaced it. Add a Conversation section to the task detail page
that renders the dialog as chat bubbles, below the Success criteria
section. Gated on parsed turns, so single-shot tasks (which never
write conversation.log) are unaffected.

- lib/runs.ts: readConversationLog (twin of readLogTail) + a pure
  parseConversation parser returning ConversationTurn[].
- _sections.tsx: ConversationSection renders turns as bubbles
  (user right/gray, agent left/blue).
- page.tsx: fetch + parse, render below Success criteria.

Tests mirror repo conventions: unit tests for parseConversation
(alongside parseMessages) and a render test for ConversationSection
(alongside message-timeline). readConversationLog is left untested,
matching its untested twin readLogTail.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 14, 2026

Copy link
Copy Markdown

Claude finished @CarlesUIPath's task in 1m 33s —— View job


Code Review in Progress

Todo List:

  • Read .github/code_review.md for review standards
  • Review CLAUDE.md for project conventions
  • Get full diff with git diff origin/main...HEAD
  • Read and analyze each changed file in full context
  • Perform cross-file consistency checks
  • Analyze what might be missing
  • Format and post the review

@bai-uipath bai-uipath left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve. Clean, well-scoped feature — parser, presentation, and page wiring live in the right seams, and it's verified against the actual writer. All the design calls here are the ones I'd make.

  • Parser matches the writer exactly. parseConversation targets the format _log_conversation emits (orchestrator.py:1843) — header shape, the metadata separator (both U+2014), and the header\n{body}\n\n framing all line up. The always-present opener metadata (pinned initial_prompt / simulator-generated opener) parses correctly.
  • The no-simulation default is right, and better than described. Gating on parsed-turns (conversation.length > 0) rather than file existence means single-shot tasks render nothing, and empty-file / no-header-garbage / read-error cases all collapse to [] and hide cleanly — no crash, no empty section.
  • Dialogue-only rendering is the correct split. Tool-use and thinking are stripped on the Python side before the log is written, so bubbles show the clean final utterance; the per-turn caption surfaces N tool calls + cost, and the full trajectory already lives in the message-timeline / CostExplorerSection below. Not redundant with the timeline.
  • Tests mirror repo conventions. Parser unit tests beside parseMessages, render test beside message-timeline; two-turn / empty / metadata / multi-line all covered. Appropriately scoped.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants