feat(api): flag re-executed investigation lanes; correct the TTFT comment - #639
Merged
Merged
Conversation
…ment A lane step whose result is lost to a retry boundary is re-run by the workflow engine on its own schedule - minutes later, with no failure recorded anywhere. The re-execution's spans were indistinguishable from new work, so a session view showed a silent multi-minute hole followed by what looked like fresh lanes. The step now reads the lane row before claiming it (past "queued" means a prior execution ran) and stamps maple.hypothesis.rerun on the lane's root span. Also corrects the annotateModelCallTiming comment: the openai-chat protocol emits no step-start, and role-only chunks, keep-alives and hidden reasoning produce no events, so TTFT is measured to the first token-bearing frame - which on reasoning models spans the silent reasoning phase. The behavior is intended; the comment claimed first-byte semantics it never had.
JeremyFunk
force-pushed
the
feat/investigation-lane-rerun-visibility
branch
from
August 27, 2026 06:53
9787d41 to
6f1d67d
Compare
JeremyFunk
had a problem deploying
to
pr-preview
August 27, 2026 11:58 — with 
GitHub Actions
Failure
GitHub Actions
Failure
🍁 Maple PR previewWarning Preview cleanup could not be confirmed. The Alchemy teardown outcome was Final commit |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Investigating a TTFT/idle anomaly on an incident-investigation session surfaced two things worth landing:
1. Re-executed hypothesis lanes are now stamped
maple.hypothesis.rerun.A lane step whose result is lost to a retry boundary is re-run by the workflow engine on its own schedule — observed in production as a 4m23s silent hole followed by two lanes re-running with identical hypothesis ids, every span
Ok, no failure recorded anywhere. The step now reads the lane row before claiming it (status pastqueuedmeans a prior execution already ran) and passesrerunthrough to the lane agent, which stamps it on theinvestigation.hypothesisroot span. Session views can then label the second wave as a re-run instead of anonymous idle-then-new-work.Costs one
SELECTper lane per attempt.2. The
annotateModelCallTimingcomment claimed first-byte semantics TTFT never had.The openai-chat protocol emits no
step-start(only open-responses does), and role-only chunks, keep-alive comments and hidden reasoning produce noLLMEvents — sogen_ai.response.time_to_first_chunkmeasures to the first token-bearing frame. On reasoning models that includes the whole silent reasoning phase, which is why tool-call-only steps show TTFT near the span's full duration. The behavior is intended (first token to arrive client-side); the comment now says what is actually measured.Verification
hypothesis-*step callback (modeling the engine re-execution) and asserts the second execution seesrerun: truewhile the first seesfalse.InvestigationFanoutWorkflow.run.test.ts: 16/16 pass.turbo typecheck --filter=@maple/api: clean.Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.