Skip to content

feat(loop): richer Langfuse signals — OpenRouter cost/cache + optional input preview - #44

Merged
angel-manuel merged 1 commit into
masterfrom
feat/loop-langfuse-openrouter-cost-cache
Apr 17, 2026
Merged

angel-manuel merged 1 commit into
masterfrom
feat/loop-langfuse-openrouter-cost-cache

Conversation

@angel-manuel

Copy link
Copy Markdown
Contributor

Summary

  • Parse OpenRouter's richer usage shape off the SSE wire: top-level cost, plus cache tokens in both Anthropic-flat (cache_read_input_tokens, cache_creation_input_tokens) and OpenAI-nested (prompt_tokens_details.cached_tokens) forms. A custom Deserialize on Usage collapses both into the flat public shape (cache_read_tokens = max of the two sources). Fields flow into the Langfuse generation span (metadata.cost_usd, cache_read_tokens, cache_creation_tokens) and into trace.finalize(...total_cost...) — previously all hardcoded to 0.
  • Add an opt-in LANGFUSE_CAPTURE_INPUT flag that attaches a redacted chat-log snapshot (ported from sister-repo overfolder/agent-runner's build_context_snapshot) to the generation input. Adapted for overloop's Content enum: non-text blocks render as [image]/[audio]/[media] placeholders so no base64/data-URI payload leaks. Off by default — the default posture keeps prompt content local.
  • No breaking changes on wire, REST, or CLI surfaces. Usage public shape stays flat (added fields default to 0 on absent input).

Test plan

  • cargo test -p overloop — 184 tests pass (121 unit + integration suites)
  • cargo clippy --workspace --all-targets -- -D warnings clean
  • cargo fmt --all
  • 4 new Usage deser tests cover Anthropic-flat, OpenAI-nested, both-absent, both-present-picks-max
  • 7 new preview.rs tests cover detail-window, skip-count summary, per-role truncation, multimodal placeholder rendering (no base64 leak), char-boundary safety
  • test_langfuse.rs success-generation test now asserts metadata.cost_usd round-trip; new generation_input_preview_string_roundtrips covers the string-preview branch
  • LANGFUSE_CAPTURE_INPUT toggle parses 1/true/yes/TRUE as truthy, everything else as false
  • Manual: run with LANGFUSE_CAPTURE_INPUT=1 + real creds, eyeball Langfuse UI for [USER]:/[AGENT]: snapshot + non-zero metadata.cost_usd

🤖 Generated with Claude Code

…put preview

Parse OpenRouter's richer `usage` shape off the SSE wire: top-level
`cost`, plus cache tokens in both Anthropic-flat
(`cache_read_input_tokens`, `cache_creation_input_tokens`) and
OpenAI-nested (`prompt_tokens_details.cached_tokens`) forms. A custom
`Deserialize` on `Usage` collapses both into the flat public shape
(cache_read = max of the two sources). Previously-dropped data now
flows into the Langfuse generation span (`metadata.cost_usd`,
`cache_read_tokens`, `cache_creation_tokens`) and into
`trace.finalize(...total_cost...)`.

Add an opt-in `LANGFUSE_CAPTURE_INPUT` flag that attaches a redacted
chat-log snapshot (ported from overfolder's `build_context_snapshot`)
to the generation `input` field. Adapted for overloop's `Content`
enum: non-text blocks render as `[image]`/`[audio]`/`[media]`
placeholders so no base64/data-URI payload leaks. Off by default —
the default posture keeps prompt content local.

- New `loop/src/observability/preview.rs` + 7 unit tests.
- 4 new deser tests covering Anthropic-flat, OpenAI-nested, both
  absent, and both-present-picks-max.
- `test_langfuse.rs`: existing success test now asserts cost_usd
  round-trip; new test covers string-preview round-trip.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@sentry

sentry Bot commented Apr 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.93487% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.81%. Comparing base (8b1a9d0) to head (d024af0).

Files with missing lines Patch % Lines
loop/src/observability/preview.rs 95.94% 6 Missing ⚠️
loop/src/agentic_loop.rs 96.00% 1 Missing ⚠️
loop/src/main.rs 0.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #44      +/-   ##
==========================================
+ Coverage   91.62%   91.81%   +0.19%     
==========================================
  Files          49       50       +1     
  Lines        6432     6685     +253     
==========================================
+ Hits         5893     6138     +245     
- Misses        539      547       +8     
Files with missing lines Coverage Δ
loop/src/acp.rs 91.12% <100.00%> (+0.05%) ⬆️
loop/src/config.rs 100.00% <100.00%> (ø)
loop/src/llm/types.rs 99.58% <100.00%> (+0.13%) ⬆️
loop/src/observability/langfuse.rs 99.60% <100.00%> (+<0.01%) ⬆️
loop/src/agentic_loop.rs 92.37% <96.00%> (+0.08%) ⬆️
loop/src/main.rs 0.00% <0.00%> (ø)
loop/src/observability/preview.rs 95.94% <95.94%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@angel-manuel
angel-manuel merged commit 349b5f5 into master Apr 17, 2026
8 checks passed
@angel-manuel
angel-manuel deleted the feat/loop-langfuse-openrouter-cost-cache branch April 17, 2026 11:08
angel-manuel added a commit that referenced this pull request Apr 18, 2026
Extract AgentRegistryProvider, MessageQueueProvider, and
StreamBrokerProvider traits from the concrete in-memory
implementations. Add a Redis/Valkey-backed implementation behind
the `redis` cargo feature that enables multiple overacp-server
instances to share routing state via:

- Ownership leases (SET EX + Lua CAS heartbeat) for tunnel pinning
- XREADGROUP inbox streams for cross-instance message delivery
  (produce-unconditionally pattern)
- XAUTOCLAIM + DLQ for orphan recovery
- Redis pub/sub for SSE fan-out and control signals (takeover,
  force-disconnect)

Design aligned with overfolder-on-overacp vision doc: tunnel-holder
is owner, no sticky LB required, lock-symmetric across deployment
modes.

Includes two-instance smoke test (tests/smoke-ha.sh) and design
doc (docs/design/ha.md).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
angel-manuel added a commit that referenced this pull request Apr 18, 2026
Extract AgentRegistryProvider, MessageQueueProvider, and
StreamBrokerProvider traits from the concrete in-memory
implementations. Add a Redis/Valkey-backed implementation behind
the `redis` cargo feature that enables multiple overacp-server
instances to share routing state via:

- Ownership leases (SET EX + Lua CAS heartbeat) for tunnel pinning
- XREADGROUP inbox streams for cross-instance message delivery
  (produce-unconditionally pattern)
- XAUTOCLAIM + DLQ for orphan recovery
- Redis pub/sub for SSE fan-out and control signals (takeover,
  force-disconnect)

Design aligned with overfolder-on-overacp vision doc: tunnel-holder
is owner, no sticky LB required, lock-symmetric across deployment
modes.

Includes two-instance smoke test (tests/smoke-ha.sh) and design
doc (docs/design/ha.md).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
angel-manuel added a commit that referenced this pull request Apr 18, 2026
* feat(server): optional Redis/Valkey backend for multi-instance HA (#44)

Extract AgentRegistryProvider, MessageQueueProvider, and
StreamBrokerProvider traits from the concrete in-memory
implementations. Add a Redis/Valkey-backed implementation behind
the `redis` cargo feature that enables multiple overacp-server
instances to share routing state via:

- Ownership leases (SET EX + Lua CAS heartbeat) for tunnel pinning
- XREADGROUP inbox streams for cross-instance message delivery
  (produce-unconditionally pattern)
- XAUTOCLAIM + DLQ for orphan recovery
- Redis pub/sub for SSE fan-out and control signals (takeover,
  force-disconnect)

Design aligned with overfolder-on-overacp vision doc: tunnel-holder
is owner, no sticky LB required, lock-symmetric across deployment
modes.

Includes two-instance smoke test (tests/smoke-ha.sh) and design
doc (docs/design/ha.md).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(tests): skip smoke-ha when binary lacks redis feature

The HA smoke test requires the server binary to be built with
--features redis. When compiled without the feature,
OVERACP_REDIS_URL is silently ignored and both instances run
in-memory mode, causing cross-instance assertions to fail.

Add a pre-flight check that greps the binary for the redis
enablement log string and exits 0 (skip) if absent.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(redis_backend): address PR review findings

- inbox: fix XAUTOCLAIM deserialization for Redis 7.0+ (3-element
  response) and process pending entries before new messages so
  auto-claimed messages are actually delivered
- queue: make drain atomic via Lua script to prevent XRANGE+DEL
  race where a concurrent push loses a message
- registry: make deliver atomic via Lua script to prevent
  EXISTS+XADD race where agent disconnect between the two commands
  orphans the message

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: exclude redis_backend from codecov patch coverage

The Redis backend module requires a live Redis/Valkey instance for
integration testing and cannot be exercised by cargo test alone.
Exclude it from the patch coverage target to prevent informational
codecov checks from blocking PRs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant