fix: resolve routing tier for fallback and affinity-reuse decisions

[thossullivan]

Closes #276.

Why

A routing decision made by the cascade fallback, or served from an affinity pin, logs without its routing tier while a judged decision to the same target logs with one:

fall-through selected ok-strong (confidence 1.000); routing tier: strong
fall-through selected ft-strong (confidence 0.000)

#338 preserves routing-tier and fallback details in the decision reasoning and #413 moved that reasoning into the log, so a decision that omits the tier loses detail the log is meant to carry.

What

with_routing_tier asks only the deciding classifier for the tier. Neither DefaultTarget nor AffinityRouter implements Classifier::routing_tier, so the trait default returns None and the tier is dropped. This resolves the tier across the cascade when the deciding classifier defines none.

How

One lookup in crates/libsy/src/algorithms/fall_through.rs: fall back to the first classifier in the cascade that defines a tier for the selected target. TaskClassifier sits in the cascade in both cases, so the fallback and affinity paths resolve the same weak/strong labels a judged decision would produce.

Test Plan

• cargo fmt --all --check
• cargo clippy --workspace --all-targets --all-features -- -D warnings
• cargo test -p switchyard-protocol -p switchyard-libsy -p switchyard-llm-client -p switchyard-server
• git diff --check
• uv run ruff check .
• uv run mypy switchyard
• uv run pytest tests/ -m "not integration"

Two tests in crates/libsy-llm-client/tests/observability.rs cover the fallback and affinity-reuse paths. Both fail without the fall_through.rs change.

Note on this branch

This was rebased at maintainer request and became a rewrite rather than a replay. The original commits patched FallThroughDecision.tier and asserted a tier attribute on switchyard.requests; #338 removed both, so there was nothing left for them to apply to. The defect they described survives in the decision log, and this branch addresses it there.

[coderabbitai]

Walkthrough

FallThrough::route now finds a tier from the classifier cascade when the deciding classifier lacks one. Observability tests cover unavailable-judge fallback and session-affinity tier labels.

Changes

Tier Routing and Observability

Layer / File(s)	Summary
Cascade tier resolution crates/libsy/src/algorithms/fall_through.rs	FallThrough::route checks the classifier cascade for a tier associated with the resolved target when the deciding classifier provides none.
Fallback and affinity observability crates/libsy/tests/observability.rs	Added a client stub and tests for classifier timeouts, strong-tier fallback, and weak-tier session-affinity metrics.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Poem

A rabbit watched the tiers align,
Through cascading paths they shine.
When judges pause, the route goes on,
Strong and weak labels both stay drawn.
Two requests share one metric trail,
Hop, hop—fallbacks never fail!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: resolving routing tiers for fallback and affinity-reuse decisions.

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/libsy/tests/observability.rs`:
- Around line 1233-1245: Extend the observability test around the two-request
loop to track non-routed judge calls in ClassifierClient, then assert that the
second request does not increase that count. Keep the existing routing_tier
assertions, and use the recorded call count to verify the second request reuses
the session-affinity decision path rather than invoking the judge again.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d2d5e19e-2405-44c7-8282-6f94390ca9cb

📥 Commits

Reviewing files that changed from the base of the PR and between a9c04b3 and 50c8be2.

📒 Files selected for processing (2)

• crates/libsy/src/algorithms/fall_through.rs
• crates/libsy/tests/observability.rs

[ayushag-nv]

@thossullivan Thanks for putting up the PR. Can you put up a Issue and link a PR to that.

[thossullivan]

Filed #276 for this and linked it in the description. Thanks for the look!

[ayushag-nv]

@thossullivan Thanks for putting up the PR. Can you please rebase over main once

[thossullivan]

@ayushag-nv sorry for the delay, bad storms knocked out our power for about a week!

Rebased. It became a rewrite rather than a replay.

The original commits patched FallThroughDecision.tier and asserted a tier attribute on switchyard.requests. #338 removed both, so there was nothing left to replay. You were right that the tier field is gone (like you mentioned in the issue).

The gap survives in the decision log, where #338 moved the detail and #413 turned it into a log message. with_routing_tier asks only the deciding classifier, and neither DefaultTarget nor AffinityRouter implements routing_tier, so on 61ee1b3:

​ fall-through selected ok-strong (confidence 1.000); routing tier: strong fall-through selected ft-strong (confidence 0.000) ​

The branch is now one commit that resolves the tier across the cascade when the deciding classifier defines none, plus two tests covering the fallback and affinity-reuse paths that fail without it.

If you would rather the tier stay out of this path entirely, I am still happy to close or work on the cleanup instead!

[nachiketb-nvidia]

looks good! there's a conflict i see

quick rebase oughtta fix it

Thanks!

Sorry about the rebases, we were doing a core library refactor, making it nice for integrators

[thossullivan]

@nachiketb-nvidia No problem at all with the rebases! Love the progress on the repo :)

Rebased onto main and reduced to the single test as requested. The refactor actually made this even simpler.

With the tier now a structured field on the Model selected event rather than text appended to the message, the change is one lookup (i.e., ask the cascade when the deciding classifier defines none). The old tests asserted on message text that no longer exists, so they needed rewriting regardless.