apollo_dashboard: exclude observers from strk_to_usd_rate_frozen alert

[matanl-starkware]

Problem

strk_to_usd_rate_frozen fires permanently on observer nodes (seen on apollo-sepolia-alpha-0, the one sepolia-alpha node whose validator_id is outside the committee).

Root cause: oracle_rate_frozen_alert hardcodes ObserverApplicability::Applicable for both exchange-rate oracles, but the two oracles are queried on different consensus paths:

• eth_to_strk is queried on the proposal validation path (validate_proposal.rs → get_l1_prices_in_fri_and_wei), so every node — observers included — keeps its gauge moving. Applicable is correct.
• strk_to_usd is queried only on the proposer build path (build_proposal → compute_proposer_fee_proposal → resolve_fee_target), for the SNIP-35 fee proposal. Validation bounds-checks the proposer's fee_proposal_fri against the local median without an oracle read. An observer never proposes, so its snip35_strk_usd_rate gauge is registered at 0 (ExchangeRateOracleClient::new registers metrics unconditionally) and never updated — present-but-flat by construction, so sum(changes(...[1h])) < 1 is permanently true.

Not an outage: on the observer, snip35_strk_usd_success_count and _error_count are both flat 0 (zero attempts), while all validators read the rate normally via round-robin proposership.

Fix

Parameterize oracle_rate_frozen_alert with ObserverApplicability and pass NotApplicable for strk_to_usd (Applicable for eth_to_strk, unchanged behavior). This matches the oracle success/error-count alerts, which already exclude observers. Regenerated dev_grafana_alerts.json; the only expression change is strk_to_usd_rate_frozen gaining the and on() (is_observer == 0) clause.

Verification

• SEED=0 cargo test -p apollo_dashboard — 16 passed (includes the generated-JSON drift test)
• cargo clippy -p apollo_dashboard --all-targets -- -D warnings — clean
• scripts/rust_fmt.sh — clean

🤖 Generated with Claude Code

[cursor]

PR Summary

Low Risk
Alert-definition-only change; validator monitoring behavior improves with no runtime node logic changes.

Overview
Stops false-positive strk_to_usd_rate_frozen alerts on observer nodes by scoping that frozen-rate check to non-observers only.

oracle_rate_frozen_alert now takes ObserverApplicability: eth_to_strk stays Applicable (rate is read on proposal validation on every node); strk_to_usd uses NotApplicable because the SNIP-35 rate is only fetched when building a proposal, so observers keep a flat gauge and looked “frozen” forever. Generated dev_grafana_alerts.json adds and on() (is_observer == 0) to the strk_to_usd expression only; eth_to_strk_rate_frozen is unchanged.

^{Reviewed by Cursor Bugbot for commit 51204e2. Bugbot is set up for automated code reviews on this repo. Configure here.}

[reviewable-StarkWare]

This change is 

[asaf-sw]

@asaf-sw reviewed 2 files and all commit messages.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on matanl-starkware).

[asaf-sw]

@asaf-sw+AGNT made 1 comment.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on matanl-starkware).

---

crates/apollo_dashboard/src/alert_scenarios/l1_gas_prices.rs line 197 at r1 (raw file):

/// "frozen"); only a present-but-flat gauge trips this.
///
/// Observer applicability depends on which consensus path queries the oracle. Eth to Strk is

Suggestion, take it or leave it, your call.

These lines explain the two specific oracles, but they sit on the generic helper, which after this change takes applicability from the caller and does not care which oracle it is given. The same explanation is also in the PR description, almost word for word.

One option is to leave the helper's existing doc as it was and put one line at each call site, which have none today:

/// Eth to Strk is queried on the validate path too, so every node's gauge moves.
pub(crate) fn get_eth_to_strk_rate_frozen_alert() -> Alert {

/// Strk to Usd is queried only when building a proposal; observers never propose, so their gauge
/// stays flat and the alert excludes them.
pub(crate) fn get_strk_to_usd_rate_frozen_alert() -> Alert {

That puts each fact next to the code it applies to, and drops the "must exclude them" and "by construction" phrasing, which argues for the change rather than saying what the code does.

[asaf-sw]

crates/apollo_dashboard/src/alert_scenarios/l1_gas_prices.rs line 197 at r1 (raw file):

Previously, asaf-sw wrote…

Suggestion, take it or leave it, your call.

These lines explain the two specific oracles, but they sit on the generic helper, which after this change takes applicability from the caller and does not care which oracle it is given. The same explanation is also in the PR description, almost word for word.

One option is to leave the helper's existing doc as it was and put one line at each call site, which have none today:

/// Eth to Strk is queried on the validate path too, so every node's gauge moves.
pub(crate) fn get_eth_to_strk_rate_frozen_alert() -> Alert {

/// Strk to Usd is queried only when building a proposal; observers never propose, so their gauge
/// stays flat and the alert excludes them.
pub(crate) fn get_strk_to_usd_rate_frozen_alert() -> Alert {

That puts each fact next to the code it applies to, and drops the "must exclude them" and "by construction" phrasing, which argues for the change rather than saying what the code does.

PTAL

[asaf-sw]

crates/apollo_dashboard/src/alert_scenarios/l1_gas_prices.rs line 197 at r1 (raw file):

Previously, asaf-sw wrote…

PTAL

bug

[asaf-sw]

Reviewable status: all files reviewed, 1 unresolved discussion (waiting on matanl-starkware).

[matanl-starkware]

@matanl-starkware reviewed 2 files and all commit messages, and resolved 1 discussion.
Reviewable status: 0 of 2 files reviewed, all discussions resolved (waiting on asaf-sw).

[matanl-starkware]

@matanl-starkware reviewed 2 files and all commit messages, and made 1 comment.
Reviewable status: 0 of 2 files reviewed, all discussions resolved (waiting on asaf-sw).

[asaf-sw]

@asaf-sw reviewed 2 files and all commit messages.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on matanl-starkware).