Repository navigation
feat(relay): a stock relay keeps 10% of memory free #9269
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
This file was deleted.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| # [M] A publisher's live edge across many tracks is bounded in bytes | ||
|
|
||
| ## Goal | ||
|
|
||
| A publisher holding many live tracks, each with one finished group and nothing | ||
| more written, cannot make the relay hold more than a bound set by the | ||
| relay's own settings. The cache pool's byte budget, which the headroom | ||
| governor moves, today reaches only what a writing track sheds; the newest | ||
| frame of every live track is kept whatever the budget. | ||
|
|
||
| ## Plan | ||
|
|
||
| Found by the review of the relay's default headroom: `Pool::resize` only | ||
| moves the target, a track repays debt when it next writes, a group keeps its | ||
| newest frame while open, and the latest group of every track is exempt from | ||
| idle expiry. So N tracks that each wrote one large frame and went quiet hold | ||
| N frames past any capacity, until the publisher or a subscriber ends them. The | ||
| default headroom bounds a track that keeps writing, not this shape. | ||
|
|
||
| Decide the bound: reclaiming a quiet track's live edge (a late joiner then | ||
| needs a flight or a new group), a per-session or per-broadcast track limit, or | ||
| charging the held live edge against the session's credit. Measure it with | ||
| many tracks of one frame each against a small headroom on the built binary. | ||
|
|
||
| ## Waiting | ||
|
|
||
| - Maintainer: pick the bound. Recommend a per-session live-edge byte budget | ||
| (charge a track's retained newest frame to the session that opened it), since | ||
| evicting a live edge changes what a subscriber is served and a track-count | ||
| limit does not bound bytes; alternative is reclaiming idle live edges. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -10,6 +10,10 @@ use futures::FutureExt; | |
| use moq_net::cache; | ||
| use serde::{Deserialize, Serialize}; | ||
|
|
||
| /// The headroom a relay keeps free when `headroom` is unset, so an authenticated | ||
| /// publisher opening tracks can't grow the relay without bound. | ||
| const DEFAULT_HEADROOM: &str = "10%"; | ||
|
|
||
| /// How often the headroom governor re-samples system memory. | ||
| const GOVERNOR_INTERVAL: Duration = Duration::from_secs(5); | ||
|
|
||
|
|
@@ -18,29 +22,32 @@ const GOVERNOR_INTERVAL: Duration = Duration::from_secs(5); | |
| /// Non-latest groups stay cached until they sit unaccessed past the wall-clock | ||
| /// LRU window (`duration`, 30s by default) or the pool runs out of room, | ||
| /// whichever comes first (the latest group of every track is always retained). | ||
| /// With none of the knobs set the pool is unbounded in bytes and only the | ||
| /// default LRU window bounds memory. | ||
| /// With none of the knobs set the pool keeps 10% of system memory free | ||
| /// ([`DEFAULT_HEADROOM`]); `headroom = "0"` with no `capacity` leaves it unbounded in | ||
| /// bytes, so only the LRU window bounds memory. | ||
| #[derive(usage_rs::Args, Clone, Debug, Default, Deserialize, Serialize)] | ||
| #[usage(unknown_flags = "error", args_override_self = false)] | ||
| #[serde(default, deny_unknown_fields)] | ||
| #[non_exhaustive] | ||
| pub struct Config { | ||
| /// Target bytes of cached group payload, e.g. "8GiB", "512MB", or a | ||
| /// percentage of memory like "75%" (respecting the cgroup limit when set). | ||
| /// Unbounded when unset. | ||
| /// Unbounded when unset, apart from what `headroom` frees. | ||
| /// | ||
| /// A target that usage converges toward as tracks write, not a hard limit, and it | ||
| /// counts cached group bytes (payload plus a fixed cost per group), not process | ||
| /// RSS; leave some slack below physical memory or combine with `headroom`. | ||
| /// RSS; leave some slack below physical memory. `headroom` still applies on top. | ||
| #[usage(long = "cache-capacity", env = "MOQ_CACHE_CAPACITY", setting = "cache.capacity")] | ||
| pub capacity: Option<String>, | ||
|
|
||
| /// Keep at least this much system memory available, e.g. "2GiB" or "10%". | ||
| /// Defaults to "10%" (respecting the cgroup limit when set); "0" disables it. | ||
| /// | ||
| /// Enables a background governor that re-sizes the cache every few seconds | ||
| /// Runs a background governor that re-sizes the cache every few seconds | ||
| /// so the cache soaks up idle memory but is the first thing reclaimed when | ||
| /// the rest of the system needs it. Combine with `capacity` to also cap the | ||
| /// absolute size. | ||
| /// the rest of the system needs it. `capacity` also caps the absolute size. | ||
| /// "0" removes the governor, leaving the cache unbounded unless `capacity` | ||
| /// is set: an authenticated publisher can then grow the relay without limit. | ||
| #[usage(long = "cache-headroom", env = "MOQ_CACHE_HEADROOM", setting = "cache.headroom")] | ||
| pub headroom: Option<String>, | ||
|
|
||
|
|
@@ -107,14 +114,15 @@ impl Config { | |
| .with_expiry(duration.unwrap_or(cache::DEFAULT_EXPIRY)); | ||
| let pool = cache::Pool::new(config); | ||
|
|
||
| let headroom = parse_limit(self.headroom.as_deref().unwrap_or(DEFAULT_HEADROOM))?; | ||
| let mut governed = None; | ||
| if let Some(headroom) = self.headroom.as_deref() { | ||
| let headroom = parse_limit(headroom)?; | ||
| anyhow::ensure!(headroom > 0, "cache headroom must be non-zero"); | ||
| if headroom > 0 { | ||
| tracing::info!(?capacity, headroom, "cache governor enabled"); | ||
| governed = Some(governor(system()?, pool.downgrade(), capacity, headroom).boxed()); | ||
| } else if let Some(capacity) = capacity { | ||
| tracing::info!(capacity, "cache capacity set"); | ||
| } else { | ||
| tracing::warn!("cache headroom is 0 and no capacity is set: the cache is unbounded"); | ||
| } | ||
|
|
||
| if let Some(duration) = duration { | ||
|
|
@@ -131,13 +139,14 @@ impl Config { | |
|
|
||
| /// Parse a size limit: either absolute bytes ("8GiB", "512MB", "1073741824") or | ||
| /// a percentage of total memory ("75%"), honoring the cgroup limit when set. | ||
| /// Zero is a valid limit, which `headroom` reads as no governor. | ||
| fn parse_limit(value: &str) -> anyhow::Result<u64> { | ||
| let value = value.trim(); | ||
| if let Some(percent) = value.strip_suffix('%') { | ||
| let percent: f64 = percent.trim().parse()?; | ||
| anyhow::ensure!( | ||
| percent > 0.0 && percent <= 100.0, | ||
| "memory percentage must be within (0, 100], got {percent}" | ||
| (0.0..=100.0).contains(&percent), | ||
| "memory percentage must be within [0, 100], got {percent}" | ||
| ); | ||
| return Ok((total_memory()? as f64 * percent / 100.0) as u64); | ||
| } | ||
|
|
@@ -152,12 +161,27 @@ fn parse_limit(value: &str) -> anyhow::Result<u64> { | |
| fn total_memory() -> anyhow::Result<u64> { | ||
| let mut sys = system()?; | ||
| sys.refresh_memory(); | ||
| Ok(match sys.cgroup_limits() { | ||
| Ok(match cgroup_limits(&mut sys) { | ||
| Some(limits) => limits.total_memory, | ||
| None => sys.total_memory(), | ||
| }) | ||
| } | ||
|
|
||
| /// The memory limits of this process's own cgroup, such as a systemd unit's `MemoryMax`. | ||
| /// | ||
| /// Not [`sysinfo::System::cgroup_limits`], which reads the cgroup root this process can | ||
| /// see: that is the container's own only under a private cgroup namespace, so a service | ||
| /// limited by its parent cgroup would be sized against the whole host. | ||
| fn cgroup_limits(sys: &mut sysinfo::System) -> Option<sysinfo::CGroupLimits> { | ||
| let pid = sysinfo::get_current_pid().ok()?; | ||
| sys.refresh_processes_specifics( | ||
| sysinfo::ProcessesToUpdate::Some(&[pid]), | ||
| false, | ||
| sysinfo::ProcessRefreshKind::nothing(), | ||
| ); | ||
| sys.process(pid)?.cgroup_limits() | ||
| } | ||
|
|
||
| fn system() -> anyhow::Result<sysinfo::System> { | ||
| sysinfo::System::new().map_err(|err| anyhow::anyhow!("failed to read system memory: {err}")) | ||
| } | ||
|
|
@@ -184,7 +208,7 @@ async fn governor(mut sys: sysinfo::System, pool: cache::PoolWeak, capacity: Opt | |
|
|
||
| sys.refresh_memory(); | ||
|
|
||
| let available = match sys.cgroup_limits() { | ||
| let available = match cgroup_limits(&mut sys) { | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Apply host memory pressure in addition to the cgroup limit. Under a systemd Use the lower of the two values. The cgroup limit still applies when host memory is plentiful: Suggested fix
|
||
| Some(limits) => limits.free_memory, | ||
| None => sys.available_memory(), | ||
| }; | ||
|
|
@@ -218,6 +242,29 @@ mod tests { | |
| assert_eq!(cache.duration, None); | ||
| } | ||
|
|
||
| /// A stock relay is governed; only an explicit zero headroom leaves the pool unbounded. | ||
| #[test] | ||
| fn headroom_defaults_to_a_governor_and_zero_disables_it() { | ||
| assert!(Config::default().init().unwrap().governor.is_some()); | ||
|
|
||
| for zero in ["0", "0%", "0GiB"] { | ||
| let config = Config { | ||
| headroom: Some(zero.to_string()), | ||
| ..Default::default() | ||
| }; | ||
| assert!(config.init().unwrap().governor.is_none(), "headroom {zero:?}"); | ||
| } | ||
|
|
||
| let capped = Config { | ||
| capacity: Some("1GiB".to_string()), | ||
| ..Default::default() | ||
| }; | ||
| assert!( | ||
| capped.init().unwrap().governor.is_some(), | ||
| "an explicit capacity keeps the default headroom" | ||
| ); | ||
| } | ||
|
|
||
| #[test] | ||
| fn explicit_duration_configures_both_bounds() { | ||
| let duration = Duration::from_secs(5); | ||
|
|
@@ -239,6 +286,7 @@ mod tests { | |
|
|
||
| #[test] | ||
| fn parse_limit_percent() { | ||
| assert_eq!(parse_limit("0%").unwrap(), 0); | ||
| let half = parse_limit("50%").unwrap(); | ||
| let full = parse_limit("100%").unwrap(); | ||
| assert!(half > 0); | ||
|
|
@@ -249,7 +297,7 @@ mod tests { | |
| #[test] | ||
| fn parse_limit_rejects_garbage() { | ||
| assert!(parse_limit("lots").is_err()); | ||
| assert!(parse_limit("0%").is_err()); | ||
| assert!(parse_limit("-1%").is_err()); | ||
| assert!(parse_limit("150%").is_err()); | ||
| } | ||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Describe headroom as a target, not a memory bound.
A publisher can retain the newest frames on many quiet live tracks. The governor changes the byte target, but resizing does not synchronously reclaim those frames. The cache implementation explicitly protects the live edge. Thus, enabling headroom does not establish the bound claimed here. (raw.githubusercontent.com)
State that the governor targets free memory every five seconds and cannot guarantee protection from bursts or retained live-edge data. Correct the equivalent claims in
cache.rsandconfig.rs.As per coding guidelines, “State a claim's conditions with it” and “Describe only what ships.”
Source: Coding guidelines