diff --git a/SECURITY.md b/SECURITY.md index 3249cd152..89e72b459 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -48,24 +48,55 @@ operator. Use hosted accounts only for public demos; keep wallets, private data and recovery material on your own device until hosted protection passes its acceptance gate. Independent review and installed proof remain security gates. +## Capability revocation and key rotation + +Runtime stores the capability revocation epoch in `capability_store/epoch` +inside its data home. A successful epoch advance commits the new value before +publishing it in memory. Capability validation reads the stored epoch and +rejects tokens with a lower epoch, including after another process advances the +same store. +Epoch writes are serialized across processes and preserve the highest stored +or observed value. A stale process that persists its state preserves a newer +epoch already committed by another process. + +`elastos emergency rotate` advances that durable epoch before publishing the +replacement signing key. An epoch error stops rotation before key publication. +If key publication fails after the epoch advances, the command reports the +partial result and retains the higher epoch. The epoch and key are separate +writes. + +A running Runtime loads the replacement signing key only after restart. For a +complete rotation, stop Runtime for the affected data home, run +`elastos emergency rotate`, then start Runtime with the same data home. Confirm +that Runtime loads the stored key and starts successfully before resuming use. +Command completion confirms the persisted changes; the operator owns the +restart. Runtime can use an in-memory key when it cannot load or save a signing +key, so resolve signing-key warnings before relying on authority across restarts. + +An established capability store requires a readable, valid epoch at startup. +Capability validation also fails closed when that epoch is unavailable or +invalid. For recovery, stop all Runtime processes using the affected data home +and preserve the capability store. Restore a valid unsigned 64-bit epoch at or +above the latest committed epoch established from trusted, current recovery data. A higher +epoch preserves committed revocations and can invalidate additional tokens; the +maximum value leaves no room for another epoch advance. Preserve the epoch and +any existing lock file. If only `epoch.lock` is missing, stop every Runtime +process using that data home, then restart Runtime to recreate the lock without +changing the valid stored epoch. Resetting the epoch or deleting the store can +discard committed revocations. If the latest committed epoch cannot be +established, obtain recovery assistance before starting Runtime. + +Upgrade note: Older Runtime versions could start at epoch zero when a stored +epoch was empty or malformed. Runtime now refuses that state; if an upgrade +exposes it, preserve the store and use the recovery procedure above before +restarting. A truncated value that still parses as a number cannot be detected +from its syntax alone, so establish the recovery lower bound from trusted, +current data. + ## Open Findings The following security-relevant findings remain open in the current runtime and are documented here for transparency. -### Capability state and key rotation are not restart safe - -**Severity:** Medium -**Files:** `elastos/crates/elastos-runtime/src/capability/manager.rs`, `elastos/crates/elastos-server/src/security_cmd.rs` -**Status:** Open - -Capability signing-key creation and `elastos emergency rotate` log persistence -errors but continue with in-memory state. The emergency command can therefore -report success even when the new key was not written. Rotation also advances a -fresh in-process capability store rather than a durable epoch record. Until the -key and revocation state are committed atomically, operators must verify the -persisted key and restart result instead of treating command completion as a -durable rotation receipt. - ### Host-plane Carrier providers use Runtime admission **Severity:** Medium diff --git a/docs/COMMAND_MATRIX.md b/docs/COMMAND_MATRIX.md index 2d558b953..ecdff1f60 100644 --- a/docs/COMMAND_MATRIX.md +++ b/docs/COMMAND_MATRIX.md @@ -53,7 +53,7 @@ target. | Configuration | `elastos config show`, `elastos config set` | Reads or writes local configuration. | | Identity | `elastos identity show`, `elastos identity nickname get`, `elastos identity nickname set`, `elastos identity arm-owner`, `elastos identity arm-owner-recovery` | Reads or updates the local DID-backed profile. `nickname set` prompts only when no value is supplied and a TTY is available. `arm-owner` admits one first-owner passkey at an exact public HTTPS Home address. `arm-owner-recovery` prints a one-time code for recovering this Home's admin with the Recovery Kit; use the command shown on the sign-in screen. | | TLS | `elastos tls trust`, `elastos tls regen` | Prints trust instructions or regenerates the local leaf certificate. | -| Emergency | `elastos emergency rotate` | Attempts to persist a new signing key. The active Runtime changes only after restart, and current persistence failure is not fail-closed. See the [security finding](../SECURITY.md#capability-state-and-key-rotation-are-not-restart-safe). | +| Emergency | `elastos emergency rotate` | Advances the durable capability revocation epoch, then persists a new signing key. Runtime validation reads the stored epoch; a running Runtime requires a restart to load the new key. The command reports persistence errors, including an epoch advance followed by a key publication failure. See [rotation, upgrade and recovery guidance](../SECURITY.md#capability-revocation-and-key-rotation). | | Trusted sources | `elastos source add`, `elastos source list`, `elastos source show`, `elastos source switch-channel`, `elastos source verify` | Manages local trusted-release source state. | | Updates | `elastos update` | Checks the trusted source over Carrier only. No local runtime is required. See [INSTALL.md](INSTALL.md#how-updates-reach-your-home). | | Offline principal-root maintenance | hidden `elastos principal-root-migrate`, hidden `elastos principal-root-upgrade` | Operates on an explicit data directory. The Runtime must be offline and the command requires explicit backup inputs. | diff --git a/elastos/crates/elastos-runtime/src/bootstrap/runtime_builder.rs b/elastos/crates/elastos-runtime/src/bootstrap/runtime_builder.rs index 488756b0e..0d7813af0 100755 --- a/elastos/crates/elastos-runtime/src/bootstrap/runtime_builder.rs +++ b/elastos/crates/elastos-runtime/src/bootstrap/runtime_builder.rs @@ -575,6 +575,39 @@ mod tests { assert!(runtime.stop().await.is_err()); } + #[tokio::test] + async fn test_runtime_shutdown_preserves_an_externally_advanced_epoch() { + let directory = tempfile::tempdir().unwrap(); + let config = RuntimeConfig { + data_dir: directory.path().to_path_buf(), + enable_audit: false, + ..Default::default() + }; + let mut runtime = + ElastosRuntime::build(config, Arc::new(MockComputeProvider), Arc::new(NullFetcher)) + .await + .unwrap(); + runtime.start().await.unwrap(); + let store_path = directory.path().join("capability_store"); + let external = CapabilityStore::with_persistence(&store_path) + .await + .unwrap(); + assert_eq!(external.advance_epoch().unwrap(), 1); + assert_eq!(runtime.capability_store.current_epoch(), 0); + + runtime.stop().await.unwrap(); + + let reloaded = CapabilityStore::with_persistence(&store_path) + .await + .unwrap(); + assert_eq!( + reloaded.current_epoch(), + 1, + "shutdown must preserve the externally advanced epoch" + ); + assert!(!reloaded.is_epoch_valid(0)); + } + #[tokio::test] async fn test_runtime_accessors() { let temp_dir = tempfile::tempdir().unwrap(); diff --git a/elastos/crates/elastos-runtime/src/capability/manager.rs b/elastos/crates/elastos-runtime/src/capability/manager.rs index 88a49978b..f72d731fa 100644 --- a/elastos/crates/elastos-runtime/src/capability/manager.rs +++ b/elastos/crates/elastos-runtime/src/capability/manager.rs @@ -345,6 +345,23 @@ impl CapabilityManager { token } + /// Whether an issued token is revoked or has no uses remaining. + /// + /// This is only a bookkeeping query for already-granted tokens: it neither + /// validates authority nor consumes a use. Callers must still use `validate` + /// before exercising a capability. + pub(crate) async fn is_revoked_or_exhausted(&self, token: &CapabilityToken) -> bool { + if !self.store.is_epoch_valid(token.constraints.epoch) + || self.store.is_token_revoked(&token.id).await + { + return true; + } + match token.constraints.max_uses { + Some(max_uses) => self.store.has_exceeded_uses(&token.id, max_uses).await, + None => false, + } + } + /// Validate a token for use /// /// This performs all 12 validation checks as specified in PHASE3.md. @@ -590,42 +607,70 @@ impl CapabilityManager { /// Revoke all tokens by advancing the epoch /// /// All tokens with epoch < new_epoch will be rejected. - /// Returns the new epoch. - pub fn revoke_all(&self, reason: &str) -> u64 { - let old_epoch = self.store.current_epoch(); - let new_epoch = self.store.advance_epoch(); - self.audit_log.epoch_advance(old_epoch, new_epoch, reason); - new_epoch + /// Returns the persisted epoch, or the write/rename/overflow error. + pub fn revoke_all(&self, reason: &str) -> std::io::Result { + let new_epoch = self.store.advance_epoch()?; + self.audit_log + .epoch_advance(new_epoch - 1, new_epoch, reason); + Ok(new_epoch) } - /// Rotate the signing key: generate a new key, advance the epoch, persist. + /// Rotate the signing key after epoch advancement succeeds. /// - /// All previously issued tokens become invalid (epoch revocation). - /// Returns the new public key bytes. - pub fn rotate_signing_key(&mut self, data_dir: &Path, reason: &str) -> [u8; 32] { + /// Epoch failure preserves the disk and memory key. If key publication + /// fails after revocation, the epoch stays advanced and the error reports + /// that partial result. This is not a multi-file atomic transaction. + pub fn rotate_signing_key( + &mut self, + data_dir: &Path, + reason: &str, + ) -> std::io::Result<[u8; 32]> { + self.rotate_signing_key_with_publish(data_dir, reason, |staged, key_path| { + staged + .persist(key_path) + .map(|_| ()) + .map_err(|error| error.error) + }) + } + + fn rotate_signing_key_with_publish( + &mut self, + data_dir: &Path, + reason: &str, + publish: impl FnOnce(tempfile::NamedTempFile, &Path) -> std::io::Result<()>, + ) -> std::io::Result<[u8; 32]> { + use std::io::Write as _; + let new_key = SigningKey::generate(&mut rand::thread_rng()); let new_pub = new_key.verifying_key(); - - // Persist new key with restrictive permissions let key_path = data_dir.join("signing_key"); - if let Err(e) = std::fs::write(&key_path, new_key.to_bytes()) { - tracing::error!("Failed to persist rotated key: {}", e); - } else { - #[cfg(unix)] - { - use std::os::unix::fs::PermissionsExt; - let _ = std::fs::set_permissions(&key_path, std::fs::Permissions::from_mode(0o600)); - } + let mut staged = tempfile::Builder::new() + .prefix(".signing-key-rotation-") + .tempfile_in(data_dir)?; + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + staged + .as_file() + .set_permissions(std::fs::Permissions::from_mode(0o600))?; } - - // Advance epoch to invalidate all existing tokens - let old_epoch = self.store.current_epoch(); - let new_epoch = self.store.advance_epoch(); + staged.write_all(&new_key.to_bytes())?; + staged.as_file().sync_all()?; + + // Until this succeeds, the live key file and in-memory key stay intact. + // NamedTempFile removes the staged key on either error path. + let new_epoch = self.revoke_all(reason)?; + let old_epoch = new_epoch - 1; + publish(staged, &key_path).map_err(|error| { + std::io::Error::new(error.kind(), format!( + "Capabilities were revoked at epoch {new_epoch}, but signing key publication failed: {error}" + )) + })?; self.audit_log.security_warning( "signing_key_rotated", &format!( - "Signing key rotated (reason: {}). Epoch advanced {} → {}. All prior tokens invalidated.", + "Signing key rotated (reason: {}). Epoch advanced {} → {}; tokens from earlier epochs are rejected. Other Runtime processes using this data directory require restart to load the new key.", reason, old_epoch, new_epoch ), ); @@ -633,8 +678,7 @@ impl CapabilityManager { let pub_bytes = new_pub.to_bytes(); self.signing_key = new_key; self.verifying_key = new_pub; - - pub_bytes + Ok(pub_bytes) } /// Get the current revocation epoch @@ -747,6 +791,250 @@ mod tests { assert!(matches!(result, Err(ValidationError::WrongAction { .. }))); } + #[tokio::test] + async fn test_revoke_all_propagates_persistence_failure_without_success_audit() { + let directory = tempfile::tempdir().unwrap(); + let store = Arc::new( + CapabilityStore::with_persistence(directory.path()) + .await + .unwrap(), + ); + let audit = Arc::new(AuditLog::new()); + let manager = CapabilityManager::new( + store.clone(), + audit.clone(), + Arc::new(MetricsManager::new()), + ); + let resource = ResourceId::new("localhost://Users/self/Documents/test"); + let token = manager.grant( + "owner", + resource.clone(), + Action::Read, + TokenConstraints::default(), + None, + ); + std::fs::create_dir(directory.path().join("epoch.tmp")).unwrap(); + assert!(manager.revoke_all("blocked write").is_err()); + assert_eq!(store.current_epoch(), 0); + assert!(audit + .recent_events_filtered(100, Some("epoch_advance")) + .is_empty()); + manager + .validate(&token, "owner", Action::Read, &resource, None) + .await + .unwrap(); + } + + #[tokio::test] + async fn test_rotation_epoch_failure_preserves_persisted_key_and_authority() { + for blocked_path in ["epoch.tmp", "epoch", "overflow"] { + let directory = tempfile::tempdir().unwrap(); + if blocked_path == "overflow" { + std::fs::write(directory.path().join("epoch"), u64::MAX.to_string()).unwrap(); + } + let store = Arc::new( + CapabilityStore::with_persistence(directory.path()) + .await + .unwrap(), + ); + let audit = Arc::new(AuditLog::new()); + let mut manager = CapabilityManager::load_or_generate( + directory.path(), + store.clone(), + audit.clone(), + Arc::new(MetricsManager::new()), + ); + let original_key = manager.public_key_bytes(); + let original_bytes = std::fs::read(directory.path().join("signing_key")).unwrap(); + let original_epoch = store.current_epoch(); + let resource = ResourceId::new("localhost://Users/self/Documents/test"); + let token = manager.grant( + "owner", + resource.clone(), + Action::Read, + TokenConstraints::default(), + None, + ); + if blocked_path != "overflow" { + if blocked_path == "epoch" { + std::fs::remove_file(directory.path().join("epoch")).unwrap(); + } + std::fs::create_dir(directory.path().join(blocked_path)).unwrap(); + } + let audit_before = audit.recent_events(100).len(); + assert!(manager + .rotate_signing_key(directory.path(), "blocked epoch") + .is_err()); + assert_eq!(manager.public_key_bytes(), original_key); + assert_eq!(store.current_epoch(), original_epoch); + assert_eq!( + std::fs::read(directory.path().join("signing_key")).unwrap(), + original_bytes + ); + assert_eq!(audit.recent_events(100).len(), audit_before); + assert!(std::fs::read_dir(directory.path()).unwrap().all(|entry| { + !entry + .unwrap() + .file_name() + .to_string_lossy() + .starts_with(".signing-key-rotation-") + })); + if blocked_path != "overflow" { + std::fs::remove_dir(directory.path().join(blocked_path)).unwrap(); + if blocked_path == "epoch" { + std::fs::write(directory.path().join("epoch"), original_epoch.to_string()) + .unwrap(); + } + } + let reloaded_store = Arc::new( + CapabilityStore::with_persistence(directory.path()) + .await + .unwrap(), + ); + let reloaded = CapabilityManager::load_or_generate( + directory.path(), + reloaded_store, + Arc::new(AuditLog::new()), + Arc::new(MetricsManager::new()), + ); + assert_eq!(reloaded.public_key_bytes(), original_key); + assert_eq!(reloaded.current_epoch(), original_epoch); + reloaded + .validate(&token, "owner", Action::Read, &resource, None) + .await + .unwrap(); + } + } + + #[tokio::test] + async fn test_rotation_publish_failure_keeps_key_and_reports_completed_revocation() { + let directory = tempfile::tempdir().unwrap(); + let store = Arc::new( + CapabilityStore::with_persistence(directory.path()) + .await + .unwrap(), + ); + let audit = Arc::new(AuditLog::new()); + let mut manager = CapabilityManager::load_or_generate( + directory.path(), + store.clone(), + audit.clone(), + Arc::new(MetricsManager::new()), + ); + let original_key = manager.public_key_bytes(); + let original_bytes = std::fs::read(directory.path().join("signing_key")).unwrap(); + let resource = ResourceId::new("localhost://Users/self/Documents/test"); + let token = manager.grant( + "owner", + resource.clone(), + Action::Read, + TokenConstraints::default(), + None, + ); + let error = manager + .rotate_signing_key_with_publish( + directory.path(), + "blocked rename", + |staged, key_path| { + assert_eq!(store.current_epoch(), 1, "revocation precedes publication"); + assert_eq!( + std::fs::read_to_string(directory.path().join("epoch")).unwrap(), + "1" + ); + assert_eq!(std::fs::read(key_path).unwrap(), original_bytes); + assert_eq!(staged.as_file().metadata().unwrap().len(), 32); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + assert_eq!( + staged.as_file().metadata().unwrap().permissions().mode() & 0o777, + 0o600 + ); + } + Err(std::io::Error::other("injected rename failure")) + }, + ) + .unwrap_err(); + assert!(error.to_string().contains("revoked at epoch 1")); + assert!(error.to_string().contains("publication failed")); + assert_eq!(manager.public_key_bytes(), original_key); + assert_eq!( + std::fs::read(directory.path().join("signing_key")).unwrap(), + original_bytes + ); + assert_eq!( + audit + .recent_events_filtered(100, Some("epoch_advance")) + .len(), + 1 + ); + assert!(std::fs::read_dir(directory.path()).unwrap().all(|entry| { + !entry + .unwrap() + .file_name() + .to_string_lossy() + .starts_with(".signing-key-rotation-") + })); + let reloaded_store = Arc::new( + CapabilityStore::with_persistence(directory.path()) + .await + .unwrap(), + ); + let mut reloaded = CapabilityManager::load_or_generate( + directory.path(), + reloaded_store, + Arc::new(AuditLog::new()), + Arc::new(MetricsManager::new()), + ); + assert_eq!(reloaded.public_key_bytes(), original_key); + assert_eq!(reloaded.current_epoch(), 1); + assert!(matches!( + reloaded + .validate(&token, "owner", Action::Read, &resource, None) + .await, + Err(ValidationError::TokenRevoked) + )); + let replacement = reloaded + .rotate_signing_key(directory.path(), "retry publication") + .unwrap(); + assert_ne!(replacement, original_key); + assert_eq!(reloaded.current_epoch(), 2); + let rotation_details = reloaded + .audit_log() + .recent_events(100) + .into_iter() + .find_map(|event| match event { + crate::primitives::audit::AuditEvent::SecurityWarning { + warning_type, + details, + .. + } if warning_type == "signing_key_rotated" => Some(details), + _ => None, + }) + .expect("successful rotation must state its scope in the audit log"); + assert!(rotation_details.contains("Epoch advanced 1 → 2")); + assert!( + rotation_details.contains("tokens from earlier epochs are rejected"), + "rotation audit must scope epoch rejection: {rotation_details}" + ); + assert!(rotation_details.contains( + "Other Runtime processes using this data directory require restart to load the new key" + )); + assert!(!rotation_details.contains("All prior tokens invalidated")); + let persisted = CapabilityManager::load_or_generate( + directory.path(), + Arc::new( + CapabilityStore::with_persistence(directory.path()) + .await + .unwrap(), + ), + Arc::new(AuditLog::new()), + Arc::new(MetricsManager::new()), + ); + assert_eq!(persisted.public_key_bytes(), replacement); + assert_eq!(persisted.current_epoch(), 2); + } + #[tokio::test] async fn test_epoch_revocation() { let manager = create_test_manager(); @@ -760,7 +1048,7 @@ mod tests { ); // Advance epoch (mass revocation) - manager.revoke_all("test revocation"); + manager.revoke_all("test revocation").unwrap(); // Token should now be invalid let result = manager @@ -863,7 +1151,7 @@ mod tests { .validate_recurring(&fresh, "principal", Action::Message, &resource, None, false) .await .is_ok()); - manager.revoke_all("epoch"); + manager.revoke_all("epoch").unwrap(); assert!(matches!( manager .validate_recurring(&fresh, "principal", Action::Message, &resource, None, false) @@ -1401,7 +1689,7 @@ mod tests { } // Advance epoch to invalidate all tokens - manager.revoke_all("epoch advance test"); + manager.revoke_all("epoch advance test").unwrap(); let mut successes = 0; let mut revoked = 0; diff --git a/elastos/crates/elastos-runtime/src/capability/pending.rs b/elastos/crates/elastos-runtime/src/capability/pending.rs index 75594750f..d03316beb 100644 --- a/elastos/crates/elastos-runtime/src/capability/pending.rs +++ b/elastos/crates/elastos-runtime/src/capability/pending.rs @@ -2,12 +2,13 @@ //! //! Tracks capability requests that are awaiting user approval (grant/deny). //! Requests have a timeout and are cleaned up periodically. -use std::collections::HashMap; +use std::collections::{HashMap, HashSet}; use std::sync::Arc; use tokio::sync::RwLock; use serde::{Deserialize, Serialize}; +use super::manager::CapabilityManager; use super::token::{Action, CapabilityToken, ResourceId}; use crate::primitives::audit::AuditLog; use crate::primitives::time::SecureTimestamp; @@ -19,10 +20,24 @@ pub const DEFAULT_REQUEST_TIMEOUT_SECS: u64 = 300; /// Maximum number of pending requests before rejecting new ones pub const MAX_PENDING_REQUESTS: usize = 1024; -/// Maximum number of pending requests per session before rejecting new ones. -/// Prevents a single session from starving other sessions. +/// Maximum outstanding requests and retained live grants per session. +/// A session keeps its live grant records within its own bounded share. pub const MAX_PENDING_PER_SESSION: usize = 32; +/// Runtime-owned request scope. Session scopes end with their authenticated +/// connection; Component Bus scopes retain their existing capsule ownership. +#[derive(Debug, Clone, PartialEq, Eq, Hash)] +pub enum RequestScope { + Session(SessionId), + Capsule(String), +} + +impl From for RequestScope { + fn from(session: SessionId) -> Self { + Self::Session(session) + } +} + /// Unique identifier for a pending request #[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)] pub struct RequestId(pub String); @@ -111,6 +126,9 @@ pub struct PendingCapabilityRequest { /// Session that made the request pub session_id: SessionId, + /// Only authenticated Session scopes participate in registry retirement. + scope: RequestScope, + /// Requested resource pub resource: ResourceId, @@ -153,6 +171,7 @@ impl PendingCapabilityRequest { Self { id: RequestId::new(), + scope: RequestScope::Session(session_id.clone()), session_id, resource, action, @@ -192,13 +211,90 @@ impl PendingCapabilityRequest { } } +#[derive(Default)] +struct PendingRequestState { + /// Requests and their index change together under the store's lock. + requests: HashMap, + /// Index from session ID to request IDs (for listing) + session_requests: HashMap>, + /// Present only when this store belongs to a capability-enabled registry. + /// Ended IDs leave this set instead of accumulating as tombstones. + active_sessions: Option>, +} + +impl PendingRequestState { + fn session_is_active(&self, session_id: &str) -> bool { + self.active_sessions + .as_ref() + .is_none_or(|active| active.contains(session_id)) + } + + fn session_admission_count(&self, scope: &RequestScope, now: &SecureTimestamp) -> usize { + self.session_requests.get(scope).map_or(0, |ids| { + ids.iter() + .filter(|id| { + self.requests + .get(*id) + .is_some_and(|request| match &request.status { + RequestStatus::Pending => !request.expires_at.is_before(now), + RequestStatus::Granted { .. } => true, + RequestStatus::Denied { .. } | RequestStatus::Expired => false, + }) + }) + .count() + }) + } + + fn cleanup_expired(&mut self, now: &SecureTimestamp) -> usize { + let mut expired = 0; + for request in self.requests.values_mut() { + if matches!(request.status, RequestStatus::Pending) && request.expires_at.is_before(now) + { + request.status = RequestStatus::Expired; + expired += 1; + } + } + expired + } + + fn cleanup_old(&mut self, now: &SecureTimestamp, retention_secs: u64) -> usize { + let previous_len = self.requests.len(); + self.requests.retain(|_, request| { + // Request expiry alone says nothing about token validity. This pass + // checks token expiry; authoritative retirement needs the manager below. + let resolved = match &request.status { + RequestStatus::Pending => false, + RequestStatus::Granted { token, .. } => { + token.expiry().is_some_and(|expiry| expiry.is_before(now)) + } + RequestStatus::Denied { .. } | RequestStatus::Expired => true, + }; + if !resolved { + return true; + } + let age_secs = now.unix_secs.saturating_sub(request.requested_at.unix_secs); + retention_secs != 0 && age_secs <= retention_secs + }); + + let removed = previous_len - self.requests.len(); + if removed != 0 { + self.prune_session_index(); + } + removed + } + + fn prune_session_index(&mut self) { + let requests = &self.requests; + self.session_requests.retain(|_, ids| { + ids.retain(|id| requests.contains_key(id)); + !ids.is_empty() + }); + } +} + /// Store for pending capability requests pub struct PendingRequestStore { - /// Pending requests indexed by request ID - requests: RwLock>, - - /// Index from session ID to request IDs (for listing) - session_requests: RwLock>>, + state: RwLock, /// Audit log audit_log: Arc, @@ -211,13 +307,82 @@ impl PendingRequestStore { /// Create a new pending request store pub fn new(audit_log: Arc) -> Self { Self { - requests: RwLock::new(HashMap::new()), - session_requests: RwLock::new(HashMap::new()), + state: RwLock::new(PendingRequestState::default()), + audit_log, + timeout_secs: DEFAULT_REQUEST_TIMEOUT_SECS, + } + } + + pub(crate) fn with_session_lifecycle(audit_log: Arc) -> Self { + Self { + state: RwLock::new(PendingRequestState { + active_sessions: Some(HashSet::new()), + ..PendingRequestState::default() + }), audit_log, timeout_secs: DEFAULT_REQUEST_TIMEOUT_SECS, } } + pub(crate) async fn register_session(&self, session_id: &str) { + let mut state = self.state.write().await; + if let Some(active) = &mut state.active_sessions { + active.insert(session_id.to_string()); + } + } + + pub(crate) async fn session_is_active(&self, session_id: &str) -> bool { + self.state.read().await.session_is_active(session_id) + } + + pub(crate) async fn deactivate_session(&self, session_id: &str) { + let mut state = self.state.write().await; + if let Some(active) = &mut state.active_sessions { + active.remove(session_id); + } + } + + pub(crate) async fn retire_session(&self, manager: &CapabilityManager, session_id: &str) { + self.retire_session_with(session_id, |token_id| async move { + manager.revoke(token_id, "Session ended").await; + }) + .await; + } + + async fn retire_session_with(&self, session_id: &str, mut revoke: F) + where + F: FnMut(super::token::TokenId) -> Fut, + Fut: std::future::Future, + { + let tokens = { + let mut state = self.state.write().await; + if let Some(active) = &mut state.active_sessions { + active.remove(session_id); + } + state + .requests + .values() + .filter(|request| { + matches!(&request.scope, RequestScope::Session(id) if id.as_str() == session_id) + }) + .filter_map(|request| request.granted_token().map(|token| *token.id())) + .collect::>() + }; + + // Inactive scopes cannot acquire new requests or grants. Retain their + // records until revocation finishes so cancellation can be retried. + // The registry retains its inactive session entry until this completes. + for token_id in tokens { + revoke(token_id).await; + } + + let mut state = self.state.write().await; + state.requests.retain(|_, request| { + !(matches!(&request.scope, RequestScope::Session(id) if id.as_str() == session_id)) + }); + state.prune_session_index(); + } + /// Shared audit sink used by Runtime-owned bridge adapters. pub fn audit_log(&self) -> Arc { self.audit_log.clone() @@ -226,112 +391,101 @@ impl PendingRequestStore { /// Create with custom timeout pub fn with_timeout(audit_log: Arc, timeout_secs: u64) -> Self { Self { - requests: RwLock::new(HashMap::new()), - session_requests: RwLock::new(HashMap::new()), + state: RwLock::new(PendingRequestState::default()), audit_log, timeout_secs, } } - /// Create a new pending request + /// Admit a request through Runtime-owned pressure reconciliation. /// - /// If the store is at capacity (MAX_PENDING_REQUESTS), expired requests are - /// cleaned up first. If still at capacity, the returned request is immediately - /// denied (not stored) to prevent request-flood DoS. - pub async fn create_request( + /// The manager must be the Runtime owner that issued this store's grants. + /// Reconciliation never consumes or grants authority. The final capacity + /// decision and insertion share one write lock after reconciliation. + pub async fn admit_request( &self, - session_id: SessionId, + capability_manager: &CapabilityManager, + scope: impl Into, resource: ResourceId, action: Action, + reason: String, ) -> PendingCapabilityRequest { - self.create_request_with_reason(session_id, resource, action, String::new()) + let scope = scope.into(); + self.cleanup_retired_grants_at_capacity(capability_manager, Some(&scope)) + .await; + self.create_request_with_reason(scope, resource, action, reason) .await } - pub async fn create_request_with_reason( + #[cfg(test)] + async fn create_request( &self, session_id: SessionId, resource: ResourceId, action: Action, + ) -> PendingCapabilityRequest { + self.create_request_with_reason(session_id.into(), resource, action, String::new()) + .await + } + + async fn create_request_with_reason( + &self, + scope: RequestScope, + resource: ResourceId, + action: Action, reason: String, ) -> PendingCapabilityRequest { - // Capacity guard: evict expired if at limit - { - let count = self.requests.read().await.len(); - if count >= MAX_PENDING_REQUESTS { - self.cleanup_expired().await; - self.cleanup_old(0).await; - } - } - // Re-check after cleanup — reject if still at capacity + let session_id = match &scope { + RequestScope::Session(session_id) => session_id.clone(), + RequestScope::Capsule(capsule_id) => SessionId(capsule_id.clone()), + }; + // Admission, cleanup, and both index updates are one state transition. + let mut state = self.state.write().await; + let now = SecureTimestamp::now(); + if state.requests.len() >= MAX_PENDING_REQUESTS + || state.session_admission_count(&scope, &now) >= MAX_PENDING_PER_SESSION { - let count = self.requests.read().await.len(); - if count >= MAX_PENDING_REQUESTS { - let mut request = PendingCapabilityRequest::new_with_reason( - session_id.clone(), - resource.clone(), - action, - reason.clone(), - self.timeout_secs, - ); - request.status = RequestStatus::Denied { - reason: "Too many pending requests".to_string(), - }; - return request; - } + state.cleanup_expired(&now); + state.cleanup_old(&now, 0); } - // Per-session rate limit: prevent one session from starving others + let denial = if matches!(scope, RequestScope::Session(_)) + && !state.session_is_active(session_id.as_str()) { - let session_requests = self.session_requests.read().await; - if let Some(ids) = session_requests.get(&session_id.0) { - let requests = self.requests.read().await; - let pending_count = ids - .iter() - .filter(|id| { - requests - .get(*id) - .is_some_and(|r| matches!(r.status, RequestStatus::Pending)) - }) - .count(); - if pending_count >= MAX_PENDING_PER_SESSION { - let mut request = PendingCapabilityRequest::new_with_reason( - session_id.clone(), - resource.clone(), - action, - reason.clone(), - self.timeout_secs, - ); - request.status = RequestStatus::Denied { - reason: "Too many pending requests for this session".to_string(), - }; - return request; - } - } - } + Some("Session has ended") + } else if state.requests.len() >= MAX_PENDING_REQUESTS { + Some("Too many pending requests") + } else { + (state.session_admission_count(&scope, &now) >= MAX_PENDING_PER_SESSION) + .then_some("Too many outstanding requests or grants for this session") + }; - let request = PendingCapabilityRequest::new_with_reason( + let mut request = PendingCapabilityRequest::new_with_reason( session_id.clone(), resource.clone(), action, reason, self.timeout_secs, ); - - // Store request - { - let mut requests = self.requests.write().await; - requests.insert(request.id.0.clone(), request.clone()); + request.scope = scope.clone(); + if let Some(reason) = denial { + request.status = RequestStatus::Denied { + reason: reason.to_string(), + }; + return request; } - // Add to session index - { - let mut session_requests = self.session_requests.write().await; - session_requests - .entry(session_id.0.clone()) - .or_default() - .push(request.id.0.clone()); - } + // Exercise the decision/insertion boundary in race tests. The writer + // remains held across this yield, so contenders cannot share a slot. + #[cfg(test)] + tokio::task::yield_now().await; + state.requests.insert(request.id.0.clone(), request.clone()); + state + .session_requests + .entry(scope) + .or_default() + .push(request.id.0.clone()); + drop(state); // Audit self.audit_log @@ -348,8 +502,8 @@ impl PendingRequestStore { /// Get a request by ID pub async fn get_request(&self, request_id: &str) -> Option { - let requests = self.requests.read().await; - requests.get(request_id).cloned() + let state = self.state.read().await; + state.requests.get(request_id).cloned() } /// Grant a pending request @@ -359,9 +513,20 @@ impl PendingRequestStore { token: CapabilityToken, duration: GrantDuration, ) -> Result<(), String> { - let mut requests = self.requests.write().await; + let mut state = self.state.write().await; + + let request = state + .requests + .get(request_id) + .ok_or_else(|| format!("Request not found: {}", request_id))?; + if matches!(request.scope, RequestScope::Session(_)) + && !state.session_is_active(request.session_id.as_str()) + { + return Err("Session has ended".to_string()); + } - let request = requests + let request = state + .requests .get_mut(request_id) .ok_or_else(|| format!("Request not found: {}", request_id))?; @@ -384,9 +549,10 @@ impl PendingRequestStore { /// Deny a pending request pub async fn deny_request(&self, request_id: &str, reason: &str) -> Result<(), String> { - let mut requests = self.requests.write().await; + let mut state = self.state.write().await; - let request = requests + let request = state + .requests .get_mut(request_id) .ok_or_else(|| format!("Request not found: {}", request_id))?; @@ -397,13 +563,15 @@ impl PendingRequestStore { request.status = RequestStatus::Denied { reason: reason.to_string(), }; + let session_id = request.session_id.to_string(); + drop(state); // Audit self.audit_log .emit(crate::primitives::audit::AuditEvent::CapabilityDenied { timestamp: SecureTimestamp::now(), request_id: request_id.to_string(), - session_id: request.session_id.to_string(), + session_id, reason: reason.to_string(), }); @@ -412,8 +580,9 @@ impl PendingRequestStore { /// List all pending requests (for shell to display) pub async fn list_pending(&self) -> Vec { - let requests = self.requests.read().await; - requests + let state = self.state.read().await; + state + .requests .values() .filter(|r| r.is_pending()) .cloned() @@ -422,14 +591,14 @@ impl PendingRequestStore { /// List pending requests for a specific session pub async fn list_session_pending(&self, session_id: &str) -> Vec { - let session_requests = self.session_requests.read().await; - let requests = self.requests.read().await; + let state = self.state.read().await; - session_requests - .get(session_id) + state + .session_requests + .get(&RequestScope::Session(SessionId::from_string(session_id))) .map(|ids| { ids.iter() - .filter_map(|id| requests.get(id)) + .filter_map(|id| state.requests.get(id)) .filter(|r| r.is_pending()) .cloned() .collect() @@ -441,80 +610,108 @@ impl PendingRequestStore { /// /// Returns the number of requests cleaned up pub async fn cleanup_expired(&self) -> usize { - let mut expired_ids = Vec::new(); - - // Find expired requests - { - let requests = self.requests.read().await; - for (id, request) in requests.iter() { - if request.is_expired() && matches!(request.status, RequestStatus::Pending) { - expired_ids.push(id.clone()); - } - } - } - - // Mark them as expired - { - let mut requests = self.requests.write().await; - for id in &expired_ids { - if let Some(request) = requests.get_mut(id) { - request.status = RequestStatus::Expired; - } - } - } - - expired_ids.len() + let mut state = self.state.write().await; + state.cleanup_expired(&SecureTimestamp::now()) } - /// Remove old completed/expired requests + /// Remove old denied/expired requests and explicitly expired grants. /// - /// Keeps requests for `retention_secs` after they're resolved + /// Retention is measured from request creation. Zero removes resolved records + /// immediately; active grants stay available for listing and revocation. pub async fn cleanup_old(&self, retention_secs: u64) -> usize { - let now = SecureTimestamp::now(); - let mut removed = Vec::new(); + let mut state = self.state.write().await; + state.cleanup_old(&SecureTimestamp::now(), retention_secs) + } - { - let mut requests = self.requests.write().await; - requests.retain(|id, request| { - // Keep pending requests - if matches!(request.status, RequestStatus::Pending) { - return true; - } + /// Reclaim retired grants under global or requesting-session pressure. + async fn cleanup_retired_grants_at_capacity( + &self, + capability_manager: &CapabilityManager, + scope: Option<&RequestScope>, + ) -> usize { + self.reclaim_retired_grants_with(scope, |token| async move { + capability_manager.is_revoked_or_exhausted(&token).await + }) + .await + } - // Remove old resolved requests - let age_secs = now.unix_secs.saturating_sub(request.requested_at.unix_secs); - if age_secs > retention_secs { - removed.push(id.clone()); - false - } else { - true - } + async fn reclaim_retired_grants_with( + &self, + scope: Option<&RequestScope>, + mut is_retired: F, + ) -> usize + where + F: FnMut(CapabilityToken) -> Fut, + Fut: std::future::Future, + { + let candidates = { + let state = self.state.read().await; + let global_pressure = state.requests.len() >= MAX_PENDING_REQUESTS; + let session_pressure = scope.is_some_and(|scope| { + state.session_admission_count(scope, &SecureTimestamp::now()) + >= MAX_PENDING_PER_SESSION }); - } + if !global_pressure && !session_pressure { + return 0; + } + state + .requests + .iter() + .filter(|(_, request)| global_pressure || Some(&request.scope) == scope) + .filter_map(|(id, request)| match &request.status { + RequestStatus::Granted { token, duration } => { + Some((id.clone(), token.as_ref().clone(), *duration)) + } + _ => None, + }) + .collect::>() + }; - // Clean up session index - if !removed.is_empty() { - let mut session_requests = self.session_requests.write().await; - for ids in session_requests.values_mut() { - ids.retain(|id| !removed.contains(id)); + // Manager queries can await without holding any pending-state guard. + let mut retired = Vec::new(); + for (id, token, duration) in candidates { + let token_id = *token.id(); + if is_retired(token).await { + retired.push((id, token_id, duration)); } } - removed.len() + let mut state = self.state.write().await; + let mut removed = 0; + for (id, token_id, duration) in retired { + // A granted record's fields are immutable until its status changes. + // Recheck status, token identity and duration before removing it. + let unchanged = state.requests.get(&id).is_some_and(|request| { + matches!(&request.status, RequestStatus::Granted { token, duration: current_duration } + if *token.id() == token_id && *current_duration == duration) + }); + if unchanged { + state.requests.remove(&id); + removed += 1; + } + } + if removed != 0 { + state.prune_session_index(); + } + removed } /// Get the number of pending requests pub async fn pending_count(&self) -> usize { - let requests = self.requests.read().await; - requests.values().filter(|r| r.is_pending()).count() + let state = self.state.read().await; + state.requests.values().filter(|r| r.is_pending()).count() } /// List granted requests for a specific session pub async fn list_session_granted(&self, session_id: &str) -> Vec { - let requests = self.requests.read().await; - requests - .values() - .filter(|r| r.session_id.to_string() == session_id && r.is_granted()) + let state = self.state.read().await; + state + .session_requests + .get(&RequestScope::Session(SessionId::from_string(session_id))) + .into_iter() + .flatten() + .filter_map(|id| state.requests.get(id)) + .filter(|request| request.is_granted()) .cloned() .collect() } @@ -523,58 +720,1147 @@ impl PendingRequestStore { /// /// This changes the status to Denied with reason "Revoked" pub async fn revoke_request(&self, request_id: &str) { - let mut requests = self.requests.write().await; - if let Some(request) = requests.get_mut(request_id) { + let mut state = self.state.write().await; + if let Some(request) = state.requests.get_mut(request_id) { if request.is_granted() { request.status = RequestStatus::Denied { reason: "Revoked by user".to_string(), }; + let session_id = request.session_id.to_string(); + drop(state); // Audit self.audit_log .emit(crate::primitives::audit::AuditEvent::CapabilityDenied { timestamp: SecureTimestamp::now(), request_id: request_id.to_string(), - session_id: request.session_id.to_string(), + session_id, reason: "Revoked by user".to_string(), }); } } } - /// Mark all granted requests as revoked + /// Mark grants from epochs older than the persisted revocation epoch revoked. /// /// Called when epoch is advanced to revoke all capabilities - pub async fn revoke_all_granted(&self) { - let mut requests = self.requests.write().await; + pub async fn revoke_all_granted_before_epoch(&self, new_epoch: u64) { + let mut state = self.state.write().await; let now = SecureTimestamp::now(); + let mut events = Vec::new(); - for (request_id, request) in requests.iter_mut() { - if request.is_granted() { + for (request_id, request) in state.requests.iter_mut() { + if request + .granted_token() + .is_some_and(|token| token.constraints().epoch() < new_epoch) + { request.status = RequestStatus::Denied { reason: "Epoch advanced - all capabilities revoked".to_string(), }; - self.audit_log - .emit(crate::primitives::audit::AuditEvent::CapabilityDenied { - timestamp: now, - request_id: request_id.clone(), - session_id: request.session_id.to_string(), - reason: "Epoch advanced - all capabilities revoked".to_string(), - }); + events.push(crate::primitives::audit::AuditEvent::CapabilityDenied { + timestamp: now, + request_id: request_id.clone(), + session_id: request.session_id.to_string(), + reason: "Epoch advanced - all capabilities revoked".to_string(), + }); } } + drop(state); + for event in events { + self.audit_log.emit(event); + } } } #[cfg(test)] mod tests { use super::*; + use crate::capability::{CapabilityStore, TokenConstraints}; + use crate::primitives::metrics::MetricsManager; + use crate::session::{SessionRegistry, SessionType}; + + fn lifecycle_store() -> ( + SessionRegistry, + Arc, + Arc, + ) { + let audit = Arc::new(AuditLog::new()); + let manager = Arc::new(CapabilityManager::new( + Arc::new(CapabilityStore::new()), + audit.clone(), + Arc::new(MetricsManager::new()), + )); + let (registry, pending) = SessionRegistry::with_capabilities(audit, manager.clone()); + (registry, pending, manager) + } + + async fn lifecycle_grant( + pending: &PendingRequestStore, + manager: &CapabilityManager, + scope: impl Into, + ) -> (PendingCapabilityRequest, CapabilityToken) { + let request = pending + .admit_request( + manager, + scope, + ResourceId::new("localhost://Users/self/Documents/lifecycle"), + Action::Read, + "session lifecycle".to_string(), + ) + .await; + assert!(request.is_pending()); + let token = manager.grant( + request.session_id.as_str(), + request.resource.clone(), + request.action, + TokenConstraints::default(), + None, + ); + pending + .grant_request(request.id.as_str(), token.clone(), GrantDuration::Session) + .await + .unwrap(); + (request, token) + } + + #[tokio::test] + async fn test_ended_sessions_release_full_global_grant_capacity() { + let (registry, pending, manager) = lifecycle_store(); + let mut grants = Vec::new(); + for _ in 0..MAX_PENDING_REQUESTS / MAX_PENDING_PER_SESSION { + let session = registry.create_session(SessionType::Capsule, None).await; + for _ in 0..MAX_PENDING_PER_SESSION { + grants.push(lifecycle_grant(&pending, &manager, session.id.clone()).await); + } + registry + .get_session_mut(&session.token, |session| { + session.last_active = SecureTimestamp::at(0); + }) + .await + .unwrap(); + } + assert_eq!(grants.len(), 32 * 32); + assert_eq!( + pending.state.read().await.requests.len(), + MAX_PENDING_REQUESTS + ); + let fresh = registry.create_session(SessionType::Capsule, None).await; + assert!(request_for(&pending, fresh.id.as_str()).await.is_denied()); + + assert_eq!(registry.cleanup_stale_sessions(600).await, 32); + let admitted = pending + .admit_request( + &manager, + fresh.id.clone(), + ResourceId::new("localhost://Users/self/Documents/fresh"), + Action::Read, + String::new(), + ) + .await; + assert!( + admitted.is_pending(), + "ended sessions must release global grant capacity" + ); + for (request, token) in grants { + assert!(pending.get_request(request.id.as_str()).await.is_none()); + assert!(manager + .validate( + &token, + request.session_id.as_str(), + request.action, + &request.resource, + None + ) + .await + .is_err()); + } + let state = pending.state.read().await; + assert_eq!(state.requests.len(), 1); + assert_eq!(state.session_requests.len(), 1); + assert_eq!(state.active_sessions.as_ref().unwrap().len(), 1); + assert_eq!( + state.session_requests[&RequestScope::Session(fresh.id.clone())], + vec![admitted.id.0] + ); + } + + #[tokio::test] + async fn test_all_session_end_paths_retire_grants_and_preserve_active_sessions() { + for mode in 0..4 { + let (registry, pending, manager) = lifecycle_store(); + let ended = registry + .create_session(SessionType::Capsule, Some("ended".to_string())) + .await; + let active = registry + .create_session(SessionType::Capsule, Some("active".to_string())) + .await; + let (request, token) = lifecycle_grant(&pending, &manager, ended.id.clone()).await; + let (live_request, live_token) = + lifecycle_grant(&pending, &manager, active.id.clone()).await; + match mode { + 0 => { + registry.invalidate_session(&ended.token).await.unwrap(); + } + 1 => { + registry.invalidate_vm_session("ended").await.unwrap(); + } + 2 => { + assert_eq!( + registry + .cleanup_dead_vm_sessions(&["active".to_string()]) + .await, + 1 + ); + } + _ => { + registry + .get_session_mut(&ended.token, |session| { + session.last_active = SecureTimestamp::at(0) + }) + .await + .unwrap(); + assert_eq!(registry.cleanup_stale_sessions(600).await, 1); + } + } + assert!(registry.validate_token(&ended.token).await.is_none()); + assert!(registry.get_vm_token("ended").await.is_none()); + assert_eq!(registry.session_count().await, 1); + assert!(pending.get_request(request.id.as_str()).await.is_none()); + assert!(request_for(&pending, ended.id.as_str()).await.is_denied()); + assert!(manager + .validate( + &token, + ended.id.as_str(), + request.action, + &request.resource, + None + ) + .await + .is_err()); + assert!(registry.validate_token(&active.token).await.is_some()); + assert_eq!( + pending.list_session_granted(active.id.as_str()).await.len(), + 1 + ); + manager + .validate( + &live_token, + active.id.as_str(), + live_request.action, + &live_request.resource, + None, + ) + .await + .unwrap(); + } + } + + #[tokio::test] + async fn test_canceled_session_retirement_fails_closed_and_cleanup_retries() { + use std::future::Future as _; + use std::task::{Context, Waker}; + + let (registry, pending, manager) = lifecycle_store(); + let session = registry + .create_session(SessionType::Shell, Some("retiring".to_string())) + .await; + let (granted, token) = lifecycle_grant(&pending, &manager, session.id.clone()).await; + let late = request_for(&pending, session.id.as_str()).await; + let late_token = manager.grant( + session.id.as_str(), + late.resource.clone(), + late.action, + TokenConstraints::default(), + None, + ); + let mut retiring = Box::pin(tokio::task::unconstrained( + pending.retire_session_with(session.id.as_str(), |_| std::future::pending::<()>()), + )); + assert!(retiring + .as_mut() + .poll(&mut Context::from_waker(Waker::noop())) + .is_pending()); + drop(retiring); + + assert!(registry.validate_token(&session.token).await.is_none()); + assert!(!registry.is_shell(&session.token).await); + assert!(!registry.has_vm_session("retiring").await); + assert!(registry.get_vm_token("retiring").await.is_none()); + assert!(registry.list_sessions().await.is_empty()); + assert_eq!(registry.session_count().await, 0); + assert!(registry + .get_session_mut(&session.token, |_| ()) + .await + .is_none()); + registry.touch_session(&session.token).await; + assert!( + request_for(&pending, session.id.as_str()).await.is_denied(), + "ended session must reject late admission" + ); + assert!( + pending + .grant_request(late.id.as_str(), late_token, GrantDuration::Session) + .await + .is_err(), + "ended session must reject a late grant" + ); + // The canceled operation preserves the identities needed for retry. + assert!(pending.get_request(granted.id.as_str()).await.is_some()); + assert!(pending.get_request(late.id.as_str()).await.is_some()); + assert_eq!(registry.cleanup_stale_sessions(u64::MAX).await, 1); + assert!(manager + .validate( + &token, + session.id.as_str(), + granted.action, + &granted.resource, + None + ) + .await + .is_err()); + let state = pending.state.read().await; + assert!(state.requests.is_empty()); + assert!(state.session_requests.is_empty()); + assert!(state.active_sessions.as_ref().unwrap().is_empty()); + } + + #[tokio::test] + async fn test_session_retirement_preserves_capsule_scope_with_same_owner_string() { + let (registry, pending, manager) = lifecycle_store(); + let session = registry.create_session(SessionType::Capsule, None).await; + let mut session_grants = Vec::new(); + for _ in 0..MAX_PENDING_PER_SESSION { + session_grants.push(lifecycle_grant(&pending, &manager, session.id.clone()).await); + } + assert!(request_for(&pending, session.id.as_str()).await.is_denied()); + let (capsule_request, capsule_token) = lifecycle_grant( + &pending, + &manager, + RequestScope::Capsule(session.id.to_string()), + ) + .await; + assert_eq!( + pending + .list_session_granted(session.id.as_str()) + .await + .len(), + MAX_PENDING_PER_SESSION + ); + assert_eq!(pending.state.read().await.session_requests.len(), 2); + registry.invalidate_session(&session.token).await.unwrap(); + for (session_request, session_token) in session_grants { + assert!(pending + .get_request(session_request.id.as_str()) + .await + .is_none()); + assert!(manager + .validate( + &session_token, + session.id.as_str(), + session_request.action, + &session_request.resource, + None + ) + .await + .is_err()); + } + assert!(pending + .list_session_granted(session.id.as_str()) + .await + .is_empty()); + assert!(pending + .get_request(capsule_request.id.as_str()) + .await + .unwrap() + .is_granted()); + manager + .validate( + &capsule_token, + session.id.as_str(), + capsule_request.action, + &capsule_request.resource, + None, + ) + .await + .unwrap(); + assert_eq!( + pending.state.read().await.session_requests + [&RequestScope::Capsule(session.id.to_string())], + vec![capsule_request.id.0] + ); + let second_capsule = lifecycle_grant( + &pending, + &manager, + RequestScope::Capsule(session.id.to_string()), + ) + .await; + assert!(pending + .get_request(second_capsule.0.id.as_str()) + .await + .unwrap() + .is_granted()); + } + + #[tokio::test] + async fn test_epoch_retirement_preserves_grant_landing_after_advance() { + let (registry, pending, manager) = lifecycle_store(); + let session = registry.create_session(SessionType::Capsule, None).await; + let (old_request, old_token) = + lifecycle_grant(&pending, &manager, session.id.clone()).await; + let epoch = manager.revoke_all("test epoch race").unwrap(); + // Deterministically place a new-epoch grant in the handler's gap. + let (new_request, new_token) = + lifecycle_grant(&pending, &manager, session.id.clone()).await; + assert_eq!(new_token.constraints().epoch(), epoch); + pending.revoke_all_granted_before_epoch(epoch).await; + assert!(pending + .get_request(old_request.id.as_str()) + .await + .unwrap() + .is_denied()); + let visible = pending.list_session_granted(session.id.as_str()).await; + assert_eq!( + visible.len(), + 1, + "new-epoch grant must remain visible and revocable" + ); + assert_eq!(visible[0].id, new_request.id); + assert!(manager + .validate( + &old_token, + session.id.as_str(), + old_request.action, + &old_request.resource, + None + ) + .await + .is_err()); + manager + .validate( + &new_token, + session.id.as_str(), + new_request.action, + &new_request.resource, + None, + ) + .await + .unwrap(); + } fn create_test_store() -> PendingRequestStore { PendingRequestStore::new(Arc::new(AuditLog::new())) } + async fn request_for(store: &PendingRequestStore, session: &str) -> PendingCapabilityRequest { + store + .create_request( + SessionId::from_string(session), + ResourceId::new("localhost://Users/self/Documents/test"), + Action::Read, + ) + .await + } + + fn token_for(request: &PendingCapabilityRequest) -> CapabilityToken { + CapabilityToken::new( + "test-capsule".to_string(), + [0u8; 32], + request.resource.clone(), + request.action, + Default::default(), + SecureTimestamp::now(), + None, + ) + } + + async fn concurrent_admissions(store: Arc, shared_session: bool) -> usize { + use std::future::Future as _; + use std::task::{Context, Waker}; + + const CONTENDERS: usize = 64; + let mut contenders = Vec::new(); + for index in 0..CONTENDERS { + let store = Arc::clone(&store); + contenders.push(Box::pin(tokio::task::unconstrained(async move { + let session = if shared_session { + "shared-session".to_string() + } else { + format!("concurrent-{index}") + }; + request_for(&store, &session).await.is_pending() + }))); + } + + // Poll every contender once before any can resume past the injected + // decision/insertion yield. Correct code keeps the first writer held, + // so the rest wait before checking capacity. If the writer is split at + // that boundary, every contender checks the same last slot first. + // Exclude Tokio's cooperative-budget yield from this first-poll probe; + // the only suspension points are the lock and our explicit boundary. + { + let mut context = Context::from_waker(Waker::noop()); + for contender in &mut contenders { + assert!(contender.as_mut().poll(&mut context).is_pending()); + } + } + let mut tasks = tokio::task::JoinSet::new(); + for contender in contenders { + tasks.spawn(contender); + } + let mut admitted = 0; + while let Some(result) = tasks.join_next().await { + admitted += usize::from(result.unwrap()); + } + admitted + } + + #[tokio::test] + async fn test_expired_requests_release_session_capacity() { + let store = create_test_store(); + for _ in 0..MAX_PENDING_PER_SESSION { + assert!(request_for(&store, "shared-session").await.is_pending()); + } + { + let mut state = store.state.write().await; + for request in state.requests.values_mut() { + request.expires_at = SecureTimestamp::at(0); + } + } + assert_eq!(store.pending_count().await, 0); + + let recovered = request_for(&store, "shared-session").await; + assert!(recovered.is_pending()); + assert!(store.get_request(recovered.id.as_str()).await.is_some()); + assert_eq!(store.list_session_pending("shared-session").await.len(), 1); + } + + #[tokio::test] + async fn test_concurrent_admission_respects_session_limit() { + let store = Arc::new(create_test_store()); + for _ in 0..MAX_PENDING_PER_SESSION - 1 { + request_for(&store, "shared-session").await; + } + + assert_eq!(concurrent_admissions(Arc::clone(&store), true).await, 1); + assert_eq!(store.pending_count().await, MAX_PENDING_PER_SESSION); + assert_eq!( + store.list_session_pending("shared-session").await.len(), + MAX_PENDING_PER_SESSION + ); + assert!(request_for(&store, "other-session").await.is_pending()); + } + + #[tokio::test] + async fn test_concurrent_admission_respects_global_limit() { + let store = Arc::new(create_test_store()); + for index in 0..MAX_PENDING_REQUESTS - 1 { + request_for(&store, &format!("prefill-{index}")).await; + } + + assert_eq!(concurrent_admissions(Arc::clone(&store), false).await, 1); + assert_eq!(store.pending_count().await, MAX_PENDING_REQUESTS); + let state = store.state.read().await; + assert_eq!(state.requests.len(), MAX_PENDING_REQUESTS); + assert_eq!(state.session_requests.len(), MAX_PENDING_REQUESTS); + } + + #[tokio::test] + async fn test_cleanup_preserves_grants_and_removes_empty_session_indexes() { + let store = create_test_store(); + let pending = request_for(&store, "mixed-session").await; + let once = request_for(&store, "mixed-session").await; + let session = request_for(&store, "grant-session").await; + let denied = request_for(&store, "mixed-session").await; + let expired = request_for(&store, "expired-session").await; + store + .grant_request(once.id.as_str(), token_for(&once), GrantDuration::Once) + .await + .unwrap(); + store + .grant_request( + session.id.as_str(), + token_for(&session), + GrantDuration::Session, + ) + .await + .unwrap(); + store + .deny_request(denied.id.as_str(), "test denial") + .await + .unwrap(); + { + let mut state = store.state.write().await; + for request in state.requests.values_mut() { + request.requested_at = SecureTimestamp::at(0); + if request.id != pending.id { + request.expires_at = SecureTimestamp::at(0); + } + } + } + + // Request expiry never expires a granted token or changes its status. + assert_eq!(store.cleanup_expired().await, 1); + assert_eq!(store.cleanup_old(60).await, 2); + assert!(store.get_request(denied.id.as_str()).await.is_none()); + assert!(store.get_request(expired.id.as_str()).await.is_none()); + assert_eq!(store.list_session_pending("mixed-session").await.len(), 1); + assert_eq!( + store.list_session_granted("mixed-session").await[0].id, + once.id + ); + assert_eq!( + store.list_session_granted("grant-session").await[0].id, + session.id + ); + assert!(!store + .state + .read() + .await + .session_requests + .contains_key(&RequestScope::Session(SessionId::from_string( + "expired-session" + )))); + + // Individual and epoch revocation make grant records eligible for cleanup. + store.revoke_request(once.id.as_str()).await; + assert_eq!(store.cleanup_old(0).await, 1); + assert!(store.list_session_granted("mixed-session").await.is_empty()); + store.revoke_all_granted_before_epoch(1).await; + assert_eq!(store.cleanup_old(0).await, 1); + assert!(!store + .state + .read() + .await + .session_requests + .contains_key(&RequestScope::Session(SessionId::from_string( + "grant-session" + )))); + assert_eq!(store.cleanup_old(0).await, 0); + assert_eq!( + store.list_session_pending("mixed-session").await[0].id, + pending.id + ); + } + + #[tokio::test] + async fn test_pressure_cleanup_preserves_grants_and_reclaims_fresh_denials() { + let store = create_test_store(); + let granted = request_for(&store, "grant-session").await; + let token = token_for(&granted); + let token_id = token.id; + store + .grant_request(granted.id.as_str(), token, GrantDuration::Session) + .await + .unwrap(); + for index in 0..MAX_PENDING_REQUESTS - 1 { + let request = request_for(&store, &format!("denied-{index}")).await; + store + .deny_request(request.id.as_str(), "test denial") + .await + .unwrap(); + } + assert_eq!( + store.state.read().await.requests.len(), + MAX_PENDING_REQUESTS + ); + + assert!(request_for(&store, "new-session").await.is_pending()); + let retained = store.get_request(granted.id.as_str()).await.unwrap(); + assert_eq!(retained.granted_token().unwrap().id, token_id); + assert_eq!(store.list_session_granted("grant-session").await.len(), 1); + let state = store.state.read().await; + assert_eq!(state.requests.len(), 2); + assert_eq!(state.session_requests.len(), 2); + } + + #[tokio::test] + async fn test_retained_live_grants_fill_only_the_owning_session_share() { + let store = create_test_store(); + for _ in 0..MAX_PENDING_PER_SESSION { + let request = request_for(&store, "grant-session").await; + assert!(request.is_pending()); + store + .grant_request( + request.id.as_str(), + token_for(&request), + GrantDuration::Session, + ) + .await + .unwrap(); + } + assert!(request_for(&store, "grant-session").await.is_denied()); + assert!(request_for(&store, "new-session").await.is_pending()); + assert_eq!( + store.list_session_granted("grant-session").await.len(), + MAX_PENDING_PER_SESSION + ); + assert_eq!(store.cleanup_old(0).await, 0); + + store.revoke_all_granted_before_epoch(1).await; + assert!(request_for(&store, "grant-session").await.is_pending()); + assert!(store.list_session_granted("grant-session").await.is_empty()); + } + + #[tokio::test] + async fn test_admission_reclaims_session_share_without_spending_live_grants() { + use crate::capability::{CapabilityStore, TokenConstraints}; + use crate::primitives::metrics::MetricsManager; + + for revoke in [false, true] { + let audit = Arc::new(AuditLog::new()); + let authority = Arc::new(CapabilityStore::new()); + let manager = CapabilityManager::new( + authority.clone(), + audit.clone(), + Arc::new(MetricsManager::new()), + ); + let store = PendingRequestStore::new(audit); + let mut grants = Vec::new(); + for _ in 0..MAX_PENDING_PER_SESSION { + let request = store + .admit_request( + &manager, + SessionId::from_string("owner"), + ResourceId::new("localhost://Users/self/Documents/test"), + Action::Read, + "bounded share".to_string(), + ) + .await; + assert!(request.is_pending()); + let token = manager.grant( + "owner", + request.resource.clone(), + request.action, + TokenConstraints::new(0, false, None, Some(1)), + None, + ); + store + .grant_request(request.id.as_str(), token.clone(), GrantDuration::Once) + .await + .unwrap(); + grants.push((request, token)); + } + let admit = |session| { + store.admit_request( + &manager, + SessionId::from_string(session), + ResourceId::new("localhost://Users/self/Documents/test"), + Action::Read, + String::new(), + ) + }; + assert!(admit("owner").await.is_denied()); + assert!(admit("another-session").await.is_pending()); + assert_eq!( + store.list_session_granted("owner").await.len(), + MAX_PENDING_PER_SESSION + ); + let (retired, token) = &grants[0]; + if revoke { + manager.revoke(*token.id(), "share recovery").await; + } else { + manager + .validate(token, "owner", Action::Read, &retired.resource, None) + .await + .unwrap(); + } + assert!(admit("owner").await.is_pending()); + assert!(store.get_request(retired.id.as_str()).await.is_none()); + assert_eq!( + store.list_session_granted("owner").await.len(), + MAX_PENDING_PER_SESSION - 1 + ); + assert_eq!( + authority.get_use_count(token.id()).await, + u32::from(!revoke) + ); + for (request, live_token) in &grants[1..] { + assert!(store + .get_request(request.id.as_str()) + .await + .unwrap() + .is_granted()); + assert_eq!(authority.get_use_count(live_token.id()).await, 0); + } + assert_eq!( + store.state.read().await.requests.len(), + MAX_PENDING_PER_SESSION + 1 + ); + } + } + + #[tokio::test] + async fn test_reclaim_queries_without_pending_lock_and_preserves_changed_records() { + let store = create_test_store(); + let mut granted = Vec::new(); + for _ in 0..4 { + let request = request_for(&store, "owner").await; + let token = token_for(&request); + store + .grant_request(request.id.as_str(), token.clone(), GrantDuration::Session) + .await + .unwrap(); + granted.push((request, token)); + } + for _ in 4..MAX_PENDING_PER_SESSION { + assert!(request_for(&store, "owner").await.is_pending()); + } + let removed = store + .reclaim_retired_grants_with( + Some(&RequestScope::Session(SessionId::from_string("owner"))), + |token| { + let store = &store; + let granted = &granted; + async move { + // This fails immediately if either a read or write guard escaped + // the snapshot scope. It does not rely on a timeout or scheduler. + let mut state = store + .state + .try_write() + .expect("manager query holds pending lock"); + let index = granted + .iter() + .position(|(_, old)| old.id() == token.id()) + .unwrap(); + let request = state + .requests + .get_mut(granted[index].0.id.as_str()) + .unwrap(); + match index { + 0 => {} + 1 => { + request.status = RequestStatus::Denied { + reason: "changed during query".into(), + } + } + 2 => { + request.status = RequestStatus::Granted { + token: Box::new(token_for(request)), + duration: GrantDuration::Session, + } + } + 3 => { + request.status = RequestStatus::Granted { + token: Box::new(token), + duration: GrantDuration::Once, + } + } + _ => unreachable!(), + } + drop(state); + tokio::task::yield_now().await; + true + } + }, + ) + .await; + assert_eq!(removed, 1); + assert!(store.get_request(granted[0].0.id.as_str()).await.is_none()); + assert!(store + .get_request(granted[1].0.id.as_str()) + .await + .unwrap() + .is_denied()); + let replacement = store.get_request(granted[2].0.id.as_str()).await.unwrap(); + assert_ne!(replacement.granted_token().unwrap().id(), granted[2].1.id()); + assert!(matches!( + store + .get_request(granted[3].0.id.as_str()) + .await + .unwrap() + .status, + RequestStatus::Granted { + duration: GrantDuration::Once, + .. + } + )); + let state = store.state.read().await; + assert_eq!(state.requests.len(), MAX_PENDING_PER_SESSION - 1); + assert_eq!( + state.session_requests[&RequestScope::Session(SessionId::from_string("owner"))].len(), + state.requests.len() + ); + } + + #[tokio::test] + async fn test_pressure_reclaims_only_authoritatively_exhausted_grants() { + use crate::capability::{CapabilityStore, TokenConstraints}; + use crate::primitives::metrics::MetricsManager; + + let audit = Arc::new(AuditLog::new()); + let counts = Arc::new(CapabilityStore::new()); + let manager = CapabilityManager::new( + counts.clone(), + audit.clone(), + Arc::new(MetricsManager::new()), + ); + let store = PendingRequestStore::new(audit); + let mut grants = Vec::new(); + for (session, max_uses, used) in [ + ("exhausted-only", Some(1), 1), + ("mixed", Some(1), 1), + ("mixed", Some(1), 0), + ("mixed", None, 0), + ("mixed", Some(2), 1), + ] { + let request = request_for(&store, session).await; + let token = manager.grant( + session, + request.resource.clone(), + request.action, + TokenConstraints::new(0, false, None, max_uses), + None, + ); + store + .grant_request(request.id.as_str(), token.clone(), GrantDuration::Once) + .await + .unwrap(); + for _ in 0..used { + manager + .validate(&token, session, request.action, &request.resource, None) + .await + .unwrap(); + } + grants.push((request, token, used)); + } + assert_eq!( + store + .cleanup_retired_grants_at_capacity(&manager, None) + .await, + 0 + ); + for index in 0..MAX_PENDING_REQUESTS - grants.len() { + assert!(request_for(&store, &format!("prefill-{index}")) + .await + .is_pending()); + } + + assert_eq!( + store + .cleanup_retired_grants_at_capacity(&manager, None) + .await, + 2 + ); + for (index, (request, token, used)) in grants.iter().enumerate() { + assert_eq!( + store.get_request(request.id.as_str()).await.is_some(), + index >= 2 + ); + assert_eq!(counts.get_use_count(token.id()).await, *used); + } + let state = store.state.read().await; + assert!(!state.session_requests.contains_key(&RequestScope::Session( + SessionId::from_string("exhausted-only") + ))); + assert_eq!( + state.session_requests[&RequestScope::Session(SessionId::from_string("mixed"))].len(), + 3 + ); + assert_eq!(state.requests.len(), MAX_PENDING_REQUESTS - 2); + assert_eq!( + state.session_requests.values().map(Vec::len).sum::(), + state.requests.len() + ); + drop(state); + + assert!(request_for(&store, "recovered").await.is_pending()); + assert!(request_for(&store, "recovered").await.is_pending()); + assert_eq!( + store + .cleanup_retired_grants_at_capacity(&manager, None) + .await, + 0 + ); + assert!(request_for(&store, "overflow").await.is_denied()); + } + + #[tokio::test] + async fn test_pressure_reclaims_revoked_grants_without_changing_authority_state() { + use crate::capability::{manager::ValidationError, CapabilityStore, TokenConstraints}; + use crate::primitives::metrics::MetricsManager; + + let audit = Arc::new(AuditLog::new()); + let authority = Arc::new(CapabilityStore::new()); + let manager = CapabilityManager::new( + authority.clone(), + audit.clone(), + Arc::new(MetricsManager::new()), + ); + let store = PendingRequestStore::new(audit); + let stale = request_for(&store, "stale-epoch").await; + let stale_token = manager.grant( + "stale-epoch", + stale.resource.clone(), + stale.action, + TokenConstraints::default(), + None, + ); + let epoch = manager.revoke_all("retire old epoch").unwrap(); + store.revoke_all_granted_before_epoch(epoch).await; + // A grant minted before revocation can finish registering afterward. + store + .grant_request( + stale.id.as_str(), + stale_token.clone(), + GrantDuration::Session, + ) + .await + .unwrap(); + + let revoked = request_for(&store, "mixed").await; + let revoked_token = manager.grant( + "mixed", + revoked.resource.clone(), + revoked.action, + TokenConstraints::new(epoch, false, None, Some(2)), + None, + ); + store + .grant_request( + revoked.id.as_str(), + revoked_token.clone(), + GrantDuration::Session, + ) + .await + .unwrap(); + manager + .validate( + &revoked_token, + "mixed", + revoked.action, + &revoked.resource, + None, + ) + .await + .unwrap(); + manager + .revoke(*revoked_token.id(), "retire individual token") + .await; + + let live = request_for(&store, "mixed").await; + let live_token = manager.grant( + "mixed", + live.resource.clone(), + live.action, + TokenConstraints::default(), + None, + ); + store + .grant_request(live.id.as_str(), live_token.clone(), GrantDuration::Session) + .await + .unwrap(); + for index in 0..MAX_PENDING_REQUESTS - 3 { + assert!(request_for(&store, &format!("prefill-{index}")) + .await + .is_pending()); + } + let revoked_before = authority.get_revoked_tokens().await; + + assert_eq!( + store + .cleanup_retired_grants_at_capacity(&manager, None) + .await, + 2 + ); + assert_eq!(authority.current_epoch(), epoch); + assert_eq!(authority.get_revoked_tokens().await, revoked_before); + assert!(authority.is_token_revoked(revoked_token.id()).await); + assert!(!authority.is_token_revoked(stale_token.id()).await); + assert_eq!(authority.get_use_count(revoked_token.id()).await, 1); + for (token, session, resource) in [ + (&stale_token, "stale-epoch", &stale.resource), + (&revoked_token, "mixed", &revoked.resource), + ] { + assert!(matches!( + manager + .validate(token, session, Action::Read, resource, None) + .await, + Err(ValidationError::TokenRevoked) + )); + } + assert_eq!(authority.get_use_count(revoked_token.id()).await, 1); + manager + .validate(&live_token, "mixed", live.action, &live.resource, None) + .await + .unwrap(); + assert_eq!(store.list_session_granted("mixed").await[0].id, live.id); + let state = store.state.read().await; + assert!(!state.requests.contains_key(stale.id.as_str())); + assert!(!state.requests.contains_key(revoked.id.as_str())); + assert!(!state.session_requests.contains_key(&RequestScope::Session( + SessionId::from_string("stale-epoch") + ))); + assert_eq!( + state.session_requests[&RequestScope::Session(SessionId::from_string("mixed"))].len(), + 1 + ); + assert_eq!(state.requests.len(), MAX_PENDING_REQUESTS - 2); + assert_eq!( + state.session_requests.values().map(Vec::len).sum::(), + state.requests.len() + ); + drop(state); + + assert!(request_for(&store, "recovered").await.is_pending()); + assert!(request_for(&store, "recovered").await.is_pending()); + assert_eq!( + store + .cleanup_retired_grants_at_capacity(&manager, None) + .await, + 0 + ); + assert!(request_for(&store, "overflow").await.is_denied()); + } + + #[tokio::test] + async fn test_cleanup_reclaims_only_explicitly_expired_grants() { + let store = create_test_store(); + let expired = request_for(&store, "expired-grant-session").await; + let live = request_for(&store, "live-grant-session").await; + let mut expired_token = token_for(&expired); + expired_token.expiry = Some(SecureTimestamp::at(0)); + let mut live_token = token_for(&live); + live_token.expiry = Some(SecureTimestamp::after_secs(300)); + store + .grant_request(expired.id.as_str(), expired_token, GrantDuration::Session) + .await + .unwrap(); + store + .grant_request(live.id.as_str(), live_token, GrantDuration::Once) + .await + .unwrap(); + + assert_eq!(store.cleanup_old(0).await, 1); + assert!(store.get_request(expired.id.as_str()).await.is_none()); + assert_eq!( + store.list_session_granted("live-grant-session").await[0].id, + live.id + ); + assert!(!store + .state + .read() + .await + .session_requests + .contains_key(&RequestScope::Session(SessionId::from_string( + "expired-grant-session" + )))); + } + + #[tokio::test] + async fn test_expired_capacity_recovers_without_waiting_for_next_second() { + let store = PendingRequestStore::with_timeout(Arc::new(AuditLog::new()), 0); + for index in 0..MAX_PENDING_REQUESTS { + request_for(&store, &format!("expired-{index}")).await; + } + let recovered = request_for(&store, "new-session").await; + assert!(!recovered.is_denied()); + assert!(store.get_request(recovered.id.as_str()).await.is_some()); + let state = store.state.read().await; + assert_eq!(state.requests.len(), 1); + assert_eq!(state.session_requests.len(), 1); + } + #[tokio::test] async fn test_create_request() { let store = create_test_store(); diff --git a/elastos/crates/elastos-runtime/src/capability/store.rs b/elastos/crates/elastos-runtime/src/capability/store.rs index 8973db7bc..9fe15fa07 100644 --- a/elastos/crates/elastos-runtime/src/capability/store.rs +++ b/elastos/crates/elastos-runtime/src/capability/store.rs @@ -7,12 +7,66 @@ use std::collections::{HashMap, HashSet}; use std::fs; +use std::io::Write as _; use std::path::{Path, PathBuf}; use std::sync::atomic::{AtomicU64, Ordering}; +use std::sync::Mutex; use tokio::sync::RwLock; use super::token::TokenId; +/// The stable lock file orders epoch writers across Runtime processes. +struct EpochFileLock(fs::File); + +impl EpochFileLock { + fn acquire(path: &Path, initialize: bool) -> std::io::Result<(Self, bool)> { + Self::acquire_inner(path, initialize).map_err(|error| { + std::io::Error::new(error.kind(), format!( + "Capability store refused: cannot acquire revocation epoch lock at {}: {error}. \ + Stop all Runtime processes that share this data directory and preserve the capability store. \ + Check directory ownership, permissions, and filesystem lock support. \ + Preserve the epoch and any existing lock file. \ + If only the lock file is missing and the stored epoch is valid, restart Runtime to recreate the lock. \ + Otherwise repair access before restarting. \ + If repair is uncertain, keep Runtime stopped and seek recovery help.", + path.join("epoch.lock").display() + )) + }) + } + + fn acquire_inner(path: &Path, initialize: bool) -> std::io::Result<(Self, bool)> { + let lock_path = path.join("epoch.lock"); + let mut options = fs::OpenOptions::new(); + options.read(true).write(true); + #[cfg(unix)] + { + use std::os::unix::fs::OpenOptionsExt as _; + options.mode(0o600); + } + let (file, created) = if initialize { + match options.create_new(true).open(&lock_path) { + Ok(file) => (file, true), + Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => { + (options.create_new(false).open(&lock_path)?, false) + } + Err(error) => return Err(error), + } + } else { + (options.open(&lock_path)?, false) + }; + file.lock()?; + Ok((Self(file), created)) + } +} + +impl Drop for EpochFileLock { + fn drop(&mut self) { + // Release at scope exit even if a concurrently spawned child briefly + // inherited this open description before its exec closes the fd. + let _ = self.0.unlock(); + } +} + /// Capability store - manages token state /// /// Owns the revocation epoch as an instance field (not process-global), @@ -22,6 +76,9 @@ pub struct CapabilityStore { /// Revocation epoch — all tokens with epoch < this value are rejected. epoch: AtomicU64, + /// Serializes every epoch snapshot, persistent write and memory update. + epoch_writer: Mutex<()>, + /// Per-token use counts use_counts: RwLock>, @@ -37,6 +94,7 @@ impl CapabilityStore { pub fn new() -> Self { Self { epoch: AtomicU64::new(0), + epoch_writer: Mutex::new(()), use_counts: RwLock::new(HashMap::new()), revoked_tokens: RwLock::new(HashSet::new()), storage_path: None, @@ -52,6 +110,7 @@ impl CapabilityStore { let store = Self { epoch: AtomicU64::new(0), + epoch_writer: Mutex::new(()), use_counts: RwLock::new(HashMap::new()), revoked_tokens: RwLock::new(HashSet::new()), storage_path: Some(path.clone()), @@ -63,7 +122,7 @@ impl CapabilityStore { Ok(store) } - /// Get the current revocation epoch + /// Get the last observed revocation epoch. Validation refreshes disk state. pub fn current_epoch(&self) -> u64 { self.epoch.load(Ordering::SeqCst) } @@ -75,40 +134,107 @@ impl CapabilityStore { /// if the process dies after persist but before in-memory update, /// the next startup loads the higher epoch and tokens stay revoked. /// Returns the new epoch value. - pub fn advance_epoch(&self) -> u64 { - let new_epoch = self.epoch.load(Ordering::SeqCst) + 1; + pub fn advance_epoch(&self) -> std::io::Result { + self.epoch_transaction( + |current| { + current + .checked_add(1) + .ok_or_else(|| std::io::Error::other("capability epoch exhausted")) + }, + |epoch| self.write_epoch(epoch), + ) + } - // Persist FIRST — crash-safe ordering + /// Load an already-persisted epoch during initialization. + /// + /// A lower value cannot restore previously revoked authority. The only + /// Runtime caller is construction-time loading; use `advance_epoch` for + /// revocation, which persists the new value before publishing it. + pub fn set_epoch(&self, epoch: u64) { + let _epoch_writer = self + .epoch_writer + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()); + self.epoch.fetch_max(epoch, Ordering::SeqCst); + } + + /// Serialize the durable snapshot, update and write across instances and + /// processes. A stale instance always starts from the durable maximum. + fn epoch_transaction( + &self, + update: impl FnOnce(u64) -> std::io::Result, + writer: impl FnOnce(u64) -> std::io::Result<()>, + ) -> std::io::Result { + let _epoch_writer = self + .epoch_writer + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()); + let _file_lock = self + .storage_path + .as_ref() + .map(|path| EpochFileLock::acquire(path, false)) + .transpose()?; if let Some(path) = &self.storage_path { - let epoch_path = path.join("epoch"); - let tmp_path = path.join("epoch.tmp"); - if let Err(e) = fs::write(&tmp_path, new_epoch.to_string()) - .and_then(|_| fs::rename(&tmp_path, &epoch_path)) - { - tracing::error!( - "CRITICAL: Failed to persist epoch to {}: {}. Epoch advance aborted.", - epoch_path.display(), - e - ); - // Do NOT advance in-memory if persistence failed — - // better to fail the revocation than to have in-memory - // and on-disk diverge. - return self.epoch.load(Ordering::SeqCst); + self.epoch + .fetch_max(Self::read_epoch(path)?, Ordering::SeqCst); + } + let next = update(self.current_epoch())?; + if let Err(error) = writer(next) { + // Rename can succeed before directory synchronization fails. Keep + // the advanced state while reporting the durability error. + if let Some(path) = &self.storage_path { + if let Ok(persisted) = Self::read_epoch(path) { + self.epoch.fetch_max(persisted, Ordering::SeqCst); + } } + return Err(error); } + self.epoch.fetch_max(next, Ordering::SeqCst); + Ok(next) + } - // Only update in-memory after successful persistence - self.epoch.store(new_epoch, Ordering::SeqCst); - new_epoch + fn read_epoch(path: &Path) -> std::io::Result { + fs::read_to_string(path.join("epoch"))? + .trim() + .parse() + .map_err(|_| { + std::io::Error::new(std::io::ErrorKind::InvalidData, "invalid capability epoch") + }) } - /// Set the epoch to a specific value (for initialization/loading) - pub fn set_epoch(&self, epoch: u64) { - self.epoch.store(epoch, Ordering::SeqCst); + fn write_epoch(&self, epoch: u64) -> std::io::Result<()> { + if let Some(path) = &self.storage_path { + let epoch_path = path.join("epoch"); + let tmp_path = path.join("epoch.tmp"); + let mut options = fs::OpenOptions::new(); + options.create(true).truncate(true).write(true); + #[cfg(unix)] + { + use std::os::unix::fs::OpenOptionsExt as _; + options.mode(0o600); + } + let mut staged = options.open(&tmp_path)?; + staged.write_all(epoch.to_string().as_bytes())?; + staged.sync_all()?; + fs::rename(&tmp_path, &epoch_path)?; + #[cfg(unix)] + fs::File::open(path).and_then(|directory| directory.sync_all()).map_err(|error| { + std::io::Error::new(error.kind(), format!( + "Capability epoch was written, but directory synchronization failed; durability is uncertain: {error}" + )) + })?; + } + Ok(()) } /// Check if a token's epoch is valid pub fn is_epoch_valid(&self, token_epoch: u64) -> bool { + if let Some(path) = &self.storage_path { + let Ok(persisted) = Self::read_epoch(path) else { + return false; + }; + self.epoch.fetch_max(persisted, Ordering::SeqCst); + } token_epoch >= self.current_epoch() } @@ -179,10 +305,9 @@ impl CapabilityStore { /// Persist state to disk pub async fn persist(&self) -> std::io::Result<()> { - if let Some(path) = &self.storage_path { - // Persist epoch - let epoch_path = path.join("epoch"); - fs::write(epoch_path, self.current_epoch().to_string())?; + if self.storage_path.is_some() { + // The synchronous transaction releases its guard before the await. + self.epoch_transaction(Ok, |epoch| self.write_epoch(epoch))?; // Persist revoked tokens self.persist_revoked_tokens().await; @@ -193,14 +318,28 @@ impl CapabilityStore { /// Load state from disk async fn load_state(&self) -> std::io::Result<()> { if let Some(path) = &self.storage_path { - // Load epoch - let epoch_path = path.join("epoch"); - if epoch_path.exists() { - if let Ok(content) = fs::read_to_string(&epoch_path) { - if let Ok(epoch) = content.trim().parse::() { - self.set_epoch(epoch); + // Only the creator of the stable lock can initialize an absent + // epoch. An interrupted initialization or removed epoch requires + // explicit repair instead of silently restoring epoch zero. + { + let (_file_lock, created) = EpochFileLock::acquire(path, true)?; + let epoch = match Self::read_epoch(path) { + Ok(epoch) => epoch, + Err(error) if created && error.kind() == std::io::ErrorKind::NotFound => { + self.write_epoch(0)?; + 0 } - } + Err(error) => { + return Err(std::io::Error::new(error.kind(), format!( + "Capability store startup refused: cannot load revocation epoch at {}: {error}. \ + Stop all Runtime processes that share this data directory and preserve the capability store. \ + Restore a valid unsigned 64-bit epoch at or above the latest committed epoch established from trusted valid state. \ + If that value cannot be established, keep Runtime stopped and seek recovery help.", + path.join("epoch").display() + ))); + } + }; + self.set_epoch(epoch); } // Load revoked tokens @@ -257,13 +396,522 @@ impl Default for CapabilityStore { mod tests { use super::*; + fn assert_epoch_writer_held(store: &CapabilityStore) { + // This is a deterministic lock-coverage probe, not a timing assertion. + let blocked = std::thread::scope(|scope| { + scope + .spawn(|| { + matches!( + store.epoch_writer.try_lock(), + Err(std::sync::TryLockError::WouldBlock) + ) + }) + .join() + .unwrap() + }); + assert!(blocked, "another epoch writer could enter the transaction"); + } + + #[tokio::test] + async fn test_epoch_transaction_serializes_snapshot_write_and_publication() { + let directory = tempfile::tempdir().unwrap(); + let store = CapabilityStore::with_persistence(directory.path()) + .await + .unwrap(); + for advance in [true, false] { + let before = store.current_epoch(); + let result = store + .epoch_transaction( + |current| { + assert_epoch_writer_held(&store); + assert_eq!(current, before); + Ok(if advance { current + 1 } else { current }) + }, + |next| { + assert_epoch_writer_held(&store); + assert_eq!(store.current_epoch(), before); + store.write_epoch(next)?; + assert_eq!( + fs::read_to_string(directory.path().join("epoch"))?, + next.to_string() + ); + assert_eq!( + store.current_epoch(), + before, + "publish only after writer success" + ); + Ok(()) + }, + ) + .unwrap(); + assert_eq!(store.current_epoch(), result); + assert!(store.epoch_writer.try_lock().is_ok()); + } + } + + #[tokio::test] + async fn test_concurrent_epoch_advance_and_snapshot_preserve_persisted_epoch() { + use std::sync::{Arc, Barrier}; + const WRITERS: usize = 8; + const ADVANCES: usize = 8; + let directory = tempfile::tempdir().unwrap(); + let store = Arc::new( + CapabilityStore::with_persistence(directory.path()) + .await + .unwrap(), + ); + let barrier = Arc::new(Barrier::new(WRITERS + 2)); + let mut handles = Vec::new(); + for _ in 0..WRITERS { + let store = Arc::clone(&store); + let barrier = Arc::clone(&barrier); + handles.push(std::thread::spawn(move || { + barrier.wait(); + (0..ADVANCES) + .map(|_| store.advance_epoch().unwrap()) + .collect::>() + })); + } + let snapshot_store = Arc::clone(&store); + let snapshot_barrier = Arc::clone(&barrier); + let snapshots = std::thread::spawn(move || { + let runtime = tokio::runtime::Builder::new_current_thread() + .build() + .unwrap(); + snapshot_barrier.wait(); + for _ in 0..WRITERS * ADVANCES { + runtime.block_on(snapshot_store.persist()).unwrap(); + } + }); + barrier.wait(); + let mut epochs = handles + .into_iter() + .flat_map(|handle| handle.join().unwrap()) + .collect::>(); + snapshots.join().unwrap(); + epochs.sort_unstable(); + let final_epoch = (WRITERS * ADVANCES) as u64; + assert_eq!(epochs, (1..=final_epoch).collect::>()); + assert_eq!(store.current_epoch(), final_epoch); + assert_eq!( + fs::read_to_string(directory.path().join("epoch")).unwrap(), + final_epoch.to_string() + ); + let reloaded = CapabilityStore::with_persistence(directory.path()) + .await + .unwrap(); + assert_eq!(reloaded.current_epoch(), final_epoch); + } + + #[tokio::test] + async fn test_epoch_write_and_rename_failures_do_not_publish_memory() { + fn assert_send(_: T) {} + for blocked_path in ["epoch.tmp", "epoch"] { + let directory = tempfile::tempdir().unwrap(); + let store = CapabilityStore::with_persistence(directory.path()) + .await + .unwrap(); + if blocked_path == "epoch" { + fs::remove_file(directory.path().join("epoch")).unwrap(); + } + fs::create_dir(directory.path().join(blocked_path)).unwrap(); + assert!(store.advance_epoch().is_err()); + assert_eq!(store.current_epoch(), 0); + // A synchronous writer guard must not make this async API !Send. + assert_send(store.persist()); + assert!(store.persist().await.is_err()); + assert_eq!(store.current_epoch(), 0); + fs::remove_dir(directory.path().join(blocked_path)).unwrap(); + if blocked_path == "epoch" { + fs::write(directory.path().join("epoch"), "0").unwrap(); + } + assert_eq!(store.advance_epoch().unwrap(), 1); + let reloaded = CapabilityStore::with_persistence(directory.path()) + .await + .unwrap(); + assert_eq!(reloaded.current_epoch(), 1); + } + } + + #[tokio::test] + async fn test_epoch_initialization_is_monotonic_and_overflow_stays_unchanged() { + let memory = CapabilityStore::new(); + memory.set_epoch(7); + memory.set_epoch(3); + assert_eq!(memory.current_epoch(), 7); + assert!(!memory.is_epoch_valid(6)); + + let directory = tempfile::tempdir().unwrap(); + fs::write(directory.path().join("epoch"), u64::MAX.to_string()).unwrap(); + let store = CapabilityStore::with_persistence(directory.path()) + .await + .unwrap(); + assert!(store.advance_epoch().is_err()); + assert_eq!(store.current_epoch(), u64::MAX); + assert_eq!( + fs::read_to_string(directory.path().join("epoch")).unwrap(), + u64::MAX.to_string() + ); + store.persist().await.unwrap(); + let reloaded = CapabilityStore::with_persistence(directory.path()) + .await + .unwrap(); + assert_eq!(reloaded.current_epoch(), u64::MAX); + } + + #[tokio::test] + async fn test_independent_epoch_writers_preserve_every_advance() { + use std::sync::{Arc, Barrier}; + const WRITERS: usize = 4; + const ADVANCES: usize = 4; + let directory = tempfile::tempdir().unwrap(); + let mut stores = Vec::new(); + for _ in 0..WRITERS { + stores.push( + CapabilityStore::with_persistence(directory.path()) + .await + .unwrap(), + ); + } + let barrier = Arc::new(Barrier::new(WRITERS)); + let handles = stores + .into_iter() + .map(|store| { + let barrier = barrier.clone(); + std::thread::spawn(move || { + barrier.wait(); + (0..ADVANCES) + .map(|_| store.advance_epoch().unwrap()) + .collect::>() + }) + }) + .collect::>(); + let mut epochs = handles + .into_iter() + .flat_map(|handle| handle.join().unwrap()) + .collect::>(); + epochs.sort_unstable(); + assert_eq!( + epochs, + (1..=(WRITERS * ADVANCES) as u64).collect::>() + ); + let reloaded = CapabilityStore::with_persistence(directory.path()) + .await + .unwrap(); + assert_eq!(reloaded.current_epoch(), (WRITERS * ADVANCES) as u64); + } + + #[test] + fn test_epoch_file_lock_is_process_wide() { + const CHILD_PATH: &str = "ELASTOS_EPOCH_LOCK_TEST_PATH"; + if let Some(path) = std::env::var_os(CHILD_PATH) { + let file = fs::OpenOptions::new() + .read(true) + .write(true) + .open(path) + .unwrap(); + assert!( + matches!(file.try_lock(), Err(fs::TryLockError::WouldBlock)), + "a second process entered an active epoch transaction" + ); + println!("EPOCH_FILE_LOCK_CHILD_VERIFIED"); + return; + } + let runtime = tokio::runtime::Builder::new_current_thread() + .build() + .unwrap(); + let directory = tempfile::tempdir().unwrap(); + let store = runtime + .block_on(CapabilityStore::with_persistence(directory.path())) + .unwrap(); + store + .epoch_transaction( + |current| { + let result = std::process::Command::new(std::env::current_exe().unwrap()) + .args([ + "--exact", + "capability::store::tests::test_epoch_file_lock_is_process_wide", + "--nocapture", + ]) + .env(CHILD_PATH, directory.path().join("epoch.lock")) + .output() + .unwrap(); + assert!( + result.status.success(), + "cross-process lock probe failed: {}", + String::from_utf8_lossy(&result.stderr) + ); + let output = String::from_utf8_lossy(&result.stdout); + assert!( + output.contains("running 1 test") + && output.contains("EPOCH_FILE_LOCK_CHILD_VERIFIED"), + "the exact child lock probe did not execute: {output}" + ); + Ok(current) + }, + |_| Ok(()), + ) + .unwrap(); + } + + #[tokio::test] + async fn test_external_epoch_advance_retires_old_tokens_in_a_running_store() { + use crate::capability::manager::ValidationError; + use crate::capability::{Action, CapabilityManager, ResourceId, TokenConstraints}; + use crate::primitives::{audit::AuditLog, metrics::MetricsManager}; + use std::sync::Arc; + + let directory = tempfile::tempdir().unwrap(); + let running = Arc::new( + CapabilityStore::with_persistence(directory.path()) + .await + .unwrap(), + ); + let manager = CapabilityManager::new( + running.clone(), + Arc::new(AuditLog::new()), + Arc::new(MetricsManager::new()), + ); + let resource = ResourceId::new("localhost://Users/self/Documents/test"); + let tokens = ["first", "second"].map(|capsule| { + manager.grant( + capsule, + resource.clone(), + Action::Read, + TokenConstraints::default(), + None, + ) + }); + assert_ne!(tokens[0].id, tokens[1].id); + let external = CapabilityStore::with_persistence(directory.path()) + .await + .unwrap(); + assert!(running.is_epoch_valid(0)); + assert_eq!(external.advance_epoch().unwrap(), 1); + for token in tokens { + assert!( + matches!( + manager + .validate(&token, &token.capsule, Action::Read, &resource, None) + .await, + Err(ValidationError::TokenRevoked) + ), + "a running store must reject tokens from the earlier durable epoch" + ); + } + assert!(running.is_epoch_valid(1)); + assert_eq!(running.current_epoch(), 1); + } + + #[tokio::test] + async fn test_epoch_lock_failure_preserves_state_and_explains_recovery() { + for missing in [true, false] { + let directory = tempfile::tempdir().unwrap(); + let store = CapabilityStore::with_persistence(directory.path()) + .await + .unwrap(); + assert_eq!(store.advance_epoch().unwrap(), 1); + let epoch_path = directory.path().join("epoch"); + let lock_path = directory.path().join("epoch.lock"); + fs::remove_file(&lock_path).unwrap(); + if !missing { + fs::create_dir(&lock_path).unwrap(); + } + let expected_kind = fs::OpenOptions::new() + .read(true) + .write(true) + .open(&lock_path) + .unwrap_err() + .kind(); + let mut errors = vec![ + store.advance_epoch().unwrap_err(), + store.persist().await.unwrap_err(), + ]; + if !missing { + errors.push( + CapabilityStore::with_persistence(directory.path()) + .await + .err() + .expect("startup must refuse an unusable epoch lock"), + ); + } + for error in errors { + assert_eq!(error.kind(), expected_kind); + let message = error.to_string(); + assert!( + message.contains("cannot acquire revocation epoch lock"), + "epoch lock failure needs context: {message}" + ); + assert!( + message.contains(&lock_path.display().to_string()), + "{message}" + ); + assert!(message.contains("Stop all Runtime processes"), "{message}"); + assert!( + message.contains("preserve the capability store"), + "{message}" + ); + assert!( + message.contains("ownership, permissions, and filesystem lock support"), + "{message}" + ); + assert!( + message.contains("Preserve the epoch and any existing lock file"), + "{message}" + ); + } + assert_eq!(store.current_epoch(), 1); + assert_eq!(fs::read_to_string(&epoch_path).unwrap(), "1"); + assert_eq!(lock_path.exists(), !missing); + if missing { + drop(store); + let restarted = CapabilityStore::with_persistence(directory.path()) + .await + .unwrap(); + assert!(lock_path.is_file()); + assert_eq!(fs::read_to_string(&epoch_path).unwrap(), "1"); + assert_eq!(restarted.current_epoch(), 1); + assert!(!restarted.is_epoch_valid(0)); + assert_eq!(restarted.advance_epoch().unwrap(), 2); + } + } + } + + #[tokio::test] + async fn test_missing_or_invalid_epoch_fails_closed_until_repaired() { + for (contents, missing_lock, repaired_epoch) in [ + (None, false, 7_u64), + (Some("invalid"), false, 7), + (None, false, 9), + (Some("invalid"), false, 9), + (Some(""), true, 7), + (Some(""), true, 9), + ] { + let directory = tempfile::tempdir().unwrap(); + let store = CapabilityStore::with_persistence(directory.path()) + .await + .unwrap(); + for _ in 0..7 { + store.advance_epoch().unwrap(); + } + let epoch_path = directory.path().join("epoch"); + let trusted_epoch: u64 = fs::read_to_string(&epoch_path).unwrap().parse().unwrap(); + if let Some(contents) = contents { + fs::write(&epoch_path, contents).unwrap(); + } else { + fs::remove_file(&epoch_path).unwrap(); + } + if missing_lock { + // Older stores can have an empty epoch without a lock file. + fs::remove_file(directory.path().join("epoch.lock")).unwrap(); + } + assert!(!store.is_epoch_valid(7)); + assert!(store.advance_epoch().is_err()); + assert!(store.persist().await.is_err()); + let error = CapabilityStore::with_persistence(directory.path()) + .await + .err() + .expect("startup must refuse unreadable durable state"); + assert_eq!( + error.kind(), + if contents.is_none() { + std::io::ErrorKind::NotFound + } else { + std::io::ErrorKind::InvalidData + } + ); + let message = error.to_string(); + assert!( + message.contains("Capability store startup refused"), + "{message}" + ); + assert!( + message.contains(&epoch_path.display().to_string()), + "{message}" + ); + assert!(message.contains("Stop all Runtime processes"), "{message}"); + assert!( + message.contains("preserve the capability store"), + "{message}" + ); + assert!( + message.contains("Restore a valid unsigned 64-bit epoch at or above the latest committed epoch established from trusted valid state"), + "{message}" + ); + assert!( + message.contains("keep Runtime stopped and seek recovery help"), + "{message}" + ); + if let Some(contents) = contents { + assert_eq!(fs::read_to_string(&epoch_path).unwrap(), contents); + } else { + assert!(!epoch_path.exists()); + } + assert!(directory.path().join("epoch.lock").is_file()); + assert_eq!(store.current_epoch(), 7); + drop(store); + + // The fixture's latest trusted epoch is the lower bound for repair. + assert!(repaired_epoch >= trusted_epoch); + fs::write(&epoch_path, repaired_epoch.to_string()).unwrap(); + let repaired = CapabilityStore::with_persistence(directory.path()) + .await + .unwrap(); + assert_eq!(repaired.current_epoch(), repaired_epoch); + for revoked_epoch in 0..repaired_epoch { + assert!(!repaired.is_epoch_valid(revoked_epoch)); + } + assert!(repaired.is_epoch_valid(repaired_epoch)); + assert_eq!(repaired.advance_epoch().unwrap(), repaired_epoch + 1); + } + } + + #[tokio::test] + async fn test_interrupted_epoch_initialization_requires_explicit_repair() { + let directory = tempfile::tempdir().unwrap(); + fs::File::create(directory.path().join("epoch.lock")).unwrap(); + assert!( + CapabilityStore::with_persistence(directory.path()) + .await + .is_err(), + "interrupted initialization must require explicit repair" + ); + assert!(!directory.path().join("epoch").exists()); + } + + #[tokio::test] + async fn test_post_write_error_keeps_the_observed_epoch_advanced() { + let directory = tempfile::tempdir().unwrap(); + let store = CapabilityStore::with_persistence(directory.path()) + .await + .unwrap(); + let error = store + .epoch_transaction( + |current| Ok(current + 1), + |next| { + store.write_epoch(next)?; + Err(std::io::Error::other("fixture: durability is uncertain")) + }, + ) + .unwrap_err(); + assert!(error.to_string().contains("durability is uncertain")); + assert_eq!( + store.current_epoch(), + 1, + "a post-write error must retain the observed advanced epoch" + ); + assert_eq!(CapabilityStore::read_epoch(directory.path()).unwrap(), 1); + store.persist().await.unwrap(); + assert_eq!(CapabilityStore::read_epoch(directory.path()).unwrap(), 1); + } + #[test] fn test_epoch() { let store = CapabilityStore::new(); assert_eq!(store.current_epoch(), 0); - let new_epoch = store.advance_epoch(); + let new_epoch = store.advance_epoch().unwrap(); assert_eq!(new_epoch, 1); assert_eq!(store.current_epoch(), 1); @@ -308,8 +956,8 @@ mod tests { let store = CapabilityStore::with_persistence(temp_dir.path()) .await .unwrap(); - store.advance_epoch(); - store.advance_epoch(); + store.advance_epoch().unwrap(); + store.advance_epoch().unwrap(); let token_id = TokenId::from_bytes([1u8; 16]); store.revoke_token(token_id).await; diff --git a/elastos/crates/elastos-runtime/src/session/registry.rs b/elastos/crates/elastos-runtime/src/session/registry.rs index 316ccbcb6..a7ef1ea0e 100644 --- a/elastos/crates/elastos-runtime/src/session/registry.rs +++ b/elastos/crates/elastos-runtime/src/session/registry.rs @@ -7,6 +7,7 @@ use std::sync::Arc; use tokio::sync::RwLock; use super::{Session, SessionType}; +use crate::capability::{CapabilityManager, PendingRequestStore}; use crate::primitives::audit::AuditLog; use crate::primitives::time::SecureTimestamp; @@ -23,6 +24,10 @@ pub struct SessionRegistry { /// Audit log for session events audit_log: Arc, + + /// Capability ownership is present for Runtime session APIs. A gateway + /// control registry without capability APIs needs only session validation. + capabilities: Option<(Arc, Arc)>, } impl SessionRegistry { @@ -33,6 +38,34 @@ impl SessionRegistry { vm_sessions: RwLock::new(HashMap::new()), default_owner: RwLock::new(None), audit_log, + capabilities: None, + } + } + + /// Build one lifecycle owner for authenticated sessions and their grants. + /// Returning the bound pending store keeps server wiring on this owner. + pub fn with_capabilities( + audit_log: Arc, + manager: Arc, + ) -> (Self, Arc) { + let pending = Arc::new(PendingRequestStore::with_session_lifecycle( + audit_log.clone(), + )); + let mut registry = Self::new(audit_log); + registry.capabilities = Some((manager, pending.clone())); + (registry, pending) + } + + async fn session_is_active(&self, session: &Session) -> bool { + match &self.capabilities { + Some((_, pending)) => pending.session_is_active(session.id.as_str()).await, + None => true, + } + } + + async fn retire_capabilities(&self, session: &Session) { + if let Some((manager, pending)) = &self.capabilities { + pending.retire_session(manager, session.id.as_str()).await; } } @@ -59,6 +92,11 @@ impl SessionRegistry { // Store session { let mut sessions = self.sessions.write().await; + if let Some((_, pending)) = &self.capabilities { + pending.register_session(session.id.as_str()).await; + } + // Registration and publication have no intervening await. A + // canceled creation cannot leave an unowned active scope. sessions.insert(session.token.clone(), session.clone()); } @@ -83,7 +121,10 @@ impl SessionRegistry { /// Validate a bearer token and return the session if valid pub async fn validate_token(&self, token: &str) -> Option { let sessions = self.sessions.read().await; - sessions.get(token).cloned() + let session = sessions.get(token)?; + self.session_is_active(session) + .await + .then(|| session.clone()) } /// Get a session by token (mutable access for updates) @@ -92,43 +133,106 @@ impl SessionRegistry { F: FnOnce(&mut Session) -> R, { let mut sessions = self.sessions.write().await; - sessions.get_mut(token).map(f) + let session = sessions.get_mut(token)?; + self.session_is_active(session).await.then(|| f(session)) } /// Update last activity for a session pub async fn touch_session(&self, token: &str) { let mut sessions = self.sessions.write().await; if let Some(session) = sessions.get_mut(token) { - session.touch(); + if self.session_is_active(session).await { + session.touch(); + } } } /// Check if a token belongs to a shell session pub async fn is_shell(&self, token: &str) -> bool { - let sessions = self.sessions.read().await; - sessions.get(token).map(|s| s.is_shell()).unwrap_or(false) + self.validate_token(token) + .await + .is_some_and(|s| s.is_shell()) } /// Invalidate a session by token pub async fn invalidate_session(&self, token: &str) -> Option { + self.invalidate_session_with_reason(token, "invalidated".to_string(), None) + .await + } + + async fn invalidate_session_with_reason( + &self, + token: &str, + reason: String, + max_idle_secs: Option, + ) -> Option { + self.invalidate_session_with_retirement( + token, + reason, + max_idle_secs, + |session| async move { + self.retire_capabilities(&session).await; + }, + ) + .await + } + + async fn invalidate_session_with_retirement( + &self, + token: &str, + reason: String, + max_idle_secs: Option, + retire: F, + ) -> Option + where + F: FnOnce(Session) -> Fut, + Fut: std::future::Future, + { let session = { - let mut sessions = self.sessions.write().await; - sessions.remove(token) + let sessions = self.sessions.write().await; + let session = sessions.get(token)?; + if let Some(max_idle_secs) = max_idle_secs { + let idle_secs = SecureTimestamp::now() + .unix_secs + .saturating_sub(session.last_active.unix_secs); + if idle_secs <= max_idle_secs && self.session_is_active(session).await { + return None; + } + } + if let Some((_, pending)) = &self.capabilities { + pending.deactivate_session(session.id.as_str()).await; + } + session.clone() }; - if let Some(ref session) = session { - // Remove VM mapping if exists - if let Some(ref vm_id) = session.vm_id { - let mut vm_sessions = self.vm_sessions.write().await; - vm_sessions.remove(vm_id); + // Deactivation is serialized with touch/creation; revocation needs no + // registry or pending-state guard. Cancellation leaves the inactive + // entry for cleanup to retry without blocking unrelated sessions. + retire(session).await; + let session = { + // Acquire both guards before removing the retry owner. Session + // and matching VM mapping removal then have no cancellation gap. + let mut sessions = self.sessions.write().await; + let mut vm_sessions = self.vm_sessions.write().await; + let session = sessions.remove(token); + if let Some(vm_id) = session.as_ref().and_then(|session| session.vm_id.as_ref()) { + if vm_sessions + .get(vm_id) + .is_some_and(|current| current == token) + { + vm_sessions.remove(vm_id); + } } + session + }; + if let Some(ref session) = session { // Audit self.audit_log .emit(crate::primitives::audit::AuditEvent::SessionDestroyed { timestamp: SecureTimestamp::now(), session_id: session.id.to_string(), - reason: "invalidated".to_string(), + reason, }); } @@ -143,28 +247,8 @@ impl SessionRegistry { }; if let Some(token) = token { - let session = { - let mut sessions = self.sessions.write().await; - sessions.remove(&token) - }; - - // Remove VM mapping - { - let mut vm_sessions = self.vm_sessions.write().await; - vm_sessions.remove(vm_id); - } - - if let Some(ref session) = session { - // Audit - self.audit_log - .emit(crate::primitives::audit::AuditEvent::SessionDestroyed { - timestamp: SecureTimestamp::now(), - session_id: session.id.to_string(), - reason: format!("vm_terminated:{}", vm_id), - }); - } - - session + self.invalidate_session_with_reason(&token, format!("vm_terminated:{}", vm_id), None) + .await } else { None } @@ -173,72 +257,53 @@ impl SessionRegistry { /// Get all active sessions (for debugging/admin) pub async fn list_sessions(&self) -> Vec { let sessions = self.sessions.read().await; - sessions.values().cloned().collect() + let mut active = Vec::new(); + for session in sessions.values() { + if self.session_is_active(session).await { + active.push(session.clone()); + } + } + active } /// Get session count pub async fn session_count(&self) -> usize { - let sessions = self.sessions.read().await; - sessions.len() + self.list_sessions().await.len() } /// Check if a VM has an active session pub async fn has_vm_session(&self, vm_id: &str) -> bool { - let vm_sessions = self.vm_sessions.read().await; - vm_sessions.contains_key(vm_id) + self.get_vm_token(vm_id).await.is_some() } /// Get session token for a VM pub async fn get_vm_token(&self, vm_id: &str) -> Option { - let vm_sessions = self.vm_sessions.read().await; - vm_sessions.get(vm_id).cloned() + let token = self.vm_sessions.read().await.get(vm_id).cloned()?; + self.validate_token(&token).await.map(|_| token) } /// Cleanup stale sessions (called periodically) /// /// Removes sessions that haven't been active for the specified duration pub async fn cleanup_stale_sessions(&self, max_idle_secs: u64) -> usize { - let now = SecureTimestamp::now(); - let mut removed = Vec::new(); - - { - let mut sessions = self.sessions.write().await; - sessions.retain(|token, session| { - let idle_secs = now.unix_secs.saturating_sub(session.last_active.unix_secs); - if idle_secs > max_idle_secs { - removed.push((token.clone(), session.clone())); - false - } else { - true - } - }); - } - - // Remove VM mappings and audit - for (token, session) in &removed { - if let Some(ref vm_id) = session.vm_id { - let mut vm_sessions = self.vm_sessions.write().await; - vm_sessions.remove(vm_id); + let tokens: Vec<_> = self.sessions.read().await.keys().cloned().collect(); + let mut removed = 0; + for token in tokens { + // The helper rechecks activity while holding the writer, then + // retries inactive sessions regardless of their last-active time. + if self + .invalidate_session_with_reason( + &token, + "stale_cleanup".to_string(), + Some(max_idle_secs), + ) + .await + .is_some() + { + removed += 1; } - - self.audit_log - .emit(crate::primitives::audit::AuditEvent::SessionDestroyed { - timestamp: SecureTimestamp::now(), - session_id: session.id.to_string(), - reason: format!( - "stale_cleanup:idle_{}s", - now.unix_secs.saturating_sub(session.last_active.unix_secs) - ), - }); - - tracing::info!( - session_id = %session.id, - token = %token[..8.min(token.len())], - "Cleaned up stale session" - ); } - - removed.len() + removed } /// Check if sessions exist for a list of VM IDs and remove dead ones @@ -256,8 +321,12 @@ impl SessionRegistry { }; let mut count = 0; - for (vm_id, _) in vm_tokens { - if self.invalidate_vm_session(&vm_id).await.is_some() { + for (vm_id, token) in vm_tokens { + if self + .invalidate_session_with_reason(&token, format!("vm_terminated:{vm_id}"), None) + .await + .is_some() + { count += 1; } } @@ -274,6 +343,93 @@ mod tests { SessionRegistry::new(Arc::new(AuditLog::new())) } + #[tokio::test] + async fn test_canceled_registry_retirement_keeps_retry_owner_and_other_sessions_responsive() { + use crate::capability::{ + Action, CapabilityStore, GrantDuration, ResourceId, TokenConstraints, + }; + use crate::primitives::metrics::MetricsManager; + use std::future::Future as _; + use std::task::{Context, Waker}; + + let audit = Arc::new(AuditLog::new()); + let manager = Arc::new(CapabilityManager::new( + Arc::new(CapabilityStore::new()), + audit.clone(), + Arc::new(MetricsManager::new()), + )); + let (registry, pending) = SessionRegistry::with_capabilities(audit, manager.clone()); + let ended = registry + .create_session(SessionType::Capsule, Some("reused-vm".to_string())) + .await; + let request = pending + .admit_request( + &manager, + ended.id.clone(), + ResourceId::new("localhost://Users/self/Documents/test"), + Action::Read, + String::new(), + ) + .await; + let token = manager.grant( + ended.id.as_str(), + request.resource.clone(), + request.action, + TokenConstraints::default(), + None, + ); + pending + .grant_request(request.id.as_str(), token.clone(), GrantDuration::Session) + .await + .unwrap(); + + let mut retirement = Box::pin(tokio::task::unconstrained( + registry.invalidate_session_with_retirement( + &ended.token, + "test paused retirement".to_string(), + None, + |_| std::future::pending::<()>(), + ), + )); + assert!(retirement + .as_mut() + .poll(&mut Context::from_waker(Waker::noop())) + .is_pending()); + // The real registry path releases its writer before async revocation. + assert!( + registry.sessions.try_read().is_ok(), + "revocation must release the registry writer" + ); + let replacement = registry + .create_session(SessionType::Capsule, Some("reused-vm".to_string())) + .await; + assert!(registry.validate_token(&replacement.token).await.is_some()); + assert!(registry.validate_token(&ended.token).await.is_none()); + drop(retirement); + assert!( + registry.sessions.read().await.contains_key(&ended.token), + "canceled retirement must retain its retry owner" + ); + assert!(pending.get_request(request.id.as_str()).await.is_some()); + assert_eq!(registry.cleanup_stale_sessions(u64::MAX).await, 1); + assert!(manager + .validate( + &token, + ended.id.as_str(), + request.action, + &request.resource, + None + ) + .await + .is_err()); + assert!(pending.get_request(request.id.as_str()).await.is_none()); + assert!(!registry.sessions.read().await.contains_key(&ended.token)); + assert_eq!( + registry.get_vm_token("reused-vm").await, + Some(replacement.token) + ); + } + #[tokio::test] async fn test_create_session() { let registry = create_test_registry(); diff --git a/elastos/crates/elastos-runtime/tests/http_api_integration.rs b/elastos/crates/elastos-runtime/tests/http_api_integration.rs index 8e2d07c2a..b4608a45d 100755 --- a/elastos/crates/elastos-runtime/tests/http_api_integration.rs +++ b/elastos/crates/elastos-runtime/tests/http_api_integration.rs @@ -110,17 +110,19 @@ async fn test_invalidate_session() { #[tokio::test] async fn test_create_capability_request() { - let (session_registry, _, pending_store) = create_test_infra(); + let (session_registry, capability_manager, pending_store) = create_test_infra(); let session = session_registry .create_session(SessionType::Capsule, None) .await; let request = pending_store - .create_request( + .admit_request( + &capability_manager, session.id.clone(), ResourceId::new("localhost://Users/self/Documents/photos/*"), Action::Read, + String::new(), ) .await; @@ -135,7 +137,7 @@ async fn test_create_capability_request() { #[tokio::test] async fn test_list_pending_requests() { - let (session_registry, _, pending_store) = create_test_infra(); + let (session_registry, capability_manager, pending_store) = create_test_infra(); // Create two sessions with requests let session1 = session_registry @@ -146,18 +148,22 @@ async fn test_list_pending_requests() { .await; pending_store - .create_request( + .admit_request( + &capability_manager, session1.id.clone(), ResourceId::new("localhost://Users/self/Documents/photos/*"), Action::Read, + String::new(), ) .await; pending_store - .create_request( + .admit_request( + &capability_manager, session2.id.clone(), ResourceId::new("localhost://Users/self/Documents/documents/*"), Action::Write, + String::new(), ) .await; @@ -174,10 +180,12 @@ async fn test_grant_capability_request() { .await; let request = pending_store - .create_request( + .admit_request( + &capability_manager, session.id.clone(), ResourceId::new("localhost://Users/self/Documents/photos/*"), Action::Read, + String::new(), ) .await; @@ -217,17 +225,19 @@ async fn test_grant_capability_request() { #[tokio::test] async fn test_deny_capability_request() { - let (session_registry, _, pending_store) = create_test_infra(); + let (session_registry, capability_manager, pending_store) = create_test_infra(); let session = session_registry .create_session(SessionType::Capsule, None) .await; let request = pending_store - .create_request( + .admit_request( + &capability_manager, session.id.clone(), ResourceId::new("localhost://Users/self/Documents/sensitive/*"), Action::Delete, + String::new(), ) .await; @@ -290,10 +300,12 @@ async fn test_cannot_grant_already_granted_request() { .await; let request = pending_store - .create_request( + .admit_request( + &capability_manager, session.id.clone(), ResourceId::new("localhost://Users/self/Documents/photos/*"), Action::Read, + String::new(), ) .await; @@ -324,17 +336,19 @@ async fn test_cannot_grant_already_granted_request() { #[tokio::test] async fn test_cannot_deny_already_denied_request() { - let (session_registry, _, pending_store) = create_test_infra(); + let (session_registry, capability_manager, pending_store) = create_test_infra(); let session = session_registry .create_session(SessionType::Capsule, None) .await; let request = pending_store - .create_request( + .admit_request( + &capability_manager, session.id.clone(), ResourceId::new("localhost://Users/self/Documents/photos/*"), Action::Read, + String::new(), ) .await; @@ -362,10 +376,12 @@ async fn test_grant_once_duration() { .await; let request = pending_store - .create_request( + .admit_request( + &capability_manager, session.id.clone(), ResourceId::new("localhost://Users/self/Documents/photos/*"), Action::Read, + String::new(), ) .await; @@ -398,17 +414,19 @@ async fn test_grant_once_duration() { #[tokio::test] async fn test_request_expiry_detection() { - let (session_registry, _, pending_store) = create_test_infra(); + let (session_registry, capability_manager, pending_store) = create_test_infra(); let session = session_registry .create_session(SessionType::Capsule, None) .await; let request = pending_store - .create_request( + .admit_request( + &capability_manager, session.id.clone(), ResourceId::new("localhost://Users/self/Documents/photos/*"), Action::Read, + String::new(), ) .await; @@ -420,7 +438,7 @@ async fn test_request_expiry_detection() { #[tokio::test] async fn test_all_action_types() { - let (session_registry, _, pending_store) = create_test_infra(); + let (session_registry, capability_manager, pending_store) = create_test_infra(); let session = session_registry .create_session(SessionType::Capsule, None) @@ -437,10 +455,12 @@ async fn test_all_action_types() { for action in actions { let request = pending_store - .create_request( + .admit_request( + &capability_manager, session.id.clone(), ResourceId::new("localhost://Users/self/Documents/test/*"), action, + String::new(), ) .await; @@ -519,7 +539,7 @@ async fn test_cleanup_dead_vm_sessions() { #[tokio::test] async fn test_multiple_requests_same_session() { - let (session_registry, _, pending_store) = create_test_infra(); + let (session_registry, capability_manager, pending_store) = create_test_infra(); let session = session_registry .create_session(SessionType::Capsule, None) @@ -527,18 +547,22 @@ async fn test_multiple_requests_same_session() { // Create multiple requests from same session let request1 = pending_store - .create_request( + .admit_request( + &capability_manager, session.id.clone(), ResourceId::new("localhost://Users/self/Documents/photos/*"), Action::Read, + String::new(), ) .await; let request2 = pending_store - .create_request( + .admit_request( + &capability_manager, session.id.clone(), ResourceId::new("localhost://Users/self/Documents/documents/*"), Action::Write, + String::new(), ) .await; @@ -568,10 +592,12 @@ async fn test_full_capability_flow() { // 3. Capsule requests capability let request = pending_store - .create_request( + .admit_request( + &capability_manager, capsule.id.clone(), ResourceId::new("localhost://Users/self/Documents/photos/*"), Action::Read, + String::new(), ) .await; @@ -908,10 +934,12 @@ async fn test_session_info_capabilities_count() { // Create and grant first capability let req1 = pending_store - .create_request( + .admit_request( + &capability_manager, session.id.clone(), ResourceId::new("localhost://Users/self/Documents/photos/*"), Action::Read, + String::new(), ) .await; @@ -937,10 +965,12 @@ async fn test_session_info_capabilities_count() { // Create and grant second capability let req2 = pending_store - .create_request( + .admit_request( + &capability_manager, session.id.clone(), ResourceId::new("localhost://Users/self/Documents/docs/*"), Action::Write, + String::new(), ) .await; @@ -966,10 +996,12 @@ async fn test_session_info_capabilities_count() { // Pending request should NOT be counted pending_store - .create_request( + .admit_request( + &capability_manager, session.id.clone(), ResourceId::new("localhost://Users/self/Documents/music/*"), Action::Read, + String::new(), ) .await; @@ -985,10 +1017,12 @@ async fn test_session_info_capabilities_count() { .create_session(SessionType::Capsule, None) .await; let other_req = pending_store - .create_request( + .admit_request( + &capability_manager, other.id.clone(), ResourceId::new("localhost://Users/self/Documents/other/*"), Action::Read, + String::new(), ) .await; let other_token = capability_manager.grant( @@ -1060,10 +1094,12 @@ async fn test_shadow_divergence_visible_in_audit_log() { .await; let request = pending_store - .create_request( + .admit_request( + &capability_manager, capsule.id.clone(), ResourceId::new("localhost://Users/self/Documents/sensitive/*"), Action::Delete, + String::new(), ) .await; @@ -1129,10 +1165,12 @@ async fn test_no_shadow_no_divergence_events() { let evaluator = PolicyEvaluator::new(Box::new(ShellPassthroughVerifier), audit_log.clone()); let request = pending_store - .create_request( + .admit_request( + &capability_manager, elastos_runtime::session::SessionId::from_string("test-session"), ResourceId::new("localhost://Users/self/Documents/photos/*"), Action::Read, + String::new(), ) .await; @@ -1177,10 +1215,12 @@ async fn test_shadow_agreement_no_divergence() { ); let request = pending_store - .create_request( + .admit_request( + &capability_manager, elastos_runtime::session::SessionId::from_string("test-session"), ResourceId::new("localhost://Users/self/Documents/photos/*"), Action::Read, + String::new(), ) .await; @@ -1211,10 +1251,12 @@ async fn test_rules_shadow_diverges_on_blocked_resource() { // Request targets a blocklisted resource segment ("admin") let request = pending_store - .create_request( + .admit_request( + &capability_manager, elastos_runtime::session::SessionId::from_string("test-session"), ResourceId::new("localhost://Users/self/Documents/admin/secrets"), Action::Read, + String::new(), ) .await; diff --git a/elastos/crates/elastos-server/src/api/gateway_browser_operator.rs b/elastos/crates/elastos-server/src/api/gateway_browser_operator.rs index f3eed6edc..e9ac76f10 100644 --- a/elastos/crates/elastos-server/src/api/gateway_browser_operator.rs +++ b/elastos/crates/elastos-server/src/api/gateway_browser_operator.rs @@ -334,7 +334,8 @@ pub(in crate::api::gateway) async fn request_admission( service.pending.cleanup_old(60).await; let pending = service .pending - .create_request_with_reason( + .admit_request( + &service.capabilities, session.id.clone(), ResourceId::new(format!("elastos://browser/pages/{page_id}/input")), Action::Write, diff --git a/elastos/crates/elastos-server/src/api/gateway_tests/collaboration_presence.rs b/elastos/crates/elastos-server/src/api/gateway_tests/collaboration_presence.rs index 8657031c7..3708c590e 100644 --- a/elastos/crates/elastos-server/src/api/gateway_tests/collaboration_presence.rs +++ b/elastos/crates/elastos-server/src/api/gateway_tests/collaboration_presence.rs @@ -762,7 +762,8 @@ async fn heartbeat_is_strict_proof_bound_and_coalesces_exact_presentation() { .presence_authority() .unwrap() .manager() - .revoke_all("test epoch"); + .revoke_all("test epoch") + .unwrap(); assert_eq!( response_json(heartbeat().await.unwrap()).await["queued"], false @@ -1034,7 +1035,8 @@ async fn people_snapshot_filters_local_and_exposes_only_signed_live_remote_prese .presence_authority() .unwrap() .manager() - .revoke_all("read-only snapshot epoch"); + .revoke_all("read-only snapshot epoch") + .unwrap(); } let state_before = file_snapshot(local_dir.path()); let snapshot = local_presence.authorized_snapshot(now).await.unwrap(); @@ -1239,7 +1241,8 @@ async fn restarted_presence_get_keeps_legacy_core_bytes_until_lifecycle_repair() .presence_authority() .unwrap() .manager() - .revoke_all("restart GET denial"); + .revoke_all("restart GET denial") + .unwrap(); state.collaboration_presence_product_port = Some(reopened.clone()); let before = file_snapshot(dir.path()); assert!(reopened diff --git a/elastos/crates/elastos-server/src/api/handlers/capability.rs b/elastos/crates/elastos-server/src/api/handlers/capability.rs index cd5038467..d975d7e5c 100755 --- a/elastos/crates/elastos-server/src/api/handlers/capability.rs +++ b/elastos/crates/elastos-server/src/api/handlers/capability.rs @@ -13,8 +13,8 @@ use serde::{Deserialize, Serialize}; use elastos_common::localhost::{is_supported_resource_scheme, is_system_only_backend_resource}; use elastos_runtime::capability::{ - pending::PendingRequestStore, Action, CapabilityManager, GrantDuration, PolicyEvaluator, - PolicyOutcome, ResourceId, TokenConstraints, + pending::PendingRequestStore, Action, CapabilityManager, CapabilityToken, GrantDuration, + PolicyEvaluator, PolicyOutcome, ResourceId, TokenConstraints, }; use elastos_runtime::session::Session; @@ -118,19 +118,24 @@ pub async fn request_capability( // For now, all requests go to pending (no auto-grant policy yet) // Future: check if session already has this capability, or if policy allows auto-grant - let request = state .pending_store - .create_request_with_reason(session.id.clone(), resource, action, input.reason) + .admit_request( + &state.capability_manager, + session.id.clone(), + resource, + action, + input.reason, + ) .await; // If pre-denied (e.g. rate limit), surface the denial immediately - if request.is_denied() { + if let elastos_runtime::capability::RequestStatus::Denied { reason } = &request.status { return Ok(Json(RequestCapabilityOutput { status: "denied".to_string(), request_id: Some(request.id.to_string()), token: None, - reason: Some("Too many pending requests".to_string()), + reason: Some(reason.clone()), })); } @@ -345,12 +350,7 @@ pub async fn grant_request( None, // No expiry for now (session-scoped) ); - // Mark request as granted - state - .pending_store - .grant_request(&input.request_id, token.clone(), duration) - .await - .map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e))?; + record_grant(&state, &input.request_id, &token, duration).await?; Ok(Json(GrantRequestOutput { success: true, @@ -359,6 +359,29 @@ pub async fn grant_request( })) } +async fn record_grant( + state: &CapabilityState, + request_id: &str, + token: &CapabilityToken, + duration: GrantDuration, +) -> Result<(), (StatusCode, String)> { + if let Err(error) = state + .pending_store + .grant_request(request_id, token.clone(), duration) + .await + { + // The owner can end while the Shell's decision is in flight. Retire + // the minted token when its request can no longer record that grant. + state + .capability_manager + .revoke(*token.id(), "Grant registration failed") + .await; + return Err((StatusCode::INTERNAL_SERVER_ERROR, error)); + } + + Ok(()) +} + // === Deny Request (Shell Only) === #[derive(Debug, Deserialize)] @@ -538,17 +561,29 @@ pub async fn revoke_all_capabilities( State(state): State, Extension(_session): Extension, // Shell check done by middleware Json(input): Json, -) -> Json { - let new_epoch = state.capability_manager.revoke_all(&input.reason); +) -> Result, (StatusCode, String)> { + let new_epoch = state + .capability_manager + .revoke_all(&input.reason) + .map_err(|error| { + tracing::error!("Failed to persist capability revocation: {error}"); + ( + StatusCode::INTERNAL_SERVER_ERROR, + "Failed to persist capability revocation".to_string(), + ) + })?; - // Mark all granted requests as revoked - state.pending_store.revoke_all_granted().await; + // A grant issued after advancement keeps its authority and visible record. + state + .pending_store + .revoke_all_granted_before_epoch(new_epoch) + .await; - Json(RevokeAllOutput { + Ok(Json(RevokeAllOutput { success: true, new_epoch, reason: input.reason, - }) + })) } // === Session Info === @@ -776,6 +811,549 @@ mod tests { } } + #[tokio::test] + async fn test_failed_grant_registration_retires_unreturned_token() { + use elastos_runtime::session::{SessionRegistry, SessionType}; + + let mut state = test_state(); + let (registry, pending) = SessionRegistry::with_capabilities( + state.pending_store.audit_log(), + state.capability_manager.clone(), + ); + state.pending_store = pending; + let session = registry.create_session(SessionType::Capsule, None).await; + let request = state + .pending_store + .admit_request( + &state.capability_manager, + session.id.clone(), + ResourceId::new("localhost://Users/self/Documents/test"), + Action::Read, + String::new(), + ) + .await; + // The handler has read the pending request and minted a token before + // the session ends; recording that stale decision must retire it. + let token = state.capability_manager.grant( + session.id.as_str(), + request.resource.clone(), + request.action, + TokenConstraints::default(), + None, + ); + registry.invalidate_session(&session.token).await.unwrap(); + let error = record_grant(&state, request.id.as_str(), &token, GrantDuration::Session) + .await + .unwrap_err(); + assert_eq!(error.0, StatusCode::INTERNAL_SERVER_ERROR); + assert!( + state + .capability_manager + .validate( + &token, + session.id.as_str(), + request.action, + &request.resource, + None + ) + .await + .is_err(), + "unreturned token must be retired when grant registration fails" + ); + } + + #[tokio::test] + async fn test_revoke_all_failure_returns_500_and_keeps_live_grants() { + use elastos_runtime::capability::{CapabilityStore, RequestStatus}; + use elastos_runtime::primitives::{audit::AuditEvent, metrics::MetricsManager}; + + for blocked_path in ["epoch.tmp", "epoch", "overflow"] { + let directory = tempfile::tempdir().unwrap(); + if blocked_path == "overflow" { + std::fs::write(directory.path().join("epoch"), u64::MAX.to_string()).unwrap(); + } + let authority = Arc::new( + CapabilityStore::with_persistence(directory.path()) + .await + .unwrap(), + ); + let mut state = test_state(); + let audit = state.pending_store.audit_log(); + state.capability_manager = Arc::new(CapabilityManager::new( + authority.clone(), + audit.clone(), + Arc::new(MetricsManager::new()), + )); + let session = Session::new_capsule("revocation-failure".to_string()); + let request = state + .pending_store + .admit_request( + &state.capability_manager, + session.id.clone(), + ResourceId::new("localhost://Users/self/Documents/test"), + Action::Read, + String::new(), + ) + .await; + let token = state.capability_manager.grant( + session.id.as_str(), + request.resource.clone(), + request.action, + TokenConstraints::default(), + None, + ); + state + .pending_store + .grant_request(request.id.as_str(), token.clone(), GrantDuration::Session) + .await + .unwrap(); + if blocked_path != "overflow" { + if blocked_path == "epoch" { + std::fs::remove_file(directory.path().join("epoch")).unwrap(); + } + std::fs::create_dir(directory.path().join(blocked_path)).unwrap(); + } + let epoch = authority.current_epoch(); + let result = revoke_all_capabilities( + State(state.clone()), + Extension(Session::new_shell("test-shell".to_string())), + Json(RevokeAllInput { + reason: "persistence failure".to_string(), + }), + ) + .await; + let (status, message) = match result { + Err(error) => error, + Ok(_) => panic!("failed persistence must not report successful revocation"), + }; + assert_eq!(status, StatusCode::INTERNAL_SERVER_ERROR); + assert_eq!(message, "Failed to persist capability revocation"); + assert_eq!(authority.current_epoch(), epoch); + assert!(matches!( + state + .pending_store + .get_request(request.id.as_str()) + .await + .unwrap() + .status, + RequestStatus::Granted { .. } + )); + assert!(audit + .recent_events(100) + .iter() + .all(|event| !matches!(event, AuditEvent::EpochAdvance { .. }))); + let validation = state + .capability_manager + .validate( + &token, + session.id.as_str(), + request.action, + &request.resource, + None, + ) + .await; + if blocked_path == "epoch" { + assert!( + validation.is_err(), + "unreadable epoch state must fail closed" + ); + } else { + validation.unwrap(); + } + if blocked_path != "overflow" { + std::fs::remove_dir(directory.path().join(blocked_path)).unwrap(); + if blocked_path == "epoch" { + std::fs::write(directory.path().join("epoch"), epoch.to_string()).unwrap(); + } + state + .capability_manager + .validate( + &token, + session.id.as_str(), + request.action, + &request.resource, + None, + ) + .await + .unwrap(); + let recovered = revoke_all_capabilities( + State(state.clone()), + Extension(Session::new_shell("test-shell".to_string())), + Json(RevokeAllInput { + reason: "retry after repair".to_string(), + }), + ) + .await + .unwrap() + .0; + assert!(recovered.success); + assert_eq!(recovered.new_epoch, epoch + 1); + assert!(state + .pending_store + .get_request(request.id.as_str()) + .await + .unwrap() + .is_denied()); + assert!(state + .capability_manager + .validate( + &token, + session.id.as_str(), + request.action, + &request.resource, + None + ) + .await + .is_err()); + } + } + } + + #[tokio::test] + async fn test_request_capacity_recovers_consumed_once_grants_without_losing_live_grants() { + use elastos_runtime::capability::{ + manager::ValidationError, pending::MAX_PENDING_REQUESTS, CapabilityToken, + }; + use elastos_runtime::session::SessionId; + + async fn request(state: &CapabilityState, session: &Session) -> RequestCapabilityOutput { + request_capability( + State(state.clone()), + Extension(session.clone()), + Json(RequestCapabilityInput { + resource: "localhost://Users/self/Documents/test".to_string(), + action: "read".to_string(), + reason: "capacity regression".to_string(), + }), + ) + .await + .unwrap() + .0 + } + + async fn grant( + state: &CapabilityState, + session: &Session, + duration: &str, + ) -> (String, CapabilityToken) { + let output = request(state, session).await; + assert_eq!(output.status, "pending"); + let request_id = output.request_id.unwrap(); + let granted = grant_request( + State(state.clone()), + Extension(Session::new_shell("test-shell".to_string())), + Json(GrantRequestInput { + request_id: request_id.clone(), + duration: duration.to_string(), + rationale: Some("capacity regression".to_string()), + }), + ) + .await + .unwrap() + .0; + assert!(granted.success); + let token = CapabilityToken::from_base64(&granted.token.unwrap()).unwrap(); + (request_id, token) + } + + let state = test_state(); + let spent_session = Session::new_capsule("spent".to_string()); + let live_session = Session::new_capsule("live".to_string()); + let next_session = Session::new_capsule("next".to_string()); + let (spent_id, spent) = grant(&state, &spent_session, "once").await; + let (unused_id, unused) = grant(&state, &live_session, "once").await; + let (live_id, live) = grant(&state, &live_session, "session").await; + assert_eq!(spent.constraints().max_uses(), Some(1)); + assert!(spent.expiry().is_none()); + for index in 0..MAX_PENDING_REQUESTS - 3 { + let pending = state + .pending_store + .admit_request( + &state.capability_manager, + SessionId::from_string(format!("prefill-{index}")), + spent.resource().clone(), + Action::Read, + String::new(), + ) + .await; + assert!(pending.is_pending()); + } + assert_eq!(request(&state, &next_session).await.status, "denied"); + + state + .capability_manager + .validate( + &spent, + spent_session.id.as_str(), + Action::Read, + spent.resource(), + None, + ) + .await + .unwrap(); + let recovered = request(&state, &next_session).await; + assert_eq!( + recovered.status, "pending", + "consumed grants must not permanently fill the store" + ); + assert!(state + .pending_store + .get_request(&recovered.request_id.unwrap()) + .await + .is_some()); + assert!(state.pending_store.get_request(&spent_id).await.is_none()); + assert!(state + .pending_store + .list_session_granted(spent_session.id.as_str()) + .await + .is_empty()); + assert_eq!( + state.pending_store.pending_count().await, + MAX_PENDING_REQUESTS - 2 + ); + assert_eq!(request(&state, &next_session).await.status, "denied"); + + let visible = list_capabilities(State(state.clone()), Extension(live_session.clone())) + .await + .0; + assert_eq!(visible.capabilities.len(), 2); + assert!(visible + .capabilities + .iter() + .any(|entry| entry.request_id == unused_id)); + assert!(visible + .capabilities + .iter() + .any(|entry| entry.request_id == live_id)); + // Pressure reconciliation must not spend an unused one-use allowance. + state + .capability_manager + .validate( + &unused, + live_session.id.as_str(), + Action::Read, + unused.resource(), + None, + ) + .await + .unwrap(); + assert!(matches!( + state + .capability_manager + .validate( + &unused, + live_session.id.as_str(), + Action::Read, + unused.resource(), + None + ) + .await, + Err(ValidationError::UseLimitExceeded { current: 1, max: 1 }) + )); + let revoked = revoke_capability( + State(state.clone()), + Extension(Session::new_shell("test-shell".to_string())), + Path(live_id), + ) + .await + .unwrap() + .0; + assert!(revoked.success); + assert!(matches!( + state + .capability_manager + .validate( + &live, + live_session.id.as_str(), + Action::Read, + live.resource(), + None + ) + .await, + Err(ValidationError::TokenRevoked) + )); + } + + #[tokio::test] + async fn test_request_capacity_recovers_late_registered_revoked_grant() { + use elastos_runtime::capability::{ + manager::ValidationError, pending::MAX_PENDING_REQUESTS, CapabilityToken, + }; + use elastos_runtime::session::SessionId; + + let state = test_state(); + let stale_session = Session::new_capsule("stale-epoch".to_string()); + let live_session = Session::new_capsule("live-epoch".to_string()); + let shell = Session::new_shell("test-shell".to_string()); + let resource = ResourceId::new("localhost://Users/self/Documents/test"); + let stale_request = state + .pending_store + .admit_request( + &state.capability_manager, + stale_session.id.clone(), + resource.clone(), + Action::Read, + String::new(), + ) + .await; + // Reproduce the grant handler's mint/register split without timing sleeps. + let stale_token = state.capability_manager.grant( + stale_session.id.as_str(), + resource.clone(), + Action::Read, + TokenConstraints::default(), + None, + ); + let revoked = revoke_all_capabilities( + State(state.clone()), + Extension(shell.clone()), + Json(RevokeAllInput { + reason: "capacity regression".to_string(), + }), + ) + .await + .unwrap() + .0; + assert!(state + .pending_store + .get_request(stale_request.id.as_str()) + .await + .unwrap() + .is_pending()); + state + .pending_store + .grant_request( + stale_request.id.as_str(), + stale_token.clone(), + GrantDuration::Session, + ) + .await + .unwrap(); + assert!(matches!( + state + .capability_manager + .validate( + &stale_token, + stale_session.id.as_str(), + Action::Read, + &resource, + None + ) + .await, + Err(ValidationError::TokenRevoked) + )); + + let live_request = state + .pending_store + .admit_request( + &state.capability_manager, + live_session.id.clone(), + resource.clone(), + Action::Read, + String::new(), + ) + .await; + let live_grant = grant_request( + State(state.clone()), + Extension(shell), + Json(GrantRequestInput { + request_id: live_request.id.to_string(), + duration: "session".to_string(), + rationale: Some("capacity regression".to_string()), + }), + ) + .await + .unwrap() + .0; + let live_token = CapabilityToken::from_base64(&live_grant.token.unwrap()).unwrap(); + assert_eq!(live_token.constraints().epoch(), revoked.new_epoch); + assert_eq!(live_token.constraints().max_uses(), None); + assert!(live_token.expiry().is_none()); + for index in 0..MAX_PENDING_REQUESTS - 2 { + assert!(state + .pending_store + .admit_request( + &state.capability_manager, + SessionId::from_string(format!("prefill-{index}")), + resource.clone(), + Action::Read, + String::new() + ) + .await + .is_pending()); + } + + let recovered = request_capability( + State(state.clone()), + Extension(Session::new_capsule("next".to_string())), + Json(RequestCapabilityInput { + resource: resource.to_string(), + action: "read".to_string(), + reason: "capacity regression".to_string(), + }), + ) + .await + .unwrap() + .0; + assert_eq!( + recovered.status, "pending", + "revoked grants must not permanently fill the store" + ); + assert!(state + .pending_store + .get_request(stale_request.id.as_str()) + .await + .is_none()); + assert_eq!( + state.pending_store.pending_count().await, + MAX_PENDING_REQUESTS - 1 + ); + assert!(state + .pending_store + .admit_request( + &state.capability_manager, + SessionId::from_string("overflow"), + resource.clone(), + Action::Read, + String::new() + ) + .await + .is_denied()); + assert_eq!(state.capability_manager.current_epoch(), revoked.new_epoch); + assert!(matches!( + state + .capability_manager + .validate( + &stale_token, + stale_session.id.as_str(), + Action::Read, + &resource, + None + ) + .await, + Err(ValidationError::TokenRevoked) + )); + state + .capability_manager + .validate( + &live_token, + live_session.id.as_str(), + Action::Read, + &resource, + None, + ) + .await + .unwrap(); + let visible = list_capabilities(State(state), Extension(live_session)) + .await + .0; + assert_eq!(visible.capabilities.len(), 1); + assert_eq!( + visible.capabilities[0].request_id, + live_request.id.to_string() + ); + } + #[tokio::test] async fn test_request_capability_denies_ipfs_backend() { let output = request_capability( diff --git a/elastos/crates/elastos-server/src/collaboration_history.rs b/elastos/crates/elastos-server/src/collaboration_history.rs index 535d7c536..7b69a780a 100644 --- a/elastos/crates/elastos-server/src/collaboration_history.rs +++ b/elastos/crates/elastos-server/src/collaboration_history.rs @@ -1063,7 +1063,7 @@ mod tests { async fn a_history_request_carries_no_presence_after_an_epoch_bump() { let now = now_secs(); let (a, _b, plane) = reachable_pair("history-epoch", now).await; - a.authority.manager().revoke_all("test epoch"); + a.authority.manager().revoke_all("test epoch").unwrap(); a.service.fetch_once(now).await.unwrap(); assert_not_shown_and_nothing_sent(&a, &plane, now); } @@ -1175,7 +1175,10 @@ mod tests { ) .unwrap(); if epoch { - a.authority.manager().revoke_all("projection smoke epoch"); + a.authority + .manager() + .revoke_all("projection smoke epoch") + .unwrap(); } else { a.decide(PresenceDecision::Hidden, now + 1).await; } @@ -1237,7 +1240,10 @@ mod tests { .await .unwrap(); } else { - self.authority.manager().revoke_all("between history peers"); + self.authority + .manager() + .revoke_all("between history peers") + .unwrap(); } response } diff --git a/elastos/crates/elastos-server/src/collaboration_presence_grant.rs b/elastos/crates/elastos-server/src/collaboration_presence_grant.rs index eaaede328..2dbde0694 100644 --- a/elastos/crates/elastos-server/src/collaboration_presence_grant.rs +++ b/elastos/crates/elastos-server/src/collaboration_presence_grant.rs @@ -1029,7 +1029,7 @@ mod tests { .await .is_some()); } - authority.manager().revoke_all("test epoch"); + authority.manager().revoke_all("test epoch").unwrap(); for person in [principal.as_str(), second] { assert_eq!(gate(&authority, data, person, ME, runtime, 60).await, None); assert_eq!(state(data, person, &scope()).unwrap(), None); diff --git a/elastos/crates/elastos-server/src/collaboration_transport.rs b/elastos/crates/elastos-server/src/collaboration_transport.rs index 2b7b04a79..dcf05d4f7 100644 --- a/elastos/crates/elastos-server/src/collaboration_transport.rs +++ b/elastos/crates/elastos-server/src/collaboration_transport.rs @@ -1510,7 +1510,7 @@ mod tests { granted_presence_driver(&a, ac.clone(), FakeCarrier::new()).await; queued_presence(&a, &port, "granted-presence").await; assert_eq!(driver.retry_outgoing_once(NOW).await.unwrap().attempted, 1); - authority.manager().revoke_all("test epoch"); + authority.manager().revoke_all("test epoch").unwrap(); tokio::time::advance(SHARED_CUSTODY_RETRY).await; assert_eq!( driver.retry_outgoing_once(NOW + 5).await.unwrap().attempted, diff --git a/elastos/crates/elastos-server/src/main.rs b/elastos/crates/elastos-server/src/main.rs index ea9eb5b1c..f22d45df9 100755 --- a/elastos/crates/elastos-server/src/main.rs +++ b/elastos/crates/elastos-server/src/main.rs @@ -1446,7 +1446,7 @@ async fn run() -> anyhow::Result<()> { } Commands::Emergency(cmd) => { - security_cmd::run_emergency(cmd)?; + security_cmd::run_emergency(cmd).await?; } Commands::Config(cmd) => { diff --git a/elastos/crates/elastos-server/src/resource_bridge.rs b/elastos/crates/elastos-server/src/resource_bridge.rs index c91eeb127..7e3cbf349 100644 --- a/elastos/crates/elastos-server/src/resource_bridge.rs +++ b/elastos/crates/elastos-server/src/resource_bridge.rs @@ -842,8 +842,11 @@ async fn handle_request(line: &str, ctx: &Option) -> Result anyhow::Result<()> { Ok(()) } -pub fn run_emergency(cmd: EmergencyCommand) -> anyhow::Result<()> { +pub async fn run_emergency(cmd: EmergencyCommand) -> anyhow::Result<()> { + let data_dir = elastos_server::sources::default_data_dir(); + run_emergency_in(&data_dir, cmd).await +} + +async fn run_emergency_in(data_dir: &std::path::Path, cmd: EmergencyCommand) -> anyhow::Result<()> { match cmd { EmergencyCommand::Rotate { reason } => { - let data_dir = elastos_server::sources::default_data_dir(); - let _ = std::fs::create_dir_all(&data_dir); - - let store = Arc::new(capability::CapabilityStore::new()); + std::fs::create_dir_all(data_dir)?; + let store = Arc::new( + capability::CapabilityStore::with_persistence(data_dir.join("capability_store")) + .await?, + ); let audit_log = Arc::new(primitives::audit::AuditLog::new()); let metrics = Arc::new(primitives::metrics::MetricsManager::new()); let mut mgr = capability::CapabilityManager::load_or_generate( - &data_dir, store, audit_log, metrics, + data_dir, store, audit_log, metrics, ); - let new_pub = mgr.rotate_signing_key(&data_dir, &reason); - println!("Signing key rotated."); + let new_pub = mgr.rotate_signing_key(data_dir, &reason)?; + println!("New signing key saved to disk."); println!(" New public key: {}", hex::encode(new_pub)); - println!(" Epoch advanced: all prior tokens invalidated."); + println!(" Durable epoch advanced: tokens from earlier epochs are rejected."); println!(" Reason: {}", reason); println!(); - println!("Restart the runtime to apply the new key."); + println!("Restart required: restart every Runtime that uses this data directory to apply the new key."); + println!("Running Runtime processes retain their loaded signing key until restart."); } } Ok(()) } + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn emergency_rotation_cli_reports_restart_requirement() { + const CHILD_DIR: &str = "ELASTOS_EMERGENCY_ROTATION_TEST_DIR"; + if let Some(directory) = std::env::var_os(CHILD_DIR) { + use clap::Parser as _; + + let cli = crate::Cli::try_parse_from([ + "elastos", + "emergency", + "rotate", + "--reason", + "isolated command fixture", + ]) + .unwrap(); + let Some(crate::Commands::Emergency(command)) = cli.command else { + panic!("the CLI must dispatch emergency rotation"); + }; + let runtime = tokio::runtime::Builder::new_current_thread() + .build() + .unwrap(); + runtime + .block_on(run_emergency_in(std::path::Path::new(&directory), command)) + .unwrap(); + return; + } + + let data_dir = tempfile::tempdir().unwrap(); + let runtime = tokio::runtime::Builder::new_current_thread() + .build() + .unwrap(); + let store_path = data_dir.path().join("capability_store"); + let store = runtime + .block_on(capability::CapabilityStore::with_persistence(&store_path)) + .unwrap(); + assert_eq!(store.advance_epoch().unwrap(), 1); + + let result = std::process::Command::new(std::env::current_exe().unwrap()) + .args([ + "--exact", + "security_cmd::tests::emergency_rotation_cli_reports_restart_requirement", + "--nocapture", + ]) + .env(CHILD_DIR, data_dir.path()) + .output() + .unwrap(); + assert!( + result.status.success(), + "rotation command failed: {}", + String::from_utf8_lossy(&result.stderr) + ); + let output = String::from_utf8_lossy(&result.stdout); + assert!(output.contains("running 1 test"), "{output}"); + assert!( + output.contains("New signing key saved to disk."), + "{output}" + ); + assert!( + output.contains("Durable epoch advanced: tokens from earlier epochs are rejected."), + "{output}" + ); + assert!( + output.contains("Restart required: restart every Runtime that uses this data directory to apply the new key."), + "{output}" + ); + assert!( + output.contains( + "Running Runtime processes retain their loaded signing key until restart." + ), + "{output}" + ); + assert!(!output.contains("all prior tokens invalidated"), "{output}"); + assert_eq!( + runtime + .block_on(capability::CapabilityStore::with_persistence(&store_path)) + .unwrap() + .current_epoch(), + 2 + ); + assert_eq!( + std::fs::read(data_dir.path().join("signing_key")) + .unwrap() + .len(), + 32 + ); + } + + #[tokio::test] + async fn emergency_rotation_uses_the_commands_persistent_store() { + let data_dir = tempfile::tempdir().unwrap(); + let store_path = data_dir.path().join("capability_store"); + let store = capability::CapabilityStore::with_persistence(&store_path) + .await + .unwrap(); + for _ in 0..7 { + store.advance_epoch().unwrap(); + } + for expected in [8, 9] { + run_emergency_in( + data_dir.path(), + EmergencyCommand::Rotate { + reason: "isolated command fixture".into(), + }, + ) + .await + .unwrap(); + let reloaded = capability::CapabilityStore::with_persistence(&store_path) + .await + .unwrap(); + assert_eq!( + reloaded.current_epoch(), + expected, + "the command must advance the durable epoch" + ); + assert!(data_dir.path().join("signing_key").is_file()); + } + // A separately loaded, older store also preserves the command result. + store.persist().await.unwrap(); + let reloaded = capability::CapabilityStore::with_persistence(&store_path) + .await + .unwrap(); + assert_eq!(reloaded.current_epoch(), 9); + } + + #[tokio::test] + async fn emergency_rotation_refuses_unreadable_state_before_creating_a_key() { + for missing in [true, false] { + let data_dir = tempfile::tempdir().unwrap(); + let store_path = data_dir.path().join("capability_store"); + let store = capability::CapabilityStore::with_persistence(&store_path) + .await + .unwrap(); + assert_eq!(store.advance_epoch().unwrap(), 1); + let epoch_path = store_path.join("epoch"); + if missing { + std::fs::remove_file(&epoch_path).unwrap(); + } else { + std::fs::write(&epoch_path, "invalid").unwrap(); + } + let error = run_emergency_in( + data_dir.path(), + EmergencyCommand::Rotate { + reason: "isolated command fixture".into(), + }, + ) + .await + .expect_err("the command must refuse unreadable durable state") + .to_string(); + assert!( + error.contains("Capability store startup refused"), + "{error}" + ); + assert!(error.contains(&epoch_path.display().to_string()), "{error}"); + assert!(error.contains("latest committed epoch"), "{error}"); + assert!(!data_dir.path().join("signing_key").exists()); + if missing { + assert!(!epoch_path.exists()); + } else { + assert_eq!(std::fs::read_to_string(&epoch_path).unwrap(), "invalid"); + } + } + } +} diff --git a/elastos/crates/elastos-server/src/server_infra.rs b/elastos/crates/elastos-server/src/server_infra.rs index 33ac418a7..7bc92e7f5 100644 --- a/elastos/crates/elastos-server/src/server_infra.rs +++ b/elastos/crates/elastos-server/src/server_infra.rs @@ -650,10 +650,6 @@ async fn setup_server_infrastructure_impl( )?; let audit_log = Arc::new(primitives::audit::AuditLog::new()); - let session_registry = Arc::new(session::SessionRegistry::new(audit_log.clone())); - session_registry - .set_default_owner(local_session_owner(&data_dir)?) - .await; let metrics = Arc::new(primitives::metrics::MetricsManager::new()); let capability_store = serve_capability_store(&data_dir).await?; let capability_manager = Arc::new(capability::CapabilityManager::load_or_generate( @@ -662,9 +658,12 @@ async fn setup_server_infrastructure_impl( audit_log.clone(), metrics.clone(), )); - let pending_store = Arc::new(capability::pending::PendingRequestStore::new( - audit_log.clone(), - )); + let (session_registry, pending_store) = + session::SessionRegistry::with_capabilities(audit_log.clone(), capability_manager.clone()); + let session_registry = Arc::new(session_registry); + session_registry + .set_default_owner(local_session_owner(&data_dir)?) + .await; let tls_config = match elastos_tls::load_or_create_tls_config(&data_dir).await { Ok(config) => { @@ -4259,7 +4258,7 @@ mod tests { ) .await .expect("grant still validates before revoke"); - manager.revoke_all("test restart revoke"); + manager.revoke_all("test restart revoke").unwrap(); token }; diff --git a/justfile b/justfile index b219d847f..fd7860f87 100644 --- a/justfile +++ b/justfile @@ -245,8 +245,17 @@ product-data: # Verify the checked-in ElastOS Bus contract and real Component fixture bus-conformance: + #!/usr/bin/env bash + set -euo pipefail node scripts/check-elastos-bus-wit.mjs - cd elastos && cargo test -p elastos-server component_runs_through_real_bus_authority_provider_and_audit_paths -- --nocapture + cd elastos + test_name=runtime::tests::component_conformance_exercises_bus_authorization_dispatch_and_audit + matches="$(cargo test --locked -p elastos-server --lib "$test_name" -- --exact --list | awk -v expected="$test_name: test" '$0 == expected { count++ } END { print count + 0 }')" + if [[ "$matches" != 1 ]]; then + echo "Bus conformance must select exactly one test; found $matches" >&2 + exit 1 + fi + cargo test --locked -p elastos-server --lib "$test_name" -- --exact --include-ignored --nocapture # Validate canonical capsule scaffolds against the current manifest and WIT contracts capsule-templates: