From 2b322d53b511018f01274856e14a1470dbdaa4e1 Mon Sep 17 00:00:00 2001 From: chhhee10 Date: Wed, 30 Sep 2026 01:05:43 +0530 Subject: [PATCH 01/41] daemon: materialise Cloud Jev policies and report policy errors DesiredState gains semanticPolicies + jevMode, DesiredPolicy gains authority + reviewedBy (all optional, skipped when empty so today's desired-state.json and active.json are byte-identical). Semantic artifacts go through the same same-origin sha-verified fetch into artifacts/.json, and both lists activate in ONE atomic active.json write. Repair-from-cache covers semantic artifacts. The poll sends policyErrors (CLI errors.json merged with the daemon's own reconcile errors, persisted in daemon-errors.json), URL-encoded and capped at 4 KiB by truncating the list, only once either side has an error state. CLOUD_POLICIES.md: new fields, error report, and the stale cloud-managed / deployments/ / cloud.json names fixed. Co-Authored-By: Claude Opus 5.5 --- crates/CLOUD_POLICIES.md | 125 +++- crates/failproofaid/src/cloud_client.rs | 439 ++++++++++++- crates/failproofaid/src/cloud_policies.rs | 757 ++++++++++++++++++++-- 3 files changed, 1225 insertions(+), 96 deletions(-) diff --git a/crates/CLOUD_POLICIES.md b/crates/CLOUD_POLICIES.md index da650e585..5d61a91da 100644 --- a/crates/CLOUD_POLICIES.md +++ b/crates/CLOUD_POLICIES.md @@ -7,26 +7,30 @@ hook path; hooks evaluate only the last verified local deployment. ## Layout ```text -~/.failproofai/policies/cloud-managed/ +~/.failproofai/policies/cloud-policies/ (FAILPROOFAI_CLOUD_POLICY_DIR overrides) ├── desired-state.json ├── active.json -├── artifacts/ -│ └── .mjs -└── deployments/ - └── / - ├── manifest.json - └── .mjs +├── errors.json written by the CLI: what it could not load +├── daemon-errors.json written by the daemon: its own reconcile errors +└── artifacts/ + ├── .mjs a JS policy (kind regex / both) + └── .json a Jev policy's declarations (kind jev / both) ``` - `desired-state.json` is the last complete desired-state snapshot received from cloud. A later slice must authenticate it with a publisher signature; SHA-256 alone proves byte identity, not publisher identity. -- `artifacts/` is a content-addressed cache. -- `deployments/` contains complete materialized policy sets. New deployments - are built without modifying the active one. -- `active.json` is an atomically replaced pointer to one complete deployment. - It is derived state and is reconstructed when it is deleted, malformed, or - disagrees with `desired-state.json`. +- `artifacts/` is a content-addressed cache and holds the only copy of each + artifact. The pre-layout-3 `generations/` / `deployments/` trees are removed + after the first successful flip. +- `active.json` is an atomically replaced pointer to one complete deployment — + its JS policies AND its Jev policies. It is derived state and is + reconstructed when it is deleted, malformed, or disagrees with + `desired-state.json`. + +The directory is `cloud-policies`, matching `cloudPoliciesDir()` in +`src/hooks/fp-home.ts`; `paths.rs`' parity test executes that module to keep +the two in step. ## Desired-state schema @@ -36,18 +40,69 @@ hook path; hooks evaluate only the last verified local deployment. "deployment": 184, "policies": [ { - "id": "block-secret-exfiltration", + "id": "no-prod-db", "version": 7, "sha256": "<64 lowercase hex characters>", - "artifactUrl": "https://..." + "artifactUrl": "/enforcement/v1/artifacts/", + "effect": "enforce", + "authority": "reviewable", + "reviewedBy": ["acme-prod-db-write"] } - ] + ], + "semanticPolicies": [ + { + "id": "no-prod-db", + "version": 7, + "sha256": "<64 lowercase hex characters>", + "artifactUrl": "/enforcement/v1/artifacts/" + } + ], + "jevMode": "observe" } ``` +The schema stays 2: every field after `policies[].effect` is optional and +omitted when empty, so a fleet with no Jev policy receives, and writes back to +disk, byte-identical JSON to what it always did. + +- `policies[]` carries the JS half of `regex` and `both` policies. A `both` + policy's entry adds `authority: "reviewable"` and `reviewedBy` (the names of + its own Jev checks); the server derives both, the client never supplies them. + `authority` must be `hard` or `reviewable`, or the whole state is refused. +- `semanticPolicies[]` carries `jev` and `both` policies. Each artifact is a + JSON array of pack-manifest semantic declarations; the daemon never parses + it, the CLI parses it with the same function it uses for a pack manifest. + Ids are unique within this list; the same id may appear in both lists (that + is what a `both` policy looks like). +- `jevMode` (`off | observe | enforce`), when present, overrides the local + `jev.json` mode — a local `off` included. Absent means Cloud does not + override. Any other value refuses the whole state. + The artifact URL is opaque to the store. A cloud client supplies downloaded -bytes through `ArtifactFetcher`; the reconciler trusts none of those bytes -until their SHA-256 matches the desired state. +bytes through `ArtifactFetcher` — the same same-origin, bearer-authenticated +fetch for both artifact kinds; the reconciler trusts none of those bytes until +their SHA-256 matches the desired state. + +## Error report + +The poll carries an optional `policyErrors` query parameter: compact JSON +`[{"id": str, "version": int|null, "kind": "regex"|"jev"|"both"|"daemon", "message": str}]`, +URL-encoded, at most 4 KiB encoded (whole entries are dropped from the end; +the JSON is never cut). It merges: + +- `daemon-errors.json` — the daemon's own reconcile errors (`kind: "daemon"`): + a refused payload, a failed fetch, a digest mismatch. Transport failures are + not reported. +- `errors.json` — written by the CLI, atomically and only when its content + changes, as `{"errors": [...]}`: a Cloud JS policy that failed to load, a + semantic artifact that failed its digest or parse, a declaration its parser + dropped, a `reviewedBy` naming a check that is not present, and + `jev_unconfigured` (Cloud set a Jev mode and the machine has no provider). + +The parameter is sent on every poll once either file exists (`[]` once the +problems are fixed, which clears them on the server) and omitted while neither +side has ever recorded an error state. The server stores the latest report per +machine and shows it on the fleet page. ## Cloud transport @@ -60,8 +115,10 @@ failproofai config --connect https://be.failproof.ai \ ``` That verifies the credentials against the server before storing anything, then -writes `~/.failproofai/cloud.json` (mode 0600). `--disconnect` removes it, -`--status` reports the connection with the token masked. +writes the `cloud` object of `~/.failproofai/credentials.json` (mode 0600; the +layout-1 `cloud.json` is still read when `credentials.json` is absent). +`--disconnect` removes it together with `active.json` and the two error files, +and `--status` reports the connection with the token masked. **The credential must not go in the service unit.** `daemon-service.ts` installs `/etc/systemd/system/failproofaid@.service` at mode 0644 — root-owned and @@ -82,7 +139,7 @@ and tests: FAILPROOFAI_CLOUD_URL=https://be.failproof.ai FAILPROOFAI_CLOUD_TOKEN= FAILPROOFAI_MACHINE_ID= -FAILPROOFAI_CLOUD_CREDENTIALS= # overrides ~/.failproofai/cloud.json +FAILPROOFAI_CLOUD_CREDENTIALS= # a standalone credentials JSON file ``` `FAILPROOFAI_CLOUD_POLICY_POLL_MS` controls the interval (30 seconds by @@ -96,24 +153,28 @@ deployment leaves the previous deployment active. ## Activation transaction -1. Validate schema, policy IDs, unique IDs, digests, and monotonic deployment. -2. Reuse a verified cache/deployment copy or fetch missing bytes. +1. Validate schema, policy IDs, unique IDs (within each list), digests, + `authority`, `jevMode`, and monotonic deployment. +2. Reuse a verified cached artifact or fetch missing bytes — JS and semantic. 3. Verify every artifact digest. -4. Materialize the complete deployment and `fsync` its files/directories. +4. Write each verified artifact atomically and `fsync` it. 5. Persist `desired-state.json`. -6. Atomically replace `active.json`. +6. Atomically replace `active.json` — both lists and `jevMode` in one write. -Any failure before step 6 leaves the previous deployment active. The worker -loads only paths named by `active.json` and independently verifies every digest -immediately before importing JavaScript. +Any failure before step 6 leaves the previous deployment active, across both +lists: a `both` policy's JS half never goes live without the Jev checks that +review it. The worker loads only paths named by `active.json` and +independently verifies every digest immediately before importing JavaScript +or parsing a semantic artifact. ## Integrity maintenance `failproofaid` runs a maintenance thread outside the hook path. It hashes the -active deployment periodically (30 seconds by default) and repairs either a -modified deployment copy or a modified content-addressed artifact from the -other verified copy. If both are gone, it retains the active manifest and -reports that a cloud re-fetch is required. +active deployment's artifacts — JS and semantic — periodically (30 seconds by +default). A lost or corrupt `active.json` is rebuilt from `desired-state.json` +and the verified cache. There is one copy of each artifact, so a modified one +cannot be repaired offline: the thread keeps the active manifest and reports +that a cloud re-fetch is required, and the next successful poll re-fetches it. `FAILPROOFAI_CLOUD_POLICY_DIR` overrides the root for tests and development. When cloud polling is disabled, `FAILPROOFAI_CLOUD_POLICY_RECONCILE_MS` diff --git a/crates/failproofaid/src/cloud_client.rs b/crates/failproofaid/src/cloud_client.rs index 84a428110..b5a2c556a 100644 --- a/crates/failproofaid/src/cloud_client.rs +++ b/crates/failproofaid/src/cloud_client.rs @@ -1,5 +1,5 @@ use crate::cloud_policies::{ - DESIRED_STATE_SCHEMA_VERSION, DesiredPolicy, DesiredState, PolicyStore, + DESIRED_STATE_SCHEMA_VERSION, DesiredPolicy, DesiredState, PolicyErrorEntry, PolicyStore, }; use reqwest::Url; use reqwest::blocking::Client; @@ -11,6 +11,146 @@ use std::time::{Duration, Instant}; const DEFAULT_POLL_MS: u64 = 30_000; const MINIMUM_POLL_MS: u64 = 100; +/// The `policyErrors` parameter's ceiling, measured URL-ENCODED — the form it +/// actually travels in. Over it, whole entries are dropped from the end: the +/// list is truncated, never the JSON. +pub const MAX_POLICY_ERRORS_ENCODED_BYTES: usize = 4096; +/// One message's ceiling, so a single pathological message cannot push every +/// other entry out of the report. +const MAX_POLICY_ERROR_MESSAGE_CHARS: usize = 500; +const MAX_POLICY_ERROR_ID_CHARS: usize = 128; + +/// Why a poll produced no usable desired state. +/// +/// Split because only one half is worth REPORTING back: a payload this daemon +/// could not accept (an unknown `effect`, an unreadable body, a schema skew) is +/// the server's to see and fix, while a refused connection or a 502 is gone by +/// the time any report could reach anyone. +#[derive(Debug)] +struct PollFailure { + message: String, + reportable: bool, +} + +impl PollFailure { + fn transport(message: String) -> Self { + Self { + message, + reportable: false, + } + } + + fn payload(message: String) -> Self { + Self { + message, + reportable: true, + } + } +} + +/// Compact JSON for the `policyErrors` parameter, within +/// [`MAX_POLICY_ERRORS_ENCODED_BYTES`] once URL-encoded. +/// +/// Long messages are shortened first (by characters, so a multi-byte message is +/// never cut mid-codepoint), then whole entries are dropped from the end until +/// the encoded form fits. What is sent is always a complete JSON array. +pub fn encode_policy_errors(entries: &[PolicyErrorEntry]) -> String { + let clipped: Vec = entries + .iter() + .map(|entry| PolicyErrorEntry { + id: clip(&entry.id, MAX_POLICY_ERROR_ID_CHARS), + version: entry.version, + kind: entry.kind.clone(), + message: clip(&entry.message, MAX_POLICY_ERROR_MESSAGE_CHARS), + }) + .collect(); + let mut keep = clipped.len(); + loop { + let json = serde_json::to_string(&clipped[..keep]).unwrap_or_else(|_| "[]".to_string()); + if keep == 0 || encoded_len(&json) <= MAX_POLICY_ERRORS_ENCODED_BYTES { + return json; + } + keep -= 1; + } +} + +fn clip(value: &str, max_chars: usize) -> String { + if value.chars().count() <= max_chars { + return value.to_string(); + } + let mut out: String = value.chars().take(max_chars.saturating_sub(1)).collect(); + out.push('…'); + out +} + +/// The length `value` takes as a query-parameter value, encoded exactly the +/// way the poll encodes it. +fn encoded_len(value: &str) -> usize { + let mut url = Url::parse("http://x.invalid/").expect("static URL parses"); + url.query_pairs_mut().append_pair("v", value); + url.query() + .map_or(0, |query| query.len().saturating_sub("v=".len())) +} + +/// The machine's current report: the daemon's own reconcile errors first, then +/// the CLI's. `None` — parameter omitted — only while NEITHER has ever written +/// an error state, so a machine that never had a problem sends nothing new, +/// and one that had a problem sends `[]` once it is fixed (which is what +/// clears it on the server). +fn current_policy_errors(store: &PolicyStore) -> Option> { + let daemon = store.read_daemon_policy_errors(); + let cli = store.read_cli_policy_errors(); + if daemon.is_none() && cli.is_none() { + return None; + } + let mut all = daemon.unwrap_or_default(); + all.extend(cli.unwrap_or_default()); + Some(all) +} + +/// A daemon error entry for a failed poll or reconcile. The version is looked +/// up in the desired state when the error names a policy it carries. +fn daemon_error( + message: String, + policy_id: Option<&str>, + desired: Option<&DesiredState>, +) -> PolicyErrorEntry { + let version = policy_id.and_then(|id| { + desired.and_then(|d| { + d.policies + .iter() + .find(|p| p.id == id) + .map(|p| p.version) + .or_else(|| { + d.semantic_policies + .iter() + .find(|p| p.id == id) + .map(|p| p.version) + }) + }) + }); + PolicyErrorEntry { + // A state-level error names no policy; `desired-state` says which + // document was refused. + id: policy_id.unwrap_or("desired-state").to_string(), + version, + kind: "daemon".to_string(), + message, + } +} + +/// Records the daemon's error state after a poll. Errors always overwrite; a +/// clean poll clears an EXISTING state to `[]` but never creates one, so the +/// report stays absent on a machine that never had a problem. +fn record_daemon_errors(store: &PolicyStore, errors: Vec) { + if errors.is_empty() && store.read_daemon_policy_errors().is_none() { + return; + } + if let Err(err) = store.write_daemon_policy_errors(&errors) { + eprintln!("[failproofaid] could not record the cloud policy error state: {err}"); + } +} + #[derive(Clone)] pub struct CloudClient { base_url: Url, @@ -339,30 +479,45 @@ impl CloudClient { /// as 0: a machine that cannot say what it is enforcing must not be recorded /// as enforcing deployment zero, and absent has to stay distinguishable from /// "reported nothing" on the far side. - pub fn desired_state(&self, applied: Option) -> Result { + /// + /// `policy_errors` is the machine's error report, already encoded as the + /// compact JSON list (see [`encode_policy_errors`]); `None` omits the + /// parameter, which the server reads as "leave what you have stored". + fn poll_desired_state( + &self, + applied: Option, + policy_errors: Option<&str>, + ) -> Result { let mut url = self .base_url .join("enforcement/v1/desired-state") - .map_err(|err| format!("failed to build desired-state URL: {err}"))?; + .map_err(|err| { + PollFailure::transport(format!("failed to build desired-state URL: {err}")) + })?; url.query_pairs_mut() .append_pair("machineId", &self.machine_id); if let Some(deployment) = applied { url.query_pairs_mut() .append_pair("appliedDeployment", &deployment.to_string()); } + // Additive, like `appliedDeployment`: `DesiredQuery` on the server is + // not `deny_unknown_fields`, so a server that predates it ignores it. + if let Some(errors) = policy_errors { + url.query_pairs_mut().append_pair("policyErrors", errors); + } self.client .get(url) .bearer_auth(&self.token) .send() .and_then(|response| response.error_for_status()) .map_err(|err| { - format!( + PollFailure::transport(format!( "desired-state request failed: {}", fpai_collect::error_chain(&err) - ) + )) })? .json::() - .map_err(|err| format!("invalid desired-state response: {err}")) + .map_err(|err| PollFailure::payload(format!("invalid desired-state response: {err}"))) .and_then(|raw| { // THE VERSION IS CHECKED BEFORE THE FIELDS, which is the whole // point of having one. `SUPPORTED_SCHEMA_VERSIONS` accepts 1 as @@ -381,19 +536,20 @@ impl CloudClient { match version { Some(v) if v == u64::from(DESIRED_STATE_SCHEMA_VERSION) => {} Some(v) => { - return Err(format!( + return Err(PollFailure::payload(format!( "server sent desired-state schemaVersion {v} but this daemon \ speaks {DESIRED_STATE_SCHEMA_VERSION} — upgrade whichever half is behind" - )); + ))); } None => { - return Err( - "desired-state response has no schemaVersion field".to_string() - ); + return Err(PollFailure::payload( + "desired-state response has no schemaVersion field".to_string(), + )); } } - serde_json::from_value::(raw) - .map_err(|err| format!("invalid desired-state response: {err}")) + serde_json::from_value::(raw).map_err(|err| { + PollFailure::payload(format!("invalid desired-state response: {err}")) + }) }) } @@ -505,22 +661,45 @@ fn poll_once(store: &PolicyStore, cloud: &CloudClient) { None } }; - match cloud.desired_state(applied) { + // Every poll carries the current report once there is one to carry — the + // previous poll's reconcile outcome plus whatever the CLI last wrote. + let report = current_policy_errors(store).map(|errors| encode_policy_errors(&errors)); + match cloud.poll_desired_state(applied, report.as_deref()) { Ok(desired) => { match store.reconcile(&desired, &|policy: &DesiredPolicy| cloud.artifact(policy)) { - Ok(outcome) - if outcome.activated || outcome.downloaded > 0 || outcome.repaired > 0 => - { - eprintln!( - "[failproofaid] cloud policy deployment {} active (downloaded {}, repaired {})", - outcome.deployment, outcome.downloaded, outcome.repaired + Ok(outcome) => { + if outcome.activated || outcome.downloaded > 0 || outcome.repaired > 0 { + eprintln!( + "[failproofaid] cloud policy deployment {} active (downloaded {}, repaired {})", + outcome.deployment, outcome.downloaded, outcome.repaired + ); + } + record_daemon_errors(store, Vec::new()); + } + Err(err) => { + eprintln!("[failproofaid] cloud policy reconcile error: {err}"); + record_daemon_errors( + store, + vec![daemon_error( + err.to_string(), + err.policy_id(), + Some(&desired), + )], ); } - Ok(_) => {} - Err(err) => eprintln!("[failproofaid] cloud policy reconcile error: {err}"), } } - Err(err) => eprintln!("[failproofaid] cloud policy poll error: {err}"), + Err(failure) => { + eprintln!( + "[failproofaid] cloud policy poll error: {}", + failure.message + ); + // A payload this daemon refused is reported; a transport failure is + // not, and leaves the recorded state as it was. + if failure.reportable { + record_daemon_errors(store, vec![daemon_error(failure.message, None, None)]); + } + } } } @@ -553,6 +732,14 @@ mod tests { // Only the fixtures construct an effect explicitly. use crate::cloud_policies::PolicyEffect; + impl CloudClient { + /// The poll without an error report, for the tests that predate it. + fn desired_state(&self, applied: Option) -> Result { + self.poll_desired_state(applied, None) + .map_err(|failure| failure.message) + } + } + // std::env::set_var is process-global, so these must not interleave with // each other or with anything else reading the same variables — including // `paths.rs`, which sets the same HOME. One crate-wide lock; see test_env.rs. @@ -872,6 +1059,210 @@ mod tests { fs::remove_dir_all(root).unwrap(); } + // ── Cloud Jev policies + the policyErrors report (CONTRACT C4/C5) ─────── + + fn error(id: &str, message: &str) -> PolicyErrorEntry { + PolicyErrorEntry { + id: id.into(), + version: Some(1), + kind: "jev".into(), + message: message.into(), + } + } + + #[test] + fn the_report_is_compact_json_under_four_kib_encoded_and_truncates_the_list() { + assert_eq!(encode_policy_errors(&[]), "[]"); + let one = encode_policy_errors(&[error("p", "bad sha")]); + assert_eq!( + one, + r#"[{"id":"p","version":1,"kind":"jev","message":"bad sha"}]"# + ); + + // Far more than fits: the list is cut, the JSON never is. + let many: Vec<_> = (0..200) + .map(|i| { + error( + &format!("policy-{i}"), + &"é needs encoding & more ".repeat(20), + ) + }) + .collect(); + let encoded = encode_policy_errors(&many); + assert!(encoded_len(&encoded) <= MAX_POLICY_ERRORS_ENCODED_BYTES); + let parsed: Vec = + serde_json::from_str(&encoded).expect("still a complete JSON array"); + assert!(!parsed.is_empty() && parsed.len() < many.len()); + assert_eq!(parsed[0].id, "policy-0", "truncated from the END"); + assert!( + parsed[0].message.chars().count() <= MAX_POLICY_ERROR_MESSAGE_CHARS, + "one long message cannot crowd out the rest" + ); + } + + #[test] + fn the_report_is_omitted_until_either_side_has_an_error_state() { + let root = std::env::temp_dir().join(format!("failproofaid-report-{}", std::process::id())); + let _ = fs::remove_dir_all(&root); + let store = PolicyStore::new(root.clone()); + + assert_eq!(current_policy_errors(&store), None, "never had one: omit"); + // A clean poll on such a machine does not create a state either. + record_daemon_errors(&store, Vec::new()); + assert!(!store.daemon_errors_path().exists()); + assert_eq!(current_policy_errors(&store), None); + + record_daemon_errors(&store, vec![daemon_error("bad".into(), Some("p"), None)]); + assert_eq!(current_policy_errors(&store).unwrap().len(), 1); + // Fixed: the state is now `[]`, which is what clears the server. + record_daemon_errors(&store, Vec::new()); + assert_eq!(current_policy_errors(&store), Some(vec![])); + + fs::write( + store.cli_errors_path(), + r#"{"errors":[{"id":"q","version":2,"kind":"both","message":"m"}]}"#, + ) + .unwrap(); + let merged = current_policy_errors(&store).unwrap(); + assert_eq!(merged.len(), 1); + assert_eq!(merged[0].id, "q"); + fs::remove_dir_all(root).ok(); + } + + #[test] + fn daemon_errors_name_the_policy_and_its_version_when_they_can() { + let desired: DesiredState = serde_json::from_str( + r#"{"schemaVersion":2,"deployment":1,"policies":[], + "semanticPolicies":[{"id":"j","version":4,"sha256":"aa","artifactUrl":"/a"}]}"#, + ) + .unwrap(); + let named = daemon_error("m".into(), Some("j"), Some(&desired)); + assert_eq!((named.id.as_str(), named.version), ("j", Some(4))); + assert_eq!(named.kind, "daemon"); + let state = daemon_error("m".into(), None, None); + assert_eq!((state.id.as_str(), state.version), ("desired-state", None)); + } + + /// The report reaches the wire as the `policyErrors` parameter, and is + /// absent when there is nothing to say. + #[test] + fn the_poll_carries_policy_errors_when_given() { + for report in [ + Some(r#"[{"id":"p","version":1,"kind":"jev","message":"x y"}]"#), + None, + ] { + let listener = TcpListener::bind("127.0.0.1:0").unwrap(); + let address = listener.local_addr().unwrap(); + let captured = Arc::new(std::sync::Mutex::new(String::new())); + let sink = captured.clone(); + let server = std::thread::spawn(move || { + let (mut stream, _) = listener.accept().unwrap(); + let mut request = [0_u8; 8192]; + let read = stream.read(&mut request).unwrap(); + *sink.lock().unwrap() = String::from_utf8_lossy(&request[..read]).to_string(); + let body = br#"{"schemaVersion":2,"deployment":7,"policies":[]}"#; + write!( + stream, + "HTTP/1.1 200 OK\r\nContent-Length: {}\r\nContent-Type: application/json\r\nConnection: close\r\n\r\n", + body.len() + ) + .unwrap(); + stream.write_all(body).unwrap(); + }); + let cloud = + CloudClient::new(&format!("http://{address}"), "t".into(), "machine-1".into()) + .unwrap(); + cloud.poll_desired_state(Some(7), report).unwrap(); + server.join().unwrap(); + let request = captured.lock().unwrap().clone(); + let first_line = request.lines().next().unwrap_or_default().to_string(); + if report.is_some() { + assert!( + first_line.contains("policyErrors=%5B%7B%22id%22%3A%22p%22"), + "URL-encoded compact JSON expected: {first_line}" + ); + assert!(first_line.contains("appliedDeployment=7")); + } else { + assert!(!first_line.contains("policyErrors"), "{first_line}"); + } + } + } + + /// A payload the daemon refuses is reportable; a transport failure is not. + #[test] + fn only_payload_failures_are_reportable() { + let port = TcpListener::bind("127.0.0.1:0") + .unwrap() + .local_addr() + .unwrap() + .port(); + let cloud = + CloudClient::new(&format!("http://127.0.0.1:{port}"), "t".into(), "m".into()).unwrap(); + assert!(!cloud.poll_desired_state(None, None).unwrap_err().reportable); + assert!(PollFailure::payload("x".into()).reportable); + } + + /// End to end over HTTP: a `jev` artifact is fetched through the same + /// same-origin, bearer-authenticated path and lands as `artifacts/.json`. + #[test] + fn fetches_a_semantic_artifact_into_the_store() { + let decls = br#"[{"name":"acme-x"}]"#.to_vec(); + let sha = Sha256::digest(&decls) + .iter() + .map(|byte| format!("{byte:02x}")) + .collect::(); + let listener = TcpListener::bind("127.0.0.1:0").unwrap(); + let address = listener.local_addr().unwrap(); + let expected_sha = sha.clone(); + let body_decls = decls.clone(); + let server = std::thread::spawn(move || { + for _ in 0..2 { + let (mut stream, _) = listener.accept().unwrap(); + let mut request = [0_u8; 4096]; + let read = stream.read(&mut request).unwrap(); + let request = String::from_utf8_lossy(&request[..read]).to_string(); + assert!( + request + .to_ascii_lowercase() + .contains("authorization: bearer tok") + ); + let body = if request.contains("desired-state") { + format!(r#"{{"schemaVersion":2,"deployment":3,"policies":[],"semanticPolicies":[{{"id":"j","version":1,"sha256":"{expected_sha}","artifactUrl":"/enforcement/v1/artifacts/{expected_sha}"}}],"jevMode":"enforce"}}"#).into_bytes() + } else { + assert!( + request + .starts_with(&format!("GET /enforcement/v1/artifacts/{expected_sha}")) + ); + body_decls.clone() + }; + write!(stream, "HTTP/1.1 200 OK\r\nContent-Length: {}\r\nContent-Type: application/json\r\nConnection: close\r\n\r\n", body.len()).unwrap(); + stream.write_all(&body).unwrap(); + } + }); + let cloud = + CloudClient::new(&format!("http://{address}"), "tok".into(), "m".into()).unwrap(); + let desired = cloud.desired_state(None).unwrap(); + let root = + std::env::temp_dir().join(format!("failproofaid-jev-http-{}", std::process::id())); + let _ = fs::remove_dir_all(&root); + let store = PolicyStore::new(root.clone()); + store + .reconcile(&desired, &|policy: &DesiredPolicy| cloud.artifact(policy)) + .unwrap(); + let active = store.read_active().unwrap().unwrap(); + assert_eq!(active.jev_mode.as_deref(), Some("enforce")); + assert_eq!( + active.semantic_policies[0].path, + format!("artifacts/{sha}.json") + ); + assert_eq!( + fs::read(root.join(&active.semantic_policies[0].path)).unwrap(), + decls + ); + server.join().unwrap(); + fs::remove_dir_all(root).ok(); + } + #[test] fn rejects_cross_origin_artifacts_before_sending_the_token() { let cloud = @@ -882,6 +1273,8 @@ mod tests { sha256: "0".repeat(64), artifact_url: "https://evil.example/artifact".into(), effect: PolicyEffect::Enforce, + authority: None, + reviewed_by: None, }; assert!(cloud.artifact(&policy).unwrap_err().contains("outside")); } diff --git a/crates/failproofaid/src/cloud_policies.rs b/crates/failproofaid/src/cloud_policies.rs index 8acd7993c..03976426c 100644 --- a/crates/failproofaid/src/cloud_policies.rs +++ b/crates/failproofaid/src/cloud_policies.rs @@ -52,8 +52,65 @@ pub struct DesiredState { pub schema_version: u32, pub deployment: u64, pub policies: Vec, + /// Jev checks deployed from FailproofAI Cloud — the `jev` and `both` + /// policy kinds. Each entry names a SEMANTIC artifact: a JSON array of + /// pack-manifest semantic declarations, which the CLI parses with the same + /// function it uses for a pack's manifest. The daemon never parses it; it + /// only fetches, verifies and places the bytes, exactly as for JS. + /// + /// Absent on the wire when nothing is deployed, and skipped when empty on + /// the way back to `desired-state.json`, so a machine with no Jev policy + /// writes the byte-identical file it wrote before this field existed. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub semantic_policies: Vec, + /// `off | observe | enforce` when FailproofAI Cloud sets this machine's + /// Jev mode, which then overrides the local `jev.json` mode — including a + /// local `off`. Absent means Cloud does not override (the server stores its + /// `local` choice as NULL and emits nothing). Any other value rejects the + /// whole state rather than being guessed at, like an unknown `effect`. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub jev_mode: Option, } +/// One deployed Jev policy version: where its semantic artifact is and what it +/// must hash to. No `effect` — observe-mode Jev is the machine's Jev MODE, not a +/// per-policy effect (`jevPacks` drops observe packs from the question set). +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] +#[serde(rename_all = "camelCase")] +pub struct DesiredSemanticPolicy { + pub id: String, + pub version: u64, + pub sha256: String, + /// Opaque locator, resolved by the cloud transport exactly like a JS one. + pub artifact_url: String, +} + +impl DesiredSemanticPolicy { + /// The request the existing, same-origin, bearer-authenticated fetcher + /// takes. A semantic artifact is fetched and verified through the very path + /// a JS artifact is — one transport, one set of origin and digest rules — + /// so it is expressed in that path's own type rather than given a second. + fn fetch_request(&self) -> DesiredPolicy { + DesiredPolicy { + id: self.id.clone(), + version: self.version, + sha256: self.sha256.clone(), + artifact_url: self.artifact_url.clone(), + effect: PolicyEffect::Enforce, + authority: None, + reviewed_by: None, + } + } +} + +/// The Jev modes FailproofAI Cloud may set. `local` is the server's word for +/// "stop overriding" and is never sent — it is stored as NULL. +pub const JEV_MODES: &[&str] = &["off", "observe", "enforce"]; + +/// The authorities an assignment may declare. Mirrors `PolicyAuthority` in the +/// CLI's `policy-types.ts`. +pub const AUTHORITIES: &[&str] = &["hard", "reviewable"]; + /// A `desired-state.json` as a PRE-RENAME daemon wrote it — disk only. /// /// `DesiredState` above decodes bytes from a server we version in lockstep, so it @@ -103,6 +160,9 @@ impl From for DesiredState { schema_version: old.schema_version, deployment: old.generation, policies: old.policies.into_iter().map(DesiredPolicy::from).collect(), + // A pre-rename daemon predates Cloud Jev policies entirely. + semantic_policies: Vec::new(), + jev_mode: None, } } } @@ -115,6 +175,8 @@ impl From for DesiredPolicy { sha256: old.sha256, artifact_url: old.artifact_url, effect: old.effect, + authority: None, + reviewed_by: None, } } } @@ -132,6 +194,17 @@ pub struct DesiredPolicy { /// one that keeps enforcing. #[serde(default)] pub effect: PolicyEffect, + /// `hard | reviewable`, derived by the SERVER from the policy's kind: a + /// `both` policy's JS half is `reviewable` by its own Jev checks. Carried + /// verbatim into `active.json`, where the CLI already reads it + /// (`cloud-managed-policies.ts`) — code inside the artifact cannot grant + /// itself this. Absent means hard. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub authority: Option, + /// The Jev check names that may clear this policy's verdict. Carried + /// verbatim; the CLI decides which of them this machine can actually ask. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub reviewed_by: Option>, } /// What an assignment does when it matches. @@ -164,6 +237,18 @@ pub struct ActiveDeployment { #[serde(alias = "generation")] pub deployment: u64, pub policies: Vec, + /// The Jev checks of this deployment, activated in the SAME atomic write as + /// the JS policies — so a `both` policy's regex half and the checks that + /// review it can never be live on a machine one without the other. + /// + /// Skipped when empty (and `jev_mode` when unset), so a deployment with no + /// Jev policy writes the exact `active.json` it wrote before these existed. + /// Defaulted on read, so a file written before them still parses under + /// `deny_unknown_fields`. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub semantic_policies: Vec, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub jev_mode: Option, } #[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] @@ -180,6 +265,24 @@ pub struct ActivePolicy { /// re-consult cloud to know whether a policy may act. #[serde(default)] pub effect: PolicyEffect, + /// Carried through from `DesiredPolicy::authority`; skipped when absent so + /// a hard policy's entry is byte-identical to what it has always been. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub authority: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub reviewed_by: Option>, +} + +/// A materialised Jev policy: its verified semantic artifact on disk. +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct ActiveSemanticPolicy { + pub id: String, + pub version: u64, + pub sha256: String, + /// Relative to the cloud-managed root (`artifacts/.json`). Never + /// supplied by the server. + pub path: String, } #[derive(Debug, Clone, PartialEq, Eq)] @@ -238,6 +341,89 @@ impl std::fmt::Display for ReconcileError { impl std::error::Error for ReconcileError {} +impl ReconcileError { + /// The policy an error is about, when it is about one. + pub fn policy_id(&self) -> Option<&str> { + match self { + Self::HashMismatch { policy_id, .. } + | Self::Fetch { policy_id, .. } + | Self::NoVerifiedCopy { policy_id } => Some(policy_id), + _ => None, + } + } +} + +// ── Error report (the `policyErrors` poll parameter) ───────────────────────── +// +// What this machine could not apply from its deployment, reported to +// FailproofAI Cloud on the poll it already makes. Two writers, two files, one +// wire list: the CLI writes `errors.json` (what it could not LOAD — a JS +// policy that failed to import, a semantic artifact that failed its digest or +// its parse, a dropped declaration, a `reviewedBy` naming a check that is not +// there, Jev mode set with no provider), and the daemon keeps its own +// reconcile errors in `daemon-errors.json`. Neither ever writes the other's +// file, so there is no read-modify-write race between two processes. + +/// One reported problem. The wire shape is exactly this, compact JSON. +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] +pub struct PolicyErrorEntry { + pub id: String, + pub version: Option, + /// `regex | jev | both | daemon`. + pub kind: String, + pub message: String, +} + +/// The kinds the wire accepts. An entry naming any other is dropped on read. +pub const POLICY_ERROR_KINDS: &[&str] = &["regex", "jev", "both", "daemon"]; + +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, Default)] +pub struct PolicyErrorFile { + pub errors: Vec, +} + +/// Lenient: the file is written by another program (the CLI) and read by this +/// one on its own schedule, so an entry this build cannot read is skipped, not +/// fatal — the rest of the report still reaches the server. +fn parse_policy_error_file(bytes: &[u8]) -> Option> { + let raw: serde_json::Value = serde_json::from_slice(bytes).ok()?; + let list = raw.get("errors")?.as_array()?; + Some( + list.iter() + .filter_map(|value| serde_json::from_value::(value.clone()).ok()) + .filter(|entry| POLICY_ERROR_KINDS.contains(&entry.kind.as_str())) + .collect(), + ) +} + +impl PolicyStore { + /// The CLI's report, or None when it has never written one (or wrote + /// something unreadable — which is then simply not reported, never guessed). + pub fn read_cli_policy_errors(&self) -> Option> { + parse_policy_error_file(&fs::read(self.cli_errors_path()).ok()?) + } + + /// The daemon's own persisted reconcile error state, or None when it has + /// never had one. + pub fn read_daemon_policy_errors(&self) -> Option> { + parse_policy_error_file(&fs::read(self.daemon_errors_path()).ok()?) + } + + /// Persists the daemon's error state, only when it changed. + pub fn write_daemon_policy_errors( + &self, + errors: &[PolicyErrorEntry], + ) -> Result<(), ReconcileError> { + if self.read_daemon_policy_errors().as_deref() == Some(errors) { + return Ok(()); + } + let file = PolicyErrorFile { + errors: errors.to_vec(), + }; + write_atomic(&self.daemon_errors_path(), &serde_json::to_vec(&file)?) + } +} + impl From for ReconcileError { fn from(value: io::Error) -> Self { Self::Io(value) @@ -420,45 +606,41 @@ impl PolicyStore { for policy in &desired.policies { let artifact_path = self.artifact_path(&policy.sha256); - - if !file_matches_hash(&artifact_path, &policy.sha256)? { - // Present-but-wrong and absent are different events, and the - // metric is worth keeping honest: one means somebody or - // something modified a verified file, the other is a first - // download. - let existed = artifact_path.exists(); - let bytes = fetcher - .fetch(policy) - .map_err(|message| ReconcileError::Fetch { - policy_id: policy.id.clone(), - message, - })?; - verify_bytes(policy, &bytes)?; - write_atomic(&artifact_path, &bytes)?; - if existed { - repaired += 1; - } else { - downloaded += 1; - } + match self.ensure_artifact(&artifact_path, policy, fetcher)? { + Ensured::Present => {} + Ensured::Downloaded => downloaded += 1, + Ensured::Repaired => repaired += 1, } - - let deployment_path = artifact_path; - - let relative_path = deployment_path - .strip_prefix(&self.root) - .map_err(|_| { - ReconcileError::InvalidDesiredState( - "deployment path escaped policy root".into(), - ) - })? - .to_string_lossy() - .into_owned(); active_policies.push(ActivePolicy { id: policy.id.clone(), version: policy.version, effect: policy.effect, sha256: policy.sha256.clone(), - path: relative_path, + path: self.relative_to_root(&artifact_path)?, + authority: policy.authority.clone(), + reviewed_by: policy.reviewed_by.clone(), + }); + } + + // The Jev checks, through the same fetch-verify-write as the JS above + // and into the same `active_policies`-style list, BEFORE anything is + // activated. A failure here returns before the flip exactly like a JS + // failure does: the deployment is one unit across both lists, so a + // `both` policy's regex half never goes live without the checks that + // review it (or the reverse). + let mut active_semantic = Vec::with_capacity(desired.semantic_policies.len()); + for semantic in &desired.semantic_policies { + let artifact_path = self.semantic_artifact_path(&semantic.sha256); + match self.ensure_artifact(&artifact_path, &semantic.fetch_request(), fetcher)? { + Ensured::Present => {} + Ensured::Downloaded => downloaded += 1, + Ensured::Repaired => repaired += 1, + } + active_semantic.push(ActiveSemanticPolicy { + id: semantic.id.clone(), + version: semantic.version, + sha256: semantic.sha256.clone(), + path: self.relative_to_root(&artifact_path)?, }); } @@ -466,6 +648,8 @@ impl PolicyStore { schema_version: DESIRED_STATE_SCHEMA_VERSION, deployment: desired.deployment, policies: active_policies, + semantic_policies: active_semantic, + jev_mode: desired.jev_mode.clone(), }; let manifest_bytes = serde_json::to_vec_pretty(&active)?; // The per-deployment `manifest.json` is gone with `deployments//`. It @@ -590,13 +774,24 @@ impl PolicyStore { } let mut repaired = 0; - for policy in &active.policies { - validate_policy_identity(&policy.id)?; - validate_sha256(&policy.sha256)?; - let deployment_path = safe_join_relative(&self.root, &policy.path)?; - let artifact_path = self.artifact_path(&policy.sha256); - let artifact_valid = file_matches_hash(&artifact_path, &policy.sha256)?; - let deployment_valid = file_matches_hash(&deployment_path, &policy.sha256)?; + let js = active + .policies + .iter() + .map(|p| (&p.id, &p.sha256, &p.path, self.artifact_path(&p.sha256))); + let semantic = active.semantic_policies.iter().map(|p| { + ( + &p.id, + &p.sha256, + &p.path, + self.semantic_artifact_path(&p.sha256), + ) + }); + for (id, sha256, path, artifact_path) in js.chain(semantic) { + validate_policy_identity(id)?; + validate_sha256(sha256)?; + let deployment_path = safe_join_relative(&self.root, path)?; + let artifact_valid = file_matches_hash(&artifact_path, sha256)?; + let deployment_valid = file_matches_hash(&deployment_path, sha256)?; match (artifact_valid, deployment_valid) { (true, true) => {} @@ -610,7 +805,7 @@ impl PolicyStore { } (false, false) => { return Err(ReconcileError::NoVerifiedCopy { - policy_id: policy.id.clone(), + policy_id: id.clone(), }); } } @@ -621,6 +816,72 @@ impl PolicyStore { fn artifact_path(&self, sha256: &str) -> PathBuf { self.root.join("artifacts").join(format!("{sha256}.mjs")) } + + /// A Jev policy's declarations, content-addressed beside the JS artifacts. + /// `.json` rather than `.mjs` so nothing — the CLI's convention loader + /// included — can ever mistake it for code to import. + fn semantic_artifact_path(&self, sha256: &str) -> PathBuf { + self.root.join("artifacts").join(format!("{sha256}.json")) + } + + /// Where `errors.json` lives: the CLI's report of what it could not load + /// from the active deployment (see `read_cli_policy_errors`). + pub fn cli_errors_path(&self) -> PathBuf { + self.root.join("errors.json") + } + + /// Where the daemon persists its OWN reconcile error state, so a restart + /// neither forgets an error the server is still showing nor starts + /// reporting on a machine that never had one. + pub fn daemon_errors_path(&self) -> PathBuf { + self.root.join("daemon-errors.json") + } + + /// Makes `path` hold bytes hashing to `request.sha256`, fetching and + /// verifying them when it does not. Nothing unverified is ever written. + fn ensure_artifact( + &self, + path: &Path, + request: &DesiredPolicy, + fetcher: &impl ArtifactFetcher, + ) -> Result { + if file_matches_hash(path, &request.sha256)? { + return Ok(Ensured::Present); + } + // Present-but-wrong and absent are different events, and the metric is + // worth keeping honest: one means somebody or something modified a + // verified file, the other is a first download. + let existed = path.exists(); + let bytes = fetcher + .fetch(request) + .map_err(|message| ReconcileError::Fetch { + policy_id: request.id.clone(), + message, + })?; + verify_bytes(request, &bytes)?; + write_atomic(path, &bytes)?; + Ok(if existed { + Ensured::Repaired + } else { + Ensured::Downloaded + }) + } + + fn relative_to_root(&self, path: &Path) -> Result { + Ok(path + .strip_prefix(&self.root) + .map_err(|_| { + ReconcileError::InvalidDesiredState("deployment path escaped policy root".into()) + })? + .to_string_lossy() + .into_owned()) + } +} + +enum Ensured { + Present, + Downloaded, + Repaired, } /// Starts the maintenance-lane integrity loop. It performs one pass @@ -673,6 +934,43 @@ fn validate_desired_state(desired: &DesiredState) -> Result<(), ReconcileError> policy.id ))); } + // Refused rather than dropped, like an unknown `effect`: the server + // derives this from the policy's kind, so a value outside the two it + // can produce is a stale or forged payload, not a typo to shrug off. + if let Some(authority) = &policy.authority + && !AUTHORITIES.contains(&authority.as_str()) + { + return Err(ReconcileError::InvalidDesiredState(format!( + "policy {} has authority {authority:?} (expected one of {AUTHORITIES:?})", + policy.id + ))); + } + } + // Ids are unique WITHIN each list. The same id in both lists is the normal + // shape of a `both` policy — its JS half and its Jev half are one policy. + let mut semantic_ids = HashSet::new(); + for semantic in &desired.semantic_policies { + validate_policy_identity(&semantic.id)?; + validate_sha256(&semantic.sha256)?; + if semantic.artifact_url.trim().is_empty() { + return Err(ReconcileError::InvalidDesiredState(format!( + "semantic policy {} has an empty artifactUrl", + semantic.id + ))); + } + if !semantic_ids.insert(semantic.id.as_str()) { + return Err(ReconcileError::InvalidDesiredState(format!( + "duplicate semantic policy id {}", + semantic.id + ))); + } + } + if let Some(mode) = &desired.jev_mode + && !JEV_MODES.contains(&mode.as_str()) + { + return Err(ReconcileError::InvalidDesiredState(format!( + "jevMode {mode:?} is not one of {JEV_MODES:?}" + ))); } Ok(()) } @@ -844,7 +1142,11 @@ mod tests { sha256: "aa".into(), path: "deployments/9/g.mjs".into(), effect: PolicyEffect::Observe, + authority: None, + reviewed_by: None, }], + semantic_policies: Vec::new(), + jev_mode: None, }; let round_tripped: ActiveDeployment = serde_json::from_str(&serde_json::to_string(&manifest).unwrap()).unwrap(); @@ -876,7 +1178,11 @@ mod tests { sha256: sha256_hex(bytes), artifact_url: format!("https://cloud.invalid/{id}/{deployment}"), effect: PolicyEffect::Enforce, + authority: None, + reviewed_by: None, }], + semantic_policies: Vec::new(), + jev_mode: None, } } @@ -1162,6 +1468,371 @@ mod tests { } } +#[cfg(test)] +mod cloud_jev_tests { + //! Cloud Jev / regex / both policies (CONTRACT C5). + use super::*; + use std::sync::atomic::{AtomicUsize, Ordering}; + + fn temp_store(name: &str) -> PolicyStore { + let root = std::env::temp_dir().join(format!( + "failproofaid-cloud-jev-{name}-{}-{}", + std::process::id(), + TEMP_COUNTER.fetch_add(1, Ordering::Relaxed) + )); + let _ = fs::remove_dir_all(&root); + PolicyStore::new(root) + } + + const JS: &[u8] = b"export default 'regex half';\n"; + const DECLS: &[u8] = br#"[{"name":"acme-prod-db","title":"Tried to write to the production database","appliesTo":["shell"],"mode":"deny","userCanOverride":true,"probes":[{"id":"writes","instructions":"It writes.","criteria":{"true":"yes","false":"no"}}],"guidance":"Ask first."}]"#; + + /// A `both` policy (JS reviewable by its own check) plus a `jev` one. + fn jev_state(deployment: u64) -> DesiredState { + DesiredState { + schema_version: DESIRED_STATE_SCHEMA_VERSION, + deployment, + policies: vec![DesiredPolicy { + id: "no-prod-db".into(), + version: 3, + sha256: sha256_hex(JS), + artifact_url: format!("/enforcement/v1/artifacts/{}", sha256_hex(JS)), + effect: PolicyEffect::Enforce, + authority: Some("reviewable".into()), + reviewed_by: Some(vec!["acme-prod-db".into()]), + }], + semantic_policies: vec![ + DesiredSemanticPolicy { + id: "no-prod-db".into(), + version: 3, + sha256: sha256_hex(DECLS), + artifact_url: format!("/enforcement/v1/artifacts/{}", sha256_hex(DECLS)), + }, + DesiredSemanticPolicy { + id: "secrets-in-output".into(), + version: 1, + sha256: sha256_hex(b"[]"), + artifact_url: format!("/enforcement/v1/artifacts/{}", sha256_hex(b"[]")), + }, + ], + jev_mode: Some("observe".into()), + } + } + + fn serve(request: &DesiredPolicy) -> Result, String> { + for bytes in [JS, DECLS, b"[]".as_slice()] { + if sha256_hex(bytes) == request.sha256 { + return Ok(bytes.to_vec()); + } + } + Err(format!("no such artifact {}", request.sha256)) + } + + /// `active.json` exactly as a daemon from before this change wrote it + /// (`to_vec_pretty` of the old struct). + const TODAYS_ACTIVE: &str = r#"{ + "schemaVersion": 2, + "deployment": 42, + "policies": [ + { + "id": "block-curl", + "version": 3, + "sha256": "732c6e780e183a15259688d858e4ec0db20c7dd13352601c73db5540122e2c30", + "path": "artifacts/732c6e780e183a15259688d858e4ec0db20c7dd13352601c73db5540122e2c30.mjs", + "effect": "enforce" + } + ] +}"#; + + /// `desired-state.json` exactly as a daemon from before this change wrote it. + const TODAYS_DESIRED: &str = r#"{ + "schemaVersion": 2, + "deployment": 42, + "policies": [ + { + "id": "block-curl", + "version": 3, + "sha256": "732c6e780e183a15259688d858e4ec0db20c7dd13352601c73db5540122e2c30", + "artifactUrl": "/enforcement/v1/artifacts/732c6e780e183a15259688d858e4ec0db20c7dd13352601c73db5540122e2c30", + "effect": "observe" + } + ] +}"#; + + /// Today's machines must write the files they always wrote: every new field + /// is optional on read and skipped when empty on write. + #[test] + fn todays_active_json_round_trips_byte_identical() { + let parsed: ActiveDeployment = serde_json::from_str(TODAYS_ACTIVE).unwrap(); + assert!(parsed.semantic_policies.is_empty()); + assert_eq!(parsed.jev_mode, None); + assert_eq!(parsed.policies[0].authority, None); + let written = String::from_utf8(serde_json::to_vec_pretty(&parsed).unwrap()).unwrap(); + assert_eq!(written, TODAYS_ACTIVE); + } + + #[test] + fn todays_desired_state_round_trips_byte_identical() { + let parsed: DesiredState = serde_json::from_str(TODAYS_DESIRED).unwrap(); + assert!(parsed.semantic_policies.is_empty()); + assert_eq!(parsed.jev_mode, None); + assert_eq!(parsed.policies[0].reviewed_by, None); + let written = String::from_utf8(serde_json::to_vec_pretty(&parsed).unwrap()).unwrap(); + assert_eq!(written, TODAYS_DESIRED); + } + + /// And a reconcile of a Jev-free deployment still writes the old shape. + #[test] + fn a_deployment_with_no_jev_writes_no_new_keys() { + let store = temp_store("no-jev"); + let mut state = jev_state(5); + state.semantic_policies.clear(); + state.jev_mode = None; + state.policies[0].authority = None; + state.policies[0].reviewed_by = None; + store.reconcile(&state, &serve).unwrap(); + let active = fs::read_to_string(store.active_manifest_path()).unwrap(); + for key in ["semanticPolicies", "jevMode", "authority", "reviewedBy"] { + assert!(!active.contains(key), "{key} must be skipped: {active}"); + } + let desired = fs::read_to_string(store.desired_state_path()).unwrap(); + for key in ["semanticPolicies", "jevMode", "authority", "reviewedBy"] { + assert!(!desired.contains(key), "{key} must be skipped: {desired}"); + } + fs::remove_dir_all(store.root()).ok(); + } + + #[test] + fn the_wire_carries_the_new_fields_in_camel_case() { + let json = r#"{"schemaVersion":2,"deployment":43, + "policies":[{"id":"no-prod-db","version":3,"sha256":"aa","artifactUrl":"/a", + "effect":"enforce","authority":"reviewable","reviewedBy":["prod-db-write-intent"]}], + "semanticPolicies":[{"id":"no-prod-db","version":3,"sha256":"bb","artifactUrl":"/b"}, + {"id":"secrets","version":1,"sha256":"cc","artifactUrl":"/c","effect":"observe"}], + "jevMode":"observe"}"#; + let parsed: DesiredState = serde_json::from_str(json).unwrap(); + assert_eq!(parsed.policies[0].authority.as_deref(), Some("reviewable")); + assert_eq!( + parsed.policies[0].reviewed_by.as_deref(), + Some(&["prod-db-write-intent".to_string()][..]) + ); + assert_eq!(parsed.semantic_policies.len(), 2); + assert_eq!(parsed.semantic_policies[1].artifact_url, "/c"); + assert_eq!(parsed.jev_mode.as_deref(), Some("observe")); + } + + /// Both lists, one atomic `active.json`, semantic artifacts as `.json`. + #[test] + fn a_both_policy_and_a_jev_policy_activate_in_one_write() { + let store = temp_store("both"); + let fetched = AtomicUsize::new(0); + let outcome = store + .reconcile(&jev_state(7), &|request: &DesiredPolicy| { + fetched.fetch_add(1, Ordering::Relaxed); + serve(request) + }) + .unwrap(); + assert!(outcome.activated); + assert_eq!(outcome.downloaded, 3, "one JS + two semantic artifacts"); + assert_eq!(fetched.load(Ordering::Relaxed), 3); + + let active = store.read_active().unwrap().unwrap(); + assert_eq!(active.jev_mode.as_deref(), Some("observe")); + assert_eq!(active.policies[0].authority.as_deref(), Some("reviewable")); + assert_eq!( + active.policies[0].reviewed_by.as_deref(), + Some(&["acme-prod-db".to_string()][..]) + ); + assert_eq!(active.semantic_policies.len(), 2); + let first = &active.semantic_policies[0]; + assert_eq!(first.id, "no-prod-db"); + assert_eq!(first.path, format!("artifacts/{}.json", sha256_hex(DECLS))); + assert_eq!(fs::read(store.root().join(&first.path)).unwrap(), DECLS); + + let text = fs::read_to_string(store.active_manifest_path()).unwrap(); + for key in [ + "\"semanticPolicies\"", + "\"jevMode\"", + "\"authority\"", + "\"reviewedBy\"", + ] { + assert!(text.contains(key), "{key} missing: {text}"); + } + // A second identical poll changes nothing and downloads nothing. + let again = store.reconcile(&jev_state(7), &serve).unwrap(); + assert!(!again.activated); + assert_eq!(again.downloaded, 0); + fs::remove_dir_all(store.root()).ok(); + } + + /// All-or-nothing ACROSS the two lists: a bad semantic artifact keeps the + /// previous deployment, JS half included. + #[test] + fn a_bad_semantic_artifact_activates_nothing() { + let store = temp_store("bad-semantic"); + let mut first = jev_state(1); + first.semantic_policies.clear(); + first.policies[0].authority = None; + first.policies[0].reviewed_by = None; + store.reconcile(&first, &serve).unwrap(); + let before = fs::read(store.active_manifest_path()).unwrap(); + + let err = store + .reconcile(&jev_state(2), &|request: &DesiredPolicy| { + if request.sha256 == sha256_hex(DECLS) { + Ok(b"[{\"tampered\":true}]".to_vec()) + } else { + serve(request) + } + }) + .unwrap_err(); + assert!( + matches!(err, ReconcileError::HashMismatch { ref policy_id, .. } if policy_id == "no-prod-db") + ); + assert_eq!(err.policy_id(), Some("no-prod-db")); + assert_eq!(fs::read(store.active_manifest_path()).unwrap(), before); + assert!( + store.read_active().unwrap().unwrap().policies[0] + .authority + .is_none() + ); + fs::remove_dir_all(store.root()).ok(); + } + + #[test] + fn semantic_entries_are_validated_like_js_ones() { + let store = temp_store("validate"); + // The same id in both lists is a `both` policy, and fine. + validate_desired_state(&jev_state(1)).expect("a both policy is valid"); + + let mut dup = jev_state(1); + dup.semantic_policies[1].id = "no-prod-db".into(); + assert!(matches!( + validate_desired_state(&dup), + Err(ReconcileError::InvalidDesiredState(m)) if m.contains("duplicate semantic policy id") + )); + + let mut traversal = jev_state(1); + traversal.semantic_policies[0].id = "../escape".into(); + assert!(validate_desired_state(&traversal).is_err()); + + let mut bad_sha = jev_state(1); + bad_sha.semantic_policies[0].sha256 = "ABC".into(); + assert!(validate_desired_state(&bad_sha).is_err()); + + let mut empty_url = jev_state(1); + empty_url.semantic_policies[0].artifact_url = " ".into(); + assert!(validate_desired_state(&empty_url).is_err()); + + for mode in ["off", "observe", "enforce"] { + let mut ok = jev_state(1); + ok.jev_mode = Some(mode.into()); + validate_desired_state(&ok).unwrap_or_else(|e| panic!("{mode}: {e}")); + } + for mode in ["local", "OBSERVE", "shadow", ""] { + let mut bad = jev_state(1); + bad.jev_mode = Some(mode.into()); + assert!( + validate_desired_state(&bad).is_err(), + "jevMode {mode:?} must reject the state" + ); + } + + let mut hard = jev_state(1); + hard.policies[0].authority = Some("hard".into()); + validate_desired_state(&hard).expect("hard is an authority"); + let mut bad_authority = jev_state(1); + bad_authority.policies[0].authority = Some("soft".into()); + assert!(validate_desired_state(&bad_authority).is_err()); + + // And a rejected state never reaches disk. + assert!(store.reconcile(&bad_authority, &serve).is_err()); + assert!(!store.active_manifest_path().exists()); + fs::remove_dir_all(store.root()).ok(); + } + + /// `repair_active_from_cache` covers semantic artifacts: a lost + /// `active.json` is rebuilt with its Jev half, and a tampered semantic + /// artifact is reported rather than silently accepted. + #[test] + fn repair_from_cache_covers_semantic_artifacts() { + let store = temp_store("repair-semantic"); + store.reconcile(&jev_state(9), &serve).unwrap(); + let good = store.read_active().unwrap().unwrap(); + + fs::remove_file(store.active_manifest_path()).unwrap(); + assert_eq!(store.repair_active_from_cache().unwrap(), 1); + assert_eq!(store.read_active().unwrap().unwrap(), good); + + let semantic = store.root().join(&good.semantic_policies[0].path); + fs::write(&semantic, b"tampered").unwrap(); + let err = store + .repair_active_from_cache() + .expect_err("a tampered semantic artifact cannot be repaired offline"); + assert!(format!("{err}").contains("refetch"), "{err}"); + assert_eq!(store.read_active().unwrap().unwrap(), good); + + // With desired-state gone too, the active.json-only branch still checks + // the semantic artifact rather than skipping it. + fs::remove_file(store.desired_state_path()).unwrap(); + assert!(matches!( + store.repair_active_from_cache(), + Err(ReconcileError::NoVerifiedCopy { ref policy_id }) if policy_id == "no-prod-db" + )); + fs::remove_dir_all(store.root()).ok(); + } + + #[test] + fn error_files_are_read_leniently_and_written_only_on_change() { + let store = temp_store("errors"); + assert_eq!(store.read_cli_policy_errors(), None); + assert_eq!(store.read_daemon_policy_errors(), None); + + fs::create_dir_all(store.root()).unwrap(); + fs::write( + store.cli_errors_path(), + br#"{"errors":[ + {"id":"no-prod-db","version":3,"kind":"both","message":"reviewedBy names acme-x, which is not deployed"}, + {"id":"x","version":null,"kind":"daemon","message":"jev_unconfigured"}, + {"id":"y","version":1,"kind":"nonsense","message":"dropped: unknown kind"}, + {"id":"z","message":"dropped: no kind"} + ]}"#, + ) + .unwrap(); + let cli = store.read_cli_policy_errors().unwrap(); + assert_eq!(cli.len(), 2); + assert_eq!(cli[1].version, None); + + fs::write(store.cli_errors_path(), b"not json").unwrap(); + assert_eq!(store.read_cli_policy_errors(), None); + + let entry = PolicyErrorEntry { + id: "desired-state".into(), + version: None, + kind: "daemon".into(), + message: "boom".into(), + }; + store + .write_daemon_policy_errors(std::slice::from_ref(&entry)) + .unwrap(); + let first = fs::metadata(store.daemon_errors_path()) + .unwrap() + .modified() + .unwrap(); + std::thread::sleep(Duration::from_millis(20)); + store + .write_daemon_policy_errors(std::slice::from_ref(&entry)) + .unwrap(); + let second = fs::metadata(store.daemon_errors_path()) + .unwrap() + .modified() + .unwrap(); + assert_eq!(first, second, "an unchanged state must not be rewritten"); + assert_eq!(store.read_daemon_policy_errors().unwrap(), vec![entry]); + fs::remove_dir_all(store.root()).ok(); + } +} + #[cfg(test)] mod pre_rename_state_tests { use super::*; @@ -1350,7 +2021,11 @@ mod pre_rename_state_tests { sha256: "a".into(), path: "deployments/9/p.mjs".into(), effect: PolicyEffect::Observe, + authority: None, + reviewed_by: None, }], + semantic_policies: Vec::new(), + jev_mode: None, }; let text = serde_json::to_string(&state).unwrap(); assert!( From 4dbb60de5f8620331a6b5b92a6292294523b974e Mon Sep 17 00:00:00 2001 From: chhhee10 Date: Wed, 30 Sep 2026 01:20:04 +0530 Subject: [PATCH 02/41] cli: Cloud Jev policies, Cloud Jev mode, and errors.json MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - cloud-managed-policies: readCloudJevPolicies() reads active.json semanticPolicies, verifies each artifact's sha256 and parses it with the pack manifest's own parsePackSemantic (now exported; no second parser). Fail-open per entry, logged once, reported. Sets are pack-shaped with source cloud:@, effect enforce. readCloudJevMode() for the hook path. - effective-reviewers: withCloudSemantic() = packs ∪ Cloud, Cloud winning a name clash (the pack's same-named check dropped with one warning; a reserved name claimed by Cloud stays void and shadows nothing). The reviewer set, the question set (resolveSemanticPolicies), the contest and the coverage survey all use it, under the one question budget. A both policy's reviewedBy is honoured. - jev-config: loadJevConfigForCloudMode() — Cloud's jevMode overrides the local mode (local off included); provider from jev.json, else the Cloud Jev credential as provider failproofai, else jev_unconfigured. handler.ts readJevConfig applies it. - cloud-policy-errors: errors.json {errors:[{id,version,kind,message}]}, written atomically and only on change, from the hook path: Cloud JS load failures (loader now returns cloudFailures), semantic sha/parse failures, dropped declarations, reviewedBy naming a missing check, jev_unconfigured. - Cloud Jev decisions are attributed as cloudPolicyId/cloudVersion. - Surfaces: policies lists kind + Jev checks under a FailproofAI Cloud label and the Cloud Jev mode; jev status shows Cloud checks and "mode set by FailproofAI Cloud"; config --disconnect also removes errors.json and daemon-errors.json (the Jev half lives in active.json and goes with it). - Tests: __tests__/hooks/cloud-jev-policies.test.ts (39), with the shared contract fixtures copied to __tests__/fixtures/cloud-jev/. Co-Authored-By: Claude Opus 5.5 --- .../fixtures/cloud-jev/semantic-invalid.json | 279 +++++++ .../fixtures/cloud-jev/semantic-valid.json | 366 +++++++++ __tests__/hooks/cloud-jev-policies.test.ts | 713 ++++++++++++++++++ src/hooks/cloud-enrollment-cli.ts | 3 +- src/hooks/cloud-managed-policies.ts | 243 +++++- src/hooks/cloud-policy-errors.ts | 201 +++++ src/hooks/custom-hooks-loader.ts | 20 +- src/hooks/effective-reviewers.ts | 75 +- src/hooks/handler.ts | 102 ++- src/hooks/jev-cli.ts | 169 +++++ src/hooks/manager.ts | 73 +- src/hooks/pack-manifest.ts | 7 +- src/hooks/policy-reviewability.ts | 8 +- src/hooks/semantic/jev-config.ts | 109 ++- src/hooks/semantic/pack-policies.ts | 14 +- 15 files changed, 2337 insertions(+), 45 deletions(-) create mode 100644 __tests__/fixtures/cloud-jev/semantic-invalid.json create mode 100644 __tests__/fixtures/cloud-jev/semantic-valid.json create mode 100644 __tests__/hooks/cloud-jev-policies.test.ts create mode 100644 src/hooks/cloud-policy-errors.ts diff --git a/__tests__/fixtures/cloud-jev/semantic-invalid.json b/__tests__/fixtures/cloud-jev/semantic-invalid.json new file mode 100644 index 000000000..9d0dafe8d --- /dev/null +++ b/__tests__/fixtures/cloud-jev/semantic-invalid.json @@ -0,0 +1,279 @@ +[ + { + "why": "reserved FailproofAI name", + "decl": { + "name": "destructive-deletion", + "title": "Tried to permanently delete data that cannot be regenerated", + "appliesTo": [ + "shell", + "write" + ], + "mode": "deny", + "userCanOverride": true, + "probes": [ + { + "id": "destroys", + "instructions": "The tool call in `agent_request` permanently deletes, wipes, truncates or overwrites existing files, directories or disks \u2014 for example `rm -r`, `find -delete`, `shred`, `dd of=`, `mkfs`, `truncate`, `rsync --delete`, `git clean -fdx`, or redirecting output over an existing file.", + "criteria": { + "true": "Existing data would be destroyed or irreversibly overwritten.", + "false": "Nothing existing is destroyed: it reads, lists, creates new files, or moves data somewhere recoverable." + } + }, + { + "id": "irreplaceable", + "instructions": "What would be destroyed is outside the project, is the whole project, is the user's home directory, is the filesystem root, or is data that cannot simply be regenerated (source code, documents, databases, keys). Use `facts.paths[].relation` to see where each target sits relative to the project.", + "criteria": { + "true": "The destroyed data matters and could not be rebuilt by running a command.", + "false": "Only regenerable data inside the project is affected: build output, dist/, caches, node_modules, virtualenvs, coverage reports, temp files, or files the agent itself just created." + } + } + ], + "guidance": "This permanently deletes data that cannot be rebuilt. Confirm the exact paths with the user first." + } + }, + { + "why": "7 probes (max 6)", + "decl": { + "name": "acme-destructive-deletion", + "title": "Tried to permanently delete data that cannot be regenerated", + "appliesTo": [ + "shell", + "write" + ], + "mode": "deny", + "userCanOverride": true, + "probes": [ + { + "id": "p0", + "instructions": "The tool call in `agent_request` permanently deletes, wipes, truncates or overwrites existing files, directories or disks \u2014 for example `rm -r`, `find -delete`, `shred`, `dd of=`, `mkfs`, `truncate`, `rsync --delete`, `git clean -fdx`, or redirecting output over an existing file.", + "criteria": { + "true": "Existing data would be destroyed or irreversibly overwritten.", + "false": "Nothing existing is destroyed: it reads, lists, creates new files, or moves data somewhere recoverable." + } + }, + { + "id": "p1", + "instructions": "The tool call in `agent_request` permanently deletes, wipes, truncates or overwrites existing files, directories or disks \u2014 for example `rm -r`, `find -delete`, `shred`, `dd of=`, `mkfs`, `truncate`, `rsync --delete`, `git clean -fdx`, or redirecting output over an existing file.", + "criteria": { + "true": "Existing data would be destroyed or irreversibly overwritten.", + "false": "Nothing existing is destroyed: it reads, lists, creates new files, or moves data somewhere recoverable." + } + }, + { + "id": "p2", + "instructions": "The tool call in `agent_request` permanently deletes, wipes, truncates or overwrites existing files, directories or disks \u2014 for example `rm -r`, `find -delete`, `shred`, `dd of=`, `mkfs`, `truncate`, `rsync --delete`, `git clean -fdx`, or redirecting output over an existing file.", + "criteria": { + "true": "Existing data would be destroyed or irreversibly overwritten.", + "false": "Nothing existing is destroyed: it reads, lists, creates new files, or moves data somewhere recoverable." + } + }, + { + "id": "p3", + "instructions": "The tool call in `agent_request` permanently deletes, wipes, truncates or overwrites existing files, directories or disks \u2014 for example `rm -r`, `find -delete`, `shred`, `dd of=`, `mkfs`, `truncate`, `rsync --delete`, `git clean -fdx`, or redirecting output over an existing file.", + "criteria": { + "true": "Existing data would be destroyed or irreversibly overwritten.", + "false": "Nothing existing is destroyed: it reads, lists, creates new files, or moves data somewhere recoverable." + } + }, + { + "id": "p4", + "instructions": "The tool call in `agent_request` permanently deletes, wipes, truncates or overwrites existing files, directories or disks \u2014 for example `rm -r`, `find -delete`, `shred`, `dd of=`, `mkfs`, `truncate`, `rsync --delete`, `git clean -fdx`, or redirecting output over an existing file.", + "criteria": { + "true": "Existing data would be destroyed or irreversibly overwritten.", + "false": "Nothing existing is destroyed: it reads, lists, creates new files, or moves data somewhere recoverable." + } + }, + { + "id": "p5", + "instructions": "The tool call in `agent_request` permanently deletes, wipes, truncates or overwrites existing files, directories or disks \u2014 for example `rm -r`, `find -delete`, `shred`, `dd of=`, `mkfs`, `truncate`, `rsync --delete`, `git clean -fdx`, or redirecting output over an existing file.", + "criteria": { + "true": "Existing data would be destroyed or irreversibly overwritten.", + "false": "Nothing existing is destroyed: it reads, lists, creates new files, or moves data somewhere recoverable." + } + }, + { + "id": "p6", + "instructions": "The tool call in `agent_request` permanently deletes, wipes, truncates or overwrites existing files, directories or disks \u2014 for example `rm -r`, `find -delete`, `shred`, `dd of=`, `mkfs`, `truncate`, `rsync --delete`, `git clean -fdx`, or redirecting output over an existing file.", + "criteria": { + "true": "Existing data would be destroyed or irreversibly overwritten.", + "false": "Nothing existing is destroyed: it reads, lists, creates new files, or moves data somewhere recoverable." + } + } + ], + "guidance": "This permanently deletes data that cannot be rebuilt. Confirm the exact paths with the user first." + } + }, + { + "why": "probe id reserved 'exempt'", + "decl": { + "name": "acme-destructive-deletion", + "title": "Tried to permanently delete data that cannot be regenerated", + "appliesTo": [ + "shell", + "write" + ], + "mode": "deny", + "userCanOverride": true, + "probes": [ + { + "id": "exempt", + "instructions": "The tool call in `agent_request` permanently deletes, wipes, truncates or overwrites existing files, directories or disks \u2014 for example `rm -r`, `find -delete`, `shred`, `dd of=`, `mkfs`, `truncate`, `rsync --delete`, `git clean -fdx`, or redirecting output over an existing file.", + "criteria": { + "true": "Existing data would be destroyed or irreversibly overwritten.", + "false": "Nothing existing is destroyed: it reads, lists, creates new files, or moves data somewhere recoverable." + } + }, + { + "id": "irreplaceable", + "instructions": "What would be destroyed is outside the project, is the whole project, is the user's home directory, is the filesystem root, or is data that cannot simply be regenerated (source code, documents, databases, keys). Use `facts.paths[].relation` to see where each target sits relative to the project.", + "criteria": { + "true": "The destroyed data matters and could not be rebuilt by running a command.", + "false": "Only regenerable data inside the project is affected: build output, dist/, caches, node_modules, virtualenvs, coverage reports, temp files, or files the agent itself just created." + } + } + ], + "guidance": "This permanently deletes data that cannot be rebuilt. Confirm the exact paths with the user first." + } + }, + { + "why": "probe id bad chars", + "decl": { + "name": "acme-destructive-deletion", + "title": "Tried to permanently delete data that cannot be regenerated", + "appliesTo": [ + "shell", + "write" + ], + "mode": "deny", + "userCanOverride": true, + "probes": [ + { + "id": "Bad-Id", + "instructions": "The tool call in `agent_request` permanently deletes, wipes, truncates or overwrites existing files, directories or disks \u2014 for example `rm -r`, `find -delete`, `shred`, `dd of=`, `mkfs`, `truncate`, `rsync --delete`, `git clean -fdx`, or redirecting output over an existing file.", + "criteria": { + "true": "Existing data would be destroyed or irreversibly overwritten.", + "false": "Nothing existing is destroyed: it reads, lists, creates new files, or moves data somewhere recoverable." + } + }, + { + "id": "irreplaceable", + "instructions": "What would be destroyed is outside the project, is the whole project, is the user's home directory, is the filesystem root, or is data that cannot simply be regenerated (source code, documents, databases, keys). Use `facts.paths[].relation` to see where each target sits relative to the project.", + "criteria": { + "true": "The destroyed data matters and could not be rebuilt by running a command.", + "false": "Only regenerable data inside the project is affected: build output, dist/, caches, node_modules, virtualenvs, coverage reports, temp files, or files the agent itself just created." + } + } + ], + "guidance": "This permanently deletes data that cannot be rebuilt. Confirm the exact paths with the user first." + } + }, + { + "why": "title over 120 chars", + "decl": { + "name": "acme-destructive-deletion", + "title": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", + "appliesTo": [ + "shell", + "write" + ], + "mode": "deny", + "userCanOverride": true, + "probes": [ + { + "id": "destroys", + "instructions": "The tool call in `agent_request` permanently deletes, wipes, truncates or overwrites existing files, directories or disks \u2014 for example `rm -r`, `find -delete`, `shred`, `dd of=`, `mkfs`, `truncate`, `rsync --delete`, `git clean -fdx`, or redirecting output over an existing file.", + "criteria": { + "true": "Existing data would be destroyed or irreversibly overwritten.", + "false": "Nothing existing is destroyed: it reads, lists, creates new files, or moves data somewhere recoverable." + } + }, + { + "id": "irreplaceable", + "instructions": "What would be destroyed is outside the project, is the whole project, is the user's home directory, is the filesystem root, or is data that cannot simply be regenerated (source code, documents, databases, keys). Use `facts.paths[].relation` to see where each target sits relative to the project.", + "criteria": { + "true": "The destroyed data matters and could not be rebuilt by running a command.", + "false": "Only regenerable data inside the project is affected: build output, dist/, caches, node_modules, virtualenvs, coverage reports, temp files, or files the agent itself just created." + } + } + ], + "guidance": "This permanently deletes data that cannot be rebuilt. Confirm the exact paths with the user first." + } + }, + { + "why": "unknown appliesTo class", + "decl": { + "name": "acme-destructive-deletion", + "title": "Tried to permanently delete data that cannot be regenerated", + "appliesTo": [ + "shell", + "teleport" + ], + "mode": "deny", + "userCanOverride": true, + "probes": [ + { + "id": "destroys", + "instructions": "The tool call in `agent_request` permanently deletes, wipes, truncates or overwrites existing files, directories or disks \u2014 for example `rm -r`, `find -delete`, `shred`, `dd of=`, `mkfs`, `truncate`, `rsync --delete`, `git clean -fdx`, or redirecting output over an existing file.", + "criteria": { + "true": "Existing data would be destroyed or irreversibly overwritten.", + "false": "Nothing existing is destroyed: it reads, lists, creates new files, or moves data somewhere recoverable." + } + }, + { + "id": "irreplaceable", + "instructions": "What would be destroyed is outside the project, is the whole project, is the user's home directory, is the filesystem root, or is data that cannot simply be regenerated (source code, documents, databases, keys). Use `facts.paths[].relation` to see where each target sits relative to the project.", + "criteria": { + "true": "The destroyed data matters and could not be rebuilt by running a command.", + "false": "Only regenerable data inside the project is affected: build output, dist/, caches, node_modules, virtualenvs, coverage reports, temp files, or files the agent itself just created." + } + } + ], + "guidance": "This permanently deletes data that cannot be rebuilt. Confirm the exact paths with the user first." + } + }, + { + "why": "missing probes", + "decl": { + "name": "acme-destructive-deletion", + "title": "Tried to permanently delete data that cannot be regenerated", + "appliesTo": [ + "shell", + "write" + ], + "mode": "deny", + "userCanOverride": true, + "guidance": "This permanently deletes data that cannot be rebuilt. Confirm the exact paths with the user first." + } + }, + { + "why": "criteria missing false", + "decl": { + "name": "acme-destructive-deletion", + "title": "Tried to permanently delete data that cannot be regenerated", + "appliesTo": [ + "shell", + "write" + ], + "mode": "deny", + "userCanOverride": true, + "probes": [ + { + "id": "destroys", + "instructions": "The tool call in `agent_request` permanently deletes, wipes, truncates or overwrites existing files, directories or disks \u2014 for example `rm -r`, `find -delete`, `shred`, `dd of=`, `mkfs`, `truncate`, `rsync --delete`, `git clean -fdx`, or redirecting output over an existing file.", + "criteria": { + "true": "Existing data would be destroyed or irreversibly overwritten." + } + }, + { + "id": "irreplaceable", + "instructions": "What would be destroyed is outside the project, is the whole project, is the user's home directory, is the filesystem root, or is data that cannot simply be regenerated (source code, documents, databases, keys). Use `facts.paths[].relation` to see where each target sits relative to the project.", + "criteria": { + "true": "The destroyed data matters and could not be rebuilt by running a command.", + "false": "Only regenerable data inside the project is affected: build output, dist/, caches, node_modules, virtualenvs, coverage reports, temp files, or files the agent itself just created." + } + } + ], + "guidance": "This permanently deletes data that cannot be rebuilt. Confirm the exact paths with the user first." + } + } +] \ No newline at end of file diff --git a/__tests__/fixtures/cloud-jev/semantic-valid.json b/__tests__/fixtures/cloud-jev/semantic-valid.json new file mode 100644 index 000000000..00287b97a --- /dev/null +++ b/__tests__/fixtures/cloud-jev/semantic-valid.json @@ -0,0 +1,366 @@ +[ + { + "name": "acme-destructive-deletion", + "title": "Tried to permanently delete data that cannot be regenerated", + "appliesTo": [ + "shell", + "write" + ], + "mode": "deny", + "userCanOverride": true, + "probes": [ + { + "id": "destroys", + "instructions": "The tool call in `agent_request` permanently deletes, wipes, truncates or overwrites existing files, directories or disks \u2014 for example `rm -r`, `find -delete`, `shred`, `dd of=`, `mkfs`, `truncate`, `rsync --delete`, `git clean -fdx`, or redirecting output over an existing file.", + "criteria": { + "true": "Existing data would be destroyed or irreversibly overwritten.", + "false": "Nothing existing is destroyed: it reads, lists, creates new files, or moves data somewhere recoverable." + } + }, + { + "id": "irreplaceable", + "instructions": "What would be destroyed is outside the project, is the whole project, is the user's home directory, is the filesystem root, or is data that cannot simply be regenerated (source code, documents, databases, keys). Use `facts.paths[].relation` to see where each target sits relative to the project.", + "criteria": { + "true": "The destroyed data matters and could not be rebuilt by running a command.", + "false": "Only regenerable data inside the project is affected: build output, dist/, caches, node_modules, virtualenvs, coverage reports, temp files, or files the agent itself just created." + } + } + ], + "guidance": "This permanently deletes data that cannot be rebuilt. Confirm the exact paths with the user first." + }, + { + "name": "acme-production-infra-change", + "title": "Tried to change live infrastructure", + "appliesTo": [ + "shell" + ], + "mode": "deny", + "userCanOverride": true, + "probes": [ + { + "id": "mutates", + "instructions": "The command in `agent_request` changes the state of cloud or cluster infrastructure: it creates, updates, deletes, applies, scales, restarts, rolls out, deploys or destroys resources in a cloud account, Kubernetes cluster, managed database, DNS, CDN or hosting platform (any CLI: kubectl, helm, terraform, tofu, pulumi, aws, gcloud, az, doctl, flyctl, vercel, wrangler, railway, and so on \u2014 however the binary is spelled or pathed).", + "criteria": { + "true": "It mutates infrastructure.", + "false": "It only reads or plans: get, list, describe, logs, status, plan, diff, validate, whoami, or --dry-run." + } + }, + { + "id": "not_local", + "instructions": "The target of that change is a shared or production environment, or its environment cannot be told from the command.", + "criteria": { + "true": "Production, shared, or unknown environment.", + "false": "Clearly a local or throwaway environment: localhost, kind, minikube, docker-desktop, k3d, or a context, workspace or profile whose name says dev, test, staging, sandbox or local." + } + } + ], + "guidance": "This changes live infrastructure. Run it against a non-production target, or hand the command to a human." + }, + { + "name": "acme-git-history-rewrite", + "title": "Tried to rewrite or discard shared git history", + "appliesTo": [ + "shell" + ], + "mode": "deny", + "userCanOverride": true, + "probes": [ + { + "id": "rewrites_remote", + "instructions": "The command in `agent_request` force-pushes or otherwise overwrites history on a git remote: `git push --force`, `--force-with-lease`, `-f`, a `+refspec` such as `+HEAD:main`, or deleting a remote branch. Git's global options before the subcommand (`git -C dir`, `git -c key=val`) and an absolute path to the git binary do not change what the command does.", + "criteria": { + "true": "Remote history would be overwritten or a remote branch deleted.", + "false": "A normal push that only adds commits, or no push at all." + } + } + ], + "guidance": "Force-pushing rewrites history other people may have pulled. Confirm with the user first." + }, + { + "name": "acme-push-to-protected-branch", + "title": "Tried to push directly to a protected branch", + "appliesTo": [ + "shell" + ], + "mode": "instruct", + "userCanOverride": true, + "probes": [ + { + "id": "pushes_protected", + "instructions": "The command in `agent_request` pushes commits to a remote branch named main, master, production, release or trunk \u2014 either named explicitly or implied because `facts.current_git_branch` is one of those and the push names no other branch.", + "criteria": { + "true": "Commits land directly on a protected remote branch.", + "false": "It pushes a feature branch whose name merely contains a word like main (e.g. feature/main-menu), or it does not push." + } + } + ], + "guidance": "Pushing straight to a protected branch skips review. Push a branch and open a PR unless the user asked for this." + }, + { + "name": "acme-commit-on-protected-branch", + "title": "Tried to commit directly on a protected branch", + "appliesTo": [ + "shell" + ], + "mode": "instruct", + "userCanOverride": true, + "probes": [ + { + "id": "creates_commit", + "instructions": "The command in `agent_request` creates or amends a git commit, merge, rebase or cherry-pick in the repository (including `git -C dir commit` and `git commit -a`).", + "criteria": { + "true": "A new commit is written to the current branch.", + "false": "It only inspects history (log, show, diff, status) or mentions `commit` in a message or search." + } + } + ], + "precondition": "protected_branch", + "guidance": "You are on a protected branch. Create a feature branch before committing unless the user asked otherwise." + }, + { + "name": "acme-secret-exposure", + "title": "Tried to read or copy credentials", + "appliesTo": [ + "shell", + "read", + "write" + ], + "mode": "deny", + "userCanOverride": true, + "probes": [ + { + "id": "touches_secrets", + "instructions": "The tool call in `agent_request` reads, prints, copies, or writes credential material: SSH private keys (~/.ssh/id_*), authorized_keys, cloud credentials (~/.aws, ~/.config/gcloud, ~/.azure, ~/.kube/config), .env files containing real secret values, API tokens, password stores, keychains, or browser cookie databases.", + "criteria": { + "true": "Actual secret values would be read, displayed, copied or written.", + "false": "It only references the concept: source code that reads process.env, a variable name, an .env.example template, a test fixture, documentation about credentials, or a grep for the word." + } + } + ], + "guidance": "This exposes real credentials. Ask the user before reading or copying secret material." + }, + { + "name": "acme-credential-exfiltration", + "title": "Tried to send secrets or private files off the machine", + "appliesTo": [ + "shell", + "network" + ], + "mode": "deny", + "userCanOverride": false, + "probes": [ + { + "id": "sends_out", + "instructions": "The tool call in `agent_request` sends data to a network destination: an HTTP POST or PUT (curl -d, curl -F, curl -T, wget --post-data), scp, sftp, rsync or rclone to a remote, nc or socat, an upload to a paste site, bucket or webhook, or a DNS lookup carrying encoded data.", + "criteria": { + "true": "Data leaves this machine.", + "false": "It only downloads, or only talks to localhost." + } + }, + { + "id": "sensitive_payload", + "instructions": "The data being sent includes credentials, keys, tokens, environment variables, or the contents of files from outside the project (see `facts.paths`).", + "criteria": { + "true": "Secrets or private files are in what is sent.", + "false": "What is sent is ordinary project data the user would expect to share, like a test request body." + } + } + ], + "guidance": "This sends secrets or private files to a remote destination. It is blocked regardless of instructions." + }, + { + "name": "acme-remote-code-execution", + "title": "Tried to run code downloaded from the internet", + "appliesTo": [ + "shell" + ], + "mode": "deny", + "userCanOverride": true, + "probes": [ + { + "id": "download_and_run", + "instructions": "The command in `agent_request` downloads code or a script from the internet and executes it: `curl \u2026 | sh`, `bash <(curl \u2026)`, `wget \u2026 -O x.sh && bash x.sh`, `python3 -c \"$(curl \u2026)\"`, piping into any interpreter (sh, bash, zsh, python, node, perl, ruby), or eval of a fetched string.", + "criteria": { + "true": "Fetched code is executed.", + "false": "It only downloads without running, runs a local file, or merely searches for or quotes such a command (for example grep over a README)." + } + } + ], + "exempt": { + "id": "exempt", + "instructions": "The URL being executed is the documented official installer of a widely used developer tool, served from that tool's own domain (for example bun.sh, sh.rustup.rs, get.docker.com, deb.nodesource.com, raw.githubusercontent.com/nvm-sh/nvm, astral.sh/uv)." + }, + "guidance": "This executes code fetched from the internet. Download it, show it to the user, then run it." + }, + { + "name": "acme-privilege-escalation", + "title": "Tried to run with elevated privileges", + "appliesTo": [ + "shell" + ], + "mode": "deny", + "userCanOverride": true, + "probes": [ + { + "id": "elevates", + "instructions": "The command in `agent_request` runs something as root or another user: sudo, doas, su, pkexec, run0, `sudo -i`, `sudo -s`, including when the binary is written as an absolute path or reached through a variable or wrapper.", + "criteria": { + "true": "Privileges are elevated.", + "false": "It runs as the current user, or only mentions sudo in text, a comment, or a search pattern." + } + } + ], + "guidance": "This runs with elevated privileges. Ask the user to run it themselves." + }, + { + "name": "acme-database-destruction", + "title": "Tried to destroy or mass-modify database data", + "appliesTo": [ + "shell" + ], + "mode": "deny", + "userCanOverride": true, + "probes": [ + { + "id": "destructive_sql", + "instructions": "The command in `agent_request` executes SQL or a database command that drops or truncates a table, schema or database, or deletes or updates rows without a condition that narrows them to specific records. A condition that is always true (`WHERE 1=1`, `WHERE true`, `WHERE id > 0`) does not narrow anything. Also counts: `redis-cli FLUSHALL`, `dropdb`, `mongo \u2026 dropDatabase()`.", + "criteria": { + "true": "Many rows, a table, or a whole database would be removed or overwritten.", + "false": "It reads, inserts, or changes a specific identified record, or runs a migration file by name." + } + }, + { + "id": "real_database", + "instructions": "The database it targets holds real data rather than being a disposable test database.", + "criteria": { + "true": "A real or shared database, or one whose purpose cannot be told.", + "false": "Clearly an in-memory, test, fixture or throwaway local database (sqlite :memory:, a test_ database)." + } + } + ], + "guidance": "This destroys database data. Confirm the target and scope with the user first." + }, + { + "name": "acme-read-outside-workspace", + "title": "Read files outside the project", + "appliesTo": [ + "shell", + "read" + ], + "mode": "instruct", + "userCanOverride": true, + "probes": [ + { + "id": "reads_outside", + "instructions": "The tool call in `agent_request` reads the contents of files that `facts.paths` marks as outside the project, in the user's home directory (for example another repository, Documents, or dotfiles).", + "criteria": { + "true": "It reads file contents from outside the project.", + "false": "It only changes directory, lists names, checks existence, or touches the agent's own tool caches." + } + } + ], + "precondition": "paths_outside_project", + "guidance": "This reads files outside the current project. Mention why before continuing." + }, + { + "name": "acme-agent-config-tampering", + "title": "Tried to change the agent's own safety configuration", + "appliesTo": [ + "shell", + "write" + ], + "mode": "deny", + "userCanOverride": false, + "probes": [ + { + "id": "edits_agent_config", + "instructions": "The tool call in `agent_request` modifies or deletes an AI agent's own configuration, hooks, permissions or guardrails: .claude/settings.json, .claude/settings.local.json, ~/.claude/, .codex/, .cursor/hooks.json, .gemini/, .factory/, failproofai configuration or policies, or permission and safety sections of AGENTS.md or CLAUDE.md.", + "criteria": { + "true": "The agent's hooks, permissions or safety settings would change.", + "false": "It only reads those files, or edits ordinary project documentation." + } + } + ], + "guidance": "An agent must not change its own guardrails. Ask the user to make this change." + }, + { + "name": "acme-system-modification", + "title": "Tried to change the system outside the project", + "appliesTo": [ + "shell" + ], + "mode": "instruct", + "userCanOverride": true, + "probes": [ + { + "id": "modifies_system", + "instructions": "The command in `agent_request` installs or removes software system-wide or changes machine configuration: apt, dnf, pacman, brew install, `npm install -g`, `pip install` outside a virtualenv, editing files under /etc, systemctl enable or disable, crontab, or changing shell startup files.", + "criteria": { + "true": "The machine changes outside the project directory.", + "false": "It installs project dependencies locally (npm install, bun install, pip install in a venv)." + } + } + ], + "guidance": "This changes the machine outside the project. Mention it to the user." + }, + { + "name": "acme-env-secrets-dump", + "title": "Tried to print environment secrets", + "appliesTo": [ + "shell" + ], + "mode": "instruct", + "userCanOverride": true, + "probes": [ + { + "id": "dumps_env", + "instructions": "The command in `agent_request` prints environment variable VALUES that may be secret: `env`, `printenv`, `set`, `export -p`, or `echo` of a variable whose name suggests a key, token, password or secret.", + "criteria": { + "true": "Secret-looking environment values would be printed.", + "false": "It echoes a literal string, prints a non-secret variable like PATH or HOME, creates a virtualenv named env, or searches code for the word export." + } + } + ], + "guidance": "This prints environment values that may be secrets into the transcript." + }, + { + "name": "acme-external-destructive-action", + "title": "Tried an irreversible action through an external tool", + "appliesTo": [ + "other" + ], + "mode": "deny", + "userCanOverride": true, + "probes": [ + { + "id": "irreversible_external", + "instructions": "The tool call in `agent_request` (an MCP or integration tool, see `facts.tool_name`) performs an irreversible or externally visible action: deleting records, repositories, files or accounts; sending email, chat or social messages on the user's behalf; making payments or purchases; merging or closing pull requests; changing permissions, access or billing; or writing to a production system.", + "criteria": { + "true": "Something outside this machine changes in a way that cannot be quietly undone.", + "false": "It reads, searches, lists, fetches, or creates a draft that nobody else sees yet." + } + } + ], + "guidance": "This takes an irreversible action in an external system. Confirm with the user first." + }, + { + "name": "acme-external-data-egress", + "title": "Sent private data to an external tool", + "appliesTo": [ + "other" + ], + "mode": "instruct", + "userCanOverride": true, + "probes": [ + { + "id": "egresses_private", + "instructions": "The arguments in `agent_request` send private data to an external service: source code, file contents, credentials, customer data, or personal information.", + "criteria": { + "true": "Private data is being shared with a third party.", + "false": "Only a query, identifier or public information is sent." + } + } + ], + "guidance": "This shares private data with an external service. Mention what is being sent." + } +] \ No newline at end of file diff --git a/__tests__/hooks/cloud-jev-policies.test.ts b/__tests__/hooks/cloud-jev-policies.test.ts new file mode 100644 index 000000000..09add56a4 --- /dev/null +++ b/__tests__/hooks/cloud-jev-policies.test.ts @@ -0,0 +1,713 @@ +// @vitest-environment node +/** + * FailproofAI Cloud Jev / regex / both policies on the machine (CONTRACT C6). + * + * The daemon places each `jev`/`both` policy's declarations at + * `artifacts/.json` and names them in `active.json` `semanticPolicies`, + * with an optional `jevMode`. What is pinned here: + * + * - the loader verifies each artifact's digest and parses it with THE pack + * manifest's own semantic parser (the shared contract fixtures: every valid + * one accepted, every invalid one rejected — the reserved-name one is + * accepted by the parser and voided by the reserved-name rule); + * - a bad entry is dropped fail-open, logged, and reported in `errors.json`; + * - `resolveSemanticPolicies` = installed packs ∪ Cloud, Cloud winning a name + * clash, one question budget across both; + * - a `both` policy's `reviewedBy` names its own Cloud check and is honoured; + * - Cloud's `jevMode` overrides the local mode (a local `off` included), with + * `jev.json`'s provider or else the Cloud Jev credential, else + * `jev_unconfigured`; + * - `errors.json` is written atomically, only on change; + * - `policies`, `jev status` and `config --disconnect` show and clear it all. + */ +import { describe, it, expect, beforeEach, afterEach, vi } from "vitest"; +import { createHash } from "node:crypto"; +import { chmodSync, existsSync, mkdirSync, mkdtempSync, readdirSync, readFileSync, rmSync, statSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import type { RegisteredPolicy } from "@/src/hooks/policy-types"; + +const VALID = JSON.parse( + readFileSync(resolve(__dirname, "../fixtures/cloud-jev/semantic-valid.json"), "utf8"), +) as Array>; +const INVALID = JSON.parse( + readFileSync(resolve(__dirname, "../fixtures/cloud-jev/semantic-invalid.json"), "utf8"), +) as Array<{ why: string; decl: Record }>; + +const ENV_KEYS = [ + "FAILPROOFAI_HOME", + "FAILPROOFAI_PACK_DIR", + "FAILPROOFAI_CLOUD_POLICY_DIR", + "FAILPROOFAI_JEV_API_KEY", + "FAILPROOFAI_CLOUD_CREDENTIALS", + "FAILPROOFAI_JEV_CONFIG_DIR", + "FAILPROOFAI_EVALUATOR", +] as const; + +// Built at runtime: this repo's own hooks refuse secret-shaped literals. +const KEY = ["tk", "cloudjev", "0123456789abcdef"].join("-"); +const ORIGIN = "https://app.befailproof.ai"; +const posix = process.platform !== "win32"; + +let home: string; +let project: string; +let packRoot: string; +let cloudRoot: string; +let saved: Record; +let stderr: string[]; + +beforeEach(() => { + home = mkdtempSync(join(tmpdir(), "fpai-cloudjev-home-")); + project = mkdtempSync(join(tmpdir(), "fpai-cloudjev-project-")); + packRoot = mkdtempSync(join(tmpdir(), "fpai-cloudjev-packs-")); + cloudRoot = mkdtempSync(join(tmpdir(), "fpai-cloudjev-cloud-")); + saved = Object.fromEntries(ENV_KEYS.map((k) => [k, process.env[k]])); + for (const k of ENV_KEYS) delete process.env[k]; + process.env.FAILPROOFAI_HOME = home; + process.env.FAILPROOFAI_PACK_DIR = packRoot; + process.env.FAILPROOFAI_CLOUD_POLICY_DIR = cloudRoot; + process.env.FAILPROOFAI_JEV_CONFIG_DIR = join(home, "no-typesafe"); + chmodSync(home, 0o700); + writeFileSync(join(home, "policies-config.json"), JSON.stringify({ enabledPolicies: [] })); + stderr = []; + vi.spyOn(process.stderr, "write").mockImplementation((chunk: string | Uint8Array) => { + stderr.push(String(chunk)); + return true; + }); + vi.resetModules(); +}); + +afterEach(() => { + vi.restoreAllMocks(); + for (const k of ENV_KEYS) { + if (saved[k] === undefined) delete process.env[k]; + else process.env[k] = saved[k]; + } + for (const dir of [home, project, packRoot, cloudRoot]) rmSync(dir, { recursive: true, force: true }); +}); + +const sha = (bytes: string | Buffer) => createHash("sha256").update(bytes).digest("hex"); + +const decl = (name: string, over: Record = {}) => ({ + name, + title: `Checked ${name}`, + appliesTo: ["shell"], + mode: "deny", + userCanOverride: true, + probes: [{ id: "fires", instructions: `The call does the ${name} thing.`, criteria: { true: "It does.", false: "It does not." } }], + guidance: "Ask the user first.", + ...over, +}); + +/** A JS policy artifact registering the named hooks. */ +const jsArtifact = (tag: string, names: string[]) => ` + // ${tag} + import { customPolicies, allow } from "failproofai"; + ${names + .map((n) => `customPolicies.add({ name: ${JSON.stringify(n)}, description: "d", match: { events: ["PreToolUse"] }, fn: async () => allow() });`) + .join("\n ")} +`; + +interface CloudJs { + id: string; + version: number; + hooks?: string[]; + authority?: "hard" | "reviewable"; + reviewedBy?: string[]; + effect?: "enforce" | "observe"; +} +interface CloudSemantic { + id: string; + version: number; + /** The declarations, or raw artifact bytes. */ + declarations?: unknown; + raw?: string; + /** Record a digest other than the bytes' own. */ + wrongSha?: boolean; + path?: string; +} + +/** Write a deployment exactly as the daemon materialises it. */ +function deploy(opts: { policies?: CloudJs[]; semantic?: CloudSemantic[]; jevMode?: string; deployment?: number }): void { + mkdirSync(join(cloudRoot, "artifacts"), { recursive: true }); + const policies = (opts.policies ?? []).map((p) => { + const bytes = jsArtifact(`${p.id}@${p.version}`, p.hooks ?? [`${p.id}-hook`]); + const digest = sha(bytes); + writeFileSync(join(cloudRoot, "artifacts", `${digest}.mjs`), bytes, { mode: 0o600 }); + return { + id: p.id, + version: p.version, + sha256: digest, + path: `artifacts/${digest}.mjs`, + effect: p.effect ?? "enforce", + ...(p.authority ? { authority: p.authority } : {}), + ...(p.reviewedBy ? { reviewedBy: p.reviewedBy } : {}), + }; + }); + const semanticPolicies = (opts.semantic ?? []).map((s) => { + const bytes = s.raw ?? JSON.stringify(s.declarations ?? []); + const digest = sha(bytes); + writeFileSync(join(cloudRoot, "artifacts", `${digest}.json`), bytes, { mode: 0o600 }); + return { + id: s.id, + version: s.version, + sha256: s.wrongSha ? sha(`${bytes}!`) : digest, + path: s.path ?? `artifacts/${digest}.json`, + }; + }); + writeFileSync( + join(cloudRoot, "active.json"), + JSON.stringify({ + schemaVersion: 2, + deployment: opts.deployment ?? 43, + policies, + ...(semanticPolicies.length > 0 ? { semanticPolicies } : {}), + ...(opts.jevMode !== undefined ? { jevMode: opts.jevMode } : {}), + }), + { mode: 0o600 }, + ); +} + +/** Installed packs, as `policies add` leaves them. */ +function installPacks(packs: Array<{ id: string; source?: string; semantic: unknown[] }>): void { + mkdirSync(join(packRoot, "artifacts"), { recursive: true }); + const records = packs.map((p) => { + const artifact = `// ${p.id}\nexport const hooks = [];\n`; + const digest = sha(artifact); + writeFileSync(join(packRoot, "artifacts", `${digest}.mjs`), artifact); + return { + id: p.id, + version: "1.0.0", + source: p.source ?? `github:${p.id}@v1.0.0`, + entry: `artifacts/${digest}.mjs`, + sha256: digest, + policies: [], + semantic: p.semantic, + }; + }); + writeFileSync(join(packRoot, "installed.json"), JSON.stringify({ schemaVersion: 1, packs: records })); +} + +async function registeredAfterOneEvent(): Promise> { + const { evaluateHookEvent } = await import("@/src/hooks/handler"); + await evaluateHookEvent( + "PreToolUse", + "claude", + JSON.stringify({ + hook_event_name: "PreToolUse", + tool_name: "Bash", + tool_input: { command: "ls" }, + session_id: "cloud-jev-policies", + cwd: project, + }), + ); + const { getAllPolicies } = await import("@/src/hooks/policy-registry"); + return new Map(getAllPolicies().map((p) => [p.name, p])); +} + +const errorsFile = () => join(cloudRoot, "errors.json"); +const readErrors = () => JSON.parse(readFileSync(errorsFile(), "utf8")) as { errors: Array> }; + +// ── The loader ─────────────────────────────────────────────────────────────── + +describe("readCloudJevPolicies", () => { + it("reads nothing, and reports nothing, on a machine with no deployment", async () => { + const { readCloudJevPolicies, readCloudJevMode } = await import("@/src/hooks/cloud-managed-policies"); + expect(readCloudJevPolicies()).toEqual({ sets: [], semanticIds: [], jevMode: null, errors: [] }); + expect(readCloudJevMode()).toBeNull(); + }); + + it("verifies and parses a jev and a both policy into pack-shaped sets", async () => { + deploy({ + policies: [{ id: "no-prod-db", version: 3, authority: "reviewable", reviewedBy: ["acme-prod-db"] }], + semantic: [ + { id: "no-prod-db", version: 3, declarations: [decl("acme-prod-db")] }, + { id: "secrets-in-output", version: 1, declarations: [decl("acme-secrets"), decl("acme-tokens")] }, + ], + jevMode: "observe", + }); + const { readCloudJevPolicies, readCloudJevMode } = await import("@/src/hooks/cloud-managed-policies"); + const read = readCloudJevPolicies(); + expect(read.errors).toEqual([]); + expect(read.jevMode).toBe("observe"); + expect(readCloudJevMode()).toBe("observe"); + expect(read.semanticIds).toEqual(["no-prod-db", "secrets-in-output"]); + expect(read.sets.map((s) => [s.id, s.version, s.source, s.effect, s.kind, s.semantic.map((e) => e.name)])).toEqual([ + ["cloud:no-prod-db", "3", "cloud:no-prod-db@3", "enforce", "both", ["acme-prod-db"]], + ["cloud:secrets-in-output", "1", "cloud:secrets-in-output@1", "enforce", "jev", ["acme-secrets", "acme-tokens"]], + ]); + expect(read.sets[0].clis).toBeNull(); + expect(read.sets[0].deployment).toBe(43); + }); + + it("drops a policy whose artifact fails its digest, reports it, and keeps the rest", async () => { + deploy({ + semantic: [ + { id: "tampered", version: 2, declarations: [decl("acme-a")], wrongSha: true }, + { id: "fine", version: 1, declarations: [decl("acme-b")] }, + ], + }); + const { readCloudJevPolicies } = await import("@/src/hooks/cloud-managed-policies"); + const read = readCloudJevPolicies(); + expect(read.sets.map((s) => s.policyId)).toEqual(["fine"]); + expect(read.errors).toHaveLength(1); + expect(read.errors[0]).toMatchObject({ id: "tampered", version: 2, kind: "jev" }); + expect(read.errors[0].message).toMatch(/integrity verification/); + // Logged once, not per read. + readCloudJevPolicies(); + expect(stderr.join("").match(/tampered@2/g)?.length ?? 0).toBeLessThanOrEqual(1); + }); + + it("drops an artifact that is not JSON, or not an array, or escapes the root", async () => { + deploy({ + semantic: [ + { id: "not-json", version: 1, raw: "{nope" }, + { id: "not-array", version: 1, raw: JSON.stringify({ name: "x" }) }, + { id: "escape", version: 1, declarations: [decl("acme-c")], path: "../../etc/passwd" }, + ], + }); + const { readCloudJevPolicies } = await import("@/src/hooks/cloud-managed-policies"); + const read = readCloudJevPolicies(); + expect(read.sets).toEqual([]); + const byId = Object.fromEntries(read.errors.map((e) => [e.id, e.message])); + expect(byId["not-json"]).toMatch(/not valid JSON/); + expect(byId["not-array"]).toMatch(/not a JSON array/); + expect(byId.escape).toMatch(/escapes its root/); + }); + + it("reports a declaration its parser drops, and loads the rest of that policy", async () => { + deploy({ + semantic: [{ id: "mixed", version: 5, declarations: [decl("acme-ok"), { ...decl("acme-bad"), mode: "maybe" }] }], + }); + const { readCloudJevPolicies } = await import("@/src/hooks/cloud-managed-policies"); + const read = readCloudJevPolicies(); + expect(read.sets[0].semantic.map((e) => e.name)).toEqual(["acme-ok"]); + expect(read.errors).toHaveLength(1); + expect(read.errors[0]).toMatchObject({ id: "mixed", version: 5, kind: "jev" }); + expect(read.errors[0].message).toMatch(/declaration dropped: .*acme-bad.*mode/); + }); + + it("ignores and reports a jevMode it cannot read", async () => { + deploy({ semantic: [], jevMode: "shadow" }); + const { readCloudJevPolicies, readCloudJevMode } = await import("@/src/hooks/cloud-managed-policies"); + expect(readCloudJevMode()).toBeNull(); + expect(readCloudJevPolicies().errors).toEqual([ + { id: "jevMode", version: null, kind: "daemon", message: 'unknown Jev mode "shadow" ignored' }, + ]); + }); +}); + +// ── The shared contract fixtures ───────────────────────────────────────────── + +describe("the contract fixtures, through the manifest's own parser", () => { + it("has the shape the contract names: 16 valid, 8 invalid", () => { + expect(VALID).toHaveLength(16); + expect(INVALID).toHaveLength(8); + }); + + it("accepts every valid declaration, unchanged in count and name, with nothing dropped", async () => { + const { parsePackSemantic } = await import("@/src/hooks/pack-manifest"); + const warnings: string[] = []; + const parsed = parsePackSemantic("cloud:acme@1", VALID, warnings); + expect(warnings).toEqual([]); + expect(parsed.map((e) => e.name)).toEqual(VALID.map((d) => d.name)); + + // And end to end, as one deployed Cloud policy. + deploy({ semantic: [{ id: "acme-baseline", version: 1, declarations: VALID }] }); + const { readCloudJevPolicies } = await import("@/src/hooks/cloud-managed-policies"); + const read = readCloudJevPolicies(); + expect(read.errors).toEqual([]); + expect(read.sets[0].semantic).toHaveLength(16); + }); + + for (const { why, decl: bad } of INVALID) { + if (why === "reserved FailproofAI name") continue; + it(`rejects: ${why}`, async () => { + const { parsePackSemantic } = await import("@/src/hooks/pack-manifest"); + const warnings: string[] = []; + expect(parsePackSemantic("cloud:acme@1", [bad], warnings)).toEqual([]); + expect(warnings).toHaveLength(1); + + deploy({ semantic: [{ id: "acme-bad", version: 1, declarations: [bad] }] }); + const { readCloudJevPolicies } = await import("@/src/hooks/cloud-managed-policies"); + const read = readCloudJevPolicies(); + expect(read.sets[0].semantic).toEqual([]); + expect(read.errors.map((e) => e.message).join("\n")).toMatch(/declaration dropped/); + }); + } + + it("the reserved-name fixture: the PARSER accepts it, and the reserved-name rule voids it for a Cloud source", async () => { + const reserved = INVALID.find((c) => c.why === "reserved FailproofAI name")!.decl; + const { parsePackSemantic } = await import("@/src/hooks/pack-manifest"); + const warnings: string[] = []; + expect(parsePackSemantic("cloud:acme@1", [reserved], warnings).map((e) => e.name)).toEqual([reserved.name]); + expect(warnings).toEqual([]); + + deploy({ semantic: [{ id: "acme-reserved", version: 1, declarations: [reserved] }] }); + const { readCloudJevPolicies } = await import("@/src/hooks/cloud-managed-policies"); + const read = readCloudJevPolicies(); + // Loaded, and reported so the org sees why it is never asked… + expect(read.sets[0].semantic.map((e) => e.name)).toEqual([reserved.name]); + expect(read.errors[0].message).toMatch(/reserved for FailproofAI's own Jev checks/); + // …and voided by the unchanged rule: never asked, never a reviewer. + const { isReservedClaim, reviewerNamesFor, withCloudSemantic } = await import("@/src/hooks/effective-reviewers"); + expect(isReservedClaim(read.sets[0], reserved.name as string)).toBe(true); + const sources = withCloudSemantic([], read.sets).sources; + expect(reviewerNamesFor(sources).has(reserved.name as string)).toBe(false); + const { semanticPoliciesFromPacks } = await import("@/src/hooks/semantic/pack-policies"); + expect(semanticPoliciesFromPacks(sources).policies.map((p) => p.name)).not.toContain(reserved.name); + }); +}); + +// ── packs ∪ Cloud ──────────────────────────────────────────────────────────── + +describe("resolveSemanticPolicies: installed packs ∪ FailproofAI Cloud", () => { + it("asks both, and Cloud wins a name clash — the pack's same-named check is dropped, with one warning", async () => { + installPacks([{ id: "acme/checks", semantic: [decl("acme-shared", { title: "The pack's version" }), decl("acme-pack-only")] }]); + deploy({ semantic: [{ id: "org-checks", version: 4, declarations: [decl("acme-shared", { title: "The org's version" }), decl("acme-cloud-only")] }] }); + const { resolveSemanticPolicies } = await import("@/src/hooks/semantic/pack-policies"); + const policies = resolveSemanticPolicies("claude"); + const byName = Object.fromEntries(policies.map((p) => [p.name, p])); + expect(Object.keys(byName).sort()).toEqual(["acme-cloud-only", "acme-pack-only", "acme-shared"]); + expect(byName["acme-shared"].title).toBe("The org's version"); + expect(byName["acme-shared"].origin).toEqual({ packId: "cloud:org-checks", packVersion: "4" }); + expect(byName["acme-pack-only"].origin?.packId).toBe("acme/checks"); + const warned = stderr.join("").match(/declares Jev check acme-shared, which FailproofAI Cloud policy cloud:org-checks@4 also declares/g); + expect(warned).toHaveLength(1); + resolveSemanticPolicies("claude"); + expect(stderr.join("").match(/declares Jev check acme-shared/g)).toHaveLength(1); + }); + + it("a Cloud claim to a reserved name shadows nothing: FailproofAI's own pack keeps it", async () => { + const reserved = INVALID.find((c) => c.why === "reserved FailproofAI name")!.decl; + installPacks([{ id: "FailproofAI/jev-policies", source: "github:FailproofAI/jev-policies@v0.2.0", semantic: [reserved] }]); + deploy({ semantic: [{ id: "org-copy", version: 1, declarations: [{ ...reserved, title: "An org's copy" }] }] }); + const { resolveSemanticPolicies } = await import("@/src/hooks/semantic/pack-policies"); + const [only] = resolveSemanticPolicies("claude"); + expect(only.name).toBe(reserved.name); + expect(only.origin?.packId).toBe("FailproofAI/jev-policies"); + expect(only.title).toBe(reserved.title); + }); + + it("spends ONE question budget across packs and Cloud", async () => { + const renamed = (prefix: string) => VALID.map((d) => ({ ...d, name: `${prefix}-${d.name as string}` })); + deploy({ + semantic: [ + { id: "org-a", version: 1, declarations: renamed("a") }, + { id: "org-b", version: 1, declarations: renamed("b") }, + { id: "org-c", version: 1, declarations: renamed("c") }, + ], + }); + const { readCloudJevPolicies } = await import("@/src/hooks/cloud-managed-policies"); + const { withCloudSemantic } = await import("@/src/hooks/effective-reviewers"); + const { semanticPoliciesFromPacks, MAX_PACK_QUESTION_CHARS, questionChars } = await import("@/src/hooks/semantic/pack-policies"); + const resolved = semanticPoliciesFromPacks(withCloudSemantic([], readCloudJevPolicies().sets).sources); + const spent = readCloudJevPolicies() + .sets.flatMap((s) => s.semantic) + .filter((e) => resolved.policies.some((p) => p.name === e.name)) + .reduce((n, e) => n + questionChars(e), 0); + expect(spent).toBeLessThanOrEqual(MAX_PACK_QUESTION_CHARS); + expect(resolved.policies.length).toBeLessThan(48); + expect(resolved.errors.join("\n")).toMatch(/pack cloud:org-[abc] semantic policy .* was dropped: its questions need/); + }); +}); + +// ── Reviewers: a `both` policy's reviewedBy ────────────────────────────────── + +describe("a both policy's reviewedBy names its own Cloud check", () => { + it("is honoured: the JS half registers reviewable, and nothing is reported", async () => { + deploy({ + policies: [{ id: "no-prod-db", version: 3, hooks: ["block-prod-db"], authority: "reviewable", reviewedBy: ["acme-prod-db"] }], + semantic: [{ id: "no-prod-db", version: 3, declarations: [decl("acme-prod-db")] }], + }); + const registered = await registeredAfterOneEvent(); + const policy = registered.get("cloud/no-prod-db@3/block-prod-db"); + expect(policy?.authority).toBe("reviewable"); + expect(policy?.reviewedBy).toEqual(["acme-prod-db"]); + const { effectiveReviewerNames, jevChecksInstalled } = await import("@/src/hooks/effective-reviewers"); + expect(effectiveReviewerNames().has("acme-prod-db")).toBe(true); + expect(jevChecksInstalled()).toBe(true); + expect(existsSync(errorsFile())).toBe(false); + }); + + it("stays hard when the check is not deployed, and errors.json says why", async () => { + deploy({ + policies: [{ id: "no-prod-db", version: 3, hooks: ["block-prod-db"], authority: "reviewable", reviewedBy: ["acme-prod-db"] }], + }); + const registered = await registeredAfterOneEvent(); + expect(registered.get("cloud/no-prod-db@3/block-prod-db")?.authority).toBe("hard"); + expect(readErrors().errors).toEqual([ + { + id: "no-prod-db", + version: 3, + kind: "regex", + message: "reviewedBy names acme-prod-db, which is not a Jev check this machine can ask, so the policy stays hard", + }, + ]); + }); +}); + +// ── errors.json ────────────────────────────────────────────────────────────── + +describe("errors.json", () => { + const entry = { id: "p", version: 1, kind: "jev" as const, message: "m" }; + + it("is written only on a managed machine, only on change, atomically and owner-only", async () => { + const { writeCloudPolicyErrors, _resetCloudPolicyErrorsCacheForTest } = await import("@/src/hooks/cloud-policy-errors"); + // No deployment: never a file. + expect(writeCloudPolicyErrors([entry])).toBe("skipped"); + expect(existsSync(errorsFile())).toBe(false); + + deploy({}); + // Nothing to say and nothing to clear: still no file, so the poll stays silent. + expect(writeCloudPolicyErrors([])).toBe("skipped"); + expect(existsSync(errorsFile())).toBe(false); + + expect(writeCloudPolicyErrors([entry])).toBe("written"); + expect(readErrors()).toEqual({ errors: [entry] }); + if (posix) expect(statSync(errorsFile()).mode & 0o777).toBe(0o600); + const before = statSync(errorsFile()).mtimeMs; + expect(writeCloudPolicyErrors([entry])).toBe("unchanged"); + _resetCloudPolicyErrorsCacheForTest(); + expect(writeCloudPolicyErrors([{ ...entry }])).toBe("unchanged"); + expect(statSync(errorsFile()).mtimeMs).toBe(before); + + // Fixed: `[]` replaces the list, which is what clears the server's copy. + expect(writeCloudPolicyErrors([])).toBe("written"); + expect(readErrors()).toEqual({ errors: [] }); + expect(readdirSync(cloudRoot).filter((n) => n.includes(".tmp-"))).toEqual([]); + }); + + it("collects every kind of problem, deduplicated, with the right kind", async () => { + const { collectCloudPolicyErrors, JEV_UNCONFIGURED } = await import("@/src/hooks/cloud-policy-errors"); + const errors = collectCloudPolicyErrors({ + manifestError: null, + jsPolicies: [ + { id: "both-one", version: 2, effect: "enforce", sha256: "a", path: "p", deployment: 1, authority: "reviewable", reviewedBy: ["acme-x", "acme-y"] }, + { id: "regex-one", version: 1, effect: "enforce", sha256: "b", path: "q", deployment: 1 }, + ], + jsFailures: new Map([["regex-one", { type: "syntax_error" as const, reason: "Unexpected token" }]]), + jev: { + sets: [], + semanticIds: ["both-one"], + jevMode: "enforce", + errors: [{ id: "both-one", version: 2, kind: "both", message: "not loaded: bad" }, { id: "both-one", version: 2, kind: "both", message: "not loaded: bad" }], + }, + reviewerNames: new Set(["acme-x"]), + jevUnconfigured: JEV_UNCONFIGURED, + }); + expect(errors).toEqual([ + { id: "both-one", version: 2, kind: "both", message: "reviewedBy names acme-y, which is not a Jev check this machine can ask, so the policy stays hard" }, + { id: "regex-one", version: 1, kind: "regex", message: "policy did not load (syntax_error): Unexpected token" }, + { id: "both-one", version: 2, kind: "both", message: "not loaded: bad" }, + { id: "jevMode", version: null, kind: "daemon", message: "jev_unconfigured" }, + ]); + }); + + it("the hook path reports a broken Cloud Jev artifact, and clears the report once it is fixed", async () => { + deploy({ semantic: [{ id: "org-checks", version: 7, declarations: [decl("acme-a")], wrongSha: true }] }); + await registeredAfterOneEvent(); + expect(readErrors().errors).toEqual([ + expect.objectContaining({ id: "org-checks", version: 7, kind: "jev", message: expect.stringMatching(/integrity verification/) }), + ]); + + deploy({ semantic: [{ id: "org-checks", version: 8, declarations: [decl("acme-a")] }] }); + vi.resetModules(); + await registeredAfterOneEvent(); + expect(readErrors()).toEqual({ errors: [] }); + }); + + it("the hook path reports a Cloud JS policy that fails to load, as kind both when it has Jev checks", async () => { + mkdirSync(join(cloudRoot, "artifacts"), { recursive: true }); + const broken = "export default {{{ not javascript"; + const digest = sha(broken); + writeFileSync(join(cloudRoot, "artifacts", `${digest}.mjs`), broken); + const decls = JSON.stringify([decl("acme-z")]); + writeFileSync(join(cloudRoot, "artifacts", `${sha(decls)}.json`), decls); + writeFileSync( + join(cloudRoot, "active.json"), + JSON.stringify({ + schemaVersion: 2, + deployment: 9, + policies: [{ id: "broken", version: 1, sha256: digest, path: `artifacts/${digest}.mjs`, effect: "enforce" }], + semanticPolicies: [{ id: "broken", version: 1, sha256: sha(decls), path: `artifacts/${sha(decls)}.json` }], + }), + ); + await registeredAfterOneEvent(); + const [problem] = readErrors().errors; + expect(problem).toMatchObject({ id: "broken", version: 1, kind: "both" }); + expect(problem.message).toMatch(/policy did not load/); + }); +}); + +// ── Jev mode set by FailproofAI Cloud ──────────────────────────────────────── + +describe("Cloud's jevMode overrides the local mode", () => { + const writeJev = (obj: unknown, mode = 0o600) => { + const file = join(home, "jev.json"); + writeFileSync(file, JSON.stringify(obj), { mode }); + chmodSync(file, mode); + }; + const connect = async () => { + const { readCredentials, writeCredentials, writeJevCloudCredential } = await import("@/src/hooks/fp-config"); + writeCredentials({ ...readCredentials(), ingest: { url: `${ORIGIN}/v1/events`, key: KEY } }); + writeJevCloudCredential({ url: ORIGIN, key: KEY }); + }; + const load = async (mode: "off" | "observe" | "enforce") => + (await import("@/src/hooks/semantic/jev-config")).loadJevConfigForCloudMode(mode); + + it("Cloud off: no Jev, even over a working local config, and nothing to report", async () => { + writeJev({ provider: "typesafe", apiKey: KEY, mode: "enforce" }); + expect(await load("off")).toEqual({ config: null, provider: null, problem: null }); + }); + + it("Cloud observe over a local enforce: jev.json's provider, Cloud's mode", async () => { + writeJev({ provider: "typesafe", apiKey: KEY, mode: "enforce" }); + const r = await load("observe"); + expect(r.provider).toBe("jev.json"); + expect(r.config).toMatchObject({ provider: "typesafe", mode: "observe", apiKey: KEY }); + }); + + it("Cloud enforce over a local OFF: the local off is overridden, its routing and key reused", async () => { + writeJev({ provider: "typesafe", apiKey: KEY, mode: "off" }); + const { loadJevConfig } = await import("@/src/hooks/semantic/jev-config"); + expect(loadJevConfig()).toBeNull(); + const r = await load("enforce"); + expect(r.config).toMatchObject({ provider: "typesafe", mode: "enforce", apiKey: KEY }); + }); + + it("Cloud enforce over a local Cloud-provider off, with the Cloud credential", async () => { + await connect(); + writeJev({ provider: "failproofai", baseUrl: `${ORIGIN}/enforcement/v1/jev`, mode: "off" }); + const r = await load("enforce"); + expect(r.config).toMatchObject({ provider: "failproofai", mode: "enforce", apiKey: KEY, credentialOrigin: ORIGIN }); + }); + + it("no jev.json: the Cloud Jev credential answers as provider failproofai", async () => { + await connect(); + const r = await load("enforce"); + expect(r.provider).toBe("cloud-credential"); + expect(r.problem).toBeNull(); + expect(r.config).toMatchObject({ + provider: "failproofai", + baseUrl: `${ORIGIN}/enforcement/v1/jev`, + mode: "enforce", + apiKey: KEY, + credentialOrigin: ORIGIN, + }); + const { validateLoadedJevConfig } = await import("@/src/hooks/semantic/jev-config"); + expect(validateLoadedJevConfig(r.config!).ok).toBe(true); + }); + + it("neither: no Jev, and jev_unconfigured", async () => { + expect(await load("observe")).toEqual({ config: null, provider: null, problem: "jev_unconfigured" }); + }); + + it.skipIf(!posix)("a jev.json that is unusable is not swapped for the Cloud credential", async () => { + await connect(); + writeJev({ provider: "typesafe", apiKey: KEY, mode: "enforce" }, 0o644); + const r = await load("enforce"); + expect(r.config).toBeNull(); + expect(r.problem).toMatch(/^jev_unconfigured: jev\.json is refused/); + }); + + it("keeps the plain-http rule: enforce is refused on a loopback http endpoint, observe runs", async () => { + writeJev({ provider: "custom", apiKey: KEY, baseUrl: "http://127.0.0.1:4000/v1", mode: "observe" }); + expect((await load("observe")).config?.mode).toBe("observe"); + const r = await load("enforce"); + expect(r.config).toBeNull(); + expect(r.problem).toMatch(/^jev_unconfigured: .*plain http/); + }); + + it("the hook path reports jev_unconfigured when Cloud sets a mode nothing can answer", async () => { + deploy({ semantic: [{ id: "org-checks", version: 1, declarations: [decl("acme-a")] }], jevMode: "observe" }); + await registeredAfterOneEvent(); + expect(readErrors().errors).toEqual([{ id: "jevMode", version: null, kind: "daemon", message: "jev_unconfigured" }]); + }); + + it("no Cloud mode: the local file decides, exactly as before", async () => { + deploy({ semantic: [{ id: "org-checks", version: 1, declarations: [decl("acme-a")] }] }); + const { readCloudJevMode } = await import("@/src/hooks/cloud-managed-policies"); + expect(readCloudJevMode()).toBeNull(); + await registeredAfterOneEvent(); + expect(existsSync(errorsFile())).toBe(false); + }); +}); + +// ── Surfaces ───────────────────────────────────────────────────────────────── + +describe("surfaces", () => { + const RENDER = { render: { cols: 120, color: false }, readModelList: async () => ({ ok: false as const, reason: "no list read in tests" }) }; + + it("jev status: Cloud checks, and the mode set by FailproofAI Cloud", async () => { + const { readCredentials, writeCredentials, writeJevCloudCredential } = await import("@/src/hooks/fp-config"); + writeCredentials({ ...readCredentials(), ingest: { url: `${ORIGIN}/v1/events`, key: KEY } }); + writeJevCloudCredential({ url: ORIGIN, key: KEY }); + deploy({ + policies: [{ id: "no-prod-db", version: 3, authority: "reviewable", reviewedBy: ["acme-prod-db"] }], + semantic: [{ id: "no-prod-db", version: 3, declarations: [decl("acme-prod-db")] }], + jevMode: "enforce", + }); + const { runJevCommand } = await import("@/src/hooks/jev-cli"); + const human = await runJevCommand(["status"], RENDER); + const text = human.lines.join("\n"); + expect(text).toMatch(/on · enforce — mode set by FailproofAI Cloud/); + expect(text).toMatch(/FailproofAI Cloud Jev checks/); + expect(text).toMatch(/no-prod-db v3\s+acme-prod-db · reviews its own regex policy/); + expect(text).not.toMatch(/AgentEye/); + + const json = JSON.parse((await runJevCommand(["status", "--json"], RENDER)).json as string); + expect(json).toMatchObject({ + status: "ok", + mode: "enforce", + modeSetBy: "FailproofAI Cloud", + provider: "failproofai", + providerSource: "cloud-credential", + cloud: { jevMode: "enforce", checks: [{ policy: "no-prod-db", version: 3, kind: "both", checks: ["acme-prod-db"] }] }, + }); + }); + + it("jev status with Cloud checks and no Cloud mode: the local answer, plus the checks", async () => { + deploy({ semantic: [{ id: "org-checks", version: 2, declarations: [decl("acme-a")] }] }); + const { runJevCommand } = await import("@/src/hooks/jev-cli"); + const json = JSON.parse((await runJevCommand(["status", "--json"], RENDER)).json as string); + expect(json.status).toBe("absent"); + expect(json.cloud).toMatchObject({ jevMode: null, checks: [{ policy: "org-checks", version: 2, kind: "jev", checks: ["acme-a"] }] }); + }); + + it("policies lists Cloud jev and both policies with a FailproofAI Cloud label", async () => { + deploy({ + policies: [{ id: "no-prod-db", version: 3, authority: "reviewable", reviewedBy: ["acme-prod-db"] }, { id: "plain", version: 1 }], + semantic: [ + { id: "no-prod-db", version: 3, declarations: [decl("acme-prod-db")] }, + { id: "secrets", version: 1, declarations: [decl("acme-secrets")] }, + ], + jevMode: "observe", + }); + const out: string[] = []; + vi.spyOn(console, "log").mockImplementation((...a: unknown[]) => void out.push(a.map(String).join(" "))); + vi.spyOn(process.stdout, "write").mockImplementation((chunk: unknown) => { + out.push(...String(chunk).split("\n")); + return true; + }); + const { listHooks } = await import("@/src/hooks/manager"); + await listHooks(project); + const text = out.join("\n").replace(/\x1B\[[0-9;]*m/g, ""); + expect(text).toMatch(/Cloud-managed — deployment 43 · FailproofAI Cloud/); + expect(text).toMatch(/no-prod-db\s+v3\s+both\s+acme-prod-db/); + expect(text).toMatch(/plain\s+v1\s+regex/); + expect(text).toMatch(/secrets\s+v1\s+jev\s+acme-secrets/); + expect(text).toMatch(/Jev mode: observe — mode set by FailproofAI Cloud/); + }); + + it("config --disconnect's clear takes the Jev policies, the Jev mode and both error reports with active.json", async () => { + deploy({ semantic: [{ id: "org-checks", version: 1, declarations: [decl("acme-a")] }], jevMode: "enforce" }); + writeFileSync(errorsFile(), JSON.stringify({ errors: [] })); + writeFileSync(join(cloudRoot, "daemon-errors.json"), JSON.stringify({ errors: [] })); + const { clearActiveCloudManagedPolicies, readCloudJevPolicies, readCloudJevMode } = await import("@/src/hooks/cloud-managed-policies"); + expect(clearActiveCloudManagedPolicies()).toBe(true); + for (const name of ["active.json", "errors.json", "daemon-errors.json"]) expect(existsSync(join(cloudRoot, name))).toBe(false); + expect(readCloudJevPolicies().sets).toEqual([]); + expect(readCloudJevMode()).toBeNull(); + }); +}); diff --git a/src/hooks/cloud-enrollment-cli.ts b/src/hooks/cloud-enrollment-cli.ts index 19952b150..eeb9b2655 100644 --- a/src/hooks/cloud-enrollment-cli.ts +++ b/src/hooks/cloud-enrollment-cli.ts @@ -393,7 +393,8 @@ export function runDisconnectCommand(): CommandResult { "Disconnected from FailproofAI Cloud.", "", stoppedManaged - ? " Cloud-managed policies stop being enforced and stop being refreshed.\n" + + ? " Cloud-managed policies stop being enforced and stop being refreshed — their Jev checks\n" + + " and any Jev mode FailproofAI Cloud set go with them.\n" + " Local builtin, custom and convention policies are unaffected." : " Local builtin, custom and convention policies are unaffected.", ...jevLines, diff --git a/src/hooks/cloud-managed-policies.ts b/src/hooks/cloud-managed-policies.ts index 6ab53beb4..23b3b518a 100644 --- a/src/hooks/cloud-managed-policies.ts +++ b/src/hooks/cloud-managed-policies.ts @@ -8,8 +8,15 @@ import { createHash } from "node:crypto"; import { existsSync, readFileSync, realpathSync, rmSync } from "node:fs"; import { isAbsolute, relative, resolve } from "node:path"; import { cloudPoliciesDir } from "./fp-home"; -import { authorityFieldsOf } from "./policy-authority"; +import { hookLogWarn } from "./hook-logger"; +import { authorityFieldsOf, SEMANTIC_REVIEWER_NAMES } from "./policy-authority"; import type { PolicyAuthority } from "./policy-types"; +// A cycle (pack-manifest imports `resolveManagedPath` from here), and a safe +// one: neither module touches the other's exports while it is being evaluated, +// only inside functions. It is the price of the contract's one rule for this +// file — a Cloud Jev declaration is parsed by THE SAME function a pack manifest's +// semantic entries are, never by a second parser that could drift from it. +import { parsePackSemantic, type SemanticManifestEntry } from "./pack-manifest"; /** * Active-manifest schema versions this reader accepts. @@ -163,7 +170,19 @@ export function resolveManagedPath(root: string, candidate: string): string { * Returns true when a manifest was actually removed. */ export function clearActiveCloudManagedPolicies(): boolean { - const activePath = resolve(cloudManagedPolicyRoot(), "active.json"); + const root = cloudManagedPolicyRoot(); + // The error reports go with it: they describe a deployment this machine no + // longer has, and a reconnect (possibly to another organisation) must not + // open by reporting the old one's problems. Cloud Jev policies and the Cloud + // Jev mode live IN active.json, so removing it clears them too. + for (const name of [CLOUD_POLICY_ERRORS_FILE, DAEMON_POLICY_ERRORS_FILE]) { + try { + rmSync(resolve(root, name), { force: true }); + } catch { + // Best-effort, like the manifest below. + } + } + const activePath = resolve(root, "active.json"); if (!existsSync(activePath)) return false; try { rmSync(activePath, { force: true }); @@ -175,6 +194,226 @@ export function clearActiveCloudManagedPolicies(): boolean { } } +/** The CLI's error report, beside `active.json`. See `cloud-policy-errors.ts`. */ +export const CLOUD_POLICY_ERRORS_FILE = "errors.json"; +/** The daemon's own reconcile error state (`PolicyStore::daemon_errors_path`). */ +export const DAEMON_POLICY_ERRORS_FILE = "daemon-errors.json"; + +// ── FailproofAI Cloud Jev policies ─────────────────────────────────────────── +// +// A Cloud policy of kind `jev` or `both` carries Jev checks: a JSON array of +// pack-manifest semantic declarations, which the daemon places at +// `artifacts/.json` and names in `active.json` `semanticPolicies`. They +// arrive in the same atomic write as the JS policies, so a `both` policy's +// regex half and the checks that review it are live together or not at all. +// +// Read FAIL-OPEN, per entry, like a pack's semantic entries and unlike the JS +// reader above: a Jev check is what CLEARS a reviewable regex verdict, so losing +// one leaves the regex deny standing — noisier, never weaker — and one bad +// artifact must not take the rest of the deployment's checks with it. Every +// drop is logged once and returned as an error for `errors.json`. + +/** The Jev modes Cloud may set. `local` never reaches a machine (stored NULL). */ +export type CloudJevMode = "off" | "observe" | "enforce"; + +/** What `errors.json` reports, one entry per problem (CONTRACT C4/C6). */ +export type CloudPolicyErrorKind = "regex" | "jev" | "both" | "daemon"; +export interface CloudPolicyError { + id: string; + version: number | null; + kind: CloudPolicyErrorKind; + message: string; +} + +/** + * One deployed Cloud Jev policy, shaped like a pack so everything that consumes + * packs' semantic sets — the reviewer set, the question set, the contest, the + * budget — takes it unchanged. + */ +export interface CloudSemanticPolicySet { + /** `cloud:`: pack-shaped, and never a valid pack id, so the two cannot collide. */ + id: string; + /** Pack-shaped: the Cloud version as a string. */ + version: string; + /** + * `cloud:@`. NOT first-party (`isFirstPartyPack` wants + * `github:FailproofAI/`), so FailproofAI's sixteen reserved check names are + * void here exactly as for any other pack. + */ + source: string; + /** Always `enforce`: an observe rollout of Jev is the Jev MODE, not an effect. */ + effect: "enforce"; + /** Every agent. */ + clis: null; + semantic: SemanticManifestEntry[]; + /** The Cloud policy id and version, for attribution and error reports. */ + policyId: string; + policyVersion: number; + /** `both` when the same id also deployed a JS half, else `jev`. */ + kind: "jev" | "both"; + sha256: string; + deployment: number; +} + +export interface CloudJevRead { + sets: CloudSemanticPolicySet[]; + /** Every policy id `semanticPolicies` names, loaded or not — what makes a JS policy `both`. */ + semanticIds: string[]; + /** Cloud's Jev mode override, or null when Cloud does not set one. */ + jevMode: CloudJevMode | null; + errors: CloudPolicyError[]; +} + +const EMPTY_CLOUD_JEV: CloudJevRead = { sets: [], semanticIds: [], jevMode: null, errors: [] }; +const warnedCloudJev = new Set(); + +function warnCloudJevOnce(message: string): void { + if (warnedCloudJev.has(message)) return; + warnedCloudJev.add(message); + hookLogWarn(message); +} + +/** Forget which warnings were already said. Tests only. */ +export function _resetCloudJevWarningsForTest(): void { + warnedCloudJev.clear(); +} + +function parseCloudJevMode(raw: unknown): CloudJevMode | null { + return raw === "off" || raw === "observe" || raw === "enforce" ? raw : null; +} + +/** `active.json` as untyped JSON, schema-checked, or null when there is none. Throws on a bad one. */ +function readActiveRaw(): Record | null { + const activePath = resolve(cloudManagedPolicyRoot(), "active.json"); + if (!existsSync(activePath)) return null; + const raw: unknown = JSON.parse(readFileSync(activePath, "utf8")); + if (!raw || typeof raw !== "object" || Array.isArray(raw)) throw new Error("active manifest is not an object"); + const record = raw as Record; + if (!ACCEPTED_ACTIVE_SCHEMA_VERSIONS.includes(record.schemaVersion as number)) { + throw new Error(`unsupported active manifest schema ${String(record.schemaVersion)}`); + } + return record; +} + +/** + * Cloud's Jev mode override from `active.json`, or null — no deployment, no + * override, or a file that cannot be read (which is then simply not an + * override: the local `jev.json` decides, as it does on a machine Cloud never + * touched). Cheap and never throws: `readJevConfig` asks on every gate event. + */ +export function readCloudJevMode(): CloudJevMode | null { + try { + return parseCloudJevMode(readActiveRaw()?.jevMode); + } catch { + return null; + } +} + +/** + * The Cloud Jev policies `active.json` names, each artifact read, SHA-256 + * verified and parsed with the pack manifest's own semantic parser. Never + * throws; see the section note for what a failure costs. + */ +export function readCloudJevPolicies(): CloudJevRead { + let active: Record | null; + try { + active = readActiveRaw(); + } catch { + // The JS reader reports an unreadable manifest itself (and the handler files + // it); saying it twice would be one problem reported as two. + return EMPTY_CLOUD_JEV; + } + if (!active) return EMPTY_CLOUD_JEV; + + const errors: CloudPolicyError[] = []; + const rawMode = active.jevMode; + const jevMode = parseCloudJevMode(rawMode); + if (rawMode !== undefined && rawMode !== null && jevMode === null) { + // The daemon refuses such a state, so this is a hand-edited or foreign file. + errors.push({ id: "jevMode", version: null, kind: "daemon", message: `unknown Jev mode ${JSON.stringify(rawMode)} ignored` }); + } + const deployment = typeof active.deployment === "number" ? active.deployment : 0; + const jsIds = new Set( + Array.isArray(active.policies) + ? active.policies.flatMap((p) => (p && typeof p === "object" && typeof (p as { id?: unknown }).id === "string" ? [(p as { id: string }).id] : [])) + : [], + ); + const entries = active.semanticPolicies; + if (entries === undefined) return { sets: [], semanticIds: [], jevMode, errors }; + if (!Array.isArray(entries)) { + errors.push({ id: "semanticPolicies", version: null, kind: "daemon", message: "active manifest semanticPolicies is not an array" }); + return { sets: [], semanticIds: [], jevMode, errors }; + } + const semanticIds = entries.flatMap((e) => + e && typeof e === "object" && typeof (e as { id?: unknown }).id === "string" ? [(e as { id: string }).id] : [], + ); + + const root = cloudManagedPolicyRoot(); + const sets: CloudSemanticPolicySet[] = []; + const seen = new Set(); + for (const value of entries) { + const entry = (value && typeof value === "object" ? value : {}) as Record; + const id = typeof entry.id === "string" ? entry.id : String(entry.id); + const version = Number.isSafeInteger(entry.version) && (entry.version as number) >= 0 ? (entry.version as number) : null; + const kind: "jev" | "both" = jsIds.has(id) ? "both" : "jev"; + const fail = (message: string): void => { + warnCloudJevOnce(`FailproofAI Cloud Jev policy ${id}${version === null ? "" : `@${version}`}: ${message}`); + errors.push({ id, version, kind, message }); + }; + try { + if (!POLICY_ID_RE.test(id) || id === "." || id === "..") throw new Error(`unsafe policy id ${JSON.stringify(entry.id)}`); + if (version === null) throw new Error("invalid version"); + if (typeof entry.sha256 !== "string" || !SHA256_RE.test(entry.sha256)) throw new Error("invalid SHA-256"); + if (seen.has(id)) throw new Error("duplicate semantic policy id"); + seen.add(id); + const path = resolveManagedPath(root, typeof entry.path === "string" ? entry.path : ""); + const bytes = readFileSync(path); + const actual = createHash("sha256").update(bytes).digest("hex"); + if (actual !== entry.sha256) { + throw new Error(`failed integrity verification: expected ${entry.sha256}, got ${actual}`); + } + let declarations: unknown; + try { + declarations = JSON.parse(bytes.toString("utf8")); + } catch (err) { + throw new Error(`artifact is not valid JSON: ${err instanceof Error ? err.message : String(err)}`); + } + if (!Array.isArray(declarations)) throw new Error("artifact is not a JSON array of Jev declarations"); + + const source = `cloud:${id}@${version}`; + const dropped: string[] = []; + // THE manifest parser: the same per-entry rules, the same 24-entry cap and + // the same duplicate-name refusal a pack's `semantic` gets. + const semantic = parsePackSemantic(source, declarations, dropped); + for (const reason of dropped) fail(`declaration dropped: ${reason}`); + for (const decl of semantic) { + // Kept (the unchanged reserved-name rule voids it downstream, exactly as + // for any pack that is not FailproofAI's) and reported, so the org sees + // why a check it deployed is never asked. + if (SEMANTIC_REVIEWER_NAMES.has(decl.name)) { + fail(`${decl.name} is a name reserved for FailproofAI's own Jev checks, so it is never asked here`); + } + } + sets.push({ + id: `cloud:${id}`, + version: String(version), + source, + effect: "enforce", + clis: null, + semantic, + policyId: id, + policyVersion: version, + kind, + sha256: entry.sha256, + deployment, + }); + } catch (err) { + fail(`not loaded: ${err instanceof Error ? err.message : String(err)}`); + } + } + return { sets, semanticIds, jevMode, errors }; +} + export function readActiveCloudManagedPolicies(): CloudManagedPolicyArtifact[] { const root = cloudManagedPolicyRoot(); const activePath = resolve(root, "active.json"); diff --git a/src/hooks/cloud-policy-errors.ts b/src/hooks/cloud-policy-errors.ts new file mode 100644 index 000000000..9cedb7a58 --- /dev/null +++ b/src/hooks/cloud-policy-errors.ts @@ -0,0 +1,201 @@ +/** + * `cloud-policies/errors.json` — what this machine could not load from its + * FailproofAI Cloud deployment, for the daemon to report on its next poll + * (`policyErrors`, CONTRACT C4–C6). + * + * Nothing here changes a verdict. Cloud policies are read fail-open: a JS + * policy that fails to import, a Jev artifact that fails its digest, a + * declaration the parser drops — each costs that one policy on this machine and + * nothing else, which is the right trade on the hook path and the wrong thing + * to keep quiet about. Before this file the only trace was a line in the local + * hook log, so a fleet page read "applied" for a deployment that was not. + * + * ## One writer per file + * + * The CLI writes `errors.json`; the daemon keeps its own reconcile errors in + * `daemon-errors.json` and merges the two onto the wire. Neither writes the + * other's file, so there is no read-modify-write race between processes. + * + * ## Written on change, atomically + * + * Computed on every hook evaluation of a Cloud-managed machine, so the write + * has to be almost always a no-op: the new content is compared with what is on + * disk (and with what this process last wrote, to skip even that read in the + * warm worker) and written only when it differs — tmp file + rename, owner-only, + * so the daemon never reads half a file. + * + * A machine that never had a problem never gets the file at all: an empty list + * is written only to REPLACE a non-empty one, which is what clears the report + * on the server. That keeps the poll parameter absent on a healthy fleet. + */ +import { randomUUID } from "node:crypto"; +import { chmodSync, existsSync, readFileSync, renameSync, rmSync, writeFileSync } from "node:fs"; +import { resolve } from "node:path"; +import { + CLOUD_POLICY_ERRORS_FILE, + cloudManagedPolicyRoot, + type CloudJevRead, + type CloudManagedPolicyArtifact, + type CloudPolicyError, + type CloudPolicyErrorKind, +} from "./cloud-managed-policies"; +import type { PolicyLoadFailure } from "./custom-hooks-loader"; + +export type { CloudPolicyError, CloudPolicyErrorKind } from "./cloud-managed-policies"; + +/** The message the contract fixes for "Cloud set a Jev mode and nothing can answer". */ +export const JEV_UNCONFIGURED = "jev_unconfigured"; + +export function cloudPolicyErrorsPath(): string { + return resolve(cloudManagedPolicyRoot(), CLOUD_POLICY_ERRORS_FILE); +} + +const KINDS: ReadonlySet = new Set(["regex", "jev", "both", "daemon"]); + +/** The file as the CLI last wrote it, or null when absent or unreadable. */ +export function readCloudPolicyErrors(): CloudPolicyError[] | null { + try { + const raw = JSON.parse(readFileSync(cloudPolicyErrorsPath(), "utf8")) as { errors?: unknown }; + if (!Array.isArray(raw.errors)) return null; + return raw.errors.filter( + (e): e is CloudPolicyError => + !!e && + typeof e === "object" && + typeof (e as CloudPolicyError).id === "string" && + typeof (e as CloudPolicyError).message === "string" && + KINDS.has((e as CloudPolicyError).kind) && + ((e as CloudPolicyError).version === null || Number.isSafeInteger((e as CloudPolicyError).version)), + ); + } catch { + return null; + } +} + +/** Text this process last wrote, so the warm worker's steady state reads nothing. */ +let lastWritten: { path: string; text: string } | null = null; + +/** Tests only. */ +export function _resetCloudPolicyErrorsCacheForTest(): void { + lastWritten = null; +} + +function serialize(errors: ReadonlyArray): string { + // Field order fixed, so equal lists are equal text. + const list = errors.map((e) => ({ id: e.id, version: e.version, kind: e.kind, message: e.message })); + return `${JSON.stringify({ errors: list }, null, 2)}\n`; +} + +/** + * Replace `errors.json` with `errors` when that changes it. Never throws. + * + * - `"skipped"`: nothing to report and no file to clear, or no deployment here + * (no `active.json` — a disconnected or never-managed machine gets no file). + * - `"unchanged"`: the file already says exactly this. + * - `"written"`: it did not, and now does. + * - `"failed"`: the write itself failed (logged nowhere: the next event retries). + */ +export function writeCloudPolicyErrors(errors: ReadonlyArray): "written" | "unchanged" | "skipped" | "failed" { + try { + const root = cloudManagedPolicyRoot(); + if (!existsSync(resolve(root, "active.json"))) return "skipped"; + const path = cloudPolicyErrorsPath(); + const text = serialize(errors); + if (lastWritten && lastWritten.path === path && lastWritten.text === text && existsSync(path)) return "unchanged"; + let current: string | null = null; + try { + current = readFileSync(path, "utf8"); + } catch { + current = null; + } + if (current === text) { + lastWritten = { path, text }; + return "unchanged"; + } + if (current === null && errors.length === 0) return "skipped"; + const tmp = `${path}.tmp-${process.pid}-${randomUUID()}`; + try { + writeFileSync(tmp, text, { mode: 0o600, flag: "wx" }); + chmodSync(tmp, 0o600); + renameSync(tmp, path); + } catch (err) { + try { + rmSync(tmp, { force: true }); + } catch { + // Nothing useful to do. + } + throw err; + } + lastWritten = { path, text }; + return "written"; + } catch { + // A report that cannot be written must not cost a hook its answer. + return "failed"; + } +} + +export interface CloudPolicyErrorInputs { + /** `readActiveCloudManagedPolicies` threw: the whole JS half is not enforcing. */ + manifestError?: string | null; + /** What the JS reader returned. */ + jsPolicies: ReadonlyArray; + /** Import failures by Cloud policy id (`LoadAllResult.cloudFailures`). */ + jsFailures?: ReadonlyMap; + /** The Cloud Jev read (its own drops are already in `errors`). */ + jev: CloudJevRead; + /** + * The Jev check names this machine can ask, for every agent — not the + * per-agent registration set, or the report would flip with each event's + * agent and rewrite the file on every call. + */ + reviewerNames: ReadonlySet; + /** Set when Cloud chose a Jev mode and no provider can answer. */ + jevUnconfigured?: string | null; +} + +/** + * Everything wrong with this machine's Cloud deployment, as `errors.json` + * entries: deduplicated, in a stable order. Pure. + */ +export function collectCloudPolicyErrors(input: CloudPolicyErrorInputs): CloudPolicyError[] { + const out: CloudPolicyError[] = []; + const semanticIds = new Set(input.jev.semanticIds); + const kindOf = (id: string): CloudPolicyErrorKind => (semanticIds.has(id) ? "both" : "regex"); + + if (input.manifestError) { + out.push({ id: "active.json", version: null, kind: "daemon", message: `Cloud policies could not be loaded: ${input.manifestError}` }); + } + for (const policy of input.jsPolicies) { + const failure = input.jsFailures?.get(policy.id); + if (failure) { + out.push({ id: policy.id, version: policy.version, kind: kindOf(policy.id), message: `policy did not load (${failure.type}): ${failure.reason}` }); + } + // A `reviewedBy` naming a check this machine cannot ask: the policy stays + // hard, and a `both` policy's regex half then denies what its own Jev + // check was deployed to clear. + if (policy.authority === "reviewable" && policy.reviewedBy) { + const missing = policy.reviewedBy.filter((name) => !input.reviewerNames.has(name)); + if (missing.length > 0) { + out.push({ + id: policy.id, + version: policy.version, + kind: kindOf(policy.id), + message: + `reviewedBy names ${missing.join(", ")}, which ${missing.length === 1 ? "is not a Jev check" : "are not Jev checks"} ` + + `this machine can ask, so the policy stays hard`, + }); + } + } + } + out.push(...input.jev.errors); + if (input.jevUnconfigured) { + out.push({ id: "jevMode", version: null, kind: "daemon", message: input.jevUnconfigured }); + } + + const seen = new Set(); + return out.filter((e) => { + const key = `${e.id}\u0000${e.version}\u0000${e.kind}\u0000${e.message}`; + if (seen.has(key)) return false; + seen.add(key); + return true; + }); +} diff --git a/src/hooks/custom-hooks-loader.ts b/src/hooks/custom-hooks-loader.ts index 965f78f2e..1e1e80e8d 100644 --- a/src/hooks/custom-hooks-loader.ts +++ b/src/hooks/custom-hooks-loader.ts @@ -334,6 +334,13 @@ export interface LoadAllResult { * Only present for a path more than one pack resolved to. */ packAliases: Map; + /** + * Import failures of FailproofAI Cloud JS policies, keyed by Cloud policy id + * (every id behind a shared artifact). Fail-open like the rest of the Cloud + * read — nothing denies over them — but reported to Cloud in `errors.json`, + * so a deployment that is not actually enforcing says so. + */ + cloudFailures?: Map; } export function customPolicyId(file: string, name: string): string { @@ -456,8 +463,12 @@ export async function loadAllCustomHooks( // codebase exists to remove. It is also announced, so an operator can give the // two policies distinguishable source instead of living with the merge. const cloudManagedByPath = new Map(); + /** Every Cloud policy id behind each artifact path, for `cloudFailures`. */ + const cloudIdsByPath = new Map(); + const cloudFailures = new Map(); for (const policy of opts?.cloudManagedPolicies ?? []) { const key = resolve(policy.path); + cloudIdsByPath.set(key, [...(cloudIdsByPath.get(key) ?? []), policy.id]); const existing = cloudManagedByPath.get(key); if (!existing) { cloudManagedByPath.set(key, policy); @@ -660,6 +671,9 @@ export async function loadAllCustomHooks( if (failure && pack) { for (const id of packIdsByPath.get(absPath) ?? [pack.id]) packFailures.set(id, failure); } + if (failure && cloudManaged) { + for (const id of cloudIdsByPath.get(absPath) ?? [cloudManaged.id]) cloudFailures.set(id, failure); + } for (const hook of getCustomHooks().slice(hooksBefore)) { const tagged = hook as CustomHook & { __policyId?: string; @@ -680,6 +694,10 @@ export async function loadAllCustomHooks( } } else { hookLogWarn(`custom policy path not found: ${absPath}`); + if (cloudManaged) { + const missing = { type: "path_missing", reason: `path missing: ${absPath}` } as const; + for (const id of cloudIdsByPath.get(absPath) ?? [cloudManaged.id]) cloudFailures.set(id, missing); + } if (pack) { const missing = { type: "path_missing", reason: `path missing: ${absPath}` } as const; for (const id of packIdsByPath.get(absPath) ?? [pack.id]) packFailures.set(id, missing); @@ -830,5 +848,5 @@ export async function loadAllCustomHooks( const winner = packByPath.get(path); if (winner && ids.length > 1) packAliases.set(winner.id, ids); } - return { hooks: allHooks, conventionSources, packFailures, packAliases }; + return { hooks: allHooks, conventionSources, packFailures, packAliases, cloudFailures }; } diff --git a/src/hooks/effective-reviewers.ts b/src/hooks/effective-reviewers.ts index 0ccfac1d6..ecb5ce493 100644 --- a/src/hooks/effective-reviewers.ts +++ b/src/hooks/effective-reviewers.ts @@ -1,12 +1,14 @@ /** * The semantic checks a `reviewedBy` may name ON THIS MACHINE. * - * ## Only what an installed pack declares + * ## Only what an installed pack — or FailproofAI Cloud — declares * * The npm package ships no Jev checks, so no name is a reviewer by default. A * `reviewedBy` is honoured only for checks some installed pack declares — in * practice `FailproofAI/jev-policies`, which the core pack's fifteen - * `reviewable` policies name. With no such pack the set is EMPTY, every policy + * `reviewable` policies name — or a FailproofAI Cloud Jev policy deployed to + * this machine declares (a `both` policy's JS half names its own checks; see + * `withCloudSemantic`). With no such pack the set is EMPTY, every policy * registers `hard`, and nothing Jev says can clear a regex verdict. That is the * fail-safe direction, and it is also what keeps Jev inert on a machine that * never installed its checks: `jevChecksInstalled` below is the hook path's @@ -36,9 +38,12 @@ */ import { readInstalledPacks, type ResolvedPack } from "./pack-manifest"; import { NO_REVIEWERS, SEMANTIC_REVIEWER_NAMES } from "./policy-authority"; +import { readCloudJevPolicies, type CloudJevRead, type CloudSemanticPolicySet } from "./cloud-managed-policies"; let cached: ReadonlySet | null = null; let cachedContested: ReadonlyMap = new Map(); +/** The Cloud Jev read behind `cached`, for the same registration pass. */ +let cachedCloud: CloudJevRead | null = null; /** The agent the current registration pass is for; see {@link forgetEffectiveReviewerNames}. */ let reviewerCli: string | undefined; @@ -174,9 +179,9 @@ export function effectiveReviewerNames(): ReadonlySet { let names: ReadonlySet = NO_REVIEWERS; cachedContested = new Map(); try { - const packs = jevPacks(readInstalledPacks().packs, reviewerCli); - names = reviewerNamesFor(packs); - cachedContested = contestedSemanticNames(packs); + const sources = withCloudSemantic(jevPacks(readInstalledPacks().packs, reviewerCli), cloudJevForPass().sets).sources; + names = reviewerNamesFor(sources); + cachedContested = contestedSemanticNames(sources); } catch { // See above: an unreadable manifest declares no checks. } @@ -184,6 +189,63 @@ export function effectiveReviewerNames(): ReadonlySet { return names; } +/** + * The FailproofAI Cloud Jev policies `active.json` names, read once per + * registration pass — the same lifetime as the reviewer set built from them, so + * the two cannot describe different deployments. Never throws. + */ +export function cloudJevForPass(): CloudJevRead { + if (!cachedCloud) cachedCloud = readCloudJevPolicies(); + return cachedCloud; +} + +/** + * Installed packs ∪ FailproofAI Cloud Jev policies, as ONE list of semantic + * sources for the reviewer set, the question set and the contest. + * + * ## Cloud wins a name clash + * + * A check name both a Cloud policy and an installed pack declare is the Cloud + * one's: the pack's same-named entry is dropped here (and `shadowed` says so, + * once per pack and name). Left in, the two declarations would CONTEST the name + * and it would be asked for nobody — so a pack could switch off an org's + * centrally deployed check by declaring its name, which is the same "local + * config cannot disable a central assignment" rule Cloud JS policies follow. + * + * A reserved name is the exception, in both directions: Cloud is never + * first-party, so its claim to one of FailproofAI's sixteen names is void + * (`isReservedClaim`) and shadows nothing — FailproofAI's own pack keeps it. + * + * Cloud sets come first in the list; `semanticPoliciesFromPacks` still spends + * the one question budget on first-party packs before anyone else. Pure. + */ +export function withCloudSemantic & { source?: string }>( + packs: ReadonlyArray, + cloud: ReadonlyArray, +): { sources: Array; shadowed: string[] } { + if (cloud.length === 0) return { sources: [...packs], shadowed: [] }; + const cloudNames = new Map(); + for (const set of cloud) { + for (const entry of set.semantic) { + if (!isReservedClaim(set, entry.name) && !cloudNames.has(entry.name)) cloudNames.set(entry.name, set.source); + } + } + const shadowed: string[] = []; + const kept = packs.map((pack) => { + const semantic = pack.semantic ?? []; + const clashing = semantic.filter((entry) => cloudNames.has(entry.name)); + if (clashing.length === 0) return pack; + for (const entry of clashing) { + shadowed.push( + `pack ${pack.id} declares Jev check ${entry.name}, which FailproofAI Cloud policy ` + + `${cloudNames.get(entry.name)} also declares — the Cloud one is asked, the pack's is not`, + ); + } + return { ...pack, semantic: semantic.filter((entry) => !cloudNames.has(entry.name)) }; + }); + return { sources: [...cloud, ...kept], shadowed }; +} + /** The contest behind {@link effectiveReviewerNames}' answer, so a refusal can name it. */ export function contestedReviewerNames(): ReadonlyMap { effectiveReviewerNames(); @@ -211,7 +273,7 @@ export function jevChecksInstalled(): boolean { */ export function jevChecksDeclared(): boolean { try { - return reviewerNamesFor(jevPacks(readInstalledPacks().packs)).size > 0; + return reviewerNamesFor(withCloudSemantic(jevPacks(readInstalledPacks().packs), readCloudJevPolicies().sets).sources).size > 0; } catch { return false; } @@ -246,5 +308,6 @@ export function reviewerNamesFor( */ export function forgetEffectiveReviewerNames(cli?: string): void { cached = null; + cachedCloud = null; reviewerCli = cli; } diff --git a/src/hooks/handler.ts b/src/hooks/handler.ts index 4a789bde1..1cf7431b8 100644 --- a/src/hooks/handler.ts +++ b/src/hooks/handler.ts @@ -37,8 +37,16 @@ import type { TwoTierReview } from "./semantic/combine"; import type { JevActivityFields } from "./semantic/combine"; import type { JevConfig } from "./semantic/jev-config"; import { clearPolicies, registerPolicy, getPoliciesForEvent } from "./policy-registry"; -import { loadAllCustomHooks } from "./custom-hooks-loader"; -import { contestedReviewerNames, effectiveReviewerNames, jevChecksInstalled } from "./effective-reviewers"; +import { loadAllCustomHooks, type LoadAllResult } from "./custom-hooks-loader"; +import { + cloudJevForPass, + contestedReviewerNames, + effectiveReviewerNames, + jevChecksInstalled, + jevPacks, + reviewerNamesFor, + withCloudSemantic, +} from "./effective-reviewers"; import { authorityDeclarationFor, refusedAuthorityWarning, @@ -56,7 +64,8 @@ import { resolveTranscriptPath } from "./resolve-transcript-path"; import { getInstanceId } from "../../lib/telemetry-id"; import { hookLogInfo, hookLogWarn } from "./hook-logger"; import { readStdinPayload } from "./read-stdin"; -import { readActiveCloudManagedPolicies, type CloudManagedPolicyArtifact } from "./cloud-managed-policies"; +import { readActiveCloudManagedPolicies, readCloudJevMode, type CloudManagedPolicyArtifact } from "./cloud-managed-policies"; +import { cloudPolicyErrorsPath, collectCloudPolicyErrors, writeCloudPolicyErrors } from "./cloud-policy-errors"; import { hasInstalledRegexPacks, readInstalledPacks, type PackError, type ResolvedPack } from "./pack-manifest"; import { missingGuards, packFailureReason, combinedGuardMatch, guardsCover } from "./pack-failclosed"; import { readActivePause, type ActivePause } from "./session-pause"; @@ -284,9 +293,15 @@ function jevForcedOff(opts: EvaluateHookEventOptions | undefined): boolean { */ async function readJevConfig(): Promise<{ config: JevConfig; defaultMode: TwoTierReview["mode"] } | null> { try { - if (!existsSync(jevConfigFile())) return null; - const { loadJevConfig, DEFAULT_JEV_MODE } = await import("./semantic/jev-config"); - const config = loadJevConfig(); + // FailproofAI Cloud's mode, when its deployment sets one, OVERRIDES the + // local one — a local `off` included — and may run Jev with no `jev.json` + // at all, on this machine's Cloud Jev credential (`loadJevConfigForCloudMode`). + // Read off `active.json`, which is small and read on this path already. + const cloudMode = readCloudJevMode(); + if (cloudMode === "off") return null; + if (cloudMode === null && !existsSync(jevConfigFile())) return null; + const { loadJevConfig, loadJevConfigForCloudMode, DEFAULT_JEV_MODE } = await import("./semantic/jev-config"); + const config = cloudMode ? loadJevConfigForCloudMode(cloudMode).config : loadJevConfig(); // `loadJevConfig` never returns a switched-off config; this says so again at // the one place it matters, because an `off` that got through would fall to // the default mode below — which is `enforce`. @@ -299,6 +314,65 @@ async function readJevConfig(): Promise<{ config: JevConfig; defaultMode: TwoTie } } +/** + * Where a Jev decision's check came from, as activity-row fields. A FailproofAI + * Cloud Jev policy is pack-shaped on the way through the resolver + * (`cloud:` / ``, see `readCloudJevPolicies`), and is filed here + * under the Cloud attribution pair a Cloud JS policy's decision uses — so the + * Cloud policy page counts it against the policy that was deployed. + */ +function jevAttribution( + origin: { packId: string; packVersion?: string } | undefined, +): { packId?: string; packVersion?: string; cloudPolicyId?: string; cloudVersion?: number } { + if (!origin) return {}; + if (origin.packId.startsWith("cloud:")) { + const version = Number(origin.packVersion); + return { + cloudPolicyId: origin.packId.slice("cloud:".length), + ...(Number.isSafeInteger(version) ? { cloudVersion: version } : {}), + }; + } + return origin; +} + +/** + * Compute and write `cloud-policies/errors.json` for this machine's Cloud + * deployment. A no-op off a Cloud-managed machine (`writeCloudPolicyErrors` + * skips without an `active.json`) and when nothing changed. Never throws. + */ +async function recordCloudPolicyErrors(input: { + manifestError: string | null; + jsPolicies: CloudManagedPolicyArtifact[]; + jsFailures: LoadAllResult["cloudFailures"] | undefined; + packs: ResolvedPack[]; +}): Promise { + try { + const jev = cloudJevForPass(); + const managed = input.manifestError !== null || input.jsPolicies.length > 0 || jev.semanticIds.length > 0 || jev.jevMode !== null; + // An unmanaged machine never reads further. A managed machine whose + // deployment just emptied still writes, so a stale report is cleared. + if (!managed && !existsSync(cloudPolicyErrorsPath())) return; + let jevUnconfigured: string | null = null; + if (jev.jevMode === "observe" || jev.jevMode === "enforce") { + const { loadJevConfigForCloudMode } = await import("./semantic/jev-config"); + jevUnconfigured = loadJevConfigForCloudMode(jev.jevMode).problem; + } + writeCloudPolicyErrors( + collectCloudPolicyErrors({ + manifestError: input.manifestError, + jsPolicies: input.jsPolicies, + jsFailures: input.jsFailures, + jev, + // Every agent's view, never this event's: see `CloudPolicyErrorInputs`. + reviewerNames: reviewerNamesFor(withCloudSemantic(jevPacks(input.packs), jev.sets).sources), + jevUnconfigured, + }), + ); + } catch (err) { + hookLogInfo(`could not record Cloud policy errors: ${err instanceof Error ? err.message : String(err)}`); + } +} + /** * Start Jev's review of this call — before the regex policies run, so the two * proceed in parallel — or return null for the regex-only path. @@ -612,10 +686,13 @@ export async function evaluateHookEvent( // than the status quo, which already failed open on five of the twelve — // just accidentally, and without saying so. let cloudManagedPolicies: CloudManagedPolicyArtifact[] = []; + /** Why the Cloud JS half did not load at all, for `errors.json`. */ + let cloudManifestError: string | null = null; try { cloudManagedPolicies = readActiveCloudManagedPolicies(); } catch (err) { const msg = err instanceof Error ? err.message : String(err); + cloudManifestError = msg; hookLogWarn( `[failproofai] cloud-managed policies could NOT be loaded and are not being enforced: ${msg}. ` + `Local policies are unaffected. Run \`failproofai config --status\` to check this machine's enrolment.`, @@ -831,6 +908,17 @@ export async function evaluateHookEvent( ); } + // What this machine could not load from its FailproofAI Cloud deployment, + // for the daemon to report on its next poll. Only on a managed machine, + // written only when it changes, and never able to cost this event its + // answer (see `cloud-policy-errors.ts`). + await recordCloudPolicyErrors({ + manifestError: cloudManifestError, + jsPolicies: cloudManagedPolicies, + jsFailures: loadResult.cloudFailures, + packs: installedPacks, + }); + // Fail closed on enforcement this machine was told it had and does not. // // Additive, and deliberately NOT a clearPolicies() the way the @@ -997,7 +1085,7 @@ export async function evaluateHookEvent( // would be false — so it is attributed to Jev itself. Leaving it out // instead filed every Jev block under "unattributed" on FailproofAI // Cloud's policy page, beside rows written before attribution existed. - ...(result.policyName && result.twoTier?.decidedByJev ? { policySource: "jev" as const, ...jevOrigin } : {}), + ...(result.policyName && result.twoTier?.decidedByJev ? { policySource: "jev" as const, ...jevAttribution(jevOrigin) } : {}), ...(result.policyName && !result.twoTier?.decidedByJev ? (() => { const attribution = policyAttribution.get(result.policyName); diff --git a/src/hooks/jev-cli.ts b/src/hooks/jev-cli.ts index 347250198..66bf02c4d 100644 --- a/src/hooks/jev-cli.ts +++ b/src/hooks/jev-cli.ts @@ -108,6 +108,7 @@ import { inspectJevConfig, jevCloudBaseUrl, jevConfigPath, + loadJevConfigForCloudMode, providerForUrl, providerHostConflict, readJevConfigFileForUpdate, @@ -144,6 +145,7 @@ import { type ReviewableCoverage, } from "./policy-reviewability"; import { jevStats, type JevStats } from "./semantic/jev-stats"; +import { readCloudJevPolicies, type CloudJevRead } from "./cloud-managed-policies"; import { readCredentials, readJevCloudCredential, type JevCloudCredential } from "./fp-config"; import type { JevRequest } from "./semantic/types"; import { TOKEN_ON_ARGV, emptyState, nextStep, note, optsFor, rows, rule, stack, title, warning, type RenderOpts } from "./tui"; @@ -1388,7 +1390,174 @@ function safeCoverage(): ReviewableCoverage | null { } } +// ── status: FailproofAI Cloud ──────────────────────────────────────────────── +// +// A Cloud deployment can carry Jev checks (`jev` / `both` policies) and can SET +// this machine's Jev mode, overriding `jev.json`'s — a local `off` included. So +// `status` answers from Cloud first: with a Cloud mode the local mode is not +// what runs, and saying "switched off in jev.json" would be false. Without one, +// the local answer stands and the Cloud checks are listed beside it. + +/** Label for everything Cloud decided here. Customer-facing: never "AgentEye". */ +const MODE_SET_BY_CLOUD = "mode set by FailproofAI Cloud"; + +function safeCloudJev(): CloudJevRead { + try { + return readCloudJevPolicies(); + } catch { + return { sets: [], semanticIds: [], jevMode: null, errors: [] }; + } +} + +/** The Cloud checks, per policy, for `--json`. */ +function cloudChecksJson(cloud: CloudJevRead): Array<{ policy: string; version: number; kind: string; checks: string[] }> { + return cloud.sets.map((set) => ({ + policy: set.policyId, + version: set.policyVersion, + kind: set.kind, + checks: set.semantic.map((entry) => entry.name), + })); +} + +/** The Cloud checks, per policy, as rows. */ +function cloudChecksBlock(cloud: CloudJevRead, opts: RenderOpts): string[] { + if (cloud.sets.length === 0 && cloud.errors.length === 0) return []; + return stack( + rule("FailproofAI Cloud Jev checks", opts), + cloud.sets.length > 0 + ? rows( + cloud.sets.map((set): [string, string] => [ + `${set.policyId} v${set.policyVersion}`, + `${set.semantic.map((entry) => entry.name).join(", ") || "(no usable checks)"}${set.kind === "both" ? " · reviews its own regex policy" : ""}`, + ]), + opts, + ) + : null, + cloud.errors.length > 0 + ? warning( + cloud.errors.map((e) => `${e.id}${e.version === null ? "" : ` v${e.version}`}: ${e.message}`), + opts, + ) + : null, + ); +} + async function status(argv: string[], opts: RenderOpts): Promise { + const cloud = safeCloudJev(); + if (cloud.jevMode === null && cloud.sets.length === 0 && cloud.errors.length === 0) return localStatus(argv, opts); + const parsed = parseFlags(argv, new Set(["--json"])); + if (typeof parsed === "string" || parsed.positionals.length > 0) return localStatus(argv, opts); + const asJson = parsed.bools.has("--json"); + + if (cloud.jevMode === null) { + // jev.json decides the mode; Cloud contributes checks. The local answer, + // with them added. + const local = await localStatus(argv, opts); + if (asJson && local.json !== undefined) { + const body = JSON.parse(local.json) as Record; + body.cloud = { jevMode: null, modeSetBy: null, checks: cloudChecksJson(cloud), errors: cloud.errors }; + return { ...local, json: JSON.stringify(body, null, 2) }; + } + return { ...local, lines: [...local.lines, ...cloudChecksBlock(cloud, opts)] }; + } + + const mode = cloud.jevMode; + const resolved = loadJevConfigForCloudMode(mode); + let stats: JevStats | null = null; + try { + stats = await jevStats(); + } catch { + stats = null; + } + const cfg = resolved.config; + const coverage = cfg ? safeCoverage() : null; + const route = cfg ? jevRoute(cfg) : null; + const running = cfg !== null; + + if (asJson) { + const body: Record = { + path: jevConfigPath(), + status: running ? "ok" : mode === "off" ? "off" : "unconfigured", + mode, + modeSetBy: "FailproofAI Cloud", + ...(cfg && route + ? { + provider: cfg.provider, + providerLabel: providerLabel(cfg.provider), + providerSource: resolved.provider, + endpoint: shownEndpoint(cfg.provider, route.endpoint), + model: route.model, + modelIsDefault: route.modelIsDefault, + timeoutMs: cfg.timeoutMs, + } + : {}), + ...(resolved.problem ? { problem: resolved.problem } : {}), + cloud: { jevMode: mode, modeSetBy: "FailproofAI Cloud", checks: cloudChecksJson(cloud), errors: cloud.errors }, + ...(coverage + ? { + reviewablePolicies: { + enabled: coverage.enabled, + reviewable: coverage.reviewable, + customPolicyFiles: coverage.customFiles, + jevChecks: coverage.jevChecks, + problem: reviewableProblem(coverage), + }, + } + : {}), + stats, + }; + return ok([], JSON.stringify(body, null, 2)); + } + + const head = + mode === "off" + ? title("failproofai jev status", `off — ${MODE_SET_BY_CLOUD}`, opts) + : running + ? title("failproofai jev status", `on · ${mode} — ${MODE_SET_BY_CLOUD}`, opts) + : title("failproofai jev status", `off — FailproofAI Cloud set ${mode}, and nothing here can answer`, opts); + return ok( + stack( + head, + note( + `FailproofAI Cloud sets this machine's Jev mode, and it overrides the mode in ${jevConfigPath()} ` + + "(an \"off\" there included) for as long as this machine is connected.", + opts, + ), + cfg && route + ? rows( + [ + ["provider", providerLabel(cfg.provider)], + ["endpoint", shownEndpoint(cfg.provider, route.endpoint)], + ["model", route.modelIsDefault ? `${route.model} (provider default)` : route.model], + ["mode", `${modeLine(mode)} (${MODE_SET_BY_CLOUD})`], + ["timeout", `${cfg.timeoutMs} ms`], + [ + "config", + resolved.provider === "cloud-credential" + ? `no ${jevConfigPath()} — this machine's FailproofAI Cloud connection answers` + : jevConfigPath(), + ], + ], + opts, + ) + : rows([["mode", mode === "off" ? `${modeLine("off")} (${MODE_SET_BY_CLOUD})` : `${mode} (${MODE_SET_BY_CLOUD}) — not running`]], opts), + resolved.problem + ? warning( + [ + `Jev cannot run here: ${resolved.problem}.`, + "Connect with a key that carries jev:evaluate (failproofai config --token ), or configure your own provider (failproofai jev setup).", + ], + opts, + ) + : null, + cloudChecksBlock(cloud, opts), + coverage ? note(reviewableSummary(coverage), opts) : null, + jevStatsLines(stats, opts), + ), + ); +} + +async function localStatus(argv: string[], opts: RenderOpts): Promise { const parsed = parseFlags(argv, new Set(["--json"])); if (typeof parsed === "string") return fail([parsed, "", ...JEV_USAGE]); if (parsed.positionals.length > 0) return fail([STRAY_ARGUMENT, "", ...JEV_USAGE]); diff --git a/src/hooks/manager.ts b/src/hooks/manager.ts index 322975be3..c3caf798e 100644 --- a/src/hooks/manager.ts +++ b/src/hooks/manager.ts @@ -25,7 +25,7 @@ import { getInstanceId, hashToId } from "../../lib/telemetry-id"; import { CliError } from "../cli-error"; import { hookLogWarn } from "./hook-logger"; import { customPoliciesDir, globalPolicyConfigFile } from "./fp-home"; -import { readActiveCloudManagedPolicies } from "./cloud-managed-policies"; +import { readActiveCloudManagedPolicies, readCloudJevPolicies } from "./cloud-managed-policies"; import { CORE_SOURCE, addPack, setPackPolicyEnabled } from "./pack-store"; import type { ResolvedPack } from "./pack-manifest"; import { hasInstalledRegexPacks, readInstalledPacks } from "./pack-manifest"; @@ -1377,27 +1377,62 @@ export async function listHooks(cwd?: string): Promise { // Read-only on purpose: these are owned by the deployment, not by local // config. `--uninstall ` cannot switch one off, and printing them // beside toggleable rows without saying so would imply it can. + // + // Each policy's KIND is shown — `regex` (JS), `jev` (Jev checks only) or + // `both` (JS reviewable by its own checks) — with the Jev check names, so a + // Jev-only policy, which registers no JS at all, is listed too. try { - const cloud = readActiveCloudManagedPolicies(); - if (cloud.length > 0) { - groups.push(rule(`Cloud-managed — deployment ${cloud[0].deployment}`, opts)); + let cloud: ReturnType = []; + try { + cloud = readActiveCloudManagedPolicies(); + } catch { + // The Jev half is read independently below; one bad half must not hide + // the other. + } + let jev: ReturnType = { sets: [], semanticIds: [], jevMode: null, errors: [] }; + try { + jev = readCloudJevPolicies(); + } catch { + // Never throws by contract; guarded anyway, for the same reason as above. + } + const setFor = (id: string) => jev.sets.find((set) => set.policyId === id); + const jevOnly = jev.sets.filter((set) => set.kind === "jev"); + if (cloud.length > 0 || jev.sets.length > 0 || jev.jevMode !== null) { + const deployment = cloud[0]?.deployment ?? jev.sets[0]?.deployment; groups.push( - table( - { - head: ["", "Policy", "Version"], - rows: cloud.map((artifact) => [ - // `observe` is evaluated and then has its verdict discarded, so a - // row that read "ON" would claim enforcement this policy - // deliberately is not doing. - chip(artifact.effect === "observe" ? "observe" : "cloud", opts), - artifact.id, - `v${artifact.version}`, - ]), - flex: 1, - }, - opts, - ), + rule(`Cloud-managed${deployment !== undefined ? ` — deployment ${deployment}` : ""} · FailproofAI Cloud`, opts), ); + const checksOf = (id: string): string => setFor(id)?.semantic.map((entry) => entry.name).join(", ") ?? ""; + const rowsOut = [ + ...cloud.map((artifact) => { + const both = jev.semanticIds.includes(artifact.id); + return [ + // `observe` is evaluated and then has its verdict discarded, so a + // row that read "ON" would claim enforcement this policy + // deliberately is not doing. + chip(artifact.effect === "observe" ? "observe" : "cloud", opts), + artifact.id, + `v${artifact.version}`, + both ? "both" : "regex", + both ? checksOf(artifact.id) || "(not loaded)" : "", + ]; + }), + ...jevOnly.map((set) => [chip("cloud", opts), set.policyId, `v${set.policyVersion}`, "jev", checksOf(set.policyId)]), + ]; + if (rowsOut.length > 0) { + groups.push(table({ head: ["", "Policy", "Version", "Kind", "Jev checks"], rows: rowsOut, flex: 4 }, opts)); + } + if (jev.jevMode !== null) { + groups.push(note(`Jev mode: ${jev.jevMode} — mode set by FailproofAI Cloud (overrides jev.json).`, opts)); + } + if (jev.errors.length > 0) { + groups.push( + warning( + jev.errors.map((e) => `${e.id}${e.version === null ? "" : ` v${e.version}`}: ${e.message}`), + opts, + ), + ); + } groups.push( note("Managed from the dashboard — not switchable with `failproofai policies`.", opts), ); diff --git a/src/hooks/pack-manifest.ts b/src/hooks/pack-manifest.ts index 2e95e1eb9..988e5065d 100644 --- a/src/hooks/pack-manifest.ts +++ b/src/hooks/pack-manifest.ts @@ -738,8 +738,13 @@ export function semanticQuestions( * Per ENTRY, not per pack: one malformed question set must not take the rest of * a publisher's semantic policies with it, for the same reason one malformed * pack does not take the other packs with it. + * + * Exported because it is also THE parser for FailproofAI Cloud Jev policies + * (`readCloudJevPolicies` in `cloud-managed-policies.ts`): a Cloud declaration + * is exactly a manifest semantic entry, and a second parser would be a second + * set of rules to drift. */ -function parsePackSemantic(packId: string, value: unknown, warnings: string[]): SemanticManifestEntry[] { +export function parsePackSemantic(packId: string, value: unknown, warnings: string[]): SemanticManifestEntry[] { if (value === undefined) return []; if (!Array.isArray(value)) { warnings.push(`pack ${packId} semantic is not an array, so none of it was loaded`); diff --git a/src/hooks/policy-reviewability.ts b/src/hooks/policy-reviewability.ts index 0b4736441..d86b6e900 100644 --- a/src/hooks/policy-reviewability.ts +++ b/src/hooks/policy-reviewability.ts @@ -55,8 +55,8 @@ * - **A session pause.** It suspends local policy for minutes, and a warning * about a policy set that is coming back shortly would be noise. */ -import { readActiveCloudManagedPolicies } from "./cloud-managed-policies"; -import { jevPacks } from "./effective-reviewers"; +import { readActiveCloudManagedPolicies, readCloudJevPolicies } from "./cloud-managed-policies"; +import { jevPacks, withCloudSemantic } from "./effective-reviewers"; import { resolve } from "node:path"; import { discoverPolicyFiles } from "./custom-hooks-loader"; import { customPoliciesDir } from "./fp-home"; @@ -180,7 +180,9 @@ export function surveyReviewableCoverage(cwd?: string): ReviewableCoverage { // contested name AND minus a check the question budget drops — which the // hook path's manifest-only `reviewerNamesFor` cannot see. Anything else and // the panel and `jev status` promise a clear that cannot happen. - const asked = semanticPoliciesFromPacks(jevPacks(packs)); + // FailproofAI Cloud Jev policies take part exactly as the hook path takes + // them (`resolveSemanticPolicies`), Cloud winning a name clash. + const asked = semanticPoliciesFromPacks(withCloudSemantic(jevPacks(packs), readCloudJevPolicies().sets).sources); reviewers = new Set(asked.policies.map((p) => p.name)); } catch { // An unreadable manifest enforces nothing and gives Jev nothing to ask; diff --git a/src/hooks/semantic/jev-config.ts b/src/hooks/semantic/jev-config.ts index 6ce4189ff..bbcf57c43 100644 --- a/src/hooks/semantic/jev-config.ts +++ b/src/hooks/semantic/jev-config.ts @@ -89,7 +89,7 @@ */ import { closeSync, constants as fsConstants, fstatSync, openSync, readSync, statSync } from "node:fs"; import { dirname } from "node:path"; -import { readJevCloudCredential, type JevCloudCredential } from "../fp-config"; +import { readCredentials, readJevCloudCredential, type JevCloudCredential } from "../fp-config"; import { jevConfigFile } from "../fp-home"; export type { JevCloudCredential } from "../fp-config"; @@ -1090,6 +1090,113 @@ export function loadJevConfig(): JevConfig | null { } } +// ── FailproofAI Cloud sets the mode ────────────────────────────────────────── + +/** Where the config for a Cloud-set mode came from. */ +export type CloudModeProvider = "jev.json" | "cloud-credential"; + +export interface CloudModeJevConfig { + /** The config to run with, its `mode` Cloud's — or null: Jev does not run. */ + config: JevConfig | null; + /** Which provider answered, when one did. */ + provider: CloudModeProvider | null; + /** + * Why nothing could answer, for `errors.json` — null when Jev runs, and null + * for `off`, which is not a problem. `jev_unconfigured` exactly when there is + * neither a `jev.json` nor a Cloud Jev credential; a `jev.json` that exists + * and cannot be used says why after the same prefix. + */ + problem: string | null; +} + +/** + * The config to run when FailproofAI Cloud sets this machine's Jev mode + * (`active.json` `jevMode`), which OVERRIDES the local mode — a local `off` + * included: an org that deploys Jev policies and sets their mode has decided, + * and a switch any process of this user can flip must not quietly undo that. + * + * The PROVIDER is still local, because a mode says how to act on answers, not + * where to ask: + * + * 1. `jev.json`'s provider when the file exists — including a file saying + * `off`, whose routing and key are kept for exactly this. Every check the + * loader applies still applies (owner-only file and directory, the Cloud + * key's origin binding, and plain http refused in enforce), re-run with + * Cloud's mode in place of the file's. + * 2. Otherwise the Cloud Jev credential (`credentials.json` `jev {url,key}`, + * only while its connection is still on this machine) as provider + * `failproofai`, at `/enforcement/v1/jev` — what `config --token` + * would have written. + * 3. Neither: no Jev, and `problem` says `jev_unconfigured`. + * + * Never throws. + */ +export function loadJevConfigForCloudMode(mode: JevConfigMode): CloudModeJevConfig { + if (mode === "off") return { config: null, provider: null, problem: null }; + try { + const inspected = inspectJevConfig(); + if (inspected.status === "ok") { + const r = validateLoadedJevConfig({ ...inspected.config, mode }); + return r.ok + ? { config: r.value, provider: "jev.json", problem: null } + : { config: null, provider: null, problem: `${JEV_UNCONFIGURED_PROBLEM}: jev.json cannot run in ${mode} mode — ${r.problem}` }; + } + if (inspected.status === "off") { + // Sound and switched off locally; its routing (and a BYOK key) are on + // disk. Re-validated whole with Cloud's mode, the credential passed only + // for the Cloud provider, exactly as `inspectJevConfig` passes it. + const file = readJevConfigFileForUpdate(); + if (!file || file.tooOpen) { + return { config: null, provider: null, problem: `${JEV_UNCONFIGURED_PROBLEM}: jev.json could not be re-read` }; + } + const raw: Record = { ...file.raw, mode }; + let r: ValidationResult; + if (raw.provider === JEV_CLOUD_PROVIDER) { + const credential = readJevCloudCredential(); + r = validateJevConfig(raw, null, credential.status === "ok" ? credential.credential : null); + } else { + r = validateJevConfig(raw, readEnvKey()); + } + return r.ok + ? { config: r.value, provider: "jev.json", problem: null } + : { config: null, provider: null, problem: `${JEV_UNCONFIGURED_PROBLEM}: jev.json — ${r.problem}` }; + } + if (inspected.status === "absent") { + const credential = readJevCloudCredential(); + if (credential.status !== "ok") return { config: null, provider: null, problem: JEV_UNCONFIGURED_PROBLEM }; + const raw = { provider: JEV_CLOUD_PROVIDER, baseUrl: jevCloudBaseUrl(cloudBaseFor(credential.credential)), mode }; + const r = validateJevConfig(raw, null, credential.credential); + return r.ok + ? { config: r.value, provider: "cloud-credential", problem: null } + : { config: null, provider: null, problem: `${JEV_UNCONFIGURED_PROBLEM}: the FailproofAI Cloud Jev credential — ${r.problem}` }; + } + // The file exists and is unusable (not connected, key missing, refused). + // Its provider is the one the owner chose, so it is not swapped for another. + const detail = (inspected as { problem?: string }).problem ?? inspected.status; + return { config: null, provider: null, problem: `${JEV_UNCONFIGURED_PROBLEM}: jev.json is ${inspected.status} — ${detail}` }; + } catch (err) { + return { config: null, provider: null, problem: `${JEV_UNCONFIGURED_PROBLEM}: ${err instanceof Error ? err.message : String(err)}` }; + } +} + +/** The contract's message for "Cloud set a Jev mode and nothing can answer". */ +export const JEV_UNCONFIGURED_PROBLEM = "jev_unconfigured"; + +/** + * The Cloud base URL the Jev route hangs off: the connection's own URL when it + * is on the credential's origin (so a Cloud served under a path prefix keeps + * it, as `config --token` writes it), else the credential's origin. + */ +function cloudBaseFor(credential: JevCloudCredential): string { + try { + const cloud = readCredentials().cloud; + if (cloud?.url && new URL(cloud.url).origin === new URL(credential.url).origin) return cloud.url; + } catch { + // Fall through to the origin. + } + return credential.url; +} + export interface JevConfigFileForUpdate { raw: Record; /** The file's permission bits (null where they mean nothing). */ diff --git a/src/hooks/semantic/pack-policies.ts b/src/hooks/semantic/pack-policies.ts index 22b19d981..8a650bbce 100644 --- a/src/hooks/semantic/pack-policies.ts +++ b/src/hooks/semantic/pack-policies.ts @@ -39,7 +39,8 @@ * regex policies cover — a machine locked out over a typo in the half of the * system whose job is to let more real work through. */ -import { contestedSemanticNames, isFirstPartyPack, isReservedClaim, jevPacks } from "../effective-reviewers"; +import { readCloudJevPolicies } from "../cloud-managed-policies"; +import { contestedSemanticNames, isFirstPartyPack, isReservedClaim, jevPacks, withCloudSemantic } from "../effective-reviewers"; import { hookLogWarn } from "../hook-logger"; import { packSemantic, @@ -254,7 +255,8 @@ export function semanticPoliciesFromPacks( const warned = new Set(); /** - * The live set, read from the installed packs. + * The live set, read from the installed packs and the FailproofAI Cloud Jev + * policies deployed to this machine (`active.json` `semanticPolicies`). * * Called from `prepareSemantic`, so only on a machine that has a Jev config and * is preparing a request. It re-reads the manifest rather than caching: the @@ -277,10 +279,14 @@ export function resolveSemanticPolicies(cli?: string): ReadonlyArray Date: Wed, 30 Sep 2026 01:32:06 +0530 Subject: [PATCH 03/41] fp: publish Jev/both policies, deploy --jev-mode, show policy errors - `fp policies publish --kind regex|jev|both [--source] [--semantic]`: sends kind + semantic (source omitted for jev); the kind follows from the inputs when omitted, a contradiction is exit 2 before any request. The semantic file gets a shape check only (1-24 named objects); the server and each machine's parser own the rules. JavaScript with Jev fields is still refused, now pointing at --kind both --semantic. - `fp fleet deploy ... --jev-mode off|observe|enforce|local`: jevMode sent only when given (absent = unchanged); --jev-mode alone is a deploy; the plan shows and carries the mode change; observe on a jev version is refused (exit 2) with the mode that does it. - Models carry kind/semantic/semanticSha256/authority/reviewedBy, deployment and machine jevMode, machine policyErrors/policyErrorsAt (emitted in --json only when the server sends them). - policies list/show print kind + Jev check names (show also the declarations); fleet list shows "N errors", fleet show the Jev mode and the reported errors. - permissions.py unchanged (the contract adds no permission). - Docs: fp README, skill, commands reference; the public Cloud CLI reference, deploy guide and FailproofAI Cloud Jev page; CHANGELOGs. - Tests: fp-cloud-cli/tests/test_cloud_jev.py (38). Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 5 + docs/policies/deploy.mdx | 11 + docs/reference/cloud-cli.mdx | 10 +- docs/reference/jev-cloud.mdx | 2 + fp-cloud-cli/CHANGELOG.md | 16 + fp-cloud-cli/README.md | 14 + fp-cloud-cli/fp_cli/client.py | 54 ++- fp-cloud-cli/fp_cli/commands/fleet_cmds.py | 41 +- fp-cloud-cli/fp_cli/commands/policies_cmds.py | 98 ++++- fp-cloud-cli/fp_cli/enforcement.py | 121 +++++- fp-cloud-cli/fp_cli/models.py | 72 +++- fp-cloud-cli/fp_cli/output.py | 87 ++++- fp-cloud-cli/fp_cli/policy_check.py | 13 +- fp-cloud-cli/skill/SKILL.md | 4 +- fp-cloud-cli/skill/references/commands.md | 23 +- fp-cloud-cli/tests/test_cloud_jev.py | 364 ++++++++++++++++++ 16 files changed, 867 insertions(+), 68 deletions(-) create mode 100644 fp-cloud-cli/tests/test_cloud_jev.py diff --git a/CHANGELOG.md b/CHANGELOG.md index e1f8e9519..0e0dad58b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,10 @@ ### Features - Uploads, `fp` calls and evaluator calls carry a request id; the daemon also sends batch and machine ids, and `fp` errors show a `ref` (#872) +- **Jev policies deploy from FailproofAI Cloud, individually, like Cloud JS policies.** A Cloud policy has a kind — `regex` (JavaScript, as before), `jev` (Jev checks only) or `both` (JavaScript reviewable by its own checks, deployed as one). The daemon verifies each policy's Jev declarations like a JS artifact and activates both halves in one atomic `active.json` write; the CLI parses them with the pack manifest's own parser and asks them beside installed packs' checks, with FailproofAI Cloud winning a name clash and one shared question budget. `failproofai policies` lists them under a FailproofAI Cloud label. +- **FailproofAI Cloud can set a machine's Jev mode**, overriding `jev.json` (a local `off` included) for as long as the machine is connected. With no `jev.json`, the machine asks through its FailproofAI Cloud Jev credential; `failproofai jev status` shows the Cloud checks and "mode set by FailproofAI Cloud". +- **Machines report the Cloud policies they could not apply.** A Cloud JS policy that fails to load, a Jev artifact that fails its digest or parse, a dropped declaration, a `reviewedBy` naming a check that is not there, or a Jev mode with no provider is written to `cloud-policies/errors.json` and sent on the daemon's next poll (`policyErrors`), so the fleet page and `fp fleet show` list it. Everything else on the machine is unchanged, and `config --disconnect` clears the reports with the deployment. +- `fp policies publish --kind regex|jev|both [--source] [--semantic]`, `fp fleet deploy --jev-mode off|observe|enforce|local`; `fp policies list/show` print the kind and Jev check names, and `fp fleet list/show` the machine's Jev mode and reported policy errors. ### Fixes @@ -14,6 +18,7 @@ ### Docs - Troubleshooting, HTTP API and Cloud CLI pages explain the `ref` / `request_id` to quote to support, and where the daemon logs them (#872) +- `crates/CLOUD_POLICIES.md` documents the Jev fields of desired-state and `active.json` and the `policyErrors` report, and its stale `cloud-managed/`, `deployments/` and `cloud.json` names are fixed; the Cloud CLI reference, the deploy guide and the FailproofAI Cloud Jev page cover Jev policies and the Cloud-set Jev mode. ### Dependencies diff --git a/docs/policies/deploy.mdx b/docs/policies/deploy.mdx index a6c7cfcfd..85331d8c9 100644 --- a/docs/policies/deploy.mdx +++ b/docs/policies/deploy.mdx @@ -70,6 +70,17 @@ A machine appears under **Admin → enforcement** once it is connected to Cloud. +## Deploy Jev checks + +A Cloud policy can carry Jev checks as well as, or instead of, JavaScript. Its kind is `jev` (Jev checks only) or `both` (JavaScript whose verdict its own checks may clear). It deploys, toggles and rolls back exactly like any other policy: a `both` policy's JavaScript and its checks always reach a machine together. + +```bash +fp policies publish prod-db-intent --kind jev --semantic ./checks.json +fp fleet deploy --add prod-db-intent --jev-mode observe +``` + +An effect applies to JavaScript only. To trial Jev checks without them deciding anything, set the machine's Jev mode to `observe`, then to `enforce` once the policy page shows what they would have done. A check that fails on a machine is dropped there and nothing else changes; the machine reports it, and `fp fleet show ` lists it under policy errors. + ## Check coverage Coverage answers whether a policy is running where the risk is. diff --git a/docs/reference/cloud-cli.mdx b/docs/reference/cloud-cli.mdx index 3bab0efb5..fed27a703 100644 --- a/docs/reference/cloud-cli.mdx +++ b/docs/reference/cloud-cli.mdx @@ -329,9 +329,9 @@ Cloud-managed policy versions. **Session-only** — every command here exits `2` | Command | Purpose | Options | | --- | --- | --- | -| `fp policies list` | List policy versions. | `--json` | -| `fp policies show POLICY_ID` | Show one policy, with its source. | — | -| `fp policies publish NAME PATH` | Mint a version from a local `.mjs`. | `--description`; `--no-verify` | +| `fp policies list` | List policy versions, with each one's kind (`regex`, `jev`, `both`) and Jev check names. | `--json` | +| `fp policies show POLICY_ID` | Show one policy, with its source and its Jev checks. | — | +| `fp policies publish NAME [PATH]` | Mint a version: JavaScript (`regex`, the default), Jev checks from a JSON file of declarations (`jev`), or both, where the JavaScript is reviewable by its own checks. | `--kind`; `--source`; `--semantic`; `--description`; `--no-verify` | | `fp policies enable POLICY_ID` | Add it back to every deployment it was removed from, minting a new generation on each. | `--yes`, `-y` | | `fp policies disable POLICY_ID` | Remove it from every deployment carrying it, minting a new generation on each. | `--yes`, `-y` | | `fp policies delete POLICY_ID` | Delete a policy version. | `--yes`, `-y` | @@ -345,8 +345,8 @@ Which machines run which policies. **Session-only**, same reason as above. | Command | Purpose | Options | | --- | --- | --- | | `fp fleet list` | List enrolled machines and their deployment generation. | — | -| `fp fleet show MACHINE_ID` | The policy set a machine currently runs. | — | -| `fp fleet deploy MACHINE_ID` | **Replaces the machine's whole policy set.** Prints the plan and asks only on an interactive terminal without `--json`. | `--add`; `--remove`; `--set`; `--create`; `--yes`, `-y` | +| `fp fleet show MACHINE_ID` | The policy set a machine currently runs, its Jev mode, and the policy errors it reported. | — | +| `fp fleet deploy MACHINE_ID` | **Replaces the machine's whole policy set.** Prints the plan and asks only on an interactive terminal without `--json`. `--jev-mode` sets the machine's Jev mode from FailproofAI Cloud (`off`, `observe`, `enforce`, or `local` to stop overriding it); omitted, the mode is unchanged. | `--add`; `--remove`; `--set`; `--jev-mode`; `--create`; `--yes`, `-y` | | `fp fleet diff MACHINE_ID` | Compare a machine against another deployment. | — | | `fp fleet history MACHINE_ID` | Past deployments for a machine. | — | | `fp fleet rollback MACHINE_ID GENERATION` | Reinstate a past generation's policy set, as a new generation. | `--yes`, `-y` | diff --git a/docs/reference/jev-cloud.mdx b/docs/reference/jev-cloud.mdx index 8a89fe118..fe67dc0a4 100644 --- a/docs/reference/jev-cloud.mdx +++ b/docs/reference/jev-cloud.mdx @@ -68,6 +68,8 @@ failproofai jev setup --mode off # keep the config, stop asking Jev The same switch is in the local dashboard: **Settings → Jev** has an on/off switch and observe/enforce. It rewrites the mode and nothing else. Hooks read the config on every tool call, so a change applies from the next one, with no restart. +FailproofAI Cloud can also set a machine's mode, per machine, from the fleet page or with `fp fleet deploy --jev-mode observe|enforce|off`. A mode set there **overrides the one in `jev.json`, including `off`**, for as long as the machine stays connected; `--jev-mode local` hands the choice back to the machine. The machine asks through its own `jev.json` provider when it has one, and otherwise through its FailproofAI Cloud connection. `failproofai jev status` says `mode set by FailproofAI Cloud`, and a machine with no way to ask reports `jev_unconfigured` to the fleet page. + ## Check what it is doing ```bash diff --git a/fp-cloud-cli/CHANGELOG.md b/fp-cloud-cli/CHANGELOG.md index 6c273f990..930d52611 100644 --- a/fp-cloud-cli/CHANGELOG.md +++ b/fp-cloud-cli/CHANGELOG.md @@ -4,6 +4,20 @@ ### Added +- `fp policies publish --kind regex|jev|both [--source f.mjs] [--semantic f.json]`: a + FailproofAI Cloud policy can carry Jev checks — `jev` (declarations only, no JavaScript) or + `both` (JavaScript reviewable by exactly its own checks; the server derives the authority). + The kind follows from what is given when `--kind` is omitted, and a kind that contradicts its + inputs is exit 2 before anything is sent. The positional source still works. +- `fp fleet deploy … --jev-mode off|observe|enforce|local` sets the machine's Jev mode from + FailproofAI Cloud (overriding the machine's own; `local` hands it back). Omitted, the request + carries no `jevMode` and the mode is unchanged; `--jev-mode` alone is a valid deploy. An + `observe` effect on a `jev` policy is refused (exit 2) — Jev is observed through the mode. +- `fp policies list/show` print each version's kind and Jev check names (`show` also the + declarations); `fp fleet list/show` print the machine's Jev mode and the policy errors it + reported, and `--json` carries `kind`/`semantic`/`semanticSha256`/`authority`/`reviewedBy`, + `jevMode` and `policyErrors`/`policyErrorsAt` when the server sends them. + - `jev:evaluate` is a known permission — Jev through FailproofAI Cloud, charged to the org's plan — so `fp keys` and `fp users` accept it instead of refusing it as unknown, and the `admin` preset carries it, as the server's built-in admin set now does. The @@ -16,6 +30,8 @@ ### Changed +- `fp policies publish`'s refusal of Jev fields in the JavaScript now says where Jev checks go: + the policy's own declarations, `--kind both --semantic checks.json`. - `fp guardrails summary` shows how many machines run Jev in each mode, reading the summary's `jev.machines.observe` and `jev.machines.enforce`. - `fp policies publish` (and `policies compose --publish`) refuses a source carrying Jev diff --git a/fp-cloud-cli/README.md b/fp-cloud-cli/README.md index f07ec157c..af11dac65 100644 --- a/fp-cloud-cli/README.md +++ b/fp-cloud-cli/README.md @@ -139,6 +139,20 @@ fp fleet deploy ci-runner-01 --add no-force-push fp guardrails summary --since 24h ``` +A policy has a **kind**: `regex` (JavaScript — the default, and what every +earlier version is), `jev` (Jev checks only, from a JSON file of declarations — +each exactly a failproofai pack manifest's semantic entry), or `both` +(JavaScript whose verdict its own Jev checks may clear, deployed as one). +`--jev-mode` sets a machine's Jev mode from FailproofAI Cloud, overriding the +machine's own; omit it and the mode is left alone. + +```bash +fp policies publish prod-db-intent --kind jev --semantic ./checks.json +fp policies publish no-prod-db --kind both --source ./rule.mjs --semantic ./checks.json +fp fleet deploy ci-runner-01 --add prod-db-intent --jev-mode observe +fp fleet show ci-runner-01 # its Jev mode, and any policy errors it reported +``` + **A deploy REPLACES a machine's whole policy set.** The server takes the full list and does not merge, so `fleet deploy` reads what the machine currently runs, applies your `--add`/`--remove`, prints the complete resulting set, and writes diff --git a/fp-cloud-cli/fp_cli/client.py b/fp-cloud-cli/fp_cli/client.py index 7368e2f9d..4276d6cb4 100644 --- a/fp-cloud-cli/fp_cli/client.py +++ b/fp-cloud-cli/fp_cli/client.py @@ -1643,22 +1643,43 @@ def list_policies(ctx: ClientContext) -> List[PolicyVersion]: def publish_policy( - ctx: ClientContext, policy_id: str, source: str, description: str = "" + ctx: ClientContext, + policy_id: str, + source: Optional[str], + description: str = "", + *, + kind: Optional[str] = None, + semantic: Optional[List[Any]] = None, ) -> PolicyVersion: """POST /api/enforcement/policies — mints a NEW VERSION; never edits in place. - Refuses Jev fields a cloud policy never reads before anything is sent, here - because `policies publish` and `policies compose --publish` both route through - it — and regardless of `--no-verify`, which only skips the syntax check. + ``kind`` is ``regex`` (JavaScript ``source`` only — what an omitted kind + means to the server), ``jev`` (Jev declarations in ``semantic``, no + ``source``) or ``both`` (the two together: the JavaScript is reviewable by + exactly its own checks). ``authority``/``reviewedBy`` are derived by the + server from the kind and are never sent. + + Refuses Jev fields in the JAVASCRIPT before anything is sent, here because + `policies publish` and `policies compose --publish` both route through it — + and regardless of `--no-verify`, which only skips the syntax check. Jev + checks belong in ``semantic``, where they take effect. """ - problem = cloud_publish_problem(source) + problem = cloud_publish_problem(source or "") if problem: raise ApiError( f"{policy_id} cannot be published as a cloud policy: {problem}", - hint=("Jev checks ship in a failproofai pack — `failproofai publish` — where " - "semanticPolicies.add and authority: \"reviewable\" take effect"), + hint=("put Jev checks in the policy's Jev declarations — `--kind both --semantic " + "checks.json` — or ship them in a failproofai pack with `failproofai publish`"), ) - body = {"id": policy_id, "source": source, "description": description} + body: Dict[str, Any] = {"id": policy_id, "description": description} + # `source` is omitted for a Jev-only policy: the server takes absent (or + # empty) as "no JavaScript", which is what a `jev` version is. + if source: + body["source"] = source + if kind is not None: + body["kind"] = kind + if semantic is not None: + body["semantic"] = semantic return PolicyVersion.from_dict(_post_json(ctx, "/api/enforcement/policies", body) or {}) @@ -1708,11 +1729,22 @@ def get_deployment(ctx: ClientContext, machine_id: str) -> Optional[Deployment]: def deploy_policies( - ctx: ClientContext, machine_id: str, policies: Sequence[PolicyRef] + ctx: ClientContext, + machine_id: str, + policies: Sequence[PolicyRef], + jev_mode: Optional[str] = None, ) -> Deployment: - """PUT /api/enforcement/deployments/{id} — REPLACES the machine's whole set.""" + """PUT /api/enforcement/deployments/{id} — REPLACES the machine's whole set. + + ``jev_mode`` (``off|observe|enforce|local``) is sent only when given: the + server reads an ABSENT ``jevMode`` as "keep the machine's current one", so a + deploy that says nothing about Jev never changes it. ``local`` stops + FailproofAI Cloud overriding the machine's own mode. + """ path = f"/api/enforcement/deployments/{machine_id}" - body = {"policies": [p.to_dict() for p in policies]} + body: Dict[str, Any] = {"policies": [p.to_dict() for p in policies]} + if jev_mode is not None: + body["jevMode"] = jev_mode return Deployment.from_dict(_request_json(ctx, "PUT", path, json_body=body) or {}) diff --git a/fp-cloud-cli/fp_cli/commands/fleet_cmds.py b/fp-cloud-cli/fp_cli/commands/fleet_cmds.py index 9e7610fbf..8addd3f7a 100644 --- a/fp-cloud-cli/fp_cli/commands/fleet_cmds.py +++ b/fp-cloud-cli/fp_cli/commands/fleet_cmds.py @@ -32,6 +32,7 @@ disabled_ids, latest_versions, plan_deploy, + version_kinds, ) from ..errors import ApiError, NotFoundError from . import _write @@ -62,6 +63,8 @@ def fleet_list(ctx: typer.Context) -> None: `intended` is the generation deployed, `applied` is the one the machine last collected, and `seen` is when it last reported anything — a machine can be in sync and dead, or alive and behind, and those are different problems. + `state` reads `N errors` when the machine reported policies it could not + apply (`fp fleet show ` lists them). A machine appears from its very first check-in, including the poll that finds nothing deployed — that is exactly the machine you are usually looking @@ -101,10 +104,17 @@ def fleet_show( machine can be told to run a policy and not yet have it; the policy list alone cannot tell you which, and that is usually the question. + Shows the Jev mode FailproofAI Cloud sets on the machine (`local` when it + sets none) and the policy errors the machine last reported: a policy it + could not load, a Jev check it could not parse, a Jev mode it has no + provider for. Anything listed there is not enforcing, whatever the + deployment says. + Needs `policies:read`. With `--json`: `{machine, deployment}` — the machine - record (including `appliedDeployment`, `drifted`, `lastSeen` and both label - fields, with raw timestamps) and the deployment, or `deployment: null` when - nothing is deployed. + record (including `appliedDeployment`, `drifted`, `lastSeen`, both label + fields, `jevMode` and `policyErrors`/`policyErrorsAt` when reported, with raw + timestamps) and the deployment, or `deployment: null` when nothing is + deployed. Example: @@ -148,6 +158,11 @@ def fleet_deploy( False, "--create", help="Allow deploying to a machine id that has not checked in yet (pre-staging).", ), + jev_mode: Optional[str] = typer.Option( + None, "--jev-mode", + help="Set the machine's Jev mode from FailproofAI Cloud: off, observe, enforce — or local " + "to stop overriding the machine's own. Omitted: unchanged.", + ), yes: bool = typer.Option(False, "--yes", "-y", help="Skip the confirmation prompt. The prompt only appears on an interactive terminal: under --json, or with stdin redirected, this command proceeds without asking."), ) -> None: """Change what a machine enforces, showing the full resulting set first. @@ -159,6 +174,15 @@ def fleet_deploy( `--set` replaces everything — the only way to drop policies you do not name. + Any kind deploys the same way: a `jev` policy puts its Jev checks on the + machine, a `both` policy its JavaScript and its checks together. An effect + applies to JavaScript only — a `jev` policy is watched through the Jev MODE + instead: `--jev-mode observe` asks Jev and logs what it says while the regex + policies decide, `enforce` lets Jev clear what its checks review, `off` stops + Jev on the machine, and `local` hands the choice back to the machine. The + mode set here overrides the machine's own, a local `off` included; without + `--jev-mode` it is left as it is. `--jev-mode` alone is a valid deploy. + **Concurrency.** The write is a full replace with no server-side lock, so the CLI records the generation it read and refuses if the result is not exactly one higher: that means somebody else deployed in between and a replace does @@ -170,17 +194,19 @@ def fleet_deploy( * `fp fleet deploy ci-runner-01 --add no-force-push` * `fp fleet deploy ci-runner-01 --add prod-guard@1:observe --remove old-rule` * `fp fleet deploy ci-runner-01 --set no-force-push --set no-secret-echo` + * `fp fleet deploy ci-runner-01 --add prod-db-intent --jev-mode observe` + * `fp fleet deploy ci-runner-01 --jev-mode local` """ state: AppState = ctx.obj deny_in_key_mode(state, "fleet deploy", _KEY_MODE_REASON) cctx = require_auth(state) - if not add and not remove and replace is None: + if not add and not remove and replace is None and jev_mode is None: # Exit 2 for the same reason `--set` with `--add` is: no flag # combination was given that this command can act on. Both are the # caller's command line, not the server's answer. raise click.UsageError( - "nothing to do — pass --add, --remove, or --set. " + "nothing to do — pass --add, --remove, --set, or --jev-mode. " "`fp fleet show ` prints the current set." ) @@ -212,6 +238,9 @@ def fleet_deploy( replace=replace, latest=latest, disabled=disabled_ids(published), + jev_mode=jev_mode, + current_jev_mode=current.jev_mode if current else None, + kinds=version_kinds(published), ) except RefUsageError as exc: # Exit 2, like every other bad flag value in this CLI (`--since`, @@ -252,7 +281,7 @@ def fleet_deploy( output.print_cancelled() return - result = api.deploy_policies(cctx, machine_id, plan.result) + result = api.deploy_policies(cctx, machine_id, plan.result, jev_mode=jev_mode) check_race(plan.base, result.deployment) if output.is_json(): diff --git a/fp-cloud-cli/fp_cli/commands/policies_cmds.py b/fp-cloud-cli/fp_cli/commands/policies_cmds.py index bc13b4161..c49bb5778 100644 --- a/fp-cloud-cli/fp_cli/commands/policies_cmds.py +++ b/fp-cloud-cli/fp_cli/commands/policies_cmds.py @@ -10,6 +10,7 @@ """ from __future__ import annotations +import json from typing import Optional import typer @@ -18,8 +19,8 @@ from .. import output, theme from .._context import GLOBALS_EPILOG, AppState, deny_in_key_mode, require_auth from .. import _click_compat as click # the Click Typer is running; see _click_compat -from ..enforcement import RefError, RefUsageError, read_source -from ..policy_check import check_syntax, run_policy +from ..enforcement import RefError, RefUsageError, parse_semantic, read_source, resolve_kind +from ..policy_check import SyntaxResult, check_syntax, run_policy from ..errors import ApiError, NotFoundError from . import _write @@ -36,11 +37,15 @@ def policies_list(ctx: typer.Context) -> None: """List every published policy version, newest of each policy first. - Shows `policy · version · state · description`, one row per VERSION — + Shows `policy · version · kind · state · description`, one row per VERSION — versions are immutable and every one stays addressable, so a policy published three times is three rows. The title counts distinct policies and captions the version total, the way the dashboard's library does. + `kind` is `regex` (JavaScript), `jev` (Jev checks only) or `both` + (JavaScript its own Jev checks may clear); a `jev`/`both` row names its + checks. + `state` is active, disabled (kept but not enforced) or archived (deleted; machines already carrying it keep it until redeployed). Needs `policies:read`. With `--json`: the server's policy list verbatim — also @@ -64,9 +69,11 @@ def policies_show( ctx: typer.Context, policy_id: str = typer.Argument(..., help="Policy id."), ) -> None: - """Show one policy, including its full source. + """Show one policy, including its full source and its Jev checks. - Needs `policies:read`. With `--json`: the policy object with `source`. + Needs `policies:read`. With `--json`: the policy object with `source`, + `kind`, and for `jev`/`both` its `semantic` declarations, `semanticSha256` + and the server-derived `authority`/`reviewedBy`. Example: @@ -96,6 +103,8 @@ def policies_show( source_bytes=len((match.source or "").encode("utf-8"))) if match.source: output.info(match.source) + if match.semantic: + output.info(json.dumps(match.semantic, indent=2)) def policies_publish( @@ -103,7 +112,19 @@ def policies_publish( policy_id: str = typer.Argument(..., help="Policy id (letters, numbers, '.', '_', '-')."), source: Optional[str] = typer.Argument( None, - help="Path to the policy source, @path, or - for stdin. Omit to paste it.", + help="Path to the JavaScript source, @path, or - for stdin. Omit to paste it. Same as --source.", + ), + kind: Optional[str] = typer.Option( + None, "--kind", + help="regex (JavaScript), jev (Jev checks only) or both (JavaScript its own Jev checks may " + "clear). Default: follows from what you give — --semantic alone is jev, with source both.", + ), + source_opt: Optional[str] = typer.Option( + None, "--source", help="The JavaScript source: a path, @path, or - for stdin.", + ), + semantic_path: Optional[str] = typer.Option( + None, "--semantic", + help="A JSON file holding the Jev declarations (a JSON array): a path, @path, or - for stdin.", ), description: str = typer.Option("", "--description", help="One-line description."), no_verify: bool = typer.Option( @@ -112,13 +133,25 @@ def policies_publish( ) -> None: """Publish a policy — mints a NEW VERSION; it never edits one in place. - The source is parse-checked with node before it is sent. Nothing downstream - does this: the server validates the id and a size ceiling, and a broken - policy otherwise fails on the machine at enforcement time. `--no-verify` - skips it; a host without node publishes with a warning rather than a block. - Jev fields a cloud policy never reads (`semanticPolicies.add`, `authority: - "reviewable"`) are refused regardless — ship those in a pack with - `failproofai publish`. + A policy has a **kind**: + + * `regex` — JavaScript (`--source`, or the positional source): what publish + has always done, and the default. + * `jev` — Jev checks only (`--semantic checks.json`): a JSON array of Jev + declarations, each `{name, title, appliesTo, mode, userCanOverride, + probes:[{id, instructions, criteria:{true,false}}], guidance, ...}` — + exactly a failproofai pack manifest's semantic entry. No JavaScript. + * `both` — the two together. FailproofAI Cloud makes the JavaScript's + verdict reviewable by exactly its own checks, and deploys them as one. + + The JavaScript is parse-checked with node before it is sent. Nothing + downstream does this: the server validates the id and a size ceiling, and a + broken policy otherwise fails on the machine at enforcement time. + `--no-verify` skips it; a host without node publishes with a warning rather + than a block. Jev fields in the JavaScript (`semanticPolicies.add`, + `authority: "reviewable"`) are refused regardless — put Jev checks in + `--semantic`. The server checks each declaration (limits, reserved + FailproofAI names, names unique in the org) and says which one it refused. Source can come from a path, `@path`, a pipe, `-`, or an interactive paste when you give none and stdin is a terminal. @@ -134,21 +167,42 @@ def policies_publish( * `fp policies publish no-force-push ./rule.mjs` * `cat rule.mjs | fp policies publish no-force-push` * `fp policies publish no-force-push -` — read stdin explicitly + * `fp policies publish prod-db-intent --kind jev --semantic ./checks.json` + * `fp policies publish no-prod-db --kind both --source ./rule.mjs --semantic ./checks.json` """ state: AppState = ctx.obj deny_in_key_mode(state, "policies publish", _KEY_MODE_REASON) cctx = require_auth(state) + if source is not None and source_opt is not None: + raise click.UsageError("give the JavaScript source once — positionally or with --source, not both") + source = source if source is not None else source_opt + if source == "-" and semantic_path == "-": + raise click.UsageError("only one of the JavaScript source and --semantic can come from stdin") + + # Whether JavaScript is coming is decided WITHOUT reading stdin: a Jev-only + # publish must not sit waiting for a paste nobody is going to give. + wants_source = source is not None or kind in ("regex", "both") or (kind is None and semantic_path is None) + try: + resolved_kind = resolve_kind(kind, has_source=wants_source, has_semantic=semantic_path is not None) + except RefUsageError as exc: + raise click.UsageError(str(exc)) + def _paste_prompt() -> None: output.hint("paste the policy source, then press Ctrl-D") + text: Optional[str] = None + semantic = None try: - text = read_source(source, prompt=_paste_prompt) + if semantic_path is not None: + semantic = parse_semantic(read_source(semantic_path, isatty=False)) + if wants_source: + text = read_source(source, prompt=_paste_prompt) except RefUsageError as exc: raise click.UsageError(str(exc)) except RefError as exc: raise ApiError(str(exc)) - if not text.strip(): + if wants_source and not (text or "").strip(): raise ApiError("policy source is empty — nothing to publish") # Nothing downstream parses this. The server checks the id and a size @@ -156,7 +210,9 @@ def _paste_prompt() -> None: # place for a syntax error to surface. `--no-verify` exists because a # machine without node should still be able to publish, not because # skipping is ever a good idea. - if not no_verify: + syn = SyntaxResult(ok=True, checked=False, + message="not checked" if text else "no JavaScript source") + if text and not no_verify: syn = check_syntax(text) if not syn.ok: raise ApiError( @@ -167,7 +223,13 @@ def _paste_prompt() -> None: if not syn.checked and not output.is_json(): output.warn(syn.message) - created = api.publish_policy(cctx, policy_id, text, description) + created = api.publish_policy( + cctx, policy_id, text, description, + # Sent only when it says something the server's default does not, so a + # plain JavaScript publish is the same request it has always been. + kind=None if resolved_kind == "regex" and kind is None else resolved_kind, + semantic=semantic, + ) # Which machines already carry this policy, and at which version. Publishing # deploys nothing, so this is the one thing the card must not guess at: it @@ -191,7 +253,7 @@ def _paste_prompt() -> None: output.emit_json({**created.to_dict(), "carriers": carriers, "syntax": syn.to_dict()}) return output.render_policy_published(created, carriers=carriers, - source_bytes=len(text.encode("utf-8"))) + source_bytes=len((text or "").encode("utf-8"))) def policies_enable( diff --git a/fp-cloud-cli/fp_cli/enforcement.py b/fp-cloud-cli/fp_cli/enforcement.py index 44a89fd85..3d60bde4d 100644 --- a/fp-cloud-cli/fp_cli/enforcement.py +++ b/fp-cloud-cli/fp_cli/enforcement.py @@ -28,16 +28,28 @@ """ from __future__ import annotations +import json import re import sys from dataclasses import dataclass -from typing import Dict, Iterable, List, Optional, Sequence, Tuple +from typing import Any, Dict, Iterable, List, Optional, Sequence, Tuple from .errors import ApiError from .models import PolicyRef, PolicyVersion VALID_EFFECTS = ("enforce", "observe") +#: What a policy version carries: JavaScript (`regex`), Jev checks (`jev`), or +#: JavaScript reviewable by its own Jev checks (`both`). +VALID_KINDS = ("regex", "jev", "both") + +#: The Jev modes a deployment may set. `local` stops FailproofAI Cloud +#: overriding the machine's own mode (the server stores it as no value). +JEV_MODES = ("off", "observe", "enforce", "local") + +#: The server's (and the machine's) cap on Jev declarations per policy version. +MAX_JEV_DECLARATIONS = 24 + #: `id`, `id@3`, `id:observe`, `id@3:observe`. The id charset mirrors the #: server's `safe_identifier`, so a ref this accepts is one the server will too #: — a rejection should come from the policy not existing, not from parsing. @@ -102,13 +114,29 @@ class DeployPlan: removed: List[PolicyRef] changed: List[Tuple[PolicyRef, PolicyRef]] unchanged: List[PolicyRef] + #: `--jev-mode` as given (`off|observe|enforce|local`), or None when the + #: command said nothing about Jev — which the server reads as "keep it". + jev_mode: Optional[str] = None + #: The mode FailproofAI Cloud sets on the machine now (None = local). + jev_mode_before: Optional[str] = None + + @property + def jev_mode_after(self) -> Optional[str]: + """What the machine's Cloud-set mode will be (None = local).""" + if self.jev_mode is None: + return self.jev_mode_before + return None if self.jev_mode == "local" else self.jev_mode + + @property + def jev_mode_changes(self) -> bool: + return self.jev_mode is not None and self.jev_mode_after != self.jev_mode_before @property def is_noop(self) -> bool: - return not (self.added or self.removed or self.changed) + return not (self.added or self.removed or self.changed or self.jev_mode_changes) def to_dict(self) -> Dict[str, object]: - return { + out: Dict[str, object] = { "machineId": self.machine_id, "base": self.base, "result": [p.to_dict() for p in self.result], @@ -118,6 +146,14 @@ def to_dict(self) -> Dict[str, object]: "unchanged": [p.to_dict() for p in self.unchanged], "noop": self.is_noop, } + if self.jev_mode is not None: + # `local` on both sides: the wire word, not the stored NULL. + out["jevMode"] = { + "from": self.jev_mode_before or "local", + "to": self.jev_mode_after or "local", + "changed": self.jev_mode_changes, + } + return out def latest_versions(policies: Iterable[PolicyVersion]) -> Dict[str, int]: @@ -187,13 +223,26 @@ def plan_deploy( replace: Optional[Sequence[str]] = None, latest: Optional[Dict[str, int]] = None, disabled: Optional[set] = None, + jev_mode: Optional[str] = None, + current_jev_mode: Optional[str] = None, + kinds: Optional[Dict[Tuple[str, int], str]] = None, ) -> DeployPlan: """Compute the full resulting set, plus the diff to show before writing. `replace` (`--set`) is exclusive with `add`/`remove`: mixing "these exactly" with "these as well" has no single obvious reading, and guessing one would be guessing about somebody's fleet. + + `jev_mode` is `--jev-mode`; `current_jev_mode` what Cloud sets now (None = + local). `kinds` maps `(id, version)` to the version's kind, so an `observe` + effect on a Jev-only version — which the server refuses, because a Jev + rollout is observed through the Jev MODE — is refused here, before a plan is + drawn and confirmed for a write that cannot happen. """ + if jev_mode is not None and jev_mode not in JEV_MODES: + raise RefUsageError( + f"--jev-mode {jev_mode!r} is not one of {', '.join(JEV_MODES)}" + ) latest = latest or {} current_list = list(current or []) current_map = {p.id: p for p in current_list} @@ -221,6 +270,13 @@ def plan_deploy( result_map[ref.id] = ref result = sorted(result_map.values(), key=lambda p: p.id) + for ref in result: + if ref.effect == "observe" and (kinds or {}).get((ref.id, ref.version)) == "jev": + raise RefUsageError( + f"{ref.id}@{ref.version} is a Jev policy, and an effect does not apply to Jev " + "checks — deploy it as enforce and use --jev-mode observe to watch Jev without " + "it deciding anything" + ) added, removed, changed, unchanged = [], [], [], [] for pid, ref in sorted(result_map.items()): was = current_map.get(pid) @@ -242,9 +298,68 @@ def plan_deploy( removed=removed, changed=changed, unchanged=unchanged, + jev_mode=jev_mode, + jev_mode_before=current_jev_mode, ) +def version_kinds(policies: Iterable[PolicyVersion]) -> Dict[Tuple[str, int], str]: + """`{(id, version): kind}` for every published version.""" + return {(p.id, p.version): p.kind for p in policies} + + +def resolve_kind(kind: Optional[str], *, has_source: bool, has_semantic: bool) -> str: + """The kind a publish is, checked against what it was given. + + Explicit wins; otherwise it follows from the inputs — JavaScript alone is + `regex` (what `publish` has always done), declarations alone `jev`, both + together `both`. A kind that contradicts its inputs is a usage error: a + `jev` policy has no JavaScript, and `regex` has no declarations. + """ + if kind is None: + kind = "both" if (has_source and has_semantic) else ("jev" if has_semantic else "regex") + if kind not in VALID_KINDS: + raise RefUsageError(f"--kind {kind!r} is not one of {', '.join(VALID_KINDS)}") + if kind == "regex" and has_semantic: + raise RefUsageError("--kind regex takes no --semantic; use --kind both for JavaScript plus Jev checks") + if kind == "jev" and has_source: + raise RefUsageError("--kind jev takes no JavaScript source; use --kind both for JavaScript plus Jev checks") + if kind in ("jev", "both") and not has_semantic: + raise RefUsageError(f"--kind {kind} needs --semantic with its Jev declarations") + return kind + + +def parse_semantic(text: str, *, what: str = "--semantic") -> List[Any]: + """The Jev declarations in a `--semantic` file: a JSON array of declaration + objects (a `{"semantic": [...]}` wrapper is accepted too). + + Deliberately a SHAPE check only — 1 to 24 objects, each with a name. What a + declaration may say (modes, probes, caps, reserved names) is checked by the + server, and authoritatively by each machine's own parser; a third copy of + those rules here is a third thing to drift. The server's refusal names the + declaration and the rule. + """ + try: + value = json.loads(text) + except ValueError as exc: + raise RefUsageError(f"{what} is not valid JSON: {exc}") + if isinstance(value, dict) and isinstance(value.get("semantic"), list): + value = value["semantic"] + if not isinstance(value, list): + raise RefUsageError(f"{what} must be a JSON array of Jev declarations") + if not value: + raise RefUsageError(f"{what} holds no Jev declarations") + if len(value) > MAX_JEV_DECLARATIONS: + raise RefUsageError( + f"{what} holds {len(value)} Jev declarations; a policy version carries at most " + f"{MAX_JEV_DECLARATIONS}" + ) + for i, entry in enumerate(value): + if not isinstance(entry, dict) or not isinstance(entry.get("name"), str) or not entry["name"]: + raise RefUsageError(f"{what} declaration #{i} is not an object with a name") + return value + + def check_race(base: Optional[int], returned: int) -> None: """Raise when a deploy landed on top of somebody else's. diff --git a/fp-cloud-cli/fp_cli/models.py b/fp-cloud-cli/fp_cli/models.py index c78ee8453..3caede0e0 100644 --- a/fp-cloud-cli/fp_cli/models.py +++ b/fp-cloud-cli/fp_cli/models.py @@ -724,7 +724,15 @@ def from_dict(cls, d: Dict[str, Any]) -> "AuditFinding": @dataclass class PolicyVersion: - """One published version of a policy. Versions are minted, never edited.""" + """One published version of a policy. Versions are minted, never edited. + + ``kind`` is what the version carries: ``regex`` (JavaScript only — every + version published before kinds existed), ``jev`` (Jev checks only, no + JavaScript) or ``both`` (JavaScript whose verdict its own Jev checks may + clear). ``semantic`` is the list of Jev declarations for ``jev``/``both``. + ``authority``/``reviewed_by`` are DERIVED by the server from the kind and + never sent by a client. + """ id: str version: int @@ -735,9 +743,16 @@ class PolicyVersion: created_by: Optional[str] disabled: bool archived: bool + kind: str = "regex" + semantic: Optional[List[Any]] = None + semantic_sha256: Optional[str] = None + authority: Optional[str] = None + reviewed_by: Optional[List[str]] = None @classmethod def from_dict(cls, d: Dict[str, Any]) -> "PolicyVersion": + semantic = d.get("semantic") + reviewed = d.get("reviewedBy", d.get("reviewed_by")) return cls( id=str(d.get("id", "")), version=_as_int(d.get("version"), 0), @@ -748,18 +763,43 @@ def from_dict(cls, d: Dict[str, Any]) -> "PolicyVersion": created_by=d.get("createdBy", d.get("created_by")), disabled=bool(d.get("disabled", False)), archived=bool(d.get("archived", False)), + # A server that predates kinds sends none, and everything it has is + # JavaScript — `regex` is not a guess, it is the only thing it had. + kind=str(d.get("kind") or "regex"), + semantic=semantic if isinstance(semantic, list) else None, + semantic_sha256=d.get("semanticSha256", d.get("semantic_sha256")), + authority=d.get("authority"), + reviewed_by=[str(n) for n in reviewed] if isinstance(reviewed, list) else None, ) + @property + def jev_names(self) -> List[str]: + """The names of this version's Jev checks, in declared order.""" + return [ + str(entry["name"]) + for entry in (self.semantic or []) + if isinstance(entry, dict) and isinstance(entry.get("name"), str) + ] + def to_dict(self) -> Dict[str, Any]: """The server's own shape. `vars()` would leak Python snake_case into a contract that is camelCase everywhere else, which is a difference a harness discovers at runtime rather than in review.""" - return { + out: Dict[str, Any] = { "id": self.id, "version": self.version, "description": self.description, "sha256": self.sha256, "source": self.source, "createdAt": self.created_at, "createdBy": self.created_by, "disabled": self.disabled, - "archived": self.archived, + "archived": self.archived, "kind": self.kind, } + if self.semantic is not None: + out["semantic"] = self.semantic + if self.semantic_sha256: + out["semanticSha256"] = self.semantic_sha256 + if self.authority: + out["authority"] = self.authority + if self.reviewed_by is not None: + out["reviewedBy"] = self.reviewed_by + return out @dataclass @@ -806,6 +846,9 @@ class Deployment: policies: List[PolicyRef] updated_at: str updated_by: Optional[str] + #: The Jev mode FailproofAI Cloud sets on this machine (`off|observe|enforce`), + #: or None when Cloud does not override the machine's own (`local`). + jev_mode: Optional[str] = None @classmethod def from_dict(cls, d: Dict[str, Any]) -> "Deployment": @@ -815,14 +858,18 @@ def from_dict(cls, d: Dict[str, Any]) -> "Deployment": policies=[PolicyRef.from_dict(p) for p in (d.get("policies") or [])], updated_at=str(d.get("updatedAt", d.get("updated_at", "")) or ""), updated_by=d.get("updatedBy", d.get("updated_by")), + jev_mode=d.get("jevMode", d.get("jev_mode")) or None, ) def to_dict(self) -> Dict[str, Any]: - return { + out: Dict[str, Any] = { "machineId": self.machine_id, "deployment": self.deployment, "policies": [p.to_dict() for p in self.policies], "updatedAt": self.updated_at, "updatedBy": self.updated_by, } + if self.jev_mode: + out["jevMode"] = self.jev_mode + return out @dataclass @@ -851,6 +898,13 @@ class Machine: deployed: bool policy_count: int event_count: int + #: What the machine last reported it could not apply — `[{id, version, kind, + #: message}]` — or None when it has never reported (an empty list means it + #: reported and all is well). + policy_errors: Optional[List[Dict[str, Any]]] = None + policy_errors_at: Optional[Any] = None + #: The Jev mode FailproofAI Cloud sets on this machine, or None (`local`). + jev_mode: Optional[str] = None @classmethod def from_dict(cls, d: Dict[str, Any]) -> "Machine": @@ -858,6 +912,8 @@ def _num(key: str) -> Optional[int]: v = d.get(key) return int(v) if isinstance(v, (int, float)) else None + errors = d.get("policyErrors", d.get("policy_errors")) + return cls( machine_id=str(d.get("machineId", d.get("machine_id", "")) or ""), label=d.get("label"), @@ -870,6 +926,9 @@ def _num(key: str) -> Optional[int]: deployed=bool(d.get("deployed", False)), policy_count=_as_int(d.get("policyCount"), 0), event_count=_as_int(d.get("eventCount"), 0), + policy_errors=[e for e in errors if isinstance(e, dict)] if isinstance(errors, list) else None, + policy_errors_at=d.get("policyErrorsAt", d.get("policy_errors_at")), + jev_mode=d.get("jevMode", d.get("jev_mode")) or None, ) @property @@ -892,6 +951,11 @@ def to_dict(self) -> Dict[str, Any]: "appliedAt": self.applied_at, "deployed": self.deployed, "policyCount": self.policy_count, "eventCount": self.event_count, "drifted": self.drifted, + # Emitted only when the server sent them, so a server that predates + # them reads exactly as it always did. + **({"policyErrors": self.policy_errors} if self.policy_errors is not None else {}), + **({"policyErrorsAt": self.policy_errors_at} if self.policy_errors_at is not None else {}), + **({"jevMode": self.jev_mode} if self.jev_mode else {}), } @property diff --git a/fp-cloud-cli/fp_cli/output.py b/fp-cloud-cli/fp_cli/output.py index 05e6f377b..b29c25283 100644 --- a/fp-cloud-cli/fp_cli/output.py +++ b/fp-cloud-cli/fp_cli/output.py @@ -5918,6 +5918,10 @@ def _policy_cell(ref: Any) -> Text: return t +#: A policy version's kind, as `fp policies list` colours it. +_KIND_STYLE = {"regex": theme.TEXT_DIM, "jev": theme.ACCENT, "both": theme.AMBER} + + def render_policies(items: Sequence[Any]) -> None: """``fp policies`` — every published VERSION, newest of each policy first. @@ -5938,11 +5942,22 @@ def render_policies(items: Sequence[Any]) -> None: state = Text("archived", style=theme.FAINT) elif p.disabled: state = Text("disabled", style=theme.AMBER) + kind = getattr(p, "kind", None) or "regex" + names = list(getattr(p, "jev_names", None) or []) + desc = Text() + if names: + # The checks first: for a `jev` policy they ARE the policy, and a + # long description would otherwise push them off the ellipsis. + desc.append(", ".join(names), style=theme.ACCENT) + if p.description: + desc.append(" · ", style=theme.FAINT) + desc.append(p.description or "", style=theme.TEXT_DIM) rows.append([ Text(p.id, style=theme.TEXT), Text(f"v{p.version}", style=theme.TEXT_DIM), + Text(kind, style=_KIND_STYLE.get(kind, theme.TEXT_DIM)), state, - Text(p.description or "", style=theme.TEXT_DIM), + desc, ]) distinct = len({p.id for p in items}) title = Text() @@ -5952,7 +5967,7 @@ def render_policies(items: Sequence[Any]) -> None: if len(rows) != distinct: title.append(" · ", style=theme.FAINT) title.append(f"{len(rows)} versions", style=theme.LABEL) - render_list_panel("policies", header=["policy", "version", "state", "description"], + render_list_panel("policies", header=["policy", "version", "kind", "state", "description"], rows=rows, days=set(), order=None, empty_message="no policies published — `fp policies publish `", last_col="ellipsis", title=title) @@ -5974,17 +5989,35 @@ def render_policy_published(p: Any, *, carriers: Optional[dict] = None, """ line1 = Text(p.id, style=f"bold {theme.TEXT}") line1.append(f" v{p.version}", style=f"bold {theme.ACCENT}") + kind = getattr(p, "kind", None) or "regex" + line1.append(f" {kind}", style=_KIND_STYLE.get(kind, theme.TEXT_DIM)) body = [line1] if p.description: body.append(Text(p.description, style=theme.TEXT)) + names = list(getattr(p, "jev_names", None) or []) + if names: + checks = Text("jev checks ", style=theme.LABEL) + checks.append(", ".join(names), style=theme.ACCENT) + body.append(checks) + if kind == "both": + # Derived by the server from the kind, never sent: said so the author + # knows what the JavaScript's verdict now is. + body.append(Text("its JavaScript is reviewable by exactly these checks", style=theme.LABEL)) body.append(Text()) meta = Text() if source_bytes: meta.append(f"{source_bytes:,} bytes", style=theme.LABEL) meta.append(" · ", style=theme.FAINT) - meta.append("sha256 ", style=theme.LABEL) - meta.append((p.sha256 or "")[:12] + "…", style=theme.TEXT_DIM) + if p.sha256: + meta.append("sha256 ", style=theme.LABEL) + meta.append((p.sha256 or "")[:12] + "…", style=theme.TEXT_DIM) + semantic_sha = getattr(p, "semantic_sha256", None) + if semantic_sha: + if p.sha256: + meta.append(" · ", style=theme.FAINT) + meta.append("jev sha256 ", style=theme.LABEL) + meta.append(semantic_sha[:12] + "…", style=theme.TEXT_DIM) body.append(meta) body.append(Text()) @@ -6053,8 +6086,7 @@ def render_fleet(machines: Sequence[Any]) -> None: Text(seen or "never", style=theme.FAINT if stale else theme.TEXT_DIM), Text(f"{m.event_count:,}" if m.event_count else "—", style=theme.TEXT_DIM if m.event_count else theme.FAINT), - Text("drifted" if m.drifted else ("ok" if m.deployed else "—"), - style=theme.AMBER if m.drifted else (theme.SUCCESS if m.deployed else theme.FAINT)), + _machine_state(m), ]) title = Text() title.append("fleet", style=f"bold {theme.ACCENT}") @@ -6067,6 +6099,18 @@ def render_fleet(machines: Sequence[Any]) -> None: empty_message="no machines have checked in yet", title=title) +def _machine_state(m: Any) -> Text: + """`fp fleet list`'s state cell. Reported policy errors outrank drift: a + machine that says it cannot apply part of its set is not enforcing it, + whichever generation it is on.""" + errors = getattr(m, "policy_errors", None) or [] + if errors: + n = len(errors) + return Text(f"{n} error{'s' if n != 1 else ''}", style=theme.ERROR) + return Text("drifted" if m.drifted else ("ok" if m.deployed else "—"), + style=theme.AMBER if m.drifted else (theme.SUCCESS if m.deployed else theme.FAINT)) + + def _compact_age(ms: Optional[int]) -> str: """`5h`, `7d`, `just now` — the column form of `_epoch_age`. @@ -6148,6 +6192,27 @@ def field(label: str, value: Text) -> None: act.append(f" · {machine.event_count} events", style=theme.LABEL) field("last seen", act) + mode = (getattr(dep, "jev_mode", None) if dep is not None else None) or ( + getattr(machine, "jev_mode", None) if machine is not None else None + ) + jev = Text(mode or "local", style=theme.ACCENT if mode else theme.TEXT_DIM) + jev.append(" · set by FailproofAI Cloud" if mode else " · the machine's own", style=theme.LABEL) + field("jev mode", jev) + + errors = (getattr(machine, "policy_errors", None) or []) if machine is not None else [] + if errors: + body.append(Text()) + head = Text(f"{len(errors)} policy error{'s' if len(errors) != 1 else ''}", style=f"bold {theme.ERROR}") + head.append(" · reported by the machine — these are not enforcing", style=theme.LABEL) + body.append(head) + for e in errors: + line = Text(" ") + ver = e.get("version") + line.append(f"{e.get('id', '?')}{f' v{ver}' if ver is not None else ''}", style=theme.TEXT) + line.append(f" {e.get('kind', '')}", style=theme.FAINT) + line.append(f" {e.get('message', '')}", style=theme.TEXT_DIM) + body.append(line) + body.append(Text()) pols = sorted(dep.policies, key=lambda x: x.id) if dep is not None else [] if pols: @@ -6204,6 +6269,16 @@ def render_deploy_plan(plan: Any, *, applied: bool = False) -> None: footer.append(f"polic{'y' if n == 1 else 'ies'} after this change", style=theme.LABEL) lines.append(Text()) lines.append(footer) + if getattr(plan, "jev_mode", None) is not None: + before = plan.jev_mode_before or "local" + after = plan.jev_mode_after or "local" + jev = Text(" jev mode ", style=theme.LABEL) + if plan.jev_mode_changes: + jev.append(f"{before} → ", style=theme.FAINT) + jev.append(after, style=f"bold {theme.ACCENT}") + else: + jev.append(f"{after} (unchanged)", style=theme.TEXT_DIM) + lines.append(jev) head = Text(plan.machine_id, style=f"bold {theme.TEXT}") if plan.base is not None: diff --git a/fp-cloud-cli/fp_cli/policy_check.py b/fp-cloud-cli/fp_cli/policy_check.py index d4ae90efb..bde1ab0c4 100644 --- a/fp-cloud-cli/fp_cli/policy_check.py +++ b/fp-cloud-cli/fp_cli/policy_check.py @@ -446,14 +446,17 @@ def cloud_publish_problem(source: str) -> Optional[str]: if not source or len(source) > _MAX_SCAN: return None if _find_call(source, _SEMANTIC_CALL) != -1: - return ("semanticPolicies.add does nothing in a cloud policy: Jev reads semantic checks only " - "from a failproofai pack. Remove it to publish the rest, or ship the file with " - "`failproofai publish`.") + return ("semanticPolicies.add does nothing in a cloud policy's JavaScript: Jev reads semantic " + "checks only from a failproofai pack or from the policy's own Jev declarations. Put the " + "check in a JSON file and publish with `--kind both --semantic checks.json` (or `--kind " + "jev`), or ship the file with `failproofai publish`.") # Every registration: any of them can carry a reviewable declaration. at = _find_call(source, _ADD_CALL) while at != -1: if _direct_props(source, at).get("authority") == "reviewable": - return ("a cloud policy is always hard, so Jev never clears it and authority/reviewedBy " - "are ignored. Remove them to publish it as a hard policy.") + return ("a cloud policy's JavaScript is always hard as written: its authority comes from " + "its kind, never from the source, so authority/reviewedBy here are ignored. Remove " + "them, and publish with `--kind both --semantic checks.json` to make it reviewable " + "by its own Jev checks.") at = _find_call(source, _ADD_CALL, at + len(_ADD)) return None diff --git a/fp-cloud-cli/skill/SKILL.md b/fp-cloud-cli/skill/SKILL.md index 1cb5392fd..f0fcc683a 100644 --- a/fp-cloud-cli/skill/SKILL.md +++ b/fp-cloud-cli/skill/SKILL.md @@ -196,8 +196,8 @@ you need a flag you don't already know. - `audits list|show|create|edit|delete|run|runs` — scheduled sweeps, referenced by **name**; `audits findings|finding` + the triage verbs `ack|mute|dismiss|resolve|reopen|assign` act on a finding **id**. `audits run ` only *queues* a run (poll `audits runs ` for completion). See §8. **Enforce (cloud-managed policy, session-only — see §2):** -- `policies list|show|publish|enable|disable|delete|test|compose` — policy versions. `publish` mints a version from a local `.mjs`; `test` runs one against a synthetic context locally (it applies each policy's `match` filter, so a policy that does not cover the `--event`/`--tool` you pass is reported `skipped`, not run). `enable`/`disable`/`delete` take `--yes`. -- `fleet list|show|deploy|diff|history|rollback|rename` — which machines run which policies. **`deploy` REPLACES a machine's whole set** (`--add`/`--remove` amend it, `--set` replaces, `--create` mints a deployment); it prints the plan and asks **only on an interactive terminal without `--json`** — under `--json` or with stdin redirected it applies immediately, so read `fleet show` first if you want review. +- `policies list|show|publish|enable|disable|delete|test|compose` — policy versions. `publish` mints a version of a kind — `regex` (a local `.mjs`), `jev` (`--semantic checks.json`: Jev declarations, no JavaScript) or `both` (the two, the JavaScript reviewable by its own checks); list/show print the kind and the Jev check names; `test` runs one against a synthetic context locally (it applies each policy's `match` filter, so a policy that does not cover the `--event`/`--tool` you pass is reported `skipped`, not run). `enable`/`disable`/`delete` take `--yes`. +- `fleet list|show|deploy|diff|history|rollback|rename` — which machines run which policies, and the policy errors each machine reports. **`deploy` REPLACES a machine's whole set** (`--add`/`--remove` amend it, `--set` replaces, `--create` mints a deployment, `--jev-mode off|observe|enforce|local` sets the machine's Jev mode from FailproofAI Cloud — overriding its own, and left alone when omitted); it prints the plan and asks **only on an interactive terminal without `--json`** — under `--json` or with stdin redirected it applies immediately, so read `fleet show` first if you want review. - `guardrails summary|timeline` — what enforcement actually did; `--since 1h|6h|24h|7d`, `--machine`. **Analytics & assistant:** diff --git a/fp-cloud-cli/skill/references/commands.md b/fp-cloud-cli/skill/references/commands.md index b498bc9ac..dd0d966b7 100644 --- a/fp-cloud-cli/skill/references/commands.md +++ b/fp-cloud-cli/skill/references/commands.md @@ -220,22 +220,29 @@ Built-in assistant. Chats referenced by a **short chat-id** (first 8 hex; prefix Cloud-managed enforcement, split the way the dashboard splits it: `policies` writes a version, `fleet` decides which machines run it, `guardrails` reports what it blocked. Needs `policies:read` to read, `policies:write` to change anything. ### policies -- `policies list` — one row per published VERSION (versions are immutable and all stay addressable), newest of each policy first; the title counts distinct policies and captions the version total. `state` is active / disabled / archived. JSON `{policies:[…]}` — also every version, so deduplicate on `id` for one row per policy. -- `policies show ` — the NEWEST version, including the full `source`. -- `policies publish [SOURCE] [--description "…"] [--no-verify]` — mints a **new version**; never edits one. SOURCE is a path, `@path`, `-`, a pipe, or omitted to paste on a TTY (Ctrl-D ends). The source is **parse-checked with node** before it is sent; nothing downstream does this, and a broken policy otherwise fails on the machine at enforcement time. `--no-verify` skips it, and a host without node publishes with a warning rather than a block. A source with Jev fields a cloud policy never reads — `semanticPolicies.add(...)`, or `authority: "reviewable"` on a registration — is refused before sending (**exit 1**, not skipped by `--no-verify`; commented-out code does not count): Jev checks ship in a pack via `failproofai publish`. **Publishing deploys nothing** — the version is unused until `fleet deploy` puts it on a machine. +- `policies list` — one row per published VERSION (versions are immutable and all stay addressable), newest of each policy first; the title counts distinct policies and captions the version total. `kind` is `regex` / `jev` / `both`, and a `jev`/`both` row names its Jev checks before the description. `state` is active / disabled / archived. JSON `{policies:[…]}` — also every version, so deduplicate on `id` for one row per policy; each carries `kind`, and for `jev`/`both` `semantic`, `semanticSha256`, and the server-derived `authority`/`reviewedBy`. +- `policies show ` — the NEWEST version: its kind, its Jev check names, the full `source`, and the Jev declarations. +- `policies publish [SOURCE] [--kind regex|jev|both] [--source FILE] [--semantic FILE] [--description "…"] [--no-verify]` — mints a **new version**; never edits one. A policy's **kind** says what it carries: + - `regex` — JavaScript only (SOURCE or `--source`): what publish has always done, and the default. + - `jev` — Jev checks only (`--semantic checks.json`, no JavaScript): a JSON array (or `{"semantic": [...]}`) of 1–24 Jev declarations, each exactly a failproofai pack manifest's semantic entry — `{name, title, appliesTo:[shell|write|read|network|other], mode:deny|instruct, userCanOverride, probes:[{id, instructions, criteria:{true,false}}] (≤6), guidance, precondition?, exempt?}`. + - `both` — JavaScript plus its own Jev checks, deployed and toggled as one. FailproofAI Cloud makes the JavaScript's verdict `reviewable` by exactly its own checks (`authority`/`reviewedBy` are derived by the server, never sent). + + Without `--kind` it follows from what you give (`--semantic` alone is `jev`, with source `both`); a kind that contradicts its inputs is **exit 2**, before anything is sent. `fp` checks only the shape of the file (a non-empty list of named objects, ≤24); the server checks the rest — the limits, FailproofAI's 16 reserved check names (**422 `reserved_jev_name`**), and names unique across the org — and names the declaration it refused. SOURCE is a path, `@path`, `-`, a pipe, or omitted to paste on a TTY (Ctrl-D ends); `--semantic` takes a path, `@path` or `-` (only one of the two may be stdin). The JavaScript is **parse-checked with node** before it is sent; nothing downstream does this, and a broken policy otherwise fails on the machine at enforcement time. `--no-verify` skips it, and a host without node publishes with a warning rather than a block. JavaScript carrying Jev fields — `semanticPolicies.add(...)`, or `authority: "reviewable"` on a registration — is refused before sending (**exit 1**, not skipped by `--no-verify`; commented-out code does not count): put Jev checks in `--semantic` (`--kind both`), where they take effect. **Publishing deploys nothing** — the version is unused until `fleet deploy` puts it on a machine. - `policies enable ` · `policies disable [-y]` — **disable REMOVES the policy from every deployment carrying it**, reissuing each affected machine at a new generation (visible in `fleet history`). `enable` is the exact inverse — it puts the policy back into every deployment it was removed from, reissuing those machines again. Nothing needs redeploying by hand, and `machinesUpdated` in the JSON reports the count for both directions. - `policies test [SOURCE] [--tool Bash] [--command "…"] [--file PATH] [--event PreToolUse] [--expect allow|deny|instruct]` — run a policy LOCALLY and print what it decides. Executes the real file (bare `import { deny } from "failproofai"` and all) against a synthetic context; nothing is published, nothing installed. Needs `node`. `--expect` asserts the decision and exits 1 when it differs — a correct `deny` is a PASSING test, so the decision alone never sets the exit code. JSON `{ok, decision, policies:[{name,decision,reason}], expected, met}`; `decision` is the strictest any registered policy returned. - `policies compose "" [--out FILE] [--publish ID]` — the assistant drafts policy source from plain English. Prints it and stops by default: a generated policy that deploys itself is one nobody read. `--publish` still syntax-checks first. Needs `policies:write` for BOTH the draft and the publish — `POST /api/agent/compose-policy` is `withAuth("policies:write")`; `agent:use` gates the assistant's other routes and is not checked on this one. The composer has a **30s server-side limit** — a long or vague description simply does not finish, and raising `--timeout` does not help because the cut is not client-side. Retry with something shorter and more specific. - `policies delete [-y]` — archives. **A machine already carrying the policy keeps enforcing it** until redeployed; `disable` is what stops enforcement everywhere. ### fleet -- `fleet list` — `machine · label · pol · intended · applied · seen · events · state`. `intended` is the generation deployed, `applied` what the machine last collected (they differ until it polls), and `seen` how long since it last reported anything — a machine can be in sync and dead, or alive and behind, which are different problems. JSON `{machines, deployments}` with raw epoch-ms timestamps plus the computed `drifted`. -- `fleet show ` — the set the machine is told to run, **and whether it has collected it**. Reads both the deployment and the machine record, so it reports `not yet collected` / `machine is on #N` / `collected` alongside who deployed it, when, and last-seen. A machine can be told to run a policy it has never picked up; the policy list alone cannot tell you which. JSON `{machine, deployment}` with raw timestamps and both label fields; `deployment: null` when nothing is deployed. -- `fleet deploy [--add REF]… [--remove ID]… [--set REF]… [--create] [-y]` +- `fleet list` — `machine · label · pol · intended · applied · seen · events · state`. `intended` is the generation deployed, `applied` what the machine last collected (they differ until it polls), and `seen` how long since it last reported anything — a machine can be in sync and dead, or alive and behind, which are different problems. `state` reads `N errors` when the machine reported policies it could not apply. JSON `{machines, deployments}` with raw epoch-ms timestamps plus the computed `drifted`; a machine carries `policyErrors`/`policyErrorsAt` and `jevMode` when the server has them. +- `fleet show ` — the set the machine is told to run, **and whether it has collected it**. Reads both the deployment and the machine record, so it reports `not yet collected` / `machine is on #N` / `collected` alongside who deployed it, when, and last-seen. A machine can be told to run a policy it has never picked up; the policy list alone cannot tell you which. Also the **Jev mode** FailproofAI Cloud sets on it (`local` when none) and the **policy errors** the machine last reported — `[{id, version, kind: regex|jev|both|daemon, message}]`: a policy that did not load, a Jev artifact that failed its digest or parse, a declaration its parser dropped, a `reviewedBy` naming a check that is not there, `jev_unconfigured` (a Jev mode with no provider on the machine). Whatever is listed there is not enforcing. JSON `{machine, deployment}` with raw timestamps and both label fields; `deployment: null` when nothing is deployed. +- `fleet deploy [--add REF]… [--remove ID]… [--set REF]… [--jev-mode off|observe|enforce|local] [--create] [-y]` **A deploy REPLACES the whole set.** The endpoint takes the full list and does not merge. `--add`/`--remove` are a read-modify-write: the CLI reads the current set, applies the delta, prints the complete result, writes that. `--set` replaces everything and is refused alongside `--add`/`--remove`. - REF is `id`, `id@version`, `id:effect`, or `id@version:effect`. Effect is `enforce` (default) or `observe`. A bare `--add` of an already-deployed policy **keeps its pinned version** — pass `id@version` to move it. + REF is `id`, `id@version`, `id:effect`, or `id@version:effect`. Effect is `enforce` (default) or `observe`. A bare `--add` of an already-deployed policy **keeps its pinned version** — pass `id@version` to move it. Every kind deploys the same way (a `both` policy's JavaScript and its checks go together); the effect applies to JavaScript only, so `observe` on a `jev` policy is refused (**exit 2**) — watch Jev through the mode instead. + + `--jev-mode` sets the machine's Jev mode from FailproofAI Cloud and **overrides the machine's own, a local `off` included**: `observe` asks Jev and logs it while the regex policies decide, `enforce` lets Jev clear what its checks review, `off` stops Jev there, `local` hands the choice back to the machine. Omitted, it is left as it is (the request carries no `jevMode`). `--jev-mode` alone is a valid deploy (same set, new mode); the plan shows `jev mode → `. The machine uses its own `jev.json` provider, else its FailproofAI Cloud Jev credential; with neither it reports `jev_unconfigured`. Deploying to an id that has never checked in is refused (a typo would mint a machine); `--create` allows it for pre-staging. @@ -245,7 +252,7 @@ Cloud-managed enforcement, split the way the dashboard splits it: `policies` wri **Exit codes.** A malformed ref (`bad ref!!`, `id:banana`, empty), or `--set` combined with `--add`/`--remove`, is a usage error → **exit 2**, like every other bad flag value. A ref that parses but names something that does not exist (`--add ghost-policy`, an unpublished `@version`) is **exit 1**; an unknown machine is **exit 6**. Branch on these rather than on the message. - JSON `{plan:{result,added,removed,changed,unchanged,noop}, deployment, applied}` — the plan is included so a harness does not recompute the diff. + JSON `{plan:{result,added,removed,changed,unchanged,noop[,jevMode:{from,to,changed}]}, deployment, applied}` — the plan is included so a harness does not recompute the diff. - `fleet diff [machine]` — intent vs delivery per machine, with a `drifted` flag. A machine id nobody has reported under is refused (exit 6), not rendered as an empty fleet. - `fleet history ` · `fleet rollback [-y]` — rollback mints a NEW generation carrying the old set; history stays append-only. The `change` column uses the deploy plan's vocabulary: `+` added, `-` removed, `~` same policy at a different version or effect (an enforce → observe flip is a policy that stopped blocking, so it is never "no change"). - `fleet rename "