fix(buffer): persist off the request path and at shutdown

.github/coverage-runner.php

@@ -35,14 +35,16 @@
     // of which throw) each catch every client Throwable inside RedisHandler
     // and degrade to a return value, so no Throwable can reach the buffer's
     // own catch arms.
-    'src/EventBuffer/EventBuffer.php' => [196, 197, 271, 272, 289, 290, 377, 378, 425, 426, 595, 596],
-    // stop()'s close catch: GuardAgent::$redisHandler is the final
+    'src/EventBuffer/EventBuffer.php' => [198, 199, 307, 308, 325, 326, 641, 642],
+    // persistAtShutdown()'s catch: persistPending() is total, every Redis
+    // call inside it is caught by RedisHandler (176-177). stop()'s close
+    // catch: GuardAgent::$redisHandler is the final
     // RedisHandler (initializeRedis hard-types it) whose close() catches
-    // every Throwable (252-253). asMetadataArray/asStringArray fallbacks:
+    // every Throwable (276-277). asMetadataArray/asStringArray fallbacks:
     // the inputs are SecurityEvent::$metadata / SecurityMetric::$tags array
     // properties passed through HeadersRedactor::sanitize, which returns an
-    // array for an array input at the top depth (778, 789).
-    'src/GuardAgent.php' => [252, 253, 778, 789],
+    // array for an array input at the top depth (809, 820).
+    'src/GuardAgent.php' => [176, 177, 276, 277, 809, 820],
     // read()/write() catch arms over @-suppressed filesystem calls, which
     // return false instead of throwing (76-77, 79, 95-96): is_file($path)
     // true implies a non-empty path, and the PHP 8 ValueErrors in this
@@ -63,7 +65,7 @@
     // toWire() casts metadata/tags to objects (SecurityEvent.php:155,
     // SecurityMetric.php:101) and makeRequest's payload always comes from
     // buildBatchWire mapping those toWire() outputs (512-515, 522-525).
-    'src/Transport/HttpTransport.php' => [512, 513, 514, 515, 522, 523, 524, 525],
+    'src/Transport/HttpTransport.php' => [515, 516, 517, 518, 525, 526, 527, 528],
     // json_encode of a finite float with JSON_THROW_ON_ERROR cannot fail:
     // the is_finite guard above rejects Inf/NaN before this call is reached
     // (asserted by the canonical json tests) and a finite float scalar has

.github/workflows/ci.yml

@@ -55,6 +55,9 @@ jobs:
           REDIS_HOST: 127.0.0.1
         run: php bin/test_agent.php
 
+      - name: test_shutdown
+        run: php bin/test_shutdown.php
+
   composer-audit:
     name: Composer audit
     runs-on: ubuntu-latest

AGENTS.md

@@ -28,7 +28,7 @@ All of `src/`, namespace `RenzoFranceschini\GuardAgent`:
 
 - `GuardAgent.php`: the client. `__construct(AgentConfig|array $config, ?TransportInterface $transport = null)` (the second parameter is the test seam). `start()` / `stop()` / `close()` lifecycle (start idempotent and may throw; stop idempotent and never throws); `sendEvent()` / `sendMetric()` ingest path (never throw, normalize + redact + buffer); `flushBuffer()` (never throws); `tick()` and `flushIfNeeded()` (host-driven loop, see Runtime model below); `getDynamicRules()` (TTL-cached rules fetch, never throws); `getStatus()` / `getStats()` / `healthCheck()`. Per-kind failure streaks with backoff gates (`eventsRetryAfter` / `metricsRetryAfter`), `flushConsecutiveFailures` / `statusConsecutiveFailures` / `rulesConsecutiveFailures` loop counters, `lastStatusPushOk`, `rulesFetched` / `cachedRules` / `rulesLastUpdate`.
 - `Config/AgentConfig.php`: readonly resolved config (camelCase, mirroring the TS port naming; defaults identical to the Python agent). `Config/AgentConfigResolver.php`: `resolve(array $input)` (accepts camelCase and snake_case keys), `validate()` (returns the error list), `normalizeEndpoint()` (strips trailing slashes and a legacy `/api/v1` suffix with a one-time warning, mirroring guard_agent/models.py:176-205). `Config/BufferOverflowPolicy.php`: `drop` / `block` / `raise` enum. `Config/RedisConfig.php`: url, keyPrefix (`guard:agent`), password, db, commandTimeoutMs (5000).
-- `EventBuffer/EventBuffer.php`: per-kind bounded queues with `SplObjectStorage` key maps; overflow policies; `flushEventsWithKeys()` / `flushMetricsWithKeys()` / `requeueEventsInMemory()` / `requeueMetricsInMemory()` (front requeue, tail eviction returns the evicted keys); `confirmEventRedisKeys()` / `confirmMetricRedisKeys()`; persist-on-enqueue with TTL 3600; `initializeRedis()` reloads persisted items at startup; `clearBuffer()`; `atHighWatermark()`.
+- `EventBuffer/EventBuffer.php`: per-kind bounded queues with `SplObjectStorage` key maps; overflow policies; `flushEventsWithKeys()` / `flushMetricsWithKeys()` / `requeueEventsInMemory()` / `requeueMetricsInMemory()` (front requeue, tail eviction returns the evicted keys); `confirmEventRedisKeys()` / `confirmMetricRedisKeys()`; `persistPending()` (writes unkeyed buffered items, TTL 3600; never called from add; run by `flushBuffer()` after the send, by `flushIfNeeded()` when no flush is due, and by `GuardAgent::persistAtShutdown()` from a shutdown function registered once in `initializeRedis()`); `initializeRedis()` reloads persisted items at startup; `clearBuffer()`; `atHighWatermark()`.
 - `Transport/HttpTransport.php`: cURL implementation of `TransportInterface`. Retry loop (`retryAttempts + 1` tries, local `RateLimiter` 100/60, `CircuitBreaker` 5 failures/60s), response classification, gzip compression, `Signer` HMAC, 413 split-or-drop, permanent-rejection drops, `RetryAfter` parsing (default 60s, honored up to 300s), `BatchId`, response-body summaries. `Transport/TransportInterface.php`: the seam (`sendEvents`/`sendMetrics`/`sendStatus` return bool; true = accepted or intentionally dropped, false = transient, caller requeues; `fetchDynamicRules()` returns `?DynamicRules` and never throws). `getWithRetry()` implements the GET path for `GET /api/v1/rules` with the same retry/circuit-breaker/Retry-After machinery as the POST loop (mirrors `_get_with_retry`, `_transport_send.py:238-289`). `Transport/CircuitBreaker.php`, `Transport/RateLimiter.php`, `Transport/Signer.php`.
 - `Persistence/RedisClientInterface.php`: the client seam (ping/get/set/delete/keys/close). `Persistence/StreamRedisClient.php`: dependency-free RESP2 client over `stream_socket_client` (PING, AUTH, SELECT, GET, SET EX, DEL, KEYS; command timeout via `stream_set_timeout`). `Persistence/PredisRedisClient.php` and `Persistence/ExtRedisClient.php`: optional adapters over injected `\Predis\ClientInterface`-shaped and `\Redis` instances. `Persistence/RedisHandler.php`: namespaced facade (`{keyPrefix}:{namespace}:{short}`) with the Go port's fail-open policy: 3 consecutive write failures pause writes for 30s.
 - `Model/SecurityEvent.php`, `Model/SecurityMetric.php`, `Model/AgentStatus.php`: readonly wire models with `normalize()` (camelCase or snake_case input, `InvalidEventException` on bad input) and `toWire()` (snake_case, ISO-8601 UTC with milliseconds). `Model/KnownTypes.php`: the advisory event-type list and the enforced metric-type list. `Model/DynamicRules.php`: the SaaS rules surface (mirrors DynamicRules, guard_agent/models.py:250-324, snake_case wire keys with camelCase properties, `normalize()` raising `InvalidRulesException`; `ttl` drives the client cache, `blocked_cloud_providers` deduped like the Python set, `ge=1` enforced only on the `auto_ban_*` fields). `Model/WireFormat.php`: timestamp parsing (DateTimeInterface, epoch SECONDS, or parseable string) and `isoUtc()`.
@@ -64,7 +64,7 @@ docker run --rm -v "$PWD":/app -w /app -e REDIS_HOST=host.docker.internal php:8.
 docker run --rm -v "$PWD":/app -w /app composer:2 composer validate --strict
 ```
 
-The CI-verified path (from `.github/workflows/ci.yml`): checkout, `shivammathur/setup-php` from the matrix (8.2/8.3/8.4, `mbstring` extension, coverage none), `composer install --no-interaction --no-progress`, the `php -l` sweep, then `REDIS_HOST=127.0.0.1 php bin/test_agent.php` against a `redis:7-alpine` service on port 6379, plus a `composer audit` job.
+The CI-verified path (from `.github/workflows/ci.yml`): checkout, `shivammathur/setup-php` from the matrix (8.2/8.3/8.4, `mbstring` extension, coverage none), `composer install --no-interaction --no-progress`, the `php -l` sweep, then `REDIS_HOST=127.0.0.1 php bin/test_agent.php` and `php bin/test_shutdown.php` against a `redis:7-alpine` service on port 6379, plus a `composer audit` job.
 
 ## Configuration
 
@@ -104,14 +104,15 @@ Every field except `apiKey` is optional; `AgentConfigResolver::resolve()` fills
 - **At-least-once handshake**: drain (keys aligned with items) -> send -> on success confirm (delete Redis records); on failure requeue at the FRONT in original order, retain Redis records, arm the per-kind gate for `min(flushInterval * 2^(streak - 1), 300)` seconds. Under pressure the requeue evicts the TAIL (newest items) and the caller confirms those evicted keys so nothing orphans.
 - **Response classification** (mirroring `_handle_response`): 200 with a JSON dict is evaluated (`success: false` or non-empty `errors[]` means partial failure -> whole batch requeued); 200 with an unparseable body is retried; 200 non-dict and 201 are accepted; 429 raises `RateLimitedException` with Retry-After (default 60s, honored up to 300s); 401/403 are generic retried errors; 400/404/422 are `PermanentClientException` (batch dropped, counted, never retried); 413 is `PayloadTooLargeException` (split in half recursively; a singleton that still exceeds the cap is dropped); 5xx retry with backoff; any other 4xx is logged and retried.
 - **Circuit breaker**: 5 consecutive non-permanent failures open it for 60s, then HALF_OPEN probes. 400/404/413/422 are exempt (batch-level rejections, not health signals); 429 and curl errors count.
-- **Persistence**: persist-on-enqueue under a globally-unique key `{prefix}:agent_events:event_<nanos>_<8hex>` (or `agent_metrics`), TTL 3600s. Startup reload restores items oldest-first; corrupt or missing records are warned and skipped (the TTL reclaims them). Every Redis failure is fail-open: logged, counted in `redisPersistFailures` (`durabilityDegraded`), and after 3 consecutive write failures writes pause for 30s.
+- **Persistence**: persisted by `persistPending()` (end of `flushBuffer()`, or `flushIfNeeded()`/`tick()` between flushes, or the shutdown function, never on add) under a globally-unique key `{prefix}:agent_events:event_<nanos>_<8hex>` (or `agent_metrics`), TTL 3600s. Startup reload restores items oldest-first; corrupt or missing records are warned and skipped (the TTL reclaims them). Every Redis failure is fail-open: logged, counted in `redisPersistFailures` (`durabilityDegraded`), and after 3 consecutive write failures writes pause for 30s. A shutdown function the agent registers once when Redis persistence is attached (`initializeRedis()`, including the one `start()` builds from `config.redis`) runs `persistAtShutdown()`, which calls `persistPending()`, so items buffered by a request are written to Redis when the process ends by normal end, `exit()`, an uncaught exception, a fatal error, out of memory or `max_execution_time` (proved by `bin/test_shutdown.php`). Events are lost only when PHP runs no code at all: SIGKILL (for example the OOM killer or `kill -9`), and SIGTERM unless the host installs its own handler (plain PHP skips shutdown functions on SIGTERM; with ext-pcntl a host handler that calls `exit()` makes it durable). The agent installs no signal handler because it would override the host's. It does not call `fastcgi_finish_request()` either: that would cut off output and shutdown functions the host registers later, so a host that wants the response out first calls it itself.
 - **Degraded status**: circuit OPEN, or buffer >= 90 percent occupancy, or lifetime failure rate > 10 percent. `healthCheck()` additionally fails at >= 95 percent occupancy or > 50 percent failure rate, and when not running.
 
 ## Development Commands
 
 | Command | What it runs |
 | --- | --- |
 | `composer test` | `php bin/test_agent.php` |
+| `php bin/test_shutdown.php` | child-process proof of persistence at process end (needs Redis on 6379; `REDIS_HOST=0` skips) |
 | `composer lint` | `for f in $(find src bin -name '*.php'); do php -l "$f" > /dev/null \|\| exit 1; done && echo LINT_OK` (example files are linted with `php -l` directly) |
 | `composer validate --strict` | composer.json hygiene (no version field, valid schema) |
 | `pip install mkdocs-material && mkdocs build --strict` | Build the documentation site (CI deploys it on `main`) |
@@ -124,7 +125,7 @@ Docker equivalents are in Quick Start. There is no Makefile (the sibling PHP rep
 - `REDIS_HOST` controls the persistence leg: `REDIS_HOST=0` skips the real-Redis integration (unit-only), any hostname runs it against port 6379. The fake-Redis handshake tests always run.
 - The mock-server integration smoke spawns `php -S` with `tests/mock_server.php`, which mirrors the ingestion contract: size-based 413 (decompressed body > 262144 bytes), signature verification over the uncompressed body with an optional `requireSigned` mode, scripted per-path responses from a control file (status, body, Retry-After), and a JSON state log the tests assert against.
 - Pinned HMAC vector: `Signer::signPayload('{"hello":"world"}', 'test-secret')` must equal `v1=84cc33df716ed0b0598f07437c94069ace3730358778a592bd6bbd1423d111f3`.
-- Coverage expectations when editing: config validation (all constraint branches), buffer overflow policies and requeue eviction, the handshake order (drain -> send -> confirm/requeue/gate), retry classification (200 partial, 429, 400, 401, 413 split and singleton drop, 5xx, dead endpoint), breaker open/exempt/half-open, persistence (persist-on-enqueue, TTL, reload, confirm deletes, fail-open cooldown), and the smoke contract assertions (headers, gzip, signature, endpoints).
+- Coverage expectations when editing: config validation (all constraint branches), buffer overflow policies and requeue eviction, the handshake order (drain -> send -> confirm/requeue/gate), retry classification (200 partial, 429, 400, 401, 413 split and singleton drop, 5xx, dead endpoint), breaker open/exempt/half-open, persistence (deferred persistPending, TTL, reload, confirm deletes, fail-open cooldown), and the smoke contract assertions (headers, gzip, signature, endpoints).
 - `vendor/` is never committed; `composer.lock` is (zero-dependency lock, generated with the `composer:2` image).
 
 ## Best Practices