From f303c0cc77df7021f1833ca58d5b570ff450d98f Mon Sep 17 00:00:00 2001 From: Thomas Darimont Date: Tue, 6 Oct 2026 00:46:23 +0200 Subject: [PATCH] State the processing guarantee and expose the idempotency key A SET is handled at least once, not exactly once: the javadoc of the handler and the dedup stores and both READMEs say so now, and SsfEventToken.idempotencyKey() (issuer and jti, also the dedup key) gives handlers the key for their own side effects. The SCIM-to-OIDC attribute mapping is documented as a deployment decision. Closes #28 Signed-off-by: Thomas Darimont --- CHANGELOG.md | 3 +++ README.md | 12 ++++++++---- .../java/org/easyssf/core/event/SsfEventToken.java | 11 +++++++++++ easyssf-receiver-jdbc/AGENTS.md | 4 ++-- .../easyssf/receiver/jdbc/JdbcSsfJtiDedupStore.java | 8 +++++--- .../jdbc/AbstractJdbcSharedReceiversTests.java | 4 ++-- easyssf-receiver-spring-boot-starter/README.md | 13 +++++++++---- .../org/easyssf/receiver/event/SsfEventContext.java | 9 +++++++++ .../org/easyssf/receiver/event/SsfEventHandler.java | 9 +++++++-- .../org/easyssf/receiver/set/SsfJtiDedupStore.java | 13 ++++++++----- .../receiver/event/SsfEventContextTests.java | 7 +++++++ 11 files changed, 71 insertions(+), 22 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f8c512f..43a61cb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -27,6 +27,9 @@ The notable changes of every release. The section of a version is the text of it older than the lease (`easyssf.receiver.dedup.lease`, 60 seconds) counts as abandoned and the SET is handled again. +- `SsfEventToken.idempotencyKey()` and `SsfEventContext.idempotencyKey()`, the issuer and `jti` of a + SET, as the key for the side effects of a handler: processing is at least once, which the + documentation now says in those words. - Schema migration: `easyssf-receiver-jdbc` ships versioned migration scripts in `org/easyssf/receiver/jdbc/migration/` (Flyway naming, plain SQL), and `JdbcSsfSchema.prepareTable` adds the columns a release introduced to an existing table where it may create tables diff --git a/README.md b/README.md index eae72c6..87bbe16 100644 --- a/README.md +++ b/README.md @@ -157,10 +157,14 @@ class StepUpHandler implements SsfEventHandler { } ``` -Handlers must be idempotent. If a handler throws, the SET is not acknowledged and the transmitter is -expected to deliver it again, in which case all handlers run again, possibly on another instance of -the application: with a shared dedup store, a SET is claimed while its handlers run, another instance -leaves it alone meanwhile, and a claim a crashed instance left behind is taken over after a lease. +**A SET is handled at least once, not exactly once**, so handlers must be idempotent. If a handler +throws, the SET is not acknowledged and the transmitter is expected to deliver it again, in which case +all handlers run again, possibly on another instance of the application. With a shared dedup store a +SET is claimed while its handlers run and another instance leaves it alone meanwhile, and a redelivery +of a handled SET is skipped; but an instance that dies after a handler's side effects and before the +success is recorded hands the SET to the next instance after a lease. For side effects outside the +receiver's stores, a database row, a message, an email, key them by `eventContext.idempotencyKey()`, +the issuer and `jti` of the SET. `SsfSubject` is the `sub_id` of the SET: a simple subject identifier (RFC 9493: `iss_sub`, `email`, `opaque`, `account`, `phone_number`, `did`, `uri`, `aliases`; RFC 9967: `scim`) or a complex subject whose members diff --git a/easyssf-core/src/main/java/org/easyssf/core/event/SsfEventToken.java b/easyssf-core/src/main/java/org/easyssf/core/event/SsfEventToken.java index b2f31ff..a070a62 100644 --- a/easyssf-core/src/main/java/org/easyssf/core/event/SsfEventToken.java +++ b/easyssf-core/src/main/java/org/easyssf/core/event/SsfEventToken.java @@ -28,4 +28,15 @@ public record SsfEventToken(String jti, String iss, Instant iat, List au claims = SsfCollections.copyOf(claims); } + /** + * The key that identifies this SET across deliveries: the issuer and the {@code jti}, + * which is only unique per issuer (RFC 8417, section 2.2). A handler that has side + * effects outside the receiver's stores uses it to make them idempotent, because a + * SET may be handled more than once: after a handler failed, or on another instance + * after the one that handled it died before recording the success. + */ + public String idempotencyKey() { + return this.iss + "::" + this.jti; + } + } diff --git a/easyssf-receiver-jdbc/AGENTS.md b/easyssf-receiver-jdbc/AGENTS.md index 1a071ff..21dda8e 100644 --- a/easyssf-receiver-jdbc/AGENTS.md +++ b/easyssf-receiver-jdbc/AGENTS.md @@ -16,8 +16,8 @@ The JDBC stores of the receiver in plain SQL, over the small `SsfJdbcOperations` `./mvnw -pl easyssf-receiver-jdbc -Pdatabase-tests verify` (Docker). Run them when the SQL or the schema changes. All share `AbstractJdbcSsfStoresTests`; add store tests there. `AbstractJdbcSharedReceiversTests` (same three databases) runs two receivers against one - `TestTransmitter` on shared stores: a SET handled once, acknowledgements handed over between - instances, and the race where one instance acknowledges a SET another is still handling (covered + `TestTransmitter` on shared stores: a SET handled by one instance and skipped by the others, + acknowledgements handed over between instances, and the race where one instance acknowledges a SET another is still handling (covered by the in-progress state of the dedup store, see `scratch/plans/dedup-in-progress-state.md`). - The module is an explicit Java module; test compilation adds `java.naming` because H2's data source implements a `javax.naming` interface (see `pom.xml`). Keep that if you touch the diff --git a/easyssf-receiver-jdbc/src/main/java/org/easyssf/receiver/jdbc/JdbcSsfJtiDedupStore.java b/easyssf-receiver-jdbc/src/main/java/org/easyssf/receiver/jdbc/JdbcSsfJtiDedupStore.java index 857cb5a..e077bcd 100644 --- a/easyssf-receiver-jdbc/src/main/java/org/easyssf/receiver/jdbc/JdbcSsfJtiDedupStore.java +++ b/easyssf-receiver-jdbc/src/main/java/org/easyssf/receiver/jdbc/JdbcSsfJtiDedupStore.java @@ -10,9 +10,11 @@ import org.easyssf.receiver.set.SsfJtiDedupStore; /** - * {@link SsfJtiDedupStore} that remembers processed SETs in a database table, so that a - * SET is handled once even if it is delivered to several instances of an application or - * again after a restart. + * {@link SsfJtiDedupStore} that remembers processed SETs in a database table shared by + * the instances of an application: a SET delivered to several of them or again after a + * restart is handled by one and skipped by the others, and a SET one instance is handling + * is left alone by the rest. Processing stays at least once, see + * {@link SsfJtiDedupStore}. * *

* A claim inserts the row with the state {@code IN_PROGRESS}: the primary key of the diff --git a/easyssf-receiver-jdbc/src/test/java/org/easyssf/receiver/jdbc/AbstractJdbcSharedReceiversTests.java b/easyssf-receiver-jdbc/src/test/java/org/easyssf/receiver/jdbc/AbstractJdbcSharedReceiversTests.java index 6d80730..9616d16 100644 --- a/easyssf-receiver-jdbc/src/test/java/org/easyssf/receiver/jdbc/AbstractJdbcSharedReceiversTests.java +++ b/easyssf-receiver-jdbc/src/test/java/org/easyssf/receiver/jdbc/AbstractJdbcSharedReceiversTests.java @@ -28,8 +28,8 @@ /** * Several receivers (instances of one application) polling the same stream and sharing - * the JDBC stores: a SET is handled once, and an acknowledgement one instance could not - * deliver leaves with the next poll of another. + * the JDBC stores: a SET is handled by one of them and skipped by the others, and an + * acknowledgement one instance could not deliver leaves with the next poll of another. */ abstract class AbstractJdbcSharedReceiversTests { diff --git a/easyssf-receiver-spring-boot-starter/README.md b/easyssf-receiver-spring-boot-starter/README.md index 2134a5f..2a78078 100644 --- a/easyssf-receiver-spring-boot-starter/README.md +++ b/easyssf-receiver-spring-boot-starter/README.md @@ -119,8 +119,11 @@ Things to know: **SCIM Events** (RFC 9967, see the [root README](../README.md#scim-events)): the deactivation and the deletion of a SCIM user usually mean the user may no longer be logged in. To end their local -sessions, add the events to the user events. A `scim` subject is matched with the logged-in user by -comparing attributes of the resource with claims: by default `externalId` and `id` (the `id` +sessions, add the events to the user events. Which SCIM resource is which logged-in user is not +something SCIM or OpenID Connect define: it is a decision of your deployment, about identifiers the +SCIM service provider and the OpenID Provider happen to share, and the mapping below is where you +state it. A `scim` subject is matched with the logged-in user by comparing attributes of the resource +with claims: by default `externalId` and `id` (the `id` attribute, or the last segment of the `uri`) with `sub`, and `userName` with `preferred_username`. If your SCIM service provider and your OpenID Provider share other identifiers, configure the pairs; a multi-valued attribute such as `emails` matches if any of its values does, and values compared with @@ -331,8 +334,10 @@ the state is kept in its database instead, without further configuration: instance that receives a SET another one is handling leaves it for the transmitter's redelivery, neither handling nor acknowledging it, so a handler failure on the first instance is not masked by an acknowledgement of the second. A claim older than `easyssf.receiver.dedup.lease` (60 seconds) - counts as abandoned by a crashed instance and the SET is handled again, which is one more reason for - handlers to be idempotent. Set the lease longer than your longest handler. + counts as abandoned by a crashed instance and the SET is handled again. The guarantee is therefore + at-least-once processing with concurrent deliveries suppressed, not exactly once: handlers stay + idempotent and key their side effects by `eventContext.idempotencyKey()`. Set the lease longer than + your longest handler. - **Upgrading from 0.1.0 or 0.2.0**: run `migration/V0_3_0__dedup_state_and_poll_acks.sql`, which adds the column `STATE` to `EASYSSF_PROCESSED_SET` and creates `EASYSSF_POLL_ACK`; with `initialize-schema` `embedded` or `always` the receiver does it on startup. diff --git a/easyssf-receiver/src/main/java/org/easyssf/receiver/event/SsfEventContext.java b/easyssf-receiver/src/main/java/org/easyssf/receiver/event/SsfEventContext.java index 9a793f2..e202a2c 100644 --- a/easyssf-receiver/src/main/java/org/easyssf/receiver/event/SsfEventContext.java +++ b/easyssf-receiver/src/main/java/org/easyssf/receiver/event/SsfEventContext.java @@ -25,6 +25,15 @@ public SsfEventToken eventToken() { return this.eventToken; } + /** + * The key for making the side effects of a handler idempotent, the issuer and + * {@code jti} of the SET: a SET is handled at least once, not exactly once (see + * {@link SsfEventHandler}). + */ + public String idempotencyKey() { + return this.eventToken.idempotencyKey(); + } + /** * The event type URIs contained in the SET. */ diff --git a/easyssf-receiver/src/main/java/org/easyssf/receiver/event/SsfEventHandler.java b/easyssf-receiver/src/main/java/org/easyssf/receiver/event/SsfEventHandler.java index 0bff19d..8482aeb 100644 --- a/easyssf-receiver/src/main/java/org/easyssf/receiver/event/SsfEventHandler.java +++ b/easyssf-receiver/src/main/java/org/easyssf/receiver/event/SsfEventHandler.java @@ -5,8 +5,13 @@ * {@link org.easyssf.receiver.set.SsfSetProcessor} is invoked for every SET. * *

- * Handlers must be idempotent: a SET is delivered again if any handler failed, and - * de-duplication does not survive a restart. + * A SET is handled at least once, not exactly once, so handlers must be idempotent. A SET + * is delivered again if any handler failed, and all handlers run again, possibly on + * another instance of the application; with a shared dedup store a SET is claimed while + * its handlers run and another instance leaves it alone meanwhile, but an instance that + * dies after a handler's side effects and before the success is recorded hands the SET to + * the next instance after a lease. Side effects outside the receiver's stores should be + * keyed by {@link SsfEventContext#idempotencyKey()}. */ @FunctionalInterface public interface SsfEventHandler { diff --git a/easyssf-receiver/src/main/java/org/easyssf/receiver/set/SsfJtiDedupStore.java b/easyssf-receiver/src/main/java/org/easyssf/receiver/set/SsfJtiDedupStore.java index a65be3b..e428863 100644 --- a/easyssf-receiver/src/main/java/org/easyssf/receiver/set/SsfJtiDedupStore.java +++ b/easyssf-receiver/src/main/java/org/easyssf/receiver/set/SsfJtiDedupStore.java @@ -3,8 +3,11 @@ import org.easyssf.core.event.SsfEventToken; /** - * Remembers which SETs were already processed, so that a SET delivered more than once is - * handled only once. Implement it to back it with a shared or durable store. + * Remembers which SETs were processed, so that a redelivery of a handled SET is skipped + * and a SET another instance is handling is left alone. It does not make handling exactly + * once: an instance that dies after a handler ran and before {@link #processed} hands the + * SET to the next instance after the lease, which is why handlers are idempotent. + * Implement it to back it with a shared or durable store. * *

* A SET is {@link #claim claimed} before its handlers run and marked {@link #processed @@ -68,11 +71,11 @@ default boolean seenBefore(SsfEventToken eventToken) { } /** - * The key identifying a SET. A {@code jti} is only unique per issuer (RFC 8417, - * section 2.2). + * The key identifying a SET, the same one handlers use for their side effects. + * @see SsfEventToken#idempotencyKey() */ static String key(SsfEventToken eventToken) { - return eventToken.iss() + "::" + eventToken.jti(); + return eventToken.idempotencyKey(); } } diff --git a/easyssf-receiver/src/test/java/org/easyssf/receiver/event/SsfEventContextTests.java b/easyssf-receiver/src/test/java/org/easyssf/receiver/event/SsfEventContextTests.java index 5be4b01..7e42857 100644 --- a/easyssf-receiver/src/test/java/org/easyssf/receiver/event/SsfEventContextTests.java +++ b/easyssf-receiver/src/test/java/org/easyssf/receiver/event/SsfEventContextTests.java @@ -34,6 +34,13 @@ void vendorEventTypesAreAddressedByUri() { assertThat(SsfEventTypes.aliasOf(vendorType)).isEqualTo(vendorType); } + @Test + void idempotencyKeyIsIssuerAndJti() { + SsfEventContext context = context(Map.of(SsfEventTypes.CAEP_SESSION_REVOKED, Map.of()), null); + assertThat(context.idempotencyKey()).isEqualTo("https://idp.example::jti-1"); + assertThat(context.idempotencyKey()).isEqualTo(context.eventToken().idempotencyKey()); + } + @Test void subjectIsTakenFromSubId() { SsfEventContext context = context(Map.of(SsfEventTypes.CAEP_SESSION_REVOKED, Map.of()), opaque("abc"));