Harden the receiver against a review of the Go and Python ports

CHANGELOG.md

@@ -6,8 +6,44 @@ The notable changes of every release. The section of a version is the text of it
 
 ## [Unreleased]
 
+### Security
+
+- A host name that merely starts with `127.`, such as `127.attacker.example`, counted as a loopback
+  address and was allowed to use plain `http` under the default URI policy. `SsfTransmitterUriPolicy`
+  now takes only `localhost` and names under it, IPv4 addresses in `127.0.0.0/8` and `::1` (also as
+  the IPv4-mapped `::ffff:127.0.0.1`) for loopback, the same rule as the Go library.
+- The poll endpoint is held to the URI policy of the transmitter's metadata endpoints before the
+  access token is sent to it, whether it was configured (`easyssf.receiver.poll.endpoint-url`, checked
+  at startup) or taken from the stream configuration the transmitter returned: `https`, or `http` on
+  loopback addresses only, unless `allow-insecure-http` is set. New API: `SsfPoller.setUriPolicy`,
+  which the Quarkus extension can wire to its insecure-http setting.
+- Response bodies the receiver reads from the transmitter (metadata, JWK Set, stream configurations,
+  poll responses) are bounded, 4 MiB by default; a larger response fails the call as if the
+  transmitter were unavailable instead of being buffered. New API: `JdkSsfHttpClient.setMaxResponseSize`
+  and `SsfHttpResponse.readBody` for other `SsfHttpClient` implementations, and the property
+  `easyssf.receiver.http.max-response-size` of the Spring Boot starter, which the Quarkus extension
+  can mirror.
+
+### Changed
+
+- `SsfStreamClient` checks the issuer of every stream configuration the transmitter returns, not only
+  of a created one: `getStream`, `getStreams`, `updateStream` and `replaceStream` throw
+  `SsfStreamIssuerMismatchException` as `createStream` did, as the SETs of a stream with another
+  issuer would not verify. `SsfStreamRegistrar` deletes a stream it created with the wrong issuer, as
+  before, and leaves an existing one alone; it fails without retrying either way.
+
 ### Fixed
 
+- A SET the poll response lists under a key other than its `jti` (RFC 8936, section 2.4) is reported
+  to the transmitter as `invalid_request` under that key instead of being handled and acknowledged
+  under a key that is not its own.
+- `SsfPoller.stop()` followed by `start()` while a handler was still running left the old polling
+  thread alive next to the new one once the handler returned; a polling thread now ends as soon as it
+  is no longer the poller's thread. `stop()` logs when the thread is still busy after five seconds.
+- `SsfStreamRegistrar.start()` is idempotent and `stop()` waits for the registration thread to end
+  before deleting the stream. A registration that completes after `stop()` publishes nothing and
+  deletes a stream it created on the way if the registrar deletes on shutdown; if the registrar was
+  started again meanwhile, it leaves that stream to the new registration, which adopts it.
 - The `easyssf.receiver.poll.pending-acks` gauge no longer turns `NaN` after a garbage collection.
   Micrometer refers to what a gauge reads weakly, and the poller handed it a method reference
   nothing else held on to; the poller now keeps that reference for as long as it lives, so the

README.md

@@ -310,7 +310,9 @@ verifier.setExpectedAudience("https://my-app.example");
 // metadata.resolve() also tells specVersion(), deliveryMethodsSupported(), criticalSubjectMembers(),
 // authorizationSchemes() and defaultSubjects(). The resolver requires https for the issuer and the
 // endpoints of the metadata (http on loopback addresses only); pass SsfTransmitterUriPolicy.INSECURE
-// as a fourth argument for a development setup.
+// as a fourth argument for a development setup. SsfPoller.setUriPolicy holds the poll endpoint to the
+// same rule, as the access token goes there. Response bodies are bounded, see
+// JdkSsfHttpClient.setMaxResponseSize.
 
 SsfEventHandler handler = (eventContext) -> {
     if (eventContext.hasEvent("CaepSessionRevoked")) {

easyssf-receiver-spring-boot-autoconfigure/src/main/java/org/easyssf/receiver/spring/boot/SsfReceiverProperties.java

@@ -7,10 +7,12 @@
 import java.util.Map;
 
 import org.easyssf.core.event.SsfSubject;
+import org.easyssf.receiver.http.SsfHttpResponse;
 import org.easyssf.receiver.jdbc.JdbcSsfPollAckStore;
 import org.easyssf.receiver.session.SsfSubjectClaimsMatcher;
 import org.easyssf.receiver.transmitter.SsfTransmitter;
 import org.springframework.boot.context.properties.ConfigurationProperties;
+import org.springframework.util.unit.DataSize;
 
 /**
  * Configuration properties for the OpenID Shared Signals Framework (SSF) receiver.
@@ -180,6 +182,13 @@ public static class Http {
          */
         private String userAgent;
 
+        /**
+         * The largest response body accepted from the transmitter; a larger response
+         * fails the call. Metadata, JWK Sets, stream configurations and poll responses
+         * stay far below the default of 4MB.
+         */
+        private DataSize maxResponseSize = DataSize.ofBytes(SsfHttpResponse.DEFAULT_MAX_BODY_SIZE);
+
         public boolean isUseRestClient() {
             return this.useRestClient;
         }
@@ -212,6 +221,14 @@ public void setUserAgent(String userAgent) {
             this.userAgent = userAgent;
         }
 
+        public DataSize getMaxResponseSize() {
+            return this.maxResponseSize;
+        }
+
+        public void setMaxResponseSize(DataSize maxResponseSize) {
+            this.maxResponseSize = maxResponseSize;
+        }
+
     }
 
     public static class Metrics {

easyssf-receiver-spring-boot-autoconfigure/src/main/java/org/easyssf/receiver/spring/boot/SsfTransmitterFactory.java

@@ -232,6 +232,7 @@ public static SsfHttpClient pollHttpClient(String name, SsfTransmitterProperties
         JdkSsfHttpClient pollClient = new JdkSsfHttpClient(connectTimeout, poll.getLongPollingHold().plusSeconds(15));
         if (http != null) {
             pollClient.setUserAgent(http.getUserAgent());
+            pollClient.setMaxResponseSize(http.getMaxResponseSize().toBytes());
         }
         logger.info(prefix(name) + "poll.long-polling is on, the poller calls the transmitter with the JDK HTTP "
                 + "client rather than the RestClient of the application");
@@ -260,6 +261,7 @@ public static SsfPoller poller(String name, SsfTransmitterProperties properties,
         if (poll.isLongPolling()) {
             poller.setLongPolling(poll.getLongPollingHold());
         }
+        poller.setUriPolicy(uriPolicy(properties));
         poller.setTransmitter(properties.getTransmitterIssuer());
         poller.setMetrics((metrics != null) ? metrics : SsfReceiverMetrics.NOOP);
         poller.setInterval(poll.getInterval());
@@ -373,6 +375,13 @@ private static Supplier<URI> pollEndpoint(String name, SsfTransmitterProperties
             SsfReceiverStream receiverStream) {
         URI configured = properties.getPoll().getEndpointUrl();
         if (configured != null) {
+            try {
+                uriPolicy(properties).checkEndpoint(configured, "The poll endpoint");
+            }
+            catch (IllegalArgumentException ex) {
+                throw new InvalidConfigurationPropertyValueException(prefix(name) + "poll.endpoint-url", configured,
+                        ex.getMessage());
+            }
             return () -> configured;
         }
         if (!registersStream(properties)) {

easyssf-receiver-spring-boot-autoconfigure/src/main/java/org/easyssf/receiver/spring/boot/autoconfigure/SsfReceiverAutoConfiguration.java

@@ -100,6 +100,7 @@ JdkSsfHttpClient ssfHttpClient(SsfReceiverProperties properties) {
                         : SsfReceiverProperties.Http.DEFAULT_TIMEOUT,
                 (http.getReadTimeout() != null) ? http.getReadTimeout() : SsfReceiverProperties.Http.DEFAULT_TIMEOUT);
         httpClient.setUserAgent(http.getUserAgent());
+        httpClient.setMaxResponseSize(http.getMaxResponseSize().toBytes());
         return httpClient;
     }
 

easyssf-receiver-spring-boot-autoconfigure/src/main/java/org/easyssf/receiver/spring/boot/autoconfigure/SsfReceiverRestClientAutoConfiguration.java

@@ -68,7 +68,9 @@ RestClientSsfHttpClient ssfRestClientHttpClient(RestClient.Builder restClientBui
             builder.defaultHeader(HttpHeaders.USER_AGENT, http.getUserAgent());
         }
         logger.info("The SSF transmitter is called with the RestClient of the application");
-        return new RestClientSsfHttpClient(builder.build());
+        RestClientSsfHttpClient httpClient = new RestClientSsfHttpClient(builder.build());
+        httpClient.setMaxResponseSize(http.getMaxResponseSize().toBytes());
+        return httpClient;
     }
 
     /**

easyssf-receiver-spring-boot-autoconfigure/src/main/java/org/easyssf/receiver/spring/boot/http/RestClientSsfHttpClient.java

@@ -26,11 +26,27 @@ public class RestClientSsfHttpClient implements SsfHttpClient {
 
     private final RestClient restClient;
 
+    private long maxResponseSize = SsfHttpResponse.DEFAULT_MAX_BODY_SIZE;
+
     public RestClientSsfHttpClient(RestClient restClient) {
         Assert.notNull(restClient, "restClient must not be null");
         this.restClient = restClient;
     }
 
+    /**
+     * Bounds the response bodies, {@link SsfHttpResponse#DEFAULT_MAX_BODY_SIZE 4 MiB} by
+     * default; a larger response fails the call as if none was received.
+     * @param maxResponseSize the largest response body read, in bytes
+     */
+    public void setMaxResponseSize(long maxResponseSize) {
+        Assert.isTrue(maxResponseSize > 0, "maxResponseSize must be positive");
+        this.maxResponseSize = maxResponseSize;
+    }
+
+    public long getMaxResponseSize() {
+        return this.maxResponseSize;
+    }
+
     @Override
     public SsfHttpResponse execute(SsfHttpRequest request) throws IOException {
         RestClient.RequestBodySpec spec = this.restClient.method(HttpMethod.valueOf(request.method()))
@@ -47,8 +63,9 @@ public SsfHttpResponse execute(SsfHttpRequest request) throws IOException {
                 MediaType contentType = clientResponse.getHeaders().getContentType();
                 Charset charset = (contentType != null && contentType.getCharset() != null) ? contentType.getCharset()
                         : StandardCharsets.UTF_8;
-                String body = new String(clientResponse.getBody().readAllBytes(), charset);
-                return new SsfHttpResponse(clientResponse.getStatusCode().value(), headers, body);
+                byte[] body = SsfHttpResponse.readBody(clientResponse.getBody(),
+                        clientResponse.getHeaders().getContentLength(), this.maxResponseSize, request.uri());
+                return new SsfHttpResponse(clientResponse.getStatusCode().value(), headers, new String(body, charset));
             });
         }
         catch (RestClientException ex) {

easyssf-receiver-spring-boot-autoconfigure/src/test/java/org/easyssf/receiver/spring/boot/autoconfigure/SsfReceiverAutoConfigurationTests.java

@@ -386,6 +386,15 @@ void userAgentForCallsToTheTransmitterIsConfigurable() {
                 .isEqualTo("my-receiver/1.0"));
     }
 
+    @Test
+    void responseSizeForCallsToTheTransmitterIsConfigurable() {
+        this.webContextRunner.run((context) -> assertThat(context.getBean(JdkSsfHttpClient.class).getMaxResponseSize())
+            .isEqualTo(4L * 1024 * 1024));
+        this.webContextRunner.withPropertyValues("easyssf.receiver.http.max-response-size=64KB")
+            .run((context) -> assertThat(context.getBean(JdkSsfHttpClient.class).getMaxResponseSize())
+                .isEqualTo(64 * 1024));
+    }
+
     @Test
     void transmitterIsCalledWithRestClientOfTheApplication() {
         try (TestTransmitter transmitter = new TestTransmitter()) {
@@ -536,6 +545,25 @@ void pollDeliveryReplacesPushEndpoint() {
             });
     }
 
+    @Test
+    void pollEndpointWithoutTlsIsRefusedUnlessInsecureHttpIsAllowed() {
+        this.webContextRunner
+            .withPropertyValues("easyssf.receiver.delivery-method=poll", "easyssf.receiver.poll.auto-startup=false",
+                    "easyssf.receiver.poll.endpoint-url=http://idp.example/realms/test/poll")
+            .run((context) -> {
+                assertThat(context).hasFailed();
+                assertThat(context.getStartupFailure()).rootCause()
+                    .isInstanceOf(InvalidConfigurationPropertyValueException.class)
+                    .hasMessageContaining("easyssf.receiver.poll.endpoint-url")
+                    .hasMessageContaining("must use https");
+            });
+        this.webContextRunner
+            .withPropertyValues("easyssf.receiver.delivery-method=poll", "easyssf.receiver.poll.auto-startup=false",
+                    "easyssf.receiver.poll.endpoint-url=http://idp.example/realms/test/poll",
+                    "easyssf.receiver.allow-insecure-http=true")
+            .run((context) -> assertThat(context).hasSingleBean(SsfPoller.class));
+    }
+
     @Test
     void pollDeliveryNeedsPollEndpoint() {
         this.webContextRunner.withPropertyValues("easyssf.receiver.delivery-method=poll").run((context) -> {

easyssf-receiver-spring-boot-autoconfigure/src/test/java/org/easyssf/receiver/spring/boot/http/RestClientSsfHttpClientTests.java

@@ -2,6 +2,7 @@
 
 import java.io.IOException;
 import java.net.URI;
+import java.nio.charset.StandardCharsets;
 import java.util.List;
 
 import org.easyssf.core.stream.SsfStreamConfiguration;
@@ -49,6 +50,18 @@ void returnsResponseWhateverItsStatus() throws Exception {
         assertThat(unauthorized.isSuccessful()).isFalse();
     }
 
+    @Test
+    void cutsOffResponseLargerThanTheMaximum() throws Exception {
+        RestClientSsfHttpClient bounded = new RestClientSsfHttpClient(RestClient.create());
+        URI jwksUri = URI.create(transmitter.jwksUri());
+        int size = bounded.execute(SsfHttpRequest.get(jwksUri)).body().getBytes(StandardCharsets.UTF_8).length;
+        bounded.setMaxResponseSize(size - 1);
+        assertThatIOException().isThrownBy(() -> bounded.execute(SsfHttpRequest.get(jwksUri)))
+            .withMessageContaining("larger than the " + (size - 1) + " bytes accepted");
+        bounded.setMaxResponseSize(size);
+        assertThat(bounded.execute(SsfHttpRequest.get(jwksUri)).status()).isEqualTo(200);
+    }
+
     @Test
     void reportsFailedConnectionAsIOException() {
         assertThatIOException().isThrownBy(

easyssf-receiver-spring-boot-starter/README.md

@@ -186,7 +186,8 @@ easyssf:
 ```
 
 - The poll endpoint is taken from the stream (`stream.id` or a stream managed by the receiver), or set
-  it with `easyssf.receiver.poll.endpoint-url`.
+  it with `easyssf.receiver.poll.endpoint-url`. Either way it must use `https` (`http` on loopback
+  addresses only, or with `allow-insecure-http`), as the access token is sent to it.
 - A SET is acknowledged after it was handled. An invalid SET is reported to the transmitter (`setErrs`).
   A SET that could not be handled is neither acknowledged nor reported and is delivered again.
 - If more SETs are available than `poll.max-events`, they are fetched right away.
@@ -252,6 +253,16 @@ pushed SET is answered with `503` and a polled one left unacknowledged, so the t
 it again.
 `easyssf.receiver.stream.delete-on-shutdown=true` deletes the stream when the application stops.
 
+Things to know:
+
+- **Several instances** register under the same credentials and reuse the one stream, as the
+  registrar finds it on startup. Several instances that start at the same time against a transmitter
+  with no stream yet may each create one, unless the transmitter allows a single stream per receiver
+  (Keycloak does); every event is then delivered once per stream, with a `jti` of its own, which the
+  dedup store cannot tell apart. Start one instance first, or let the transmitter manage the stream.
+- **`delete-on-shutdown` is for a single instance**: the stream is shared, so the first instance to
+  stop deletes it for the others, and the next one to start creates a new stream with a new audience.
+
 The `SsfStreamClient` bean gives access to the whole stream management API (SSF 1.0, section 8.1):
 
 ```java
@@ -434,14 +445,15 @@ name, with its own status. Switch it off with `management.health.easyssf.enabled
 | `easyssf.receiver.transmitters.<name>.*` | | Further transmitters, see [Several transmitters](#several-transmitters): every property of a transmitter, from `transmitter-issuer` to `set-validation.*`, under its name. |
 | `easyssf.receiver.transmitter-metadata-url` | derived | By default `<host>/.well-known/ssf-configuration<issuer-path>` (SSF 1.0, section 7.2), falling back to `<issuer>/.well-known/ssf-configuration`. |
 | `easyssf.receiver.transmitter-jwks-url` | from metadata | Skips metadata discovery when set. |
-| `easyssf.receiver.allow-insecure-http` | `false` | Accepts `http` for the transmitter issuer and the endpoints it publishes on any host, with a warning on startup. For development only. |
+| `easyssf.receiver.allow-insecure-http` | `false` | Accepts `http` for the transmitter issuer, the endpoints it publishes and the poll endpoint on any host, with a warning on startup. For development only. |
 | `easyssf.receiver.expected-audience` | | Every SET must contain it in `aud`. Without it the audience of the stream the receiver registers or looks up (`stream.management=receiver`, `stream.id`) is required instead, once the stream is known. One of the two, or `set-validation.allow-any-audience`, is required: a receiver that cannot know the audience of its SETs does not start. |
 | `easyssf.receiver.delivery-method` | `push` | `push` or `poll`. |
 | `easyssf.receiver.understood-subject-members` | `user, session, device, application, tenant, org_unit, group` | The members of a complex subject the application interprets; a SET with a member the transmitter declared critical that is not listed is rejected. |
 | `easyssf.receiver.event-aliases.*` | | Aliases for event type URIs, e.g. `AcmeLogin: https://events.acme.example/login`, usable wherever an event type is named. |
 | `easyssf.receiver.http.use-rest-client` | `true` | Call the transmitter with the `RestClient` of the application if it has a `RestClient.Builder`. |
 | `easyssf.receiver.http.connect-timeout` / `read-timeout` | `5s` | Calls to the transmitter. Unset, `spring.http.clients.*` applies to the `RestClient` before the 5 seconds do. |
 | `easyssf.receiver.http.user-agent` | default of the HTTP client | `User-Agent` header for calls to the transmitter. |
+| `easyssf.receiver.http.max-response-size` | `4MB` | Largest response body accepted from the transmitter (metadata, JWK Set, stream configuration, poll response); a larger one fails the call. |
 | `easyssf.receiver.transmitter-access-token` | | Static access token for stream management and polling. |
 | `easyssf.receiver.oauth2.token-uri` | | Token endpoint for the client credentials grant. |
 | `easyssf.receiver.oauth2.client-id` / `client-secret` | | |
@@ -453,8 +465,8 @@ name, with its own status. Switch it off with `management.health.easyssf.enabled
 | `easyssf.receiver.stream.id` | | Stream created at the transmitter, looked up on startup. |
 | `easyssf.receiver.stream.events-requested` | `CaepSessionRevoked`, `CaepCredentialChange` | For a stream managed by the receiver. |
 | `easyssf.receiver.stream.description` | | For a stream managed by the receiver. |
-| `easyssf.receiver.stream.delete-on-shutdown` | `false` | For a stream managed by the receiver. |
-| `easyssf.receiver.poll.endpoint-url` | from the stream | |
+| `easyssf.receiver.stream.delete-on-shutdown` | `false` | For a stream managed by the receiver. Single instance only: the stream is shared, see [Stream management](#stream-management). |
+| `easyssf.receiver.poll.endpoint-url` | from the stream | Held to the same `https` rule as the transmitter issuer. |
 | `easyssf.receiver.poll.auto-startup` | `true` | |
 | `easyssf.receiver.poll.interval` | `30s` | |
 | `easyssf.receiver.poll.initial-delay` | `1s` | |