fix(middleware): accept the full engine config and route exclusions through the engine

README.md

@@ -187,24 +187,56 @@ generic JSON body); with it disabled they only log a warning.
 | `PREST_GUARD_BLACKLIST` | empty | Blocked IPs/CIDRs (403) |
 | `PREST_GUARD_WHITELIST` | empty | Allowed IPs/CIDRs; when set, everyone else is denied |
 | `PREST_GUARD_EXEMPT_IPS` | empty | IPs/CIDRs that skip checks (still subject to active bans) |
-| `PREST_GUARD_EXCLUDE_PATHS` | empty | Extra paths whose subtree skips all guard checks |
+| `PREST_GUARD_EXCLUDE_PATHS` | empty | Extra paths whose subtree the engine scopes as excluded (detection and policy checks skip; IP bans, IP policy and rate limits still run, see below) |
 | `PREST_GUARD_TRUSTED_PROXIES` | empty | Proxy IPs/CIDRs whose `X-Forwarded-For` is trusted |
+| `PREST_GUARD_TRUSTED_PROXY_DEPTH` | engine default (`1`) | How many forwarded hops from the right belong to your own proxies |
 | `PREST_GUARD_BLOCK_CLOUD_PROVIDERS` | empty | Block datacenter ranges (`AWS,GCP,Azure`) |
 | `PREST_GUARD_MAX_BODY_BYTES` | `8192` | Body prefix the inspection layer reads |
 | `PREST_GUARD_REDIS_URL` | empty | Share rate limit and ban state across instances |
 | `PREST_GUARD_REDIS_PREFIX` | `prest_guard` | Key namespace in Redis |
 | `PREST_GUARD_REDIS_FAIL_OPEN` | `true` | Keep serving from per-instance memory when Redis is down |
+| `PREST_GUARD_ENGINE_CONFIG` | empty | Inline JSON object over the full engine config (see below) |
+| `PREST_GUARD_ENGINE_CONFIG_FILE` | empty | Path to a JSON file with the same content; set only one of the two |
+
+### Full engine config
+
+The variables above are a subset of the engine's `SecurityConfig`. Everything else (excluded detection headers, params and body fields, `Detection` tuning, `FailSecure`, endpoint rate limits, `EnforceHTTPS`, security headers, CORS, custom error responses, global behavior rules, blocked user agents, country lists, dynamic rules, emergency mode, agent settings, and the rest) is reachable through `PREST_GUARD_ENGINE_CONFIG` or `PREST_GUARD_ENGINE_CONFIG_FILE`: a JSON object whose keys are `guardcore.SecurityConfig` field names. It is decoded over the config the `PREST_GUARD_*` variables produce, so a key named in the JSON wins over the matching variable and the keys it does not name keep their values. Nested structs and maps merge into the engine defaults, slices and scalars replace them, durations are integer nanoseconds, and `ExcludePaths` can only add to the built-in defaults. Unknown keys, malformed JSON, func and interface fields, and values the engine rejects are loud config errors: an enabled guard answers 500 instead of running with a config you did not ask for.
+
+```bash
+export PREST_GUARD_ENGINE_CONFIG='{"BlockedUserAgents":["sqlmap"],"CustomErrorResponses":{"403":"blocked"},"ExcludedDetectionParams":{"_where":true}}'
+```
+
+Fields JSON cannot carry (`CustomResponseModifier`, `AgentHandler`, `GeoIPHandler`, `CustomRequestCheck`, `OnError`) are set from Go: a plugin built from a fork wraps `Load` with its own `GuardConfig.Configure`, which receives the fully built engine config last.
+
+```go
+func GuardMiddlewareLoad() negroni.Handler {
+	return Load(func() GuardConfig {
+		cfg := ParseGuardConfigFromEnv()
+		cfg.Configure = func(c *guardcore.SecurityConfig) {
+			c.CustomRequestCheck = myCheck
+		}
+		return cfg
+	})
+}
+```
+
+`CustomResponseModifier` takes effect once the pinned `nethttp-guard` release applies it (the nethttp adapter wires the engine's response modifier from the release after v1.4.0).
 
 ### Defaults that prevent outages
 
 Three defaults exist specifically because most pREST deployments sit behind
 a load balancer:
 
-- `/_health`, `/_ready`, and the docs/static paths are excluded from every
-  guard check, always. Health probes share the proxy's identity with all
-  user traffic; rate limiting or banning them takes pods out of rotation.
-  Operator entries in `EXCLUDE_PATHS` are merged with these, not replacing
-  them.
+- `/_health`, `/_ready`, and the docs/static paths are always excluded,
+  and operator entries in `EXCLUDE_PATHS` are merged with them, not
+  replacing them. Exclusion is the engine's own scoping, the same as in the
+  other adapters and the reference: detection and the route and request
+  policy checks skip these paths, and their responses still carry the
+  security headers and count in the metrics. IP bans, the IP blacklist and
+  whitelist, and rate limits are still enforced on them. Health probes share
+  the proxy's identity with all user traffic, so list the probe source in
+  `EXEMPT_IPS` (and set `TRUSTED_PROXIES`) to keep a rate limit from taking
+  pods out of rotation.
 - Auto-banning is opt-in. With an empty `TRUSTED_PROXIES`, every client
   shares the load balancer's IP, so a handful of violations would ban all
   users at once. Starting the guard without `TRUSTED_PROXIES` logs a loud
@@ -214,13 +246,7 @@ a load balancer:
 
 ### Trusted proxies
 
-When the direct peer is a trusted proxy, the client IP is resolved by
-walking `X-Forwarded-For` from the right and taking the rightmost hop that
-is not itself trusted (the leftmost entry when every hop is trusted, every
-header line joined). Forwarded headers from untrusted peers are ignored
-completely, and a malformed hop voids the whole chain. This mirrors the
-reference engine's adapter-side contract; the engine separately flags
-untrusted peers that send `X-Forwarded-For` as spoof attempts.
+The engine resolves the client identity (the reference `extract_client_ip`), the plugin only hands it the connection's `RemoteAddr`. When the direct peer is in `TRUSTED_PROXIES`, the client is the hop `TRUSTED_PROXY_DEPTH` places from the right of `X-Forwarded-For` (default 1, the entry your proxy appended; set it to the number of proxies in front of pREST). A depth that over-counts the hops falls back to the right-to-left walk past trusted entries, and a chain shorter than the depth keeps the peer. Forwarded headers from untrusted peers are ignored, and the engine flags them as spoof attempts. Earlier versions walked the chain themselves and took the rightmost untrusted hop; with several trusted proxies in a row, set the depth instead.
 
 ### Redis modes
 
@@ -254,21 +280,12 @@ inspection on, full middleware path):
 
 | Body size | Pass path (prose JSON) | Reject path (threat) |
 |---|---|---|
-| 8 KiB (default) | ~59 ms | ~59 ms |
-| 16 KiB | ~140 ms | ~139 ms |
-| 64 KiB | ~1.16 s | ~1.25 s |
-| 256 KiB | ~13 s | ~16 s |
-
-Rejection is not cheaper than passing; cost is body-size driven. Keep the
-default unless you have measured acceptance for the cost at your bound:
-eight concurrent clients posting 64 KiB bodies will saturate one core.
-
-The numbers above are large because guard-core-go does not yet port the
-reference engine's large-value truncation (Python guard-core caps
-full-pattern scans at 256 KiB and rebuilds a bounded scan budget around
-detected attack regions). Until that lands engine-side, treat
-`MAX_BODY_BYTES` as a CPU budget, not a feature dial. Rate limits and IP
-policy alone do not read bodies and cost about 4 microseconds per request.
+| 8 KiB (default) | ~28 ms | ~20 ms |
+| 16 KiB | ~47 ms | ~39 ms |
+| 64 KiB | ~0.22 s | ~0.20 s |
+| 256 KiB | ~1.2 s | ~1.2 s |
+
+Cost still grows with body size, so keep the default unless you have measured acceptance for the cost at your bound. The engine caps what it scans (its large-value truncation, `Detection.MaxBodyInspectBytes` and the scan budget), which is why the 1 MiB bound costs about the same as 256 KiB. Treat `MAX_BODY_BYTES` as a CPU budget, not a feature dial. Rate limits and IP policy alone do not read bodies and cost about 9 microseconds per request.
 
 ## Verdict codes
 

config.go

@@ -91,8 +91,13 @@ type GuardConfig struct {
 	ExcludePaths []string
 	// TrustedProxies lists proxy IPs/CIDRs whose X-Forwarded-For is trusted
 	// when resolving the real client IP. Empty means forwarded headers are
-	// ignored and the direct peer is the client.
+	// ignored and the direct peer is the client. The engine resolves the
+	// client (the reference extract_client_ip), not this plugin.
 	TrustedProxies []string
+	// TrustedProxyDepth is the engine's trusted_proxy_depth: how many
+	// forwarded hops from the right belong to your own proxies. 0 keeps the
+	// engine default (1).
+	TrustedProxyDepth int
 	// BlockCloudProviders blocks datacenter ranges ("AWS", "GCP", "Azure").
 	BlockCloudProviders []string
 
@@ -105,6 +110,19 @@ type GuardConfig struct {
 	// initialization retries in the background, and the guard goes live when
 	// Redis returns.
 	RedisFailOpen bool
+
+	// EngineConfigJSON is a JSON object whose keys are guardcore.SecurityConfig
+	// field names (see the README "Full engine config" section). It is decoded
+	// over the config the fields above produce, so it reaches every engine
+	// setting without a dedicated variable. Unknown keys and non-serializable
+	// fields are an error, never silently dropped.
+	EngineConfigJSON string
+	// Configure is the Go-level hook for a plugin built from a fork: it gets
+	// the fully built engine config last (after the fields above and
+	// EngineConfigJSON), and can set the func and interface fields JSON
+	// cannot carry (CustomResponseModifier, AgentHandler, GeoIPHandler,
+	// CustomRequestCheck, ...).
+	Configure func(*guardcore.SecurityConfig)
 }
 
 // ParseGuardConfig reads every PREST_GUARD_* variable through lookup. Each
@@ -231,6 +249,32 @@ func ParseGuardConfig(lookup func(string) (string, bool)) GuardConfig {
 		cfg.BlockCloudProviders = splitList(raw)
 	}
 
+	if raw, ok := lookup(envPrefix + "TRUSTED_PROXY_DEPTH"); ok {
+		if parsed, err := parseInt(raw); err != nil {
+			fail("PREST_GUARD_TRUSTED_PROXY_DEPTH: %q is not a valid integer", raw)
+		} else if parsed <= 0 {
+			fail("PREST_GUARD_TRUSTED_PROXY_DEPTH: %d is not positive", parsed)
+		} else {
+			cfg.TrustedProxyDepth = parsed
+		}
+	}
+
+	inline, _ := lookup(envPrefix + "ENGINE_CONFIG")
+	file, _ := lookup(envPrefix + "ENGINE_CONFIG_FILE")
+	inline, file = strings.TrimSpace(inline), strings.TrimSpace(file)
+	switch {
+	case inline != "" && file != "":
+		fail("PREST_GUARD_ENGINE_CONFIG and PREST_GUARD_ENGINE_CONFIG_FILE are both set; use one")
+	case inline != "":
+		cfg.EngineConfigJSON = inline
+	case file != "":
+		if data, err := os.ReadFile(file); err != nil {
+			fail("PREST_GUARD_ENGINE_CONFIG_FILE: %v", err)
+		} else {
+			cfg.EngineConfigJSON = string(data)
+		}
+	}
+
 	if raw, ok := lookup(envPrefix + "REDIS_URL"); ok {
 		cfg.RedisURL = strings.TrimSpace(raw)
 	}

configure_test.go

@@ -0,0 +1,38 @@
+package main
+
+import (
+	"net/http"
+	"testing"
+
+	"github.com/rennf93/guard-core-go/v4/guardcore"
+)
+
+// GuardConfig.Configure is the Go-level escape hatch for a plugin built from
+// a fork: it receives the fully built engine config (func and interface
+// fields such as CustomResponseModifier, AgentHandler or GeoIPHandler
+// included) after the env subset and the engine config JSON were applied.
+func TestConfigureHookEditsTheEngineConfigLast(t *testing.T) {
+	cfg := ParseGuardConfig(lookupMap(map[string]string{
+		"PREST_GUARD_ENABLED":       "true",
+		"PREST_GUARD_ENGINE_CONFIG": `{"BlockedUserAgents": ["badbot"]}`,
+	}))
+	cfg.Configure = func(c *guardcore.SecurityConfig) {
+		c.BlockedUserAgents = []string{"worsebot"}
+	}
+	h := Load(parseConst(cfg))
+	if code, _, _ := serveWithHeaders(t, h, "/articles", "198.51.100.2", map[string]string{"User-Agent": "badbot/1.0"}); code != http.StatusOK {
+		t.Fatalf("the hook runs after the JSON and replaces it, got %d", code)
+	}
+	if code, _, _ := serveWithHeaders(t, h, "/articles", "198.51.100.2", map[string]string{"User-Agent": "worsebot/1.0"}); code != http.StatusForbidden {
+		t.Fatalf("the hook's own setting must apply, got %d", code)
+	}
+}
+
+func TestConfigureHookRunsBeforeValidation(t *testing.T) {
+	cfg := baseEnabled(nil)
+	cfg.Configure = func(c *guardcore.SecurityConfig) { c.Blacklist = []string{"not-an-ip"} }
+	h := Load(parseConst(cfg))
+	if code, _, _ := serveWithHeaders(t, h, "/articles", "198.51.100.2", nil); code != http.StatusInternalServerError {
+		t.Fatalf("an invalid hook result must fail closed, got %d", code)
+	}
+}

exclusions.go

@@ -1,8 +1,13 @@
 package main
 
-import "strings"
+import (
+	"path"
+	"strings"
+)
 
-// pathExclusions holds normalized exclude_paths entries.
+// pathExclusions holds normalized exclude_paths entries. It is only the
+// pre-engine matcher of deferredHandler (the 503 window before an engine
+// exists); once the engine is up, exclusion scoping is the engine's own.
 type pathExclusions struct {
 	entries []string
 }
@@ -24,9 +29,11 @@ func newPathExclusions(entries []string) *pathExclusions {
 // matches reports whether path falls inside one of the exclusion subtrees,
 // using the same subtree-or-equal semantics as the engine's exclude path
 // matcher: an entry "/health" covers "/health" and anything under "/health/".
-func (p *pathExclusions) matches(path string) bool {
+func (p *pathExclusions) matches(urlPath string) bool {
+	// Clean first: "/_health/../articles" must not ride an excluded prefix.
+	urlPath = path.Clean("/" + urlPath)
 	for _, entry := range p.entries {
-		if entry == "/" || path == entry || strings.HasPrefix(path, entry+"/") {
+		if entry == "/" || urlPath == entry || strings.HasPrefix(urlPath, entry+"/") {
 			return true
 		}
 	}

exclusions_test.go

@@ -36,3 +36,13 @@ func TestPathExclusionsSubtreeSemantics(t *testing.T) {
 		t.Fatal("unrelated path must not match")
 	}
 }
+
+func TestPathExclusionsDoNotFollowDotDotOutOfASubtree(t *testing.T) {
+	ex := newPathExclusions([]string{"/_health"})
+	if ex.matches("/_health/../articles") {
+		t.Fatal("a dot-dot path leaving the subtree must not match")
+	}
+	if !ex.matches("/_health/./ready") {
+		t.Fatal("a cleaned path inside the subtree must match")
+	}
+}

go.mod

@@ -4,7 +4,7 @@ go 1.26.0
 
 require (
 	github.com/rennf93/guard-core-go/v4 v4.3.2
-	github.com/rennf93/nethttp-guard v1.3.0
+	github.com/rennf93/nethttp-guard v1.4.0
 	github.com/urfave/negroni/v3 v3.1.1
 )
 

go.sum

@@ -14,8 +14,8 @@ github.com/redis/go-redis/v9 v9.23.0 h1:/TTVdJa8BRNsybwwGGnmiYktHNlUXtbOQxlWpHBP
 github.com/redis/go-redis/v9 v9.23.0/go.mod h1:EWP2UUk+XDaI1s5nQB5mablZN6vccXlmBQTyfnF87D0=
 github.com/rennf93/guard-core-go/v4 v4.3.2 h1:UpEUaSw1MNIoPcMbikG2hPZr+rhqp8vtCDasNh0m5DQ=
 github.com/rennf93/guard-core-go/v4 v4.3.2/go.mod h1:zbSvFuYHn8Oo6bMwJ7jJ3iR8CEHBIYUvf1ALd1qMAvU=
-github.com/rennf93/nethttp-guard v1.3.0 h1:XdsT31Cg2lrqy/0kiXievANz/KaZM1LdXttK6plfC38=
-github.com/rennf93/nethttp-guard v1.3.0/go.mod h1:504XNp7GO3F/ZZ1Keiy9j7SdfYfWCHHqlGpqrdQM76A=
+github.com/rennf93/nethttp-guard v1.4.0 h1:7CArpfVqAbfk0I5HYVr1Vvcbb2P9D8RiPbv3wrgiBDE=
+github.com/rennf93/nethttp-guard v1.4.0/go.mod h1:DP766KRBdBQXFJ44j08drxWLjV9WvO1vA4gXMJqTu+k=
 github.com/stretchr/testify v1.12.1 h1:EuwCh5fleGS7H32xRwO3wRGT7DxrDhLAT6FF8MpWDWE=
 github.com/stretchr/testify v1.12.1/go.mod h1:MDEgiDPPsNp5cuIrHPPCyornHKgEVbtFUmoNlxoYthg=
 github.com/urfave/negroni/v3 v3.1.1 h1:6MS4nG9Jk/UuCACaUlNXCbiKa0ywF9LXz5dGu09v8hw=