feat: cache the frontier authn token call in redis

README.md

@@ -8,7 +8,6 @@ Kong plugin to use with [frontier](https://github.com/raystack/frontier/) auth s
 ### TODO
 - Add test cases
 - https://github.com/lunarmodules/luacheck
-- cache frontier response - https://docs.konghq.com/gateway/latest/plugin-development/entities-cache/#cache-custom-entities
 
 ### Notes
 - Add plugin configuration in kong.yml file where url is a required field
@@ -65,6 +64,158 @@ disabled = {
     default = false
 }
 ```
+### Token caching
+
+Works on Kong 3.4 and later. It only uses modules that ship with Kong and
+OpenResty, so there is nothing extra to install.
+
+The plugin exchanges the incoming cookie or bearer for a user token on every
+request. Setting `redis_host` caches that exchange in redis, so the same
+credential is not exchanged again for a few seconds. The lookup is redis first,
+then the auth server on a miss.
+
+Redis is shared by every pod, so a token is fetched once for the whole fleet
+rather than once per pod.
+
+The default ttl is 5 seconds. It is deliberately short: a cached token means a
+change to someone's access is not picked up until the entry expires.
+
+**Caching needs redis.** Without `redis_host` there is nowhere to keep a token,
+so `cache_ttl` does nothing on its own and every request goes to the auth
+server, exactly as it did before this existed.
+
+```yaml
+plugins:
+- name: frontier
+  config:
+    authn_url: ...
+    redis_host: redis.internal
+    redis_port: 6379
+```
+
+| Field | Default | What it does |
+|---|---|---|
+| `cache_ttl` | `5` | Seconds a token is reused for. `0` turns caching off. Max `300` |
+| `cache_cookie_names` | `["sid"]` | Only these cookies go into the cache key |
+| `redis_host` | unset | Setting it turns caching on |
+| `redis_port` | `6379` | |
+| `redis_timeout` | `100` | Milliseconds, for connect, send and read |
+| `redis_username` | unset | Redis 6 ACL user, if you use one |
+| `redis_password` | unset | |
+| `redis_database` | `0` | |
+| `redis_ssl` | `false` | |
+| `redis_ssl_verify` | `false` | Needs `lua_ssl_trusted_certificate` set on the gateway |
+| `redis_server_name` | unset | SNI, when using SSL |
+| `redis_key_prefix` | `frontier:authn:` | Prefix on every key |
+| `redis_breaker_seconds` | `10` | How long a worker stops trying after a failure |
+
+Connections are reused through OpenResty's own connection pool, keyed by host,
+port, database, user and whether SSL is on. Two plugin configs that mean the
+same thing share a pool; two that differ do not. The pool is not configurable,
+the same way it is not in the bundled rate limiting plugin.
+
+The timeout default is 100ms, much lower than the bundled rate limiting
+plugin's 2000ms. A healthy redis answers in well under a millisecond, so 100ms
+is already a hundred times the expected latency. This sits in the auth path, so
+a redis slower than that should be given up on rather than held onto. The cost
+of being wrong is small: the request goes to the auth server instead, and the
+worker stops trying redis for `redis_breaker_seconds`.
+
+#### How redis behaves
+
+**It never fails a request.** Redis is a cache, not an authority. A connect
+error, a timeout, a bad reply, even a raise, is logged and the plugin carries on
+to the auth server.
+
+When a command to an instance fails, the worker stops trying that instance for
+`redis_breaker_seconds`, so an outage cannot make every request pay the timeout
+first. The pause is per instance, so a fault on one redis does not stop the
+worker talking to another. A wrong password or a bad database index is a config
+mistake rather than a broken instance, so those are logged without starting the
+pause.
+
+**Treat write access to this redis as equal to being any user.** The plugin
+never checks the token signature, with or without redis. It trusts whatever the
+auth server hands back. So anything that can write these keys can put a token of
+its choosing in front of the upstream. The entries also hold live user tokens,
+which is more sensitive than something like rate limit counters. Turn on auth
+and SSL if the instance is shared or reachable from outside the cluster, and
+keep `redis_key_prefix` set so the keys cannot collide with anything else using
+it.
+
+#### What it does and does not cache
+
+- The cache key is a sha256 of the cookies named in `cache_cookie_names`, the
+  authorization header, and the config that decides what an entry means:
+  `authn_url`, `http_method`, `header_name`, `token_response_field` and
+  `cache_ttl`. The session value is never stored in plain text, and two routes
+  that would resolve a credential differently cannot share an entry.
+- Only the named cookies go into the key. Browsers send analytics and consent
+  cookies that change constantly, so keying on the whole cookie header would
+  miss on nearly every request.
+- A request with none of those credentials is never cached, so anonymous
+  requests cannot share an entry.
+- A failed exchange is never cached. A user who has just been given access is
+  not locked out for the length of the ttl.
+- The entry lives for exactly `cache_ttl`. The token is never parsed, so
+  **`cache_ttl` has to stay well under your auth server's token lifetime**, or
+  the cache will hand out tokens that have already expired. Frontier mints a
+  fresh token on every call and its `token.validity` defaults to an hour, so
+  the default of 5 seconds leaves a very wide margin. The ceiling of 300 is
+  there so a careless value cannot get close.
+- Only the authn call is cached. The authz check in `authz_url` still runs on
+  every request.
+- There is no lock, so several requests arriving together with the same new
+  credential will each fetch a token. They all write an equivalent entry, and
+  every request after that is served from redis.
+- A token is read for its `exp` when it is stored, and for the claims that
+  become headers when it is used. Nothing else about it is assumed, so a token
+  that is valid JSON but is not shaped like a JWT is refused rather than half
+  applied.
+
+#### What it costs and saves
+
+Measured against a real Frontier and a real redis, with Kong in DB-less mode.
+Absolute numbers come from docker on macOS, where container networking is slow,
+so read the gaps rather than the values.
+
+One session making requests as fast as it can for 20 seconds, `cache_ttl` at 5:
+
+| | Requests served | Auth server calls |
+|---|---|---|
+| Caching on | 455 | 4 |
+| Caching off | 262 | 262 |
+
+Four calls in a 20 second window is what a 5 second ttl should give. The same
+client also got through 1.7 times as many requests, because it was not waiting
+on an auth call every time.
+
+Kong's own CPU per request, from the container's cgroup accounting, 500 requests
+per run over one reused connection, median of 5 runs:
+
+| | Kong CPU per request | requests/sec |
+|---|---|---|
+| No plugin | 0.224 ms | 293 |
+| Plugin, redis hit | 0.330 ms | 264 |
+| Plugin, caching off | 1.731 ms | 26 |
+
+A redis hit costs 0.11ms more CPU than plain proxying, for the cookie parse, the
+hash and the redis round trip. The auth call it replaces costs 1.5ms, about
+fourteen times more.
+
+Latency, with the plain route and the cached route interleaved to cancel drift:
+
+| Path | Median | p95 |
+|---|---|---|
+| No plugin | 6.8 ms | 13.8 ms |
+| Redis hit | 7.4 ms | 16.1 ms |
+| Auth server fetch | 39.2 ms | 52.0 ms |
+
+So the redis hop adds about 0.6ms and saves about 32ms.
+
+An entry costs about 1KB in redis, for a token of roughly 950 bytes, so size it
+as `users active within the ttl window x 1KB`.
+
 - For local development linting
 ```
 brew install wget

kong-plugin-frontier-0.1.1-1.rockspec

@@ -32,6 +32,8 @@ build = {
     ["kong.plugins."..plugin_name..".jwt_decoder"] = "kong/plugins/"..plugin_name.."/jwt_decoder.lua",
     ["kong.plugins."..plugin_name..".schema"] = "kong/plugins/"..plugin_name.."/schema.lua",
     ["kong.plugins."..plugin_name..".access"] = "kong/plugins/"..plugin_name.."/access.lua",
+    ["kong.plugins."..plugin_name..".cache"] = "kong/plugins/"..plugin_name.."/cache.lua",
+    ["kong.plugins."..plugin_name..".redis"] = "kong/plugins/"..plugin_name.."/redis.lua",
     ["kong.plugins."..plugin_name..".utils"] = "kong/plugins/"..plugin_name.."/utils.lua",
   }
 }

kong/plugins/frontier/access.lua

@@ -3,6 +3,7 @@ local _M = {}
 local http = require "resty.http"
 local json = require('cjson')
 local jwt_decoder = require "kong.plugins.frontier.jwt_decoder"
+local cache = require "kong.plugins.frontier.cache"
 local kong = kong
 local ngx = ngx
 local utils = require "kong.plugins.frontier.utils"
@@ -26,12 +27,13 @@ end
 
 local function get_http_client(conf)
     local client = http.new()
-    client:set_timeouts(conf.http_connect_timeout, conf.http_read_timeout, conf.http_send_timeout)
+    local connect_timeout, send_timeout, read_timeout =
+        conf.http_connect_timeout, conf.http_send_timeout, conf.http_read_timeout
+    client:set_timeouts(connect_timeout, send_timeout, read_timeout)
     return client
 end
 
--- send a request to auth server and fetch user token in exchange of cookies
-local function check_request_identity(conf, cookies, bearer)
+local function fetch_identity_token(conf, cookies, bearer)
     local client = get_http_client(conf)
     local correlation_id = kong.request.get_header(conf.correlation_header_name)
 
@@ -59,13 +61,11 @@ local function check_request_identity(conf, cookies, bearer)
     local res, err = client:request_uri(conf.authn_url, request_options)
     if not res or err then
         kong.log.warn("failed to check request identity: ", err)
-        return fail_auth()
+        return nil, err or "no response from auth server"
     end
     if not err and res and res.status ~= 200 then
         kong.log.warn("received non 200 response status: ", res.status)
-        return kong.response.exit(ngx.HTTP_UNAUTHORIZED, unauthorized_response, {
-            ["x-upstream-status"] = res.status
-        })
+        return nil, "non 200 response status", res.status
     end
 
     kong.log.debug("check_request_identity: Received successful response with status: ", res.status)
@@ -89,6 +89,44 @@ local function check_request_identity(conf, cookies, bearer)
     end
 
     kong.log.debug("check_request_identity: Returning token: ", token and "found" or "not found")
+
+    if not token then
+        return nil, "no token in auth server response"
+    end
+
+    return token, nil, nil
+end
+
+local function check_request_identity(conf, cookies, bearer)
+    local auth_server_status
+
+    local function fetch()
+        local token, err, status = fetch_identity_token(conf, cookies, bearer)
+        auth_server_status = status
+
+        return token, err
+    end
+
+    local token, err
+
+    if conf.cache_ttl > 0 then
+        token, err = cache.get(conf, cache.build_key(conf, cookies, bearer), fetch)
+    else
+        token, err = fetch()
+    end
+
+    if not token then
+        kong.log.warn("failed to resolve user token: ", err)
+
+        if auth_server_status then
+            return kong.response.exit(ngx.HTTP_UNAUTHORIZED, unauthorized_response, {
+                ["x-upstream-status"] = auth_server_status
+            })
+        end
+
+        return fail_auth()
+    end
+
     return token
 end
 
@@ -186,6 +224,13 @@ local function append_claims_as_headers(conf, user_token)
 
     local claims = jwt.claims
 
+    local claims_are_readable = type(claims) == "table"
+
+    if not claims_are_readable then
+        kong.log.warn("token payload is not an object, cannot read claims")
+        return fail_auth()
+    end
+
     for _, header_name in pairs(conf.token_claims_to_append_as_headers) do
         local new_header = conf.frontier_header_prefix .. header_name
         local val = claims[header_name]
@@ -208,12 +253,15 @@ local function verify_organization_id_header(conf, user_token)
         end
 
         local claims = jwt.claims
-        local org_ids = claims[frontier_org_ids_claim_key]
+        local org_ids = type(claims) == "table" and claims[frontier_org_ids_claim_key] or nil
 
         local org_id_header_verified = false
-        for word in string.gmatch(org_ids, '([^,]+)') do
-            if word == request_organization_id then
-                org_id_header_verified = true
+
+        if type(org_ids) == "string" then
+            for word in string.gmatch(org_ids, '([^,]+)') do
+                if word == request_organization_id then
+                    org_id_header_verified = true
+                end
             end
         end