Repository navigation
Added callout proxy - #1286
Added callout proxy#1286
Conversation
Signed-off-by: Kristin Brown <kristin.brown@solo.io>
Deploying agentproxy with
|
| Latest commit: |
bd43580
|
| Status: | ⚡️ Build in progress... |
Signed-off-by: Kristin Brown <kristin.brown@solo.io>
Signed-off-by: Kristin Brown <kristin.brown@solo.io>
Signed-off-by: Kristin Brown <kristin.brown@solo.io>
Signed-off-by: Kristin Brown <kristin.brown@solo.io>
|
|
||
| ## Configure the callout proxy | ||
|
|
||
| You configure the callout proxy on the agentgateway proxy deployment through an {{< reuse "agw-docs/snippets/gatewayparameters.md" >}} resource, in either of the following ways. If you use both, `config.callouts.tunnel` takes precedence, and the agentgateway proxy ignores the proxy environment variables for callouts. |
There was a problem hiding this comment.
move If you use both, config.callouts.tunnel takes precedence, and the agentgateway proxy ignores the proxy environment variables for callouts. to after the options (next paragraph)
There was a problem hiding this comment.
Done. The precedence sentence now comes after the two options. I made the same move on the standalone page.
| | `HTTPS_PROXY` | The forward proxy for callouts that use TLS. | | ||
| | `HTTP_PROXY` | The forward proxy for plaintext callouts. | | ||
| | `ALL_PROXY` | The forward proxy for callouts when you omit `HTTPS_PROXY` or `HTTP_PROXY`. | | ||
| | `NO_PROXY` | A comma-separated list of destinations that bypass the forward proxy. Each entry is a hostname, an IP address, a Classless Inter-Domain Routing (CIDR) range, or `*` for every destination. A hostname also matches its subdomains. | |
There was a problem hiding this comment.
would I se NO_PROXY alongside HTTPS_PROXY, etc? Also, I think it would be good to show an example config, because I am not sure what exactly to put into these variables
There was a problem hiding this comment.
Yes, you set NO_PROXY alongside HTTPS_PROXY. I added a sentence that says so, plus an example spec.env block that sets HTTPS_PROXY, HTTP_PROXY, and NO_PROXY. The standalone page gets the same example as export commands.
| * Destinations that match `noProxy` or `NO_PROXY`. | ||
| * Loopback and link-local IP addresses, such as `127.0.0.1` and the cloud metadata server at `169.254.169.254`. A remote forward proxy cannot reach these addresses. | ||
| * Callouts to an {{< reuse "agw-docs/snippets/backend.md" >}} that has its own `tunnel` policy. These callouts use that tunnel instead. | ||
| * GCP credential fetches, when you configure the callout proxy with `config.callouts.tunnel`. To send these fetches through a forward proxy, use the environment variables instead. |
There was a problem hiding this comment.
for completion this should probably also list the jwks fetch that you have earlier
There was a problem hiding this comment.
Done. I added a bullet for remote JWKS fetches that says the control plane makes them, and links back to About callouts.
|
|
||
| To use `rawConfig` instead, replace `spec.env` with `spec.rawConfig.config.callouts.tunnel.url`, as shown in [rawConfig](#rawconfig). | ||
|
|
||
| 3. Attach the {{< reuse "agw-docs/snippets/gatewayparameters.md" >}} resource to your Gateway with `infrastructure.parametersRef`. The patch keeps the rest of your Gateway as it is. If your Gateway already references an {{< reuse "agw-docs/snippets/gatewayparameters.md" >}} resource, add the `spec.env` setting to that resource instead, because a Gateway references one resource only. |
There was a problem hiding this comment.
| 3. Attach the {{< reuse "agw-docs/snippets/gatewayparameters.md" >}} resource to your Gateway with `infrastructure.parametersRef`. The patch keeps the rest of your Gateway as it is. If your Gateway already references an {{< reuse "agw-docs/snippets/gatewayparameters.md" >}} resource, add the `spec.env` setting to that resource instead, because a Gateway references one resource only. | |
| 3. Attach the {{< reuse "agw-docs/snippets/gatewayparameters.md" >}} resource to your Gateway. The patch keeps the rest of your Gateway as it is. If your Gateway already references an {{< reuse "agw-docs/snippets/gatewayparameters.md" >}} resource, add the `spec.env` setting to that resource instead, because a Gateway can reference one {{< reuse "agw-docs/snippets/gatewayparameters.md" >}} resource only. |
There was a problem hiding this comment.
Applied.
| kubectl rollout status deployment/agentgateway-proxy -n {{< reuse "agw-docs/snippets/namespace.md" >}} | ||
| ``` | ||
|
|
||
| 5. Check the agentgateway proxy logs for the line that names the forward proxy. The label selector reads every agentgateway proxy pod, so the command also works while an old pod is still shutting down. |
There was a problem hiding this comment.
| 5. Check the agentgateway proxy logs for the line that names the forward proxy. The label selector reads every agentgateway proxy pod, so the command also works while an old pod is still shutting down. | |
| 5. Check the agentgateway proxy logs for the line that names the forward proxy. |
There was a problem hiding this comment.
Applied.
| EOF | ||
| ``` | ||
|
|
||
| 7. Create an {{< reuse "agw-docs/snippets/policy.md" >}} that sends an external authorization request for every request to the httpbin route. The `/status/200` path makes `httpbin.org` allow every request. The `http.path` value is a CEL expression, so the path is a string literal in double quotes. |
There was a problem hiding this comment.
| 7. Create an {{< reuse "agw-docs/snippets/policy.md" >}} that sends an external authorization request for every request to the httpbin route. The `/status/200` path makes `httpbin.org` allow every request. The `http.path` value is a CEL expression, so the path is a string literal in double quotes. | |
| 7. Create an {{< reuse "agw-docs/snippets/policy.md" >}} that sends an external authorization request for every request to the httpbin stand-in authorization server. The `/status/200` path makes `httpbin.org` allow every request. The `http.path` value is a CEL expression, so the path is a string literal in double quotes. |
There was a problem hiding this comment.
Applied.
| kubectl rollout restart deployment/agentgateway-proxy -n {{< reuse "agw-docs/snippets/namespace.md" >}} | ||
| kubectl rollout status deployment/agentgateway-proxy -n {{< reuse "agw-docs/snippets/namespace.md" >}} | ||
| kubectl exec -n squid deployment/squid -- cat /var/log/squid/access.log | ||
| ``` |
There was a problem hiding this comment.
I think what would be nice as another verification step when we would also show that the request got forwarded to the httpbin.org backend so you have the full flow. httpbin > squid > httpbin.org
There was a problem hiding this comment.
Good idea. The Squid log line in step 9 already shows the full path, but the page didn't explain it. I added a sentence and a small table that explains the fields: the source IP is the agentgateway pod, TCP_TUNNEL/200 means Squid opened the tunnel, and HIER_DIRECT/<ip> means Squid connected to httpbin.org. Would you also want a negative check, such as switching the path to /status/403 to show the request gets denied? I left that out for now because I haven't tested it.
| You configure the callout proxy on the agentgateway proxy deployment through an {{< reuse "agw-docs/snippets/gatewayparameters.md" >}} resource, in either of the following ways. If you use both, `config.callouts.tunnel` takes precedence, and the agentgateway proxy ignores the proxy environment variables for callouts. | ||
|
|
||
| * **[With environment variables](#environment-variables)**: Set the standard proxy environment variables, such as `HTTPS_PROXY`, in `spec.env`. Use this configuration for most setups. Note that this configuration is also the only way to send GCP credential fetches through the forward proxy. | ||
| * **[In `rawConfig`](#rawconfig)**: Set `config.callouts.tunnel` in `spec.rawConfig`. Use this way to keep the forward proxy settings separate from environment variables that other tools in the pod read. The control plane reads `rawConfig` only from an {{< reuse "agw-docs/snippets/gatewayparameters.md" >}} resource that a Gateway references. |
There was a problem hiding this comment.
Claude: rawConfig claim contradicts the controller (Kubernetes page, line 29). It says "The control plane reads rawConfig only from an AgentgatewayParameters resource that a Gateway references." ApplyToHelmValues runs for the GatewayClass parameters and then the Gateway parameters, and both apply RawConfig (controller/pkg/deployer/agentgateway_parameters.go:180 and :301-308). setIfNonNil means the Gateway's rawConfig replaces the GatewayClass's wholesale instead of merging.
There was a problem hiding this comment.
You're right, I checked agentgateway_parameters.go. The GatewayClass parameters apply first, and then the Gateway parameters, and setIfNonNil swaps in the Gateway's rawConfig as a whole. I removed the wrong sentence. The rawConfig section now says you can set it on parameters that a Gateway or a GatewayClass references, and that the Gateway's rawConfig replaces the GatewayClass's instead of merging with it.
| You configure the callout proxy on the agentgateway proxy deployment through an {{< reuse "agw-docs/snippets/gatewayparameters.md" >}} resource, in either of the following ways. If you use both, `config.callouts.tunnel` takes precedence, and the agentgateway proxy ignores the proxy environment variables for callouts. | ||
|
|
||
| * **[With environment variables](#environment-variables)**: Set the standard proxy environment variables, such as `HTTPS_PROXY`, in `spec.env`. Use this configuration for most setups. Note that this configuration is also the only way to send GCP credential fetches through the forward proxy. | ||
| * **[In `rawConfig`](#rawconfig)**: Set `config.callouts.tunnel` in `spec.rawConfig`. Use this way to keep the forward proxy settings separate from environment variables that other tools in the pod read. The control plane reads `rawConfig` only from an {{< reuse "agw-docs/snippets/gatewayparameters.md" >}} resource that a Gateway references. |
There was a problem hiding this comment.
Broken in-page anchors. The links point at #environment-variables, #rawconfig and #configuration-file, but the headings are "Option 1: …" and "Option 2: …"
There was a problem hiding this comment.
Fixed. I added explicit anchors to the headings ({#environment-variables}, {#rawconfig}, and {#configuration-file} on the standalone page). The links now resolve, and the headings can still say "Option 1" and "Option 2".
|
|
||
| You configure the callout proxy on the agentgateway proxy deployment through an {{< reuse "agw-docs/snippets/gatewayparameters.md" >}} resource, in either of the following ways. If you use both, `config.callouts.tunnel` takes precedence, and the agentgateway proxy ignores the proxy environment variables for callouts. | ||
|
|
||
| * **[With environment variables](#environment-variables)**: Set the standard proxy environment variables, such as `HTTPS_PROXY`, in `spec.env`. Use this configuration for most setups. Note that this configuration is also the only way to send GCP credential fetches through the forward proxy. |
There was a problem hiding this comment.
Claude thinks this is not true: The pages say the env vars are ignored when config.callouts.tunnel is set. That is not quite true for GCP, whose library still reads them if they are set.
There was a problem hiding this comment.
The anchors are also wrong, the built HTML has #option-1-environment-variables, #option-2-rawconfig, #option-1-configuration-file, and #option-2-environment-variables
There was a problem hiding this comment.
Correct. In callout.rs, config.callouts.tunnel makes agentgateway's own callout client skip the env vars. GCP credential fetches don't go through that client, though, and the GCP library still reads the env vars. I reworded both pages: tunnel takes precedence for most callouts, but GCP fetches never use the tunnel and still use the env vars if you set them.
| - **Streaming guardrails**: Agentgateway now rejects content if a provider guard fails while checking a streamed response or realtime connection. To keep the 1.5 behavior, set `failureMode: FailOpen` on the guard. For example, set it in `spec.backend.ai.promptGuard.response[].bedrockGuardrails` of an {{< reuse "agw-docs/snippets/policy.md" >}}. For more information, see [Provider failures]({{< link-hextra path="/documentation/llm/guardrails/overview/#provider-failures" >}}). | ||
| - **Token usage in responses**: The `usage` that agentgateway returns to clients now follows the conventions of the API that the client calls, whichever provider serves the request. Anthropic Messages responses leave cache tokens out of `input_tokens` and report them in `cache_read_input_tokens` and `cache_creation_input_tokens`. OpenAI Chat Completions and Responses include cache tokens in the input and total token counts. For example, a Chat Completions request to an Anthropic or Amazon Bedrock model that writes to the prompt cache now reports those cache tokens in `prompt_tokens`. If a client calculates usage or cost from these fields, review its counts after you upgrade. | ||
| - **Deployer ownership**: The controller no longer overwrites an existing resource of the same name that it does not own. It reports an error instead. | ||
| - **Proxy environment variables**: Callouts from the proxy, such as external authorization requests, guardrail requests, and AWS and Azure credential fetches, now use the `HTTPS_PROXY`, `HTTP_PROXY`, `ALL_PROXY`, and `NO_PROXY` environment variables. If you set these variables in `spec.env` of the {{< reuse "agw-docs/snippets/gatewayparameters.md" >}} resource for another purpose, callouts go through that proxy after you upgrade. Callouts to a Kubernetes Service go directly to the Service. Callouts to an {{< reuse "agw-docs/snippets/backend.md" >}} with a `static` host go through the proxy, even for a `svc.cluster.local` hostname, unless `NO_PROXY` lists `svc.cluster.local`. To keep all callouts direct, set `NO_PROXY` to `*`. In 1.6, only Google Cloud Platform (GCP) credential fetches used these variables. For more information, see [Callout proxy]({{< link-hextra path="/documentation/setup/customize/callout-proxy/" >}}). |
There was a problem hiding this comment.
| - **Proxy environment variables**: Callouts from the proxy, such as external authorization requests, guardrail requests, and AWS and Azure credential fetches, now use the `HTTPS_PROXY`, `HTTP_PROXY`, `ALL_PROXY`, and `NO_PROXY` environment variables. If you set these variables in `spec.env` of the {{< reuse "agw-docs/snippets/gatewayparameters.md" >}} resource for another purpose, callouts go through that proxy after you upgrade. Callouts to a Kubernetes Service go directly to the Service. Callouts to an {{< reuse "agw-docs/snippets/backend.md" >}} with a `static` host go through the proxy, even for a `svc.cluster.local` hostname, unless `NO_PROXY` lists `svc.cluster.local`. To keep all callouts direct, set `NO_PROXY` to `*`. In 1.6, only Google Cloud Platform (GCP) credential fetches used these variables. For more information, see [Callout proxy]({{< link-hextra path="/documentation/setup/customize/callout-proxy/" >}}). | |
| - **Proxy environment variables**: Callouts from the proxy, such as external authorization requests, guardrail requests, and AWS and Azure credential fetches, now use the `HTTPS_PROXY`, `HTTP_PROXY`, `ALL_PROXY`, and `NO_PROXY` environment variables to configure a forward proxy that these requests must be routed through. For more information, see [Callout proxy]({{< link-hextra path="/documentation/setup/customize/callout-proxy/" >}}). |
There was a problem hiding this comment.
I took most of this, but I kept two parts of the old text. Without them, this bullet doesn't say why it's in Behavior changes. The bullet now uses your first sentence, and then keeps "In 1.6, only GCP fetches used these variables," the warning about existing spec.env variables, and the NO_PROXY=* opt-out. I dropped the static-host detail, because the page covers it. More on why it stays in Behavior changes in the standalone thread.
| - **Telemetry**: OTLP access logs use the `agentgateway.access` instrumentation scope. The request duration metric records failed requests with an `error_type` label. | ||
| - **Istio permissions**: Set the `istio.enabled=false` Helm value to drop the Istio permissions from the controller ClusterRole. For more information, see [Istio resource discovery]({{< link-hextra path="/documentation/install/advanced/#istio-discovery" >}}). | ||
| - **Gateway name in proxy metrics**: When `monitoring.enabled` is `true`, the Helm chart creates a PodMonitor. The PodMonitor now copies the `gateway.networking.k8s.io/gateway-name` pod label onto proxy metrics. This label lets the Grafana dashboard filter by Gateway. To change the copied labels, set the `monitoring.proxy.podMonitor.podTargetLabels` Helm value. | ||
| - **Callout proxy**: You can now send the requests that the agentgateway proxy makes on its own behalf, such as external authorization requests, guardrail requests, and cloud provider credential fetches, through a forward proxy. Set the standard proxy environment variables in `spec.env` of an {{< reuse "agw-docs/snippets/gatewayparameters.md" >}} resource, or set `config.callouts.tunnel` in `spec.rawConfig`. For more information, see [Callout proxy]({{< link-hextra path="/documentation/setup/customize/callout-proxy/" >}}). |
There was a problem hiding this comment.
| - **Callout proxy**: You can now send the requests that the agentgateway proxy makes on its own behalf, such as external authorization requests, guardrail requests, and cloud provider credential fetches, through a forward proxy. Set the standard proxy environment variables in `spec.env` of an {{< reuse "agw-docs/snippets/gatewayparameters.md" >}} resource, or set `config.callouts.tunnel` in `spec.rawConfig`. For more information, see [Callout proxy]({{< link-hextra path="/documentation/setup/customize/callout-proxy/" >}}). | |
| - **Callout proxy**: You can now send the requests that the agentgateway proxy makes on its own behalf, such as external authorization requests, guardrail requests, and cloud provider credential fetches, through a forward proxy. For more information, see [Callout proxy]({{< link-hextra path="/documentation/setup/customize/callout-proxy/" >}}). |
There was a problem hiding this comment.
Applied, without the double space.
| You can configure the callout proxy in either of the following ways. If you use both, `config.callouts.tunnel` takes precedence, and agentgateway ignores the proxy environment variables for callouts. | ||
|
|
||
| * **[In the configuration file](#configuration-file)**: Set `config.callouts.tunnel`. Use this method to keep the proxy settings with the rest of your configuration. | ||
| * **[With environment variables](#environment-variables)**: Set the standard proxy environment variables, such as `HTTPS_PROXY`. Use this method to send GCP credential fetches through the proxy, or to reuse variables that your environment already sets. |
There was a problem hiding this comment.
similar comments regarding wording, link anchors, etc. as in kube
There was a problem hiding this comment.
Done. I made the same changes as on the Kubernetes page: moved the precedence sentence, pinned the anchors, added the GCP exception, and added an env var example with NO_PROXY.
|
|
||
| ### Option 1: Configuration file | ||
|
|
||
| Set `config.callouts.tunnel` in your configuration file. Settings in the `config` section take effect at startup, so restart agentgateway after you change this setting. For more information, see [Static configuration]({{< link-hextra path="/documentation/configuration/static-configuration/" >}}). |
There was a problem hiding this comment.
| Set `config.callouts.tunnel` in your configuration file. Settings in the `config` section take effect at startup, so restart agentgateway after you change this setting. For more information, see [Static configuration]({{< link-hextra path="/documentation/configuration/static-configuration/" >}}). | |
| Set `config.callouts.tunnel` in your configuration file to configure the forward proxy that you want to route requests through. Settings in the `config` section take effect at startup, so restart agentgateway after you change this setting. For more information, see [Static configuration]({{< link-hextra path="/documentation/configuration/static-configuration/" >}}). |
There was a problem hiding this comment.
Applied.
|
|
||
| | Setting | Description | | ||
| | -- | -- | | ||
| | `config.callouts.tunnel.url` | The URL of the proxy. This proxy handles every callout, whether the callout uses Transport Layer Security (TLS) or plaintext. The scheme must be `http` or `https`. With `https`, the connection to the proxy uses TLS, and the proxy certificate is verified against the system trust bundle. If you omit the port, the default is `80` for `http` and `443` for `https`. To authenticate to the proxy with basic authentication, include the credentials in the URL, such as `http://user:password@proxy.example.com:3128`. The `--validate-only` flag does not check the scheme. A URL with an unsupported scheme, such as `socks5://`, passes validation, and then agentgateway fails to start with an `invalid callouts config` error. | |
There was a problem hiding this comment.
| | `config.callouts.tunnel.url` | The URL of the proxy. This proxy handles every callout, whether the callout uses Transport Layer Security (TLS) or plaintext. The scheme must be `http` or `https`. With `https`, the connection to the proxy uses TLS, and the proxy certificate is verified against the system trust bundle. If you omit the port, the default is `80` for `http` and `443` for `https`. To authenticate to the proxy with basic authentication, include the credentials in the URL, such as `http://user:password@proxy.example.com:3128`. The `--validate-only` flag does not check the scheme. A URL with an unsupported scheme, such as `socks5://`, passes validation, and then agentgateway fails to start with an `invalid callouts config` error. | | |
| | `config.callouts.tunnel.url` | The URL and port of the forward proxy. This proxy handles every callout, whether the callout uses Transport Layer Security (TLS) or plaintext. The scheme must be `http` or `https`. With `https`, the connection to the proxy uses TLS, and the proxy certificate is verified against the system trust bundle. If you omit the port, the default is `80` for `http` and `443` for `https`. To authenticate to the proxy with basic authentication, include the credentials in the URL, such as `http://user:password@proxy.example.com:3128`. The `--validate-only` flag does not check the scheme. A URL with an unsupported scheme, such as `socks5://`, passes validation, and then agentgateway fails to start with an `invalid callouts config` error. | |
There was a problem hiding this comment.
where is the --validate-only flag set?
There was a problem hiding this comment.
Applied. About --validate-only: it's a flag on the agentgateway command, and step 2 of the example uses it. I reworded the table to "The agentgateway --validate-only command, which checks a configuration file without starting agentgateway, does not check the scheme."
|
|
||
| {{< reuse "agw-docs/snippets/prereq-agentgateway.md" >}} | ||
|
|
||
| You also need an HTTP proxy that supports the CONNECT method and logs the requests that it receives, such as Squid. The following steps assume that the proxy listens on `http://127.0.0.1:3128`. |
There was a problem hiding this comment.
we might want to give them a docker command to bring up this container in Docker
There was a problem hiding this comment.
Claude had this to say, too about the Docker command. Also, this is not in ordered steps.
With
docker run -p 3128:3128 ubuntu/squid, a request from the host returns 403 on CONNECT, because the default ACL rejects the Docker bridge address. A permissivesquid.conffixed it:http_port 3128,acl all src all,http_access allow all. Squid then returned 200 and logged the CONNECT. I ran that recipe againstcurl, not against the standalone binary. Please add a short working recipe, for example a minimalsquid.confthat allows the test client, or link to a proxy setup that works.
There was a problem hiding this comment.
Done. Before you begin is now a numbered list. Step 2 has two substeps: create a squid.conf that allows any client, and run ubuntu/squid with that file mounted and port 3128 published. The squid.conf is your recipe (http_port 3128, acl all src all, http_access allow all), and the page says why it's needed: the default config returns 403 for the Docker bridge address.
| | Setting | Description | | ||
| | -- | -- | | ||
| | `config.callouts.tunnel.url` | The address of the proxy. The loopback bypass applies to callout destinations, not to the proxy. A proxy on `127.0.0.1` still receives the callouts. | | ||
| | `jwtAuth.jwks.url` | The remote JWKS. The JWKS fetch is a callout and goes through the proxy. | |
There was a problem hiding this comment.
| | `jwtAuth.jwks.url` | The remote JWKS. The JWKS fetch is a callout and goes through the proxy. | | |
| | `jwtAuth.jwks.url` | The remote JWKS. The JWKS fetch is a callout and goes through the forward proxy. | |
There was a problem hiding this comment.
general feedback: I would try to use forward proxy as much as possible since agentgateway is also a proxy and it is sometimes hard to understand which proxy we are talking about
There was a problem hiding this comment.
Applied. I also switched to "forward proxy" across the whole standalone page and the standalone release note. The page now says "proxy" alone only in env var names and in the "callout proxy" feature name.
| - **Token usage in responses**: The `usage` that agentgateway returns to clients now follows the conventions of the API that the client calls, whichever provider serves the request. Anthropic Messages responses leave cache tokens out of `input_tokens` and report them in `cache_read_input_tokens` and `cache_creation_input_tokens`. OpenAI Chat Completions and Responses include cache tokens in the input and total token counts. For example, a Chat Completions request to an Anthropic or Amazon Bedrock model that writes to the prompt cache now reports those cache tokens in `prompt_tokens`. If a client calculates usage or cost from these fields, review its counts after you upgrade. | ||
| - **Policy service timeouts**: Calls to external services default to a timeout of 2 seconds for gRPC external authorization and 10 seconds for rate limit and external processing services. | ||
| - **Streaming guardrails**: Agentgateway now rejects content if a provider guard fails while checking a streamed response or realtime connection. To keep the 1.5 behavior, set `failureMode: failOpen` on the guard. For example, set it in `ai.promptGuard.response[].bedrockGuardrails`. For more information, see [Provider failures]({{< link-hextra path="/documentation/llm/prompt-guards/overview/#provider-failures" >}}). | ||
| - **Proxy environment variables**: Callouts, such as JWKS fetches, external authorization requests, and AWS and Azure credential fetches, now use the `HTTPS_PROXY`, `HTTP_PROXY`, `ALL_PROXY`, and `NO_PROXY` environment variables when `config.callouts.tunnel` is omitted. In 1.6, only Google Cloud Platform (GCP) credential fetches used these variables. If the environment that runs agentgateway exports these variables for other tools, callouts go through that proxy after you upgrade. To keep callouts direct, set `NO_PROXY` to `*`. For more information, see [Callout proxy]({{< link-hextra path="/documentation/configuration/callout-proxy/" >}}). |
There was a problem hiding this comment.
wondering if we need it under behavior changes. isn't this just a new feature? same comment for kube
There was a problem hiding this comment.
I'd keep it in both places. The new feature is the callout proxy itself, so it goes under New features. The behavior change is that callouts now read HTTPS_PROXY and friends with no opt-in. So if someone already exports these variables for another tool, or sets them in spec.env, their ext auth, guardrail, and AWS or Azure credential calls start going through that proxy after upgrade. 1.6.0 ignored them (I tested this). That can break a working setup, so I think upgraders need to see it under Behavior changes. Happy to drop it if you disagree, though.
| info client::callout callouts will use proxy Matcher { http: Intercept { uri: http://127.0.0.1:3128/ }, https: Intercept { uri: http://127.0.0.1:3128/ } } | ||
| ``` | ||
|
|
||
| 4. Check the access log of your proxy for a CONNECT request to `www.googleapis.com:443`, the JWKS host. The log format depends on your proxy. |
There was a problem hiding this comment.
No log example output.
Also, claude thinks that Squid writes the CONNECT log entry only when the tunnel closes, just curious if you noticed this when you tried it? if so, might be worth a line like:
"Squid writes the entry when the tunnel closes. If the log is empty, stop agentgateway with Ctrl+C, and then check again."
There was a problem hiding this comment.
Yes, I saw this in Kubernetes too, and step 10 there covers it. Added to step 4: a docker exec squid cat /var/log/squid/access.log command for the Squid container from Before you begin, an example log line, and your note about stopping agentgateway with Ctrl+C if the log is empty.
Signed-off-by: Kristin Brown <kristin.brown@solo.io>
3 participants
Documents the callout proxy from agentgateway/agentgateway#3864 (Allow sending callout requests through CONNECT tunnel), which merged to
mainon 2026-10-09. The change is not in a tagged release yet, so the docs go in themaintrees only. Both modes get a page, because the data plane is the same binary in standalone and Kubernetes.Changes
Standalone
standalone/main/documentation/configuration/callout-proxy.md. Covers:config.callouts.tunnelin the configuration file or the standard proxy environment variables, and which one takes precedence.config.callouts.tunnel, and then repeats it withHTTPS_PROXY.egress-proxy.md: one sentence that points to the new page. Egress proxy covers the opposite direction, clients sending CONNECT requests through agentgateway, so the two pages are easy to mix up.Kubernetes
kubernetes/main/documentation/setup/customize/callout-proxy.md. Covers:spec.envorspec.rawConfig.config.callouts.tunnel.statichost goes through the proxy even for asvc.cluster.localhostname.HTTPS_PROXY, and sends an external authorization callout tohttpbin.orgthrough it.main, that points to the new page. Thetunnelpolicy covers one backend; the callout proxy covers every callout, and a backend's own tunnel takes precedence.spec.envand including the static-host gotcha.Testing
Tested with the nightly build
0.0.0-alpha.c0354ea4, which includes #3864, against a CONNECT proxy that logs each request. Where the docs claim a change from 1.6, the claim was compared against the v1.6.0 release binary.Standalone (local binary):
schema/config.jsonfrommain, andagentgateway --validate-onlyaccepts the example.HTTPS_PROXYand lowercasehttps_proxyboth work.NO_PROXY=*bypasses the proxy, and so does a subdomain match inNO_PROXY.HTTPS_PROXY, which confirms the upgrade note.noProxybypasses matching hosts.Proxy-Authorizationheader.config.callouts.tunneltakes precedence over the environment variables.POST http://host/...), not a CONNECT tunnel.backendTunneluses that tunnel instead.localhost:<port>goes through the proxy, because the loopback bypass applies only to IP addresses.--validate-onlyaccepts an unsupported scheme such assocks5://, and agentgateway then fails at startup withinvalid callouts config.failed to load JWKS.Kubernetes (kind, nightly
c0354ea4controller and proxy):spec.envorspec.rawConfigpointing at an unreachable address, an external auth callout to a static AgentgatewayBackend fails withexternal authorization failed. With Squid, the Squid log showsTCP_TUNNEL/200 CONNECT httpbin.org:443.*.svc.cluster.localhost goes through the proxy, andNO_PROXY=svc.cluster.localmakes it go direct.callouts will use proxyline appears with both configuration methods.Not tested yet. After the style review, steps 8 to 10 and the cleanup's
parametersRefpath changed and were not rerun. Steps 8 and 9 were also not run against the real Gateway, because that cluster had an unrelated JWT policy on it. These are the first candidates for the doc-tests run.Resolved during review
tunnelpolicy, with JWKS as its example. The gateway-wide callout proxy is new, so it gets its own page, and the tunnel page now points to it.tunnelpolicy applies. The Kubernetes page and release note no longer list JWKS.controller/pkg/agentgateway/jwks/fetcher.gospec.envandspec.rawConfig.config.callouts.tunnelwork, and callouts to Kubernetes Services bypass the proxy.servicesbypass apply to standalone?service:reference resolves from the top-levelserviceslist, and those backends skip the callout proxy.proxy/httpproxy.rs,proxy/mod.rsinternal_call_with_policies; webhook testedaws-smithy-http-client,reqwest; JWKS tested on v1.6.0The intro also claimed that every callout uses a CONNECT tunnel. That claim was wrong for plaintext callouts, which go to the proxy as forward-proxy requests, so the intro is corrected.
Still open
mainrelease notes still use the 1.6 headings (#v16-…). Entries from after 1.6, such as #3722, #3746, and #3773, sit under those headings even thoughlatestdoesn't list them in its 1.6 notes. These entries follow the same pattern. Themainfile probably needs a next-release skeleton.ubuntu/squid:latest, not a pinned tag.🤖 Generated with Claude Code