feat: Mask secrets before sending logs to the AI provider, with a preview of what is sent

README.md

@@ -24,6 +24,8 @@ through a long console to find the one line that matters.
 - **Twelve AI providers**: Anthropic (Claude), AWS Bedrock, Azure OpenAI, Custom Okta AI, DeepSeek, Google Gemini,
   LangGraph Platform, Microsoft Foundry, Ollama, OpenAI, OpenAI Compatible (LiteLLM, OpenWebUI and other gateways) and
   Qwen, including private endpoints, company gateways and models you host yourself
+- **Secrets masked before sending**: tokens, keys and passwords in the log are replaced with `[REDACTED]`, with
+  patterns of your own and a preview of what is sent. See [Masking secrets](docs/configuration.md#masking-secrets)
 - **Folder settings**, so each team can use its own provider, language and instructions
 - **Request quotas and usage metrics** to keep provider costs in check
 - **AI auto-fix** *(experimental)*: opens a pull request on GitHub, GitLab or Bitbucket with a proposed fix
@@ -64,6 +66,8 @@ Open the console of a failed build and click **Explain Error**. The explanation
 at the top: a title that names the root cause, a summary, the steps to fix it, the log lines where it failed and, when
 there are any, practices that would have prevented it. It reads up to 200 lines of the steps that failed, or the global
 **Max Log Lines** when that is higher, and only the lines the global **Log Pattern** matches when one is set.
+**Preview what is sent**, next to the button, shows those lines as the provider would get them, with secrets masked,
+without sending anything.
 
 Once a build has an explanation, the button reads **Show explanation**, and the card says when the explanation was
 generated; the re-explain icon asks the provider again. If a request fails, the card stays open with what went wrong

docs/configuration.md

@@ -20,8 +20,70 @@ folder that sets it), then global.
 | **Automatically explain failed builds** | Explains every failed build without a pipeline change. See [Automatic Explanation](auto-explain.md) | Off |
 | **Max Log Lines** | Log lines an automatic explanation sends, shared out among the steps that failed so that each failed parallel branch is in. The Explain Error button sends at least 200; an `explainError()` step sets its own with `maxLines`. Raise it for models with large context windows | `100` |
 | **Log Pattern** | Regular expression, matched without regard to case. Only the lines it matches are sent, searched for in the whole log of each failing step; when it matches nothing, the last lines are sent as they are. Applies to automatic explanations, the Explain Error button and `explainError()` steps without a `logPattern` of their own | Empty |
+| **Mask secrets before sending** | Replaces tokens, keys and passwords with `[REDACTED]` in everything sent to the provider. See [Masking secrets](#masking-secrets) | On |
+| **Additional Redaction Patterns** | Regular expressions of your own, one per line, whose matches are masked as well | Empty |
+| **Do not send a log in which something was masked** | When something is masked, nothing is sent and nothing is explained | Off |
 | **Max Automatic Explanations per Hour** | Provider calls per hour for automatic explanations, on top of the request quota | `30` |
 
+## Masking secrets
+
+Everything sent to the AI provider is masked first: the log lines, the Custom Context, the workspace files of
+`includeWorkspaceContext` and what auto-fix sends. This happens for automatic explanations, the Explain Error button
+on the console and Stages pages, and the `explainError()` step alike. Jenkins already masks the credentials a build
+binds with `withCredentials`; masking catches the secrets that never went through a credential, such as a token in an
+environment dump, a password in a clone URL or a key a script printed.
+
+With **Mask secrets before sending** on, these are replaced with `[REDACTED]`:
+
+| What | Example | Sent as |
+|---|---|---|
+| AWS access key IDs | `AKIA…` | `[REDACTED]` |
+| GitHub, GitLab, Slack, Google, Stripe, OpenAI, Anthropic and npm tokens | `ghp_…`, `github_pat_…`, `glpat-…`, `xoxb-…`, `AIza…`, `sk_live_…`, `sk-proj-…`, `sk-ant-…`, `npm_…` | `[REDACTED]` |
+| JSON Web Tokens | `eyJ….eyJ….…` | `[REDACTED]` |
+| Private keys, PEM and PGP, from their first line to their last, whatever comes before each line (a timestamp, a BuildKit step) and also inside a JSON string. A key whose first or last line is not in the lines sent is masked as far as its base64 lines go, when nothing comes before them; a key header printed on its own, as a secret scanner does, is masked alone | `-----BEGIN … PRIVATE KEY-----` to `-----END … PRIVATE KEY-----` | `[REDACTED]` |
+| The credentials of an `Authorization` header or a bearer token | `Authorization: Bearer abc123…` | `Authorization: Bearer [REDACTED]` |
+| The password of a URL, or of `curl -u` (not `docker run -u 1000:1000`) | `https://deploy:s3cret@git.example.com`, `curl -u admin:s3cret` | `https://deploy:[REDACTED]@git.example.com` |
+| A value given to a key whose last word is `password`, `passwd`, `passphrase`, `secret` or `token`, or `key` after `api`, `access`, `private`, `secret`, `account`, `client` and the like; a key in capitals may run the words together (`PGPASSWORD`). As `KEY=value`, `key: value`, JSON, XML or a command-line option | `DB_PASSWORD=hunter2`, `"clientSecret": "…"`, `<password>…</password>`, `--token=…`, `-Dgpg.passphrase=…`, `AccountKey=…` | `DB_PASSWORD=[REDACTED]` |
+
+The key stays, so the model still sees what the value was for. The key's last word has to be one of those, so
+`TokenService.java:42`, `password_reset.py`, `io.jsonwebtoken:jjwt`, `token-service:` and `"max_tokens": 4096` are
+left alone. In a sentence, a key of one word followed by a colon and a single word is not taken for a value:
+`Invalid token: expired` says what went wrong, where `password: changeme` on a line of YAML is masked. A reference
+such as `${PASSWORD}`, a value that says there is none (`null`, `undefined`, `none`) and a value Jenkins already
+masked (`****`) are left as they are too. The rules only find secrets by
+their shape or their name: a password printed on its own, with nothing to say what it is, is sent.
+
+**Additional Redaction Patterns** takes regular expressions of your own, one per line, for what only you know is
+sensitive, such as internal host names, IP addresses or customer IDs. They apply even with the built-in rules off. A
+pattern with a group named `secret` masks that group only; any other pattern masks its whole match. Lines starting
+with `#` are left out.
+
+```text
+# Internal build hosts
+\b[a-z0-9-]+\.corp\.example\.com\b
+# Private IP addresses
+\b10(?:\.\d{1,3}){3}\b
+# A customer ID, keeping its name
+(?i)customer[_-]?id[ \t]*[:=][ \t]*(?<secret>\S+)
+```
+
+The explanation shows how many secrets were masked next to the number of lines analysed, and the build's REST API has
+it as `redactedCount`. An `explainError()` step also says it in the build's console (`[explain-error] Masked 2 secrets
+in what is sent to the AI provider.`); the Explain Error button and automatic explanations run after the build and
+leave its console alone. At `FINE`, the `io.jenkins.plugins.explain_error.ErrorExplainer` logger records each
+request: the job and build, the number of lines sent, the provider and model, and how many secrets were masked.
+
+**Do not send a log in which something was masked** is for those who would rather have no explanation than trust the
+masking: when anything is masked, in the log or in the context, nothing is sent and no quota is used. The
+`explainError()` step returns `null` and says why in the console, the Explain Error button shows why, and an automatic
+explanation leaves the build without one, saying why at `FINE` in the system log. Usage metrics count these requests
+as `blocked`.
+
+**Preview what is sent**, next to the Explain Error button on the console page, shows as plain text the log lines a
+new explanation would send, masked, without sending anything or using the quota. Anyone who can read the build's log
+can open it. The Custom Context is sent with those lines and masked with them; the preview counts what is masked in
+it and whether that would stop the request, but does not show it.
+
 ## Folder settings
 
 Teams can use their own provider, model, language and instructions. In a folder's **Configure** page, check **Set AI
@@ -74,6 +136,10 @@ unclassified:
     # enableAutoExplainOnFailure: true     # Optional, explain failed builds without pipeline changes
     # autoExplainMaxLogLines: 100          # Log lines sent per build
     # logPattern: "(?i)(error|failed)"     # Optional, only the lines it matches are sent
+    # redactSecrets: true                  # Mask tokens, keys and passwords before sending
+    # redactionPatterns: |                 # Optional, more patterns to mask, one per line
+    #   \b[a-z0-9-]+\.corp\.example\.com\b
+    # blockWhenRedacted: false             # Send nothing when something was masked
     # autoExplainMaxPerHour: 30            # Provider calls per hour for automatic explanations
 ```
 

docs/pipeline-step.md

@@ -27,8 +27,9 @@ sidebar:
 
 ![The AI Error Explanation page of a build](images/side-panel.png)
 
-A failure to explain (explanation turned off, provider misconfigured, quota exceeded, provider error) is reported in the
-console and never fails the build.
+A failure to explain (explanation turned off, provider misconfigured, quota exceeded, provider error, or a secret
+masked while [Do not send a log in which something was masked](configuration.md#masking-secrets) is on) is reported in
+the console and never fails the build.
 
 ## Return value
 
@@ -135,7 +136,8 @@ explainError(
 The files matching `workspaceContextPaths` (the default list when it is empty) are read, each cut to 8,000 bytes.
 Paths are skipped when the file name starts with `.env`, `credentials` or `secrets`, or when they lie under `.git/`,
 `.gradle/`, `target/`, `build/`, `dist/` or `node_modules/`, and absolute paths and `..` are refused. The files are sent
-to the AI provider, so leave out any that hold secrets.
+to the AI provider, [masked](configuration.md#masking-secrets) as the log is, but masking only finds secrets of a
+known shape, so leave out any files that hold secrets.
 
 The files travel with the step's `customContext`. While `includeWorkspaceContext` is on and the step sets no
 `customContext`, the folder and global Custom Context are not sent; set `customContext` on the step to keep them.

docs/usage-quota.md

@@ -148,7 +148,7 @@ The plugin exports the following metric families:
 
 | Metric | Type | Meaning |
 |---|---|---|
-| `explain_error.requests.<entryPoint>.<result>` | Counter | Total requests by entry point (`pipeline_step`, `console_action`, `run_listener`) and outcome (`success`, `provider_error`, `quota_rejected`, `throttled`, etc.) |
+| `explain_error.requests.<entryPoint>.<result>` | Counter | Total requests by entry point (`pipeline_step`, `console_action`, `run_listener`) and outcome (`success`, `provider_error`, `quota_rejected`, `throttled`, `blocked` when nothing was sent because a secret was masked, etc.) |
 | `explain_error.provider_calls.<provider>.<model>.<result>` | Counter | Provider/model-level outcomes for calls that involve a provider (`success`, `cache_hit`, `provider_error`, `quota_rejected`) |
 | `explain_error.request_duration_ms` | Histogram | End-to-end request duration in milliseconds |
 | `explain_error.input_log_lines` | Histogram | Number of input log lines processed per request |

src/main/java/io/jenkins/plugins/explain_error/ConsoleExplainErrorAction.java

@@ -152,6 +152,41 @@
         }
     }
 
+    /**
+     * Shows, as plain text, what the Explain Error button would send to the AI provider for this build: the
+     * log lines, with secrets masked. Nothing is sent and no quota is used. Anyone who can read the build's
+     * log can read this. POST only, as it reads and masks as many lines as the button may send.
+     */
+    @RequirePOST
+    public void doPreview(StaplerRequest2 req, StaplerResponse2 rsp) throws IOException {
+        run.checkPermission(Item.READ);
+        int maxLines = maxLines(req.getParameter("maxLines"));
+        String log = String.join("\n", pageExtractor(run, maxLines).getFailedStepLog());
+        LogRedactor redactor = LogRedactor.fromConfiguration();
+        String sent = redactor.redact(log);
+        int maskedInLog = redactor.count();
+        // The custom context goes with the log and is masked with it, but it is the administrator's to read
+        redactor.redact(new ErrorExplainer().inheritedCustomContext(run));
+        int masked = redactor.count();
+        rsp.setContentType("text/plain;charset=UTF-8");
+        PrintWriter writer = rsp.getWriter();
+        writer.println("# What the Explain Error button would send to the AI provider for this build.");
+        writer.println("# " + ErrorExplainer.countLines(sent) + " log lines. "
+                + (masked > 0 ? ErrorExplainer.describeRedaction(masked)
+                        : redactor.isActive() ? "Nothing to mask was found." : "Masking is turned off."));
+        if (masked > maskedInLog) {
+            writer.println("# " + (masked - maskedInLog) + " of them in the custom context, which is not shown here.");
+        }
+        if (masked > 0 && GlobalConfigurationImpl.get().isBlockWhenRedacted()) {
+            writer.println("# Nothing would be sent: the configuration says not to send a log in which"
+                    + " something was masked.");
+        }
+        writer.println("# The custom context configured for the job is sent with it, masked the same way.");
+        writer.println();
+        writer.print(sent);
+        writer.flush();
+    }
+
     /**
      * The number of log lines asked for, within {@link #MAX_LINES_LIMIT}; the default when the request does
      * not say or says something that is not a number.
@@ -187,9 +222,9 @@
         try {
             extractor.setLogPattern(PipelineLogExtractor.compileLogPattern(
                     GlobalConfigurationImpl.get().getLogPattern()));
         } catch (PatternSyntaxException e) {
             LOGGER.fine("The global log pattern is not a valid regular expression, so it is not used: "
                     + e.getDescription());
         }
         return extractor;
     }