Skip to content

feat: Mask secrets before sending logs to the AI provider, with a preview of what is sent - #263

Merged
shenxianpeng merged 3 commits into
mainfrom
feature/redact-secrets
Oct 25, 2026
Merged

shenxianpeng merged 3 commits into
mainfrom
feature/redact-secrets

Conversation

@shenxianpeng

@shenxianpeng shenxianpeng commented Oct 24, 2026 •

Copy link
Copy Markdown
Member

Description

Closes #184. Build logs often hold secrets that never went through a Jenkins credential: a token in an environment dump, a password in a clone URL, a key a script printed. Until now they were sent to the AI provider as they were.

  1. Secrets are masked before anything is sent. A new LogRedactor replaces them with [REDACTED] in the log lines, the Custom Context, the workspace files of includeWorkspaceContext and the auto-fix prompt, for automatic explanations, the Explain Error button on the console and Stages pages, and the explainError() step. Built-in rules cover AWS access key IDs, GitHub/GitLab/Slack/Google/Stripe/OpenAI/Anthropic/npm tokens, JWTs, PEM and PGP private keys (also on lines with a timestamp or BuildKit prefix, inside JSON strings, and cut short), Authorization and bearer credentials, the password of a URL or of curl -u, and values given to keys whose last word names a secret, as KEY=value, key: value, JSON, XML elements or command-line options (DB_PASSWORD=, "clientSecret":, <password>, --token=, -Dgpg.passphrase=, PGPASSWORD=). The key stays, so the model still sees what the value was for. Ordinary build output is left alone: TokenService.java:[142,17], io.jsonwebtoken:jjwt, token-service:, password_reset.py:104, max_tokens: 4096, docker run -u 1000:1000, Invalid token: expired, ${PASSWORD}, password=null and values Jenkins already masked (****).
    Anyone who can write to a build log chooses what these rules run on, so every built-in rule reads the text once (possessive quantifiers, bounded look-aheads, key names checked in code): 1,000 lines of 2,000 characters built to make a rule backtrack take at most about 3 seconds.
  2. Rules of your own. Additional Redaction Patterns takes regular expressions, one per line (internal host names, IPs, customer IDs). A group named secret masks that group only. Validated on the form.
  3. Fail closed, if wanted. Do not send a log in which something was masked sends nothing when anything was masked: the step returns null, and the console and the button say why. Off by default. Counted as a new blocked usage result.
  4. Preview. Preview what is sent, next to the Explain Error button on the console page, opens the masked lines a new explanation would send, as plain text, without calling the provider or using quota. Needs Item/Read, like the log itself, and takes a POST with the crumb (the button submits a form to a new tab; a GET gets 405).
  5. Audit. The console says how many secrets were masked, the explanation shows it next to the lines analysed, the REST API has redactedCount, and the ErrorExplainer logger records each request at FINE (job, lines, provider, model, masked count).

Mask secrets before sending is on by default. The issue proposed opt-in; on by default is proposed here because the rules only match token-shaped values and secret-named keys, so the explanation keeps the error itself, and a leak cannot be taken back. It can be turned off in the global configuration or with redactSecrets: false in CasC.

Before / After

The same pipeline run against main (f428464) and this branch, with a mock OpenAI-compatible server recording the HTTP request it receives:

sh '''echo "AWS_ACCESS_KEY_ID=AKIA..."; echo "DB_PASSWORD=hunter2hunter2"
echo "git clone https://x-access-token:ghp_...@github.com/acme/app.git"
echo "ERROR: connection to db.internal:5432 refused"; exit 1'''
Line in the request body Before After
AWS key AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE AWS_ACCESS_KEY_ID=[REDACTED]
Password DB_PASSWORD=hunter2hunter2 DB_PASSWORD=[REDACTED]
GitHub token in a URL https://x-access-token:ghp_a1B2…@github.com/… https://x-access-token:[REDACTED]@github.com/…
The error ERROR: connection to db.internal:5432 refused unchanged

With FakeAIProvider and more kinds (bearer token, RSA private key, api_key= in the step's customContext): before, all six reach the provider; after, all six are [REDACTED] and the console says [explain-error] Masked 6 secrets in what is sent to the AI provider.

Global configuration (Manage Jenkins → System):

Before After
Desktop before desktop after desktop
Mobile before mobile after mobile

Console page and explanation:

Before After
before card after card

The preview (Preview what is sent):
preview

On a phone: console mobile

Testing done

  • New LogRedactorTest (118): every built-in rule and the formats above, keys kept, counts, administrator patterns with and without a secret group (and one that only looks like it has one), 40 lines of ordinary build output that must be left alone (compiler errors, stack traces, Maven/npm/pip coordinates, LLM usage JSON, docker -u, YAML), which key names name a secret, and 19 inputs of 1,000 × 2,000 characters aimed at each rule, each with a 60 s limit (≤3 s locally).
  • New RedactionTest (14): the step, custom context, workspace files (pom.xml with <password> and a URL password), masking off, administrator patterns with the built-in rules off, blocking (no provider call, no quota used, only a blocked usage event, auto-fix skipped), nothing to mask while blocking, the automatic explanation, the console button, the preview (Item/Read can, without it cannot, GET refused, the custom context counted but not shown, nothing sent), the preview button on the console page opening it in a new window (HtmlUnit, with the crumb), form validation, and the settings through the configuration form.
  • ErrorRunListenerTest: tests of automatic explanations now wait until the explanation is done with the build before Jenkins is torn down (CI hit DirectoryNotEmptyException when the background explanation was still saving the build).
  • Reviewed independently in five rounds; every finding is fixed (backtracking in the private-key rule, false positives on compiler/dependency lines and docker -u, missed formats, prefixed key lines, the preview ignoring the custom context, docs).
  • mvn verify (JDK 21) on the review fixes: 1,055 tests, 1 failure in ConnectionDiagnosticsTest.tlsHandshakeFailuresAreReported, a timing race on a local socket under heavy load that this PR does not touch (5 of 5 reruns pass); after the last fixes, mvn verify of LogRedactorTest, RedactionTest, ErrorRunListenerTest, ConsoleToolbarTest, ExplainCardStatesTest and ConsoleExplainErrorActionTest: 167 tests, 0 failures; SpotBugs 0 bugs.
  • Manual check on a local Jenkins 2.528.3 with a mock OpenAI-compatible server: configuration page, the preview, the explanation footer, desktop and phone widths.

Submitter checklist

  • Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
  • Ensure that the pull request title represents the desired changelog entry
  • Please describe what you did
  • Link to relevant issues in GitHub or Jira
  • Link to relevant pull requests, esp. upstream and downstream changes
  • Ensure you have provided tests that demonstrate the feature works or the issue is fixed

Was generative AI tooling used to co-author this PR?
  • Yes (please specify below)

…f what is sent

Built-in rules for tokens, keys, private keys, URL and Authorization
credentials and secret-named keys, patterns of the administrator's own,
an option to send nothing when something was masked, a preview next to
the Explain Error button, and the masked count on the console, the
explanation, the REST API and the FINE log. Closes #184.
@shenxianpeng
shenxianpeng requested a review from a team as a code owner October 24, 2026 05:24
@shenxianpeng shenxianpeng added the enhancement For changelog: Minor enhancement. use `major-rfe` for changes to be highlighted label Oct 24, 2026
@shenxianpeng shenxianpeng added major-rfe For changelog: Major enhancement. Will be highlighted on the top and removed enhancement For changelog: Minor enhancement. use `major-rfe` for changes to be highlighted labels Oct 24, 2026
The private key rule took time in proportion to the square of the log,
which anyone able to write to a log could trigger through the preview.
Every built-in rule now reads the text once. Key names are split into
words, so that TokenService.java:42, io.jsonwebtoken:jjwt, max_tokens and
docker run -u 1000:1000 are left alone, while XML elements, PGP keys,
quoted options, curl -u, PGPASSWORD, sk- keys and prefixed key lines are
masked. The preview counts what is masked in the custom context, the
button counts lines after masking, and the docs say where the count and
the reason for a blocked request are shown.
The preview reads and masks as many lines as the button may send, so it
now takes a POST with the crumb: the console page submits it as a form
to a new tab, and a GET gets 405. Tests of automatic explanations now
wait until the explanation is done with the build, so that Jenkins is
not torn down while it still saves the build.
@shenxianpeng
shenxianpeng merged commit 9a6932b into main Oct 25, 2026
18 checks passed
@shenxianpeng
shenxianpeng deleted the feature/redact-secrets branch October 25, 2026 18:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

major-rfe For changelog: Major enhancement. Will be highlighted on the top

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: Log sanitization and send-before-preview for LLM payloads

2 participants