Skip to content

remote-staging-then-execute fires critical on any GitHub-raw fetch plus any later python -c, with no link between them #51

Closed

Description

@blitzcrieg1

rule_remote_staging_then_execute has two branches. The docstring describes the first and the guarantee it makes:

Requiring the fetched basename to be the executed basename is what makes host-agnostic safe: it is not "you downloaded something and later ran something", it is "you ran the thing you just downloaded".

That is true of branch one, which intersects _fetched_files with _executed_files.

Branch two drops the requirement entirely:

https://github.com/blitzcrieg1/agentmetry/blob/master/apps/orchestrator/agentmetry/core/audit/detection/rules.py#L1032

fetch_idx = next((i for i, e in enumerate(events) if _is_staging_fetch(e)), None)
...
for event in events[fetch_idx + 1:]:
    if _is_staging_fetch(event):
        continue
    if not _is_risky_exec_after_staging(event):
        continue

Any fetch from a _STAGING_HOST, followed anywhere later in the session by anything matching RISKY_EXEC_AFTER_STAGING, returns critical. The two events are never compared. It is precisely "you downloaded something and later ran something", which the docstring names as the thing to avoid.

The trade was deliberate: branch two narrows the host list instead of correlating artifacts. The problem is what is on the other list.

https://github.com/blitzcrieg1/agentmetry/blob/master/apps/orchestrator/agentmetry/core/audit/detection/traits.py#L309

RISKY_EXEC_AFTER_STAGING = re.compile(
    r"\b(bash|sh|zsh|dash)\s+[\w./~-]+\.(?:sh|bash)\b|"
    r"\bpython\d?\s+[\w./~-]+\.py\b|"
    r"\bpython\d?\s+-c\b|"          # <-- any inline python at all
    ...

python -c with no argument constraint. So the rule reduces to:

Read anything off raw.githubusercontent.com, then run python -c on anything, and get a critical.

That is a normal working day for anyone who reads a README from GitHub raw and parses JSON in the same session.

BENIGN_AFTER_STAGING does not help. It exempts npm|yarn|pnpm|pip|cargo|go with install|run|build, and nothing else.

Observed

Dispositioned false_positive today on correlation 716fbfb9-29ed-442a-bbaa-9f559faf582a:

curl -s https://raw.githubusercontent.com/ottosulin/awesome-ai-security/main/README.md | grep -n "^#\|^##" | head -25
# ... later, unrelated ...
cd /tmp/awesome-ai-security-tools && python -c "
import json
d=json.load(open('data/sections.json', encoding='utf-8'))"

Fired critical: "Staged download from public host followed by execution."

The fetched artifact was a README, and it went to grep. The python -c read data/sections.json out of a local git clone. The fetched bytes and the executed program have nothing to do with each other, and the rule had no way to notice because it never looks.

Direction

Carry the fetched artifact forward into the second half of the check, the way branch one already does. _fetched_files and _executed_files exist and are used a few lines above.

Where the later command references nothing that was fetched, either do not fire, or fire well below critical. A staging-host fetch followed by unrelated execution is weak circumstantial evidence, and pricing it the same as running the downloaded file is what makes the strong signal unreadable.

At minimum, python -c should not be a bare match. Requiring the inline script to reference something fetched, or a path under the fetch's download directory, would keep the technique covered without catching every JSON parse in the session.

Related to #50, which is the same confusion of proximity with causation on the single-command path.

Blocked

rules.py and traits.py are ruleset fingerprint inputs. Hold with #44, #49 and #50 until the dogfood gate closes.

Activity

  1. blitzcrieg1 commented on Aug 27, 2026

    @blitzcrieg1
    OwnerAuthor

    Fixed in 0.7.0, on PyPI now.

    Tightened the benign shape rather than lowering severity, and the session is in the benchmark corpus as tagged benign so a regression fails CI. The corpus is now 54 cases, 26 attack and 28 benign, with 0 missed and 0 false positives, which means something it did not mean in 0.6.0 when the corpus simply lacked the cases known to fail.

    This required breaking the detection freeze. The ruleset fingerprint moved to 15846a0915769d4a and the four-week dogfood clock restarted on 2026-08-26, earliest close 2026-09-23. Shipping criticals on these shapes to the first external tester was the more expensive of the two options.

    Details in CHANGELOG 0.7.0 and #125.

  2. blitzcrieg1 commented on Nov 15, 2026

    @blitzcrieg1
    OwnerAuthor

    Two more routes into this shape, found by replaying the dogfood trail against its dispositions. Both are fixed on the detection-pass branch (#435, commit 0a86a48), which merges after 17 November.

    1. A pipe into a filter counted as staging. 0.7.0 required the fetch to leave an artifact, but | anywhere in the command counted as one. A README fetched from GitHub raw and piped into grep -n "^## " to list its headings was the first half of a critical remote-staging-then-execute, paired with a local script the agent had written itself. The check now reads only the fetch's own pipeline: a pipe stages when it feeds something other than a filter that prints (grep, head, jq, Select-Object and similar), and a redirect, an output flag or a bare wget writes a file.

    2. The artifact check was partly dead since 0.7.0. Three of the five alternatives in STAGES_ARTIFACT contained a literal backspace character (0x08) where \b was meant, so curl -o, curl -O, --output and wget -P never matched. A fetch saved to a named file counted as staging only when a > or | came with it. Running the downloaded file by name was still caught by the same-file check, so the gap was a GitHub-raw download followed by running a different file.

    Fixing that alone surfaced two false positives the bug had been hiding: curl -s -o /dev/null -w "%{http_code}" (a status check) and a logo saved as .svg followed by an unrelated python3 -c. So output to /dev/null, stdout or $null saves nothing, and a download saved only under a data extension (svg, png, json, md and similar) is not a payload. Piping a download into a runner still stages.

    Corpus: benign_readme_piped_to_grep_then_local_script and attack_raw_fetch_teed_then_run. On the trail replay, remote-staging criticals went from 3 to 2, and every detection dispositioned resolved or risk-accepted was kept. The 2 left are also dispositioned false positive and are different shapes, still open.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    detection-ruleBehavioral sequence detection

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions