plan-skeptic v0.1.0

.github/workflows/ci.yml

@@ -0,0 +1,44 @@
+name: ci
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+permissions:
+  contents: read
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python: ["3.9", "3.13"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python }}
+      - run: python -m unittest discover -s tests -v
+
+  action:
+    # Runs the composite action against the fixtures: a clean plan must pass
+    # and a flawed one must fail, or the action is not gating anything.
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Clean plan passes
+        uses: ./
+        with:
+          plan-json: fixtures/clean/01-tags-only.json
+      - name: Flawed plan fails
+        id: flawed
+        continue-on-error: true
+        uses: ./
+        with:
+          plan-json: fixtures/flawed/02-rename-replaces-database.json
+          sarif-file: flawed.sarif
+      - name: Assert the flawed plan failed and wrote SARIF
+        run: |
+          test "${{ steps.flawed.outcome }}" = failure
+          python3 -c "import json; r=json.load(open('flawed.sarif'))['runs'][0]['results']; assert r, 'no results'"

.gitignore

@@ -0,0 +1,6 @@
+__pycache__/
+*.egg-info/
+build/
+dist/
+.venv/
+*.sarif

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Brian Teller
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -1,2 +1,113 @@
 # plan-skeptic
-Flag the parts of a Terraform/OpenTofu plan that need a human. Ten checks, SARIF output, GitHub Action.
+
+Flag the parts of a Terraform/OpenTofu plan that need a human.
+
+AI assistants write infrastructure changes that are fluent, plausible and
+occasionally destructive: a rename that replaces a database, a policy widened
+to `s3:*` to make an error go away, SSH opened "temporarily". The diff reads
+fine. The plan says what will actually happen. `plan-skeptic` reads the plan
+and points at the lines a reviewer must not skim.
+
+It is not a policy engine and does not try to replace one. It is ten
+opinionated checks about **what a change introduces**, tuned so that the output
+is short enough to be read on every pull request.
+
+```console
+$ terraform plan -out=plan.out && terraform show -json plan.out > plan.json
+$ plan-skeptic plan.json
+plan-skeptic: 2 finding(s) across 1 resource change(s)
+
+[HIGH  ] PS001 stateful-resource-replaced
+         aws_db_instance.orders: aws_db_instance will be replaced because identifier changed: destroyed, then recreated empty.
+         why a human should look: Replacement is destroy-then-create unless create_before_destroy is set. ...
+
+[MEDIUM] PS010 recovery-guard-removed
+         aws_db_instance.orders: deletion_protection changes true -> false.
+```
+
+Works the same with `tofu show -json`. No dependencies; Python 3.9+.
+
+## Install
+
+```sh
+pipx install git+https://github.com/TellersTechOrg/plan-skeptic
+# or run from a checkout
+PYTHONPATH=src python3 -m plan_skeptic plan.json
+```
+
+## GitHub Action
+
+```yaml
+- run: |
+    terraform plan -out=plan.out
+    terraform show -json plan.out > plan.json
+- uses: TellersTechOrg/plan-skeptic@v0.1.0
+  with:
+    plan-json: plan.json
+    fail-on: high          # high | medium | low | never
+- uses: github/codeql-action/upload-sarif@v3
+  if: always()
+  with:
+    sarif_file: plan-skeptic.sarif
+```
+
+Findings appear in the Security tab and inline on the pull request. Uploading
+SARIF needs `security-events: write`.
+
+## Rules
+
+| Id | Severity | Flags |
+|---|---|---|
+| <a id="ps001"></a>PS001 | high | A data-holding resource (database, bucket, table, volume, key, PVC) is destroyed or replaced, and why it is being replaced |
+| <a id="ps002"></a>PS002 | medium | Any other resource is deleted |
+| <a id="ps003"></a>PS003 | high | An IAM policy newly grants `*` / `service:*`, or uses `NotAction` in an Allow |
+| <a id="ps004"></a>PS004 | high | AdministratorAccess, PowerUserAccess or IAMFullAccess is attached |
+| <a id="ps005"></a>PS005 | high | A trust policy lets any principal assume the role without a Condition |
+| <a id="ps006"></a>PS006 | high | Ingress opened to 0.0.0.0/0 or ::/0 (medium for ports 80 and 443) |
+| <a id="ps007"></a>PS007 | high | An S3 bucket made public by ACL, bucket policy or public access block |
+| <a id="ps008"></a>PS008 | high | A database given `publicly_accessible = true` |
+| <a id="ps009"></a>PS009 | medium | Encryption at rest set to false |
+| <a id="ps010"></a>PS010 | medium | `deletion_protection` switched off, or `skip_final_snapshot` / `force_destroy` switched on |
+
+A delete paired with a create of the same type and name gets a hint to use a
+`moved` block, since that is what an unfinished refactor looks like.
+
+### What it deliberately does not do
+
+- **It reports changes, not state.** An update that leaves an existing
+  0.0.0.0/0 rule alone is not flagged. Scanning everything on every plan is
+  how a tool's output stops being read.
+- **Unknown-after-apply values never trigger a rule.** Guessing at a value the
+  plan itself cannot see would invent findings.
+- **AWS first.** Replacement detection covers AWS, GCP, Azure and Kubernetes
+  storage; the exposure and IAM rules are AWS-only in v0.1.
+
+## Options
+
+```
+plan-skeptic PLAN [--format text|json|sarif] [--output FILE]
+                  [--fail-on high|medium|low|never] [--disable RULE ...]
+                  [--list-rules]
+```
+
+Exit codes: `0` nothing at or above `--fail-on`, `1` findings at or above it,
+`2` the input could not be reviewed. A binary plan file or a state file is
+refused with exit 2 rather than reported clean.
+
+## Fixtures
+
+[`fixtures/`](fixtures/) holds seven flawed plans, each paired with the request
+that produced it, plus a clean control. They are the exercises for the
+[Confidently Wrong workshop](https://www.tellerstech.com/workshops/ai-era-infrastructure-risk-workshop/)
+and come from the same material as the book
+[*Confidently Wrong*](https://www.tellerstech.com/book/).
+
+## Development
+
+```sh
+python3 -m unittest discover -s tests
+```
+
+## License
+
+MIT

action.yml

@@ -0,0 +1,40 @@
+name: plan-skeptic
+description: Flag the parts of a Terraform/OpenTofu plan that need a human, as SARIF for code scanning.
+branding:
+  icon: alert-triangle
+  color: orange
+
+inputs:
+  plan-json:
+    description: Path to `terraform show -json` / `tofu show -json` output of a saved plan.
+    required: true
+  fail-on:
+    description: Lowest severity that fails the step (high, medium, low, never).
+    default: high
+  sarif-file:
+    description: Where to write the SARIF report. Upload it with github/codeql-action/upload-sarif.
+    default: plan-skeptic.sarif
+  disable:
+    description: Space-separated rule ids to skip, e.g. "PS002 PS010".
+    default: ""
+
+outputs:
+  sarif-file:
+    description: Path of the SARIF report.
+    value: ${{ inputs.sarif-file }}
+
+runs:
+  using: composite
+  steps:
+    - name: Run plan-skeptic
+      shell: bash
+      env:
+        PS_PLAN: ${{ inputs.plan-json }}
+        PS_FAIL_ON: ${{ inputs.fail-on }}
+        PS_SARIF: ${{ inputs.sarif-file }}
+        PS_DISABLE: ${{ inputs.disable }}
+        PYTHONPATH: ${{ github.action_path }}/src
+      run: |
+        args=(--format sarif --output "$PS_SARIF" --fail-on "$PS_FAIL_ON")
+        for rule in $PS_DISABLE; do args+=(--disable "$rule"); done
+        python3 -m plan_skeptic "${args[@]}" "$PS_PLAN"

fixtures/README.md

@@ -0,0 +1,26 @@
+# Fixtures: plans an assistant would happily hand you
+
+Each file under `flawed/` is `show -json` output of a plan that answers the
+question it was asked and does something else as well. They double as the
+exercises for the *Confidently Wrong* workshop: show the prompt and the diff,
+ask the room what they would approve, then run `plan-skeptic` on the file.
+
+`01` is real OpenTofu 1.12 output, generated from `01-quickstart-postgres.tf`
+with `tofu plan -refresh=false -out=plan.out && tofu show -json plan.out`
+(dummy credentials; a create-only plan makes no AWS calls). The others need
+prior state to produce an update or replacement, so they are written by hand in
+the same shape, trimmed to the attributes the rules read.
+
+| File | What was asked | What the plan also does | Rules |
+|---|---|---|---|
+| `01-quickstart-postgres` | "Give me a Postgres database the app can reach" | Public endpoint, unencrypted, no final snapshot, port 5432 open to the internet, `s3:*` on the side | PS003 PS006 PS008 PS009 PS010 |
+| `02-rename-replaces-database` | "Rename the database to match our naming convention" | `identifier` forces replacement, so the database is destroyed and recreated empty; deletion protection switched off so the apply succeeds | PS001 PS010 |
+| `03-iam-wildcard-creep` | "The exporter gets AccessDenied on KMS, fix it" | `s3:*` and `kms:*` on `*`; a second policy gains `NotAction: iam:*`, which allows everything except IAM | PS003 |
+| `04-open-ssh-for-debugging` | "I can't SSH in to debug, and the app can't reach Redis" | Port 22 and every port open to 0.0.0.0/0 (the pre-existing 443 rule is correctly not reported) | PS006 |
+| `05-public-bucket-for-static-site` | "Serve these assets as a static site" | Public ACL, public bucket policy, and the public access block relaxed, where CloudFront with OAC needed none of it | PS007 |
+| `06-admin-to-unblock-ci` | "CI fails with AccessDenied on deploy" | AdministratorAccess on the CI role, and a trust statement that lets any AWS account assume it | PS004 PS005 |
+| `07-refactor-without-moved-block` | "Move the orders resources into a module" | Destroys the table and log group and creates new ones at the module address, because nobody wrote a `moved` block | PS001 PS002 |
+
+`clean/01-tags-only` is the control: a tags-only change on resources that
+already carry a wildcard policy and a public 443 rule. It must produce nothing,
+because the tool reports what a plan *introduces*.

fixtures/clean/01-tags-only.json

@@ -0,0 +1,61 @@
+{
+  "format_version": "1.2",
+  "terraform_version": "1.12.5",
+  "resource_changes": [
+    {
+      "address": "aws_db_instance.orders",
+      "mode": "managed",
+      "type": "aws_db_instance",
+      "name": "orders",
+      "provider_name": "registry.opentofu.org/hashicorp/aws",
+      "change": {
+        "actions": ["update"],
+        "before": {"identifier": "orders-db", "publicly_accessible": false, "storage_encrypted": true, "deletion_protection": true, "skip_final_snapshot": false, "tags": {"team": "payments"}},
+        "after": {"identifier": "orders-db", "publicly_accessible": false, "storage_encrypted": true, "deletion_protection": true, "skip_final_snapshot": false, "tags": {"team": "payments", "cost-center": "cc-114"}},
+        "after_unknown": {}
+      }
+    },
+    {
+      "address": "aws_security_group.web",
+      "mode": "managed",
+      "type": "aws_security_group",
+      "name": "web",
+      "provider_name": "registry.opentofu.org/hashicorp/aws",
+      "change": {
+        "actions": ["update"],
+        "before": {
+          "name": "web",
+          "ingress": [{"cidr_blocks": ["0.0.0.0/0"], "from_port": 443, "ipv6_cidr_blocks": [], "protocol": "tcp", "to_port": 443}],
+          "tags": {}
+        },
+        "after": {
+          "name": "web",
+          "ingress": [{"cidr_blocks": ["0.0.0.0/0"], "from_port": 443, "ipv6_cidr_blocks": [], "protocol": "tcp", "to_port": 443}],
+          "tags": {"cost-center": "cc-114"}
+        },
+        "after_unknown": {}
+      }
+    },
+    {
+      "address": "aws_iam_policy.exporter",
+      "mode": "managed",
+      "type": "aws_iam_policy",
+      "name": "exporter",
+      "provider_name": "registry.opentofu.org/hashicorp/aws",
+      "change": {
+        "actions": ["update"],
+        "before": {"name": "report-exporter", "policy": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Action\":\"ecr:*\",\"Resource\":\"*\"}]}", "tags": {}},
+        "after": {"name": "report-exporter", "policy": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Action\":\"ecr:*\",\"Resource\":\"*\"}]}", "tags": {"cost-center": "cc-114"}},
+        "after_unknown": {}
+      }
+    },
+    {
+      "address": "data.aws_caller_identity.current",
+      "mode": "data",
+      "type": "aws_caller_identity",
+      "name": "current",
+      "provider_name": "registry.opentofu.org/hashicorp/aws",
+      "change": {"actions": ["read"], "before": null, "after": {}, "after_unknown": {"id": true}}
+    }
+  ]
+}