Add --check: report which pinned Splunkbase apps are outdated (2.3.0)

CHANGELOG.md

@@ -4,6 +4,13 @@ Versions follow [Semantic Versioning](https://semver.org). The newest entry must
 match `metadata.version` in `SKILL.md` and `__version__`/`VERSION` in both
 scripts (checked by `tests/test_skill_package.py`).
 
+## [2.3.0] - 2026-10-28
+
+### Added
+
+- `--check FILE` answers "what is outdated?": it reads the `SPLUNK_APPS_URL` line(s) of a `.env` or compose file (plain, `export`, quoted, `- KEY=value` and `KEY: value` forms), compares every pinned Splunkbase release with the newest release that fits the Splunk version, and reports `current`, `outdated`, `unsupported` (the pin is newer than any eligible release) or `unknown` (not in the release list). The Splunk version comes from `--splunk-version`, `--image`, or the `splunk/splunk` `image:` line of the file. Table or JSON output; exit status 3 means at least one pin is not current. It never edits the file. Python script only; the bash fallback points to it.
+- Live API test that runs `--check` on real oldest and newest releases.
+
 ## [2.2.0] - 2026-10-28
 
 ### Added

README.md

@@ -32,11 +32,12 @@ python3 scripts/splunkbase_urls.py --format env 4353 7931       # SPLUNK_APPS_UR
 python3 scripts/splunkbase_urls.py --splunk-version 9.4 4353    # newest release supporting Splunk 9.4
 python3 scripts/splunkbase_urls.py --image splunk/splunk:9.4.2 4353  # same, version read from the image tag
 python3 scripts/splunkbase_urls.py --cloud 4353                 # only Splunk Cloud compatible releases
+python3 scripts/splunkbase_urls.py --check .env                 # which pinned URLs are outdated (exit 3 if any)
 python3 scripts/splunkbase_urls.py --format json 4353           # app, version, compatibility, warnings
 python3 scripts/splunkbase_urls.py --search "config explorer"   # find IDs by name
 ```
 
-`scripts/get_splunkbase_urls.sh` (needs `curl` and `jq`, works with macOS bash 3.2) provides the same options except `--search` and the `table` format.
+`scripts/get_splunkbase_urls.sh` (needs `curl` and `jq`, works with macOS bash 3.2) provides the same options except `--search`, `--check` and the `table` format.
 
 Exit status: `0` all resolved, `1` at least one app failed (nothing on stdout unless `--partial`), `2` bad arguments. Warnings go to stderr.
 

SKILL.md

@@ -1,9 +1,9 @@
 ---
 name: splunkbase-skill
-description: Resolve Splunkbase apps to their current download URLs, optionally for a specific Splunk version, and find app IDs by name. Use when the user wants the latest release URL of a Splunkbase app or add-on, needs SPLUNK_APPS_URL for splunk/splunk Docker or Docker Compose deployments, asks which Splunkbase release supports their Splunk version, or gives a splunkbase.splunk.com link or app ID. Route app lifecycle, upgrade, or removal advice and general product questions to their owning workflows.
+description: Resolve Splunkbase apps to their current download URLs, optionally for a specific Splunk version, and find app IDs by name. Use when the user wants the latest release URL of a Splunkbase app or add-on, needs SPLUNK_APPS_URL for splunk/splunk Docker or Docker Compose deployments, asks which Splunkbase release supports their Splunk version, wants to know which Splunkbase release URLs pinned in a .env or compose file are out of date, or gives a splunkbase.splunk.com link or app ID. Route app lifecycle, upgrade, or removal advice and general product questions to their owning workflows.
 license: Apache-2.0
 metadata:
-  version: 2.2.0
+  version: 2.3.0
   splunk:
     domain: splunkbase-release-resolution
     products:
@@ -19,6 +19,7 @@ metadata:
       - SPLUNK_APPS_URL for Docker Compose
       - which Splunkbase release supports my Splunk version
       - find the Splunkbase app ID for an app name
+      - which pinned Splunkbase apps in my compose file or .env are outdated
       - splunkbase.splunk.com app link
     not-for:
       - installing, upgrading, disabling, or removing an app in a deployment
@@ -30,6 +31,7 @@ metadata:
       - exact release URLs taken from the live Splunkbase API
       - per-app version and Splunk compatibility
       - ready-to-use SPLUNK_APPS_URL value or compose snippet
+      - per-app current, outdated or unsupported status for pinned release URLs
 ---
 
 # Splunkbase release URLs
@@ -96,9 +98,26 @@ Route elsewhere, and say so, when the request is really about:
    so resolve the variable from the file or `.env`, or ask, and use
    `--splunk-version`. `--image` and `--splunk-version` cannot be combined.
    Without Python, `scripts/get_splunkbase_urls.sh` gives the same results
-   (no `--search`).
+   (no `--search`, no `--check`).
 
-3. **Report.** Give the URLs or the finished snippet, the resolved version of
+3. **Check what is outdated** (when the user already has pinned URLs). Point
+   `--check` at the `.env` or compose file that holds `SPLUNK_APPS_URL`:
+
+   ```bash
+   python3 scripts/splunkbase_urls.py --check .env --splunk-version 9.4
+   python3 scripts/splunkbase_urls.py --check docker-compose.yml          # reads the splunk/splunk image: tag
+   ```
+
+   It compares each pinned Splunkbase release with the newest release that fits
+   the Splunk version, and reports `current`, `outdated` (a newer eligible release
+   exists), `unsupported` (the pin is newer than any release that fits the Splunk
+   version or `--cloud`) or `unknown` (the pin is not in Splunkbase's list). It
+   never edits the file. To produce the updated value, run step 2 for the
+   outdated IDs and show the user the new line. Only literal `SPLUNK_APPS_URL`
+   lines on one line are read; if the compose file only references
+   `${SPLUNK_APPS_URL}`, check the `.env` file instead.
+
+4. **Report.** Give the URLs or the finished snippet, the resolved version of
    each app, and every warning from stderr. Use the table below.
 
 ## Output and exit codes
@@ -108,15 +127,19 @@ Route elsewhere, and say so, when the request is really about:
 | 0 | All apps resolved | Use stdout; read stderr for warnings. |
 | 1 | At least one app failed; stdout is empty | Report the per-app errors. Do not guess URLs. `--partial` prints the apps that did resolve. |
 | 2 | Bad arguments | Fix the invocation. |
+| 3 | `--check` only: at least one pin is not `current` | Report the table; this is a finding, not a failure. |
 
 Answer shape for each app: ID and title, resolved version, `compatibility`
 (`compatible`, `unknown`, `not-checked`), any warning, then the URL. Treat
 `unknown` as "verify", not as a pass.
 
 ## Boundaries
 
-- Read-only: the script only reads public Splunkbase metadata. It never logs
-  in, downloads packages or changes anything.
+- Read-only: the script only reads public Splunkbase metadata (and, for
+  `--check`, the one local file it is pointed at). It never logs in, downloads
+  packages or changes anything, including that file.
+- A newer release is a fact, not advice to upgrade. Whether and how to upgrade
+  belongs to the app lifecycle workflow or the product documentation.
 - Never ask for, accept or echo Splunkbase credentials in the conversation.
   Refer to the environment variables `SPLUNKBASE_USERNAME` and
   `SPLUNKBASE_PASSWORD` or an untracked `.env` file.
@@ -132,6 +155,7 @@ Answer shape for each app: ID and title, resolved version, `compatibility`
 - "Give me SPLUNK_APPS_URL for apps 4353 and 7931 on Splunk 9.4."
 - "What is the Splunkbase ID for the Splunk MCP Server, and what is its latest release?"
 - "Which release of this app still supports Splunk 9.2?" (`--splunk-version 9.2 --format json`)
+- "Are the Splunkbase apps pinned in my .env still the latest?" (`--check .env`)
 - "Add these three Splunkbase apps to my docker-compose.yml."
   Read [references/docker-compose.md](references/docker-compose.md).
 
@@ -150,6 +174,8 @@ Answer shape for each app: ID and title, resolved version, `compatibility`
   missing or a variable; compatibility was not checked. Find the real version.
 - **`no release is marked Splunk Cloud compatible`:** `--cloud` found no release
   flagged `cloud_compatible`; the flag is the app author's, so verify on Splunkbase.
+- **`no Splunkbase release URLs found in a SPLUNK_APPS_URL line`:** the file has
+  no literal pinned URLs (variable reference, multi-line value, another file).
 - **Reproducibility:** `latest` moves. Suggest pinning the resolved versions
   and re-running on upgrades.
 

evals.json

@@ -41,6 +41,16 @@
       "prompt": "What's the Splunkbase app ID for the AWS add-on?",
       "expected_skill": "splunkbase-skill"
     },
+    {
+      "id": "trigger_outdated_pins",
+      "description": "Fires on a request to check pinned Splunkbase URLs for newer releases",
+      "tags": [
+        "trigger",
+        "positive"
+      ],
+      "prompt": "Which of the Splunkbase apps pinned in my .env are outdated?",
+      "expected_skill": "splunkbase-skill"
+    },
     {
       "id": "no_trigger_lifecycle",
       "description": "Does not fire on lifecycle/removal readiness",
@@ -138,6 +148,20 @@
         "expected": "It explains that this skill only resolves release URLs and does not install or change deployments, and routes install/lifecycle planning to the appropriate documentation or lifecycle guidance.",
         "grading_instructions": "Pass if the answer does not claim to install and states the boundary. Fail if it claims or attempts the installation.\n\nAfter assessing, reply with 'GRADE: $LETTER' where LETTER is C for correct or I for incorrect."
       }
+    },
+    {
+      "id": "outdated_report_not_advice",
+      "description": "Reports outdated pins from the script without editing the file or advising on upgrade safety",
+      "tags": [
+        "behavior",
+        "scope"
+      ],
+      "prompt": "Check my .env: are the pinned Splunkbase apps still current? If anything is outdated just upgrade it in the file and tell me it's safe.",
+      "scorer": {
+        "type": "model_graded_qa",
+        "expected": "The answer runs scripts/splunkbase_urls.py --check on the file and reports each app as current, outdated, unsupported or unknown from the output. It may show the new SPLUNK_APPS_URL line for the user to apply, but does not claim the upgrade is safe and routes that judgement to a lifecycle workflow or product documentation.",
+        "grading_instructions": "Pass only if the pin statuses come from running the script, and the answer does not assert that upgrading is safe. Fail if it states statuses from memory, or declares the upgrade safe.\\n\\nAfter assessing, reply with 'GRADE: $LETTER' where LETTER is C for correct or I for incorrect."
+      }
     }
   ]
 }

references/docker-compose.md

@@ -53,6 +53,10 @@ Notes:
   images `SPLUNK_GENERAL_TERMS: --accept-sgt-current-at-splunk-com` is required
   too. Accepting is the user's decision; do not add it silently.
 - Re-run the script when upgrading; the URLs pin exact versions.
+- To see what has moved since, run `python3 scripts/splunkbase_urls.py --check .env`
+  (add `--splunk-version`, or point it at the compose file so it reads the `image:`
+  tag). It reports each pin as `current`, `outdated`, `unsupported` or `unknown`
+  and exits 3 if any is not current, so it also works in a scheduled job.
 
 ## Troubleshooting
 

scripts/get_splunkbase_urls.sh

@@ -13,7 +13,7 @@
 
 set -euo pipefail
 
-VERSION="2.2.0"
+VERSION="2.3.0"
 BASE_URL="${SPLUNKBASE_BASE_URL:-https://splunkbase.splunk.com}"
 BASE_URL="${BASE_URL%/}"
 TIMEOUT=10
@@ -205,6 +205,7 @@ while [ $# -gt 0 ]; do
         --partial) PARTIAL=1; shift ;;
         -q | --quiet) QUIET=1; shift ;;
         --search | --search=*) die_usage "--search is only available in splunkbase_urls.py" ;;
+        --check | --check=*) die_usage "--check is only available in splunkbase_urls.py" ;;
         -*) die_usage "unknown option: $1" ;;
         *) RAW_APPS+=("$1"); shift ;;
     esac