Skip to content

Checkov tool omits Azure Pipelines results #163

Open

Description

@ncook-hxgn

When I use the Checkov tool to scan a cloned Azure DevOps Git repo with an azure-pipelines.yml, I can see CKV_AZUREPIPELINES_ results, but when I use MicrosoftSecurityDevOps@1 in a pipeline for that very same repo, scanning the whole repo, I don't get these CKV_AZUREPIPELINES_ errors.

Don't get me wrong but, this seems like a missed easy win?
Thus, I'm not sure if this is a bug with my implementation or a missing feature. As such here are some details on my implementation, please let me know if I'm missing something obvious.

Here's my .gdcnconfig, derived from https://github.com/microsoft/security-devops-azdevops/wiki#checkov-gdnconfig-sample:

{
  "tools": [
    {
      "tool": {
        "name": "checkov",
        "version": "Latest"
      },
      "arguments": {
        "Directory": "$(Checkov.DefaultTargetDirectory)",
        "Help": false,
        "Version": false,
        "OutputType": "sarif",
        "List": false,
        "Quiet": false,
        "Compact": true,
        "RunAllExternalChecks": false,
        "Soft": false,
        "ShowConfig": false,
        "CreateBaseline": false,
        "OutputBaselineAsSkipped": false,
        "NoFailOnCrash": false,
        "EnableSecretScanAllFiles": true
      },
      "outputExtension": "sarif",
      "successfulExitCodes": [
        0
      ],
      "errorExitCodes": {
        "1": "Checkov tool found issues.",
        "2": "An error has occurred running the Checkov tool."
      },
      "outputPaths": []
    }
  ]
}

And the pipeline task:

  - task: MicrosoftSecurityDevOps@1
    env:
      GDN_RUN_WORKINGDIRECTORY: $(Build.SourcesDirectory)
    inputs:
      break: true
      publish: false

I do get the other results from Checkov, although not always on par with calling standalone Checkov itself.

There seems to be some behind-the-scenes magic at work. For instance, when using MicrosoftSecurityDevOps@1 I found CKV_AZURE_177 downgraded to a warning, where it's an error when I call checkov standalone to perform what I think is the same scan.

Here's how I've been performing a manual scan with Checkov, that seems to look deeper and be more strict than the task's call does:

checkov -d "$(Build.SourcesDirectory)" --repo-root-for-plan-enrichment "$(Build.SourcesDirectory)" --deep-analysis  -o cli -o sarif --output-file-path console,$results

Activity

  1. ncook-hxgn commented on May 13, 2026

    @ncook-hxgn
    Author

    .. OK fine, I was using TargetDirectory. When I used Directory, MSDO returns the following:

    ##[error]Error running tool 3 of 3: checkov
    ##[error]UnrecognizedArgumentException: Unrecognized argument key: Directory
    ##[error]BreakException: Guardian detected one or more breaking results.
    

    So, the documented example on the wiki seems broken.

  2. DimaBir commented on May 13, 2026

    @DimaBir
    Collaborator

    Hi ncook-hxgn — thanks for the detailed report and the side-by-side comparison, that's very helpful.

    A few things worth checking before we dig deeper:

    1. Argument key name. In your .gdnconfig the Checkov argument is named "Directory", but the wrapper's schema expects "TargetDirectory" (it translates to --directory). When the key doesn't match, the value is silently ignored and Guardian falls back to its default working directory. The wiki sample uses TargetDirectory, and the same gotcha came up in Help me with the example of using environment variables with values for checkov #121. Could you rename it and re-run?

    2. The actual Checkov invocation. In the pipeline logs, look for the line that starts with …/Microsoft.Guardian.CheckovRedist_…/tools/dist/checkov …. That line shows the exact CLI Guardian is executing — bundled version, directory, frameworks, soft-fail, etc. Sharing it (and the resulting .gdn/.r/checkov/001/checkov.sarif) would let us tell quickly whether azure_pipelines framework results are being produced and filtered out downstream, or simply never produced.

    3. --deep-analysis / --repo-root-for-plan-enrichment. These flags are not currently surfaced by the wrapper's argument schema, so the wrapped invocation won't pass them. They primarily affect Terraform plan enrichment rather than the CKV_AZUREPIPELINES_* rule family, so I wouldn't expect them to explain the missing Azure Pipelines results — but worth flagging.

    4. CKV_AZURE_177 severity. The task applies Guardian's severity mapping on top of Checkov's output, which can diverge from Checkov's defaults. That's by design rather than a bug, but if you have a specific case where the downgrade hides something material, please call that out separately.

    If you can share (a) the log line above, (b) the SARIF the task produces, and (c) the result of changing Directory → TargetDirectory, that should be enough to confirm whether this is a config issue, a wrapper issue, or a Guardian-side filter. Thanks!

  3. DimaBir commented on May 13, 2026

    @DimaBir
    Collaborator

    You're right, and apologies for the confused direction — I had your .gdnconfig and the wiki sample mixed up in my head. To confirm what you've established:

    • TargetDirectory is correct (matches samples/configs/checkov.gdnconfig in this repo).
    • Directory is rejected by the wrapper with UnrecognizedArgumentException — that's the error you just hit.
    • The wiki Home page Checkov sample is wrong: the JSON snippet uses "Directory": "" while the env-var table immediately below maps Directory → GDN_CHECKOV_TARGETDIRECTORY. The JSON snippet is the broken half — it should read "TargetDirectory". I'll get a wiki fix in for that. Thanks for catching it.

    Now, back to the original CKV_AZUREPIPELINES_* question — since your real config (with TargetDirectory) does scan and does return Checkov results, the most useful artifact would still be:

    1. The …/Microsoft.Guardian.CheckovRedist_…/tools/dist/checkov … invocation line from your task logs (so we can see the bundled version and the exact flags Guardian passed).
    2. The resulting SARIF at .gdn/.r/checkov/001/checkov.sarif (or at least its runs[0].tool.driver.rules[*].id list, filtered to CKV_AZUREPIPELINES_*) so we can tell whether the rules were executed but filtered out downstream vs never executed at all.

    That'll let me say definitively whether this is a framework/version gap in the bundled Checkov, a Guardian post-processing filter, or something else. Sorry again for the detour.

  4. DimaBir commented on May 13, 2026

    @DimaBir
    Collaborator

    Quick follow-up: the wiki is updated and live — the Checkov gdnconfig sample and the env-var table now both say TargetDirectory. Thanks again for catching that.

    The CKV_AZUREPIPELINES_* question is still open — when you have a chance, the Guardian invocation log line and .gdn/.r/checkov/001/checkov.sarif from a real run would let me give you a definitive answer.

  5. ncook-hxgn commented on May 13, 2026

    @ncook-hxgn
    Author

    1., 2. So, I found a combination of what I had working before + what I saw in the wiki example that surfaces* the CKV_AZUREPIPELINES_* errors - I changed from $(Checkov.DefaultTargetDirectory) to $(WorkingDirectory) and returned to TargetDirectory rather than the documented Directory from the sample.

    * I can see them in the cli output from Checkov as called by guardian,

    Image

    but they aren't showing up in the SAST Scans tab collapse for checkov. They are present in the Checkov collapse from my standalone call though.. (I can't get good screen shots of the omissions from the SAST Scans tab as I have quite a lot of results)..

    Standalone call's sarif in Scans tab:

    Image Image

    MicrosoftSecurityDevOps@1's results in Scans tab:

    Image Image

    So, whilst the following .gdnconfig gets Checkov to find my azure-pipelines.yml and checkov does discover and record pipeline issues to CLI, I can't see them in the final sarif.

    1. for 44 issue from standalone vs 29 from MSDO, error degredation to warning etc. Fair enough, that makes sense. I guess it is this severity mapping by Guardian that may be excluding the CKV_AZUREPIPELINES_* results from the final sarif that is rendered by the Scans tab?
        {
          "tool": {
            "name": "checkov",
            "version": "Latest"
          },
          "arguments": {
            "TargetDirectory": "$(WorkingDirectory)",
            "Help": false,
            "Version": false,
            "OutputType": "sarif",
            "List": false,
            "Quiet": false,
            "Compact": true,
            "RunAllExternalChecks": false,
            "Soft": false,
            "ShowConfig": false,
            "CreateBaseline": false,
            "OutputBaselineAsSkipped": false,
            "NoFailOnCrash": false,
            "EnableSecretScanAllFiles": true
          },
          "outputExtension": "sarif",
          "successfulExitCodes": [
            0
          ],
          "errorExitCodes": {
            "1": "Checkov tool found issues.",
            "2": "An error has occurred running the Checkov tool."
          },
          "outputPaths": []
        }
  6. ncook-hxgn commented on May 13, 2026

    @ncook-hxgn
    Author

    And here is the requested guardian call for checkov, output from pipeline log:

     Running Checkov 3.2.497
        ------------------------------------------------------------------------------
        /home/vsts/work/_msdo/packages/nuget/Microsoft.Guardian.CheckovRedist_linux_amd64.3.2.497/tools/dist/checkov --directory /home/vsts/work/1/s --output sarif --quiet --enable-secret-scan-all-files --output-file-path /home/vsts/work/1/s/.gdn/.r/checkov/001/checkov.sarif
        
        
               _               _
           ___| |__   ___  ___| | _______   __
          / __| '_ \ / _ \/ __| |/ / _ \ \ / /
         | (__| | | |  __/ (__|   < (_) \ V /
          \___|_| |_|\___|\___|_|\_\___/ \_/
        
        By Prisma Cloud | version: 3.2.497 
        Update available 3.2.497 -> 3.2.528
        Run pip3 install -U checkov to update
    
  7. ncook-hxgn commented on May 13, 2026

    @ncook-hxgn
    Author

    (or at least its runs[0].tool.driver.rules[].id list, filtered to CKV_AZUREPIPELINES_)

    Let me see if I can do some jq stuff in the pipeline to get this info, hang tight :)

  8. ncook-hxgn commented on May 13, 2026

    @ncook-hxgn
    Author

    Here's the output from the query of the checkov sarif file.

    ...
    2026-05-13T12:31:04.9150438Z ##[command]jq '[.runs[0].tool.driver.rules[].id]' /home/vsts/work/1/s/.gdn/.r/checkov/001/checkov.sarif/results_sarif.sarif
    2026-05-13T12:31:04.9184658Z [
    2026-05-13T12:31:04.9186897Z   "CKV_AZURE_212",
    2026-05-13T12:31:04.9187422Z   "CKV_AZURE_225",
    2026-05-13T12:31:04.9187855Z   "CKV_AZURE_59",
    2026-05-13T12:31:04.9188294Z   "CKV_AZURE_33",
    2026-05-13T12:31:04.9188718Z   "CKV_AZURE_44",
    2026-05-13T12:31:04.9189128Z   "CKV_AZURE_35",
    2026-05-13T12:31:04.9189556Z   "CKV_AZURE_190",
    2026-05-13T12:31:04.9189965Z   "CKV_AZURE_206",
    2026-05-13T12:31:04.9190660Z   "CKV_AZURE_72",
    2026-05-13T12:31:04.9191225Z   "CKV_AZURE_221",
    2026-05-13T12:31:04.9191640Z   "CKV_AZURE_178",
    2026-05-13T12:31:04.9192047Z   "CKV_AZURE_49",
    2026-05-13T12:31:04.9192441Z   "CKV_AZURE_97",
    2026-05-13T12:31:04.9192823Z   "CKV_AZURE_149",
    2026-05-13T12:31:04.9193456Z   "CKV_AZURE_42",
    2026-05-13T12:31:04.9193863Z   "CKV_AZURE_189",
    2026-05-13T12:31:04.9194254Z   "CKV_AZURE_110",
    2026-05-13T12:31:04.9194691Z   "CKV_AZURE_109",
    2026-05-13T12:31:04.9195357Z   "CKV_AZURE_177",
    2026-05-13T12:31:04.9195849Z   "CKV2_AZURE_33",
    2026-05-13T12:31:04.9196360Z   "CKV2_AZURE_1",
    2026-05-13T12:31:04.9196991Z   "CKV2_AZURE_38",
    2026-05-13T12:31:04.9197435Z   "CKV2_AZURE_32",
    2026-05-13T12:31:04.9197888Z   "CKV2_AZURE_40",
    2026-05-13T12:31:04.9198324Z   "CKV2_AZURE_47",
    2026-05-13T12:31:04.9198763Z   "CKV2_AZURE_41",
    2026-05-13T12:31:04.9199196Z   "CKV2_AZURE_21",
    2026-05-13T12:31:04.9199630Z   "CKV_SECRET_6",
    2026-05-13T12:31:04.9200059Z   "CKV_AZUREPIPELINES_1",
    2026-05-13T12:31:04.9200487Z   "CKV_AZUREPIPELINES_2"
    2026-05-13T12:31:04.9200886Z ]
    ...
    

    Sorry about the wonky filename, I've been playing with outputPaths in the .gdnconfigs

          "outputPaths": ["$(WorkingDirectory)"]

    I digress. I also managed to extract the ruleids from the msdo.sarif emitted by the task - the CKV_AZUREPIPELINES_1 and CKV_AZUREPIPELINES_2 are present..

    jq '[.runs[] | select(.tool.driver.name == "checkov") | .results[].ruleId]' /home/vsts/work/1/a/.gdn/msdo.sarif
    [
      "CKV_AZUREPIPELINES_1",
      "CKV_AZUREPIPELINES_2",
      "CKV_AZURE_59",
      "CKV_AZURE_33",
      "CKV_AZURE_190",
      "CKV_AZURE_206",
      "CKV_AZURE_42",
      "CKV_AZURE_189",
      "CKV_AZURE_110",
      "CKV_AZURE_109",
      "CKV2_AZURE_33",
      "CKV2_AZURE_32",
      "CKV2_AZURE_40",
      "CKV2_AZURE_47",
      "CKV2_AZURE_41",
      "CKV2_AZURE_1",
      "CKV2_AZURE_21",
      "CKV_SECRET_6",
      "CKV_AZURE_177",
      "CKV_AZURE_97",
      "CKV_SECRET_6",
      "CKV_AZURE_178",
      "CKV_AZURE_49",
      "CKV_AZURE_97",
      "CKV_AZURE_149",
      "CKV_SECRET_6",
      "CKV_AZURE_212",
      "CKV_AZURE_225",
      "CKV_AZURE_59",
      "CKV_AZURE_33",
      "CKV_AZURE_44",
      "CKV_AZURE_35",
      "CKV_AZURE_190",
      "CKV_AZURE_206",
      "CKV2_AZURE_33",
      "CKV2_AZURE_38",
      "CKV2_AZURE_40",
      "CKV2_AZURE_47",
      "CKV2_AZURE_41",
      "CKV2_AZURE_1",
      "CKV_AZURE_221",
      "CKV_AZURE_221",
      "CKV_AZURE_72",
      "CKV_AZURE_221"
    ]
    

    but yet not visible in the scans tab. That's super weird, I'm not sure how to explain that. Somehow the checkov-results.sarif file from standalone checkov calls, when uploaded to CodeAnalysisLogs, definitely shows the CSKV_AZUREPIPELINES_* stuff in the SARIF Scans tab.

  9. DimaBir commented on May 13, 2026

    @DimaBir
    Collaborator

    Thanks for the thorough data — this changes the diagnosis.

    Both queries confirm CKV_AZUREPIPELINES_1 and CKV_AZUREPIPELINES_2 do survive into msdo.sarif. So the drop isn't happening in Guardian / MSDO's SARIF processing — those findings are in the published CodeAnalysisLogs artifact. The disappearance is in the rendering layer reading msdo.sarif (the SARIF Scans Tab extension).

    Two structural hypotheses, in priority order:

    1. Orphaned results[].ruleId references. SARIF allows results[].ruleId to refer to a rule that isn't defined in runs[*].tool.driver.rules[]. Most SARIF viewers — including the Scans Tab — silently skip results whose ruleId doesn't resolve to a rule definition. Your jq queried results[].ruleId in msdo.sarif but did not query tool.driver.rules[].id. If MSDO's SARIF merge keeps the results but loses some rule definitions, AZUREPIPELINES findings would be present-but-invisible.

    2. Multi-run SARIF, viewer reading only one run. Your msdo.sarif jq used .runs[] (plural). The duplicate ruleIds in your dump (CKV_AZURE_221 × 3, CKV_SECRET_6 × 3) are a strong signal that MSDO is concatenating multiple Checkov runs into one file. If the Scans Tab renders only runs[0] (or one run per tool name), and the AZUREPIPELINES results live in a non-first run, they'd be invisible regardless of correctness.

    A single jq query disambiguates these. Could you run this against msdo.sarif (not the raw Checkov one) and share the output?

    jq '[.runs[] | select(.tool.driver.name == "checkov") | {
      rule_ids: [.tool.driver.rules[].id],
      result_ids_unique: ([.results[].ruleId] | unique)
    }]' /home/vsts/work/1/a/.gdn/msdo.sarif

    For each Checkov run in msdo.sarif it returns the rules defined in that run and the unique ruleIds the results reference. Interpretation:

    • If CKV_AZUREPIPELINES_* appears in result_ids_unique for some run but not in that same run's rule_ids → hypothesis 1 (orphaned references).
    • If they appear in both for one run and are absent from other runs → hypothesis 2 (multi-run shape mismatch).

    Either way, the fix would be on our side (or on the Scans Tab extension's side); your config is doing what it should. Thanks again for the careful reproduction — this is rapidly converging on a precise root cause.

  10. ncook-hxgn commented on May 13, 2026

    @ncook-hxgn
    Author

    Our .gdn config has configurations for trivy, terrascan, and checkov, so there are indeed three sets of results to be fed into msdo.sarif, alas, they aren't all checkov.

    jq '[.runs[] | select(.tool.driver.name == "checkov") | {
    rule_ids: [.tool.driver.rules[].id],
    result_ids_unique: ([.results[].ruleId] | unique)
    }]' /home/vsts/work/1/a/.gdn/msdo.sarif

    brb :)

  11. ncook-hxgn commented on May 13, 2026

    @ncook-hxgn
    Author
    2026-05-13T14:57:45.5464034Z ##[command]jq '[.runs[] | select(.tool.driver.name == checkov) | { rule_ids: [.tool.driver.rules[].id], result_ids_unique: ([.results[].ruleId] | unique) }]' /home/vsts/work/1/a/.gdn/msdo.sarif
    2026-05-13T14:57:45.5489460Z [
    2026-05-13T14:57:45.5489840Z   {
    2026-05-13T14:57:45.5490161Z     "rule_ids": [
    2026-05-13T14:57:45.5490496Z       "CKV_AZURE_212",
    2026-05-13T14:57:45.5490825Z       "CKV_AZURE_225",
    2026-05-13T14:57:45.5491146Z       "CKV_AZURE_59",
    2026-05-13T14:57:45.5491459Z       "CKV_AZURE_33",
    2026-05-13T14:57:45.5491780Z       "CKV_AZURE_44",
    2026-05-13T14:57:45.5492098Z       "CKV_AZURE_35",
    2026-05-13T14:57:45.5492412Z       "CKV_AZURE_190",
    2026-05-13T14:57:45.5492732Z       "CKV_AZURE_206",
    2026-05-13T14:57:45.5493053Z       "CKV_AZURE_72",
    2026-05-13T14:57:45.5493554Z       "CKV_AZURE_221",
    2026-05-13T14:57:45.5493890Z       "CKV_AZURE_178",
    2026-05-13T14:57:45.5494217Z       "CKV_AZURE_49",
    2026-05-13T14:57:45.5494659Z       "CKV_AZURE_97",
    2026-05-13T14:57:45.5494985Z       "CKV_AZURE_149",
    2026-05-13T14:57:45.5495302Z       "CKV_AZURE_42",
    2026-05-13T14:57:45.5495787Z       "CKV_AZURE_189",
    2026-05-13T14:57:45.5496712Z       "CKV_AZURE_110",
    2026-05-13T14:57:45.5497102Z       "CKV_AZURE_109",
    2026-05-13T14:57:45.5497415Z       "CKV_AZURE_177",
    2026-05-13T14:57:45.5497731Z       "CKV2_AZURE_33",
    2026-05-13T14:57:45.5498047Z       "CKV2_AZURE_1",
    2026-05-13T14:57:45.5498359Z       "CKV2_AZURE_38",
    2026-05-13T14:57:45.5498672Z       "CKV2_AZURE_32",
    2026-05-13T14:57:45.5498977Z       "CKV2_AZURE_40",
    2026-05-13T14:57:45.5499297Z       "CKV2_AZURE_47",
    2026-05-13T14:57:45.5499618Z       "CKV2_AZURE_41",
    2026-05-13T14:57:45.5499933Z       "CKV2_AZURE_21",
    2026-05-13T14:57:45.5500250Z       "CKV_SECRET_6",
    2026-05-13T14:57:45.5500576Z       "CKV_AZUREPIPELINES_1",
    2026-05-13T14:57:45.5500917Z       "CKV_AZUREPIPELINES_2"
    2026-05-13T14:57:45.5501229Z     ],
    2026-05-13T14:57:45.5501604Z     "result_ids_unique": [
    2026-05-13T14:57:45.5501997Z       "CKV2_AZURE_1",
    2026-05-13T14:57:45.5502315Z       "CKV2_AZURE_21",
    2026-05-13T14:57:45.5502644Z       "CKV2_AZURE_32",
    2026-05-13T14:57:45.5502963Z       "CKV2_AZURE_33",
    2026-05-13T14:57:45.5503289Z       "CKV2_AZURE_38",
    2026-05-13T14:57:45.5503608Z       "CKV2_AZURE_40",
    2026-05-13T14:57:45.5503931Z       "CKV2_AZURE_41",
    2026-05-13T14:57:45.5504255Z       "CKV2_AZURE_47",
    2026-05-13T14:57:45.5504583Z       "CKV_AZUREPIPELINES_1",
    2026-05-13T14:57:45.5504920Z       "CKV_AZUREPIPELINES_2",
    2026-05-13T14:57:45.5505245Z       "CKV_AZURE_109",
    2026-05-13T14:57:45.5505566Z       "CKV_AZURE_110",
    2026-05-13T14:57:45.5505889Z       "CKV_AZURE_149",
    2026-05-13T14:57:45.5506484Z       "CKV_AZURE_177",
    2026-05-13T14:57:45.5506811Z       "CKV_AZURE_178",
    2026-05-13T14:57:45.5507144Z       "CKV_AZURE_189",
    2026-05-13T14:57:45.5507465Z       "CKV_AZURE_190",
    2026-05-13T14:57:45.5507793Z       "CKV_AZURE_206",
    2026-05-13T14:57:45.5508105Z       "CKV_AZURE_212",
    2026-05-13T14:57:45.5508431Z       "CKV_AZURE_221",
    2026-05-13T14:57:45.5508772Z       "CKV_AZURE_225",
    2026-05-13T14:57:45.5509098Z       "CKV_AZURE_33",
    2026-05-13T14:57:45.5509421Z       "CKV_AZURE_35",
    2026-05-13T14:57:45.5509741Z       "CKV_AZURE_42",
    2026-05-13T14:57:45.5510067Z       "CKV_AZURE_44",
    2026-05-13T14:57:45.5510391Z       "CKV_AZURE_49",
    2026-05-13T14:57:45.5510709Z       "CKV_AZURE_59",
    2026-05-13T14:57:45.5511025Z       "CKV_AZURE_72",
    2026-05-13T14:57:45.5511330Z       "CKV_AZURE_97",
    2026-05-13T14:57:45.5511648Z       "CKV_SECRET_6"
    2026-05-13T14:57:45.5512030Z     ]
    2026-05-13T14:57:45.5512320Z   }
    2026-05-13T14:57:45.5512609Z ]
    
  12. ncook-hxgn commented on May 13, 2026

    @ncook-hxgn
    Author

    Multi-run SARIF, viewer reading only one run.

    This isn't quite correct: we run trivy, terrascan and now checkov with MicrosoftSecurityDevOps@1, and we get results for all three of those tools in the Scans tab, only the checkov results omit the AZUREPIPELINES result.

    When I run checkov "standalone" and upload the generated .sarif, I do get the CKV_AZUREPIPELINES_* issues rendered in the Scans tab.

    checkov -d "$(Build.SourcesDirectory)" --repo-root-for-plan-enrichment "$(Build.SourcesDirectory)" --deep-analysis \
       -o cli -o sarif --output-file-path console,$(Build.ArtifactStagingDirectory)/CodeAnalysisLogs/checkov-results.sarif
    echo "##vso[artifact.upload artifactname=CodeAnalysisLogs;]$(Build.ArtifactStagingDirectory)/CodeAnalysisLogs/checkov-results.sarif"
    

    Yeilds:

    Image

    As such, I'm not convinced that it's the Scans tab that's at fault.

    Edit:
    Comparing the sarif results file from checkov itself against msdo.sarif, I found both instances of the CKV_AZUREPIPELINES_2 issue. It seems I can't see it because Guardian has downgraded this to a note where checkov initially raises this as an error:

    Image

    Ahhh - the penny drops. Because they are 'note' in Guardian's opinion, the Scans tab filters them out. When I adjust the filter to include Note level issues, they appear!

    Image
  13. ncook-hxgn commented on May 13, 2026

    @ncook-hxgn
    Author

    So, I think I would reposit my overall question: please could CKV_AZUREPIPELINES_ be promoted to at least warning ?

    With all the checkmarx and trivy malarkey that's gone on recently, I think avoiding the latest tag on container images used by pipelines is probably more important than it was when the logic to downgrade this to a note was decided on originally.

  14. DimaBir commented on May 13, 2026

    @DimaBir
    Collaborator

    You nailed it — and apologies for the detour. Looking back, my very first reply flagged Guardian's severity mapping (point 4) as a possible factor, but I dropped that thread and chased structural SARIF hypotheses instead. You did the work my point 4 should have done.

    To summarize where we landed for anyone reading later:

    • Root cause: Guardian's severity policy maps CKV_AZUREPIPELINES_* to level: "note" in the published msdo.sarif. The SARIF Scans Tab extension's default severity filter hides notes, so the findings render only when you explicitly include Note in the filter.
    • Not the wrapper config, not the Scans Tab extension, not a SARIF shape problem. The findings are present in msdo.sarif and in CodeAnalysisLogs; they're just below the default visibility threshold.

    On your actual request — promoting CKV_AZUREPIPELINES_* to at least warning: that's a reasonable ask, and your reasoning lines up with where the threat landscape has moved (pinning images vs latest, untrusted third-party tasks, etc.). The severity mapping lives in Guardian's policy rather than in this task's argument schema, so it isn't something we can flip via a config option in .gdnconfig — it needs a policy update on the Guardian side. I'll flag this with the team and treat this issue as the feature/policy request going forward.

    Separately, the "findings present in the file but hidden by the default Scans Tab filter" UX is worth a follow-up of its own — even users who want note-level findings won't necessarily think to change the filter. I'll mention that alongside the severity ask.

    Re-labeled accordingly. Thanks again for the careful repro and the patience through my wrong turn.

  15. DimaBir commented on May 13, 2026

    @DimaBir
    Collaborator
  16. ncook-hxgn commented on May 14, 2026

    @ncook-hxgn
    Author

    Much appreciated Dima Birenbaum (@DimaBir)

  17. DimaBir commented on May 16, 2026

    @DimaBir
    Collaborator

    Filed the spec as a separate trackable issue so it can be assigned and worked independently: #164. This issue (#163) will close automatically when #164 is merged per the spec's definition of done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

area:taskAzure DevOps build task definition and configurationarea:toolsIndividual security tool configuration and issuesstatus:team-reviewQueued for team review and decisiontype:docsImprovements or additions to documentationtype:featureNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions