Repository navigation
Conversation
…t, complete and update acceptance tests
This was referenced Oct 9, 2026
katbyte
force-pushed
the
kt/azt005-resource-missing-acceptance-tests
branch
from
October 9, 2026 21:17
f5bafec to
96463e8
Compare
…ssing-acceptance-tests # Conflicts: # CHANGELOG.md # README.md # checks/AZT/AZT.go
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A new rule, AZT005, that reports registered resources and data sources missing their core acceptance tests.
Why: nothing today ties a registered resource to a set of tests, so a resource can ship with no tests at all and nobody notices. A survey of azurerm while designing this found 5 resources with no test, and roughly 212 resources with no requires-import check, among them the Linux and Windows function apps, private endpoint, NAT gateway, Kusto cluster, the Cosmos children and the whole Data Factory family.
Resources without their core tests. The rule starts from the registration, so a resource with zero tests is visible. A data source needs
basic. A resource needsbasicandrequiresImport, pluscompleteandupdatewhen it declares an Update handler, which the plugin SDK only allows when some field is not ForceNew. Test files are a separate package, so they are read from disk: a test counts when it is in a file named after the resource's file or its name spells the type name, with the olderAzureRMprefix, aResourcesuffix andDataSourceaffixes accepted. A step counts when a name segment starts with it, so_completeWithTagsand_updateSkuboth count. The report names the tests to add.func (r Registration) SupportedResources() map[string]*pluginsdk.Resource { return map[string]*pluginsdk.Resource{ - "azurerm_example": resourceExample(), // only TestAccExample_basic exists + "azurerm_example": resourceExample(), // basic, requiresImport, complete, updateShared code. The registration-entry walker AZP002 kept privately now lives in
lib/tf, so AZP002 and AZT005 read registrations the same way. Each entry now also carries the registered value, which is what AZT005 resolves to the declaring file.Things that are fine as they are. AZP002's behaviour, fixtures and tests are unchanged and pass on the shared walker. A resource whose Update handler is explicitly
nilis treated as having none. Entries declared in another package are skipped rather than guessed at. Checking that every config step is followed by anImportStep()is deliberately not part of this rule.Enforcing one step at a time. Every step is asked for by default. The
disableoption turns steps off, as a list in the golangci settings or repeated on the command line, so a provider can start withbasicalone and addrequiresImport, thencompleteandupdate, as each backlog clears.Run against azurerm it reports 647 entries: 600 resources and 47 data sources, with 430 missing
complete, 252requiresImport, 247updateand 170basic. Spot checks against an independent grep survey found no false positives; every disagreement was a test the grep could not see, such as_Updatewith a capital or a declaration it could not resolve. Five resources have no test of any kind: the Front Door rules engine, the Palo Alto virtual network appliance, both Sentinel AWS data connectors and the domain service replica set.The rule only reports. In azurerm the backlog is large enough that it belongs in the disable list or an azignore sweep at adoption, with enforcement on new code through
--new-from-rev. Upsert-style singletons such as transparent data encryption and security alert policies, where a requires-import check cannot fail, are legitimate azignore cases.Companion to #67, which adds AZT004 for acceptance tests not named
TestAcc.