Skip to content

add AZT005: resources and data sources must have basic, requiresImport, complete and update acceptance tests - #68

Open
katbyte wants to merge 6 commits into
mainfrom
kt/azt005-resource-missing-acceptance-tests
Open

katbyte wants to merge 6 commits into
mainfrom
kt/azt005-resource-missing-acceptance-tests

Conversation

@katbyte

@katbyte katbyte commented Oct 9, 2026 •

Copy link
Copy Markdown
Owner

A new rule, AZT005, that reports registered resources and data sources missing their core acceptance tests.

Why: nothing today ties a registered resource to a set of tests, so a resource can ship with no tests at all and nobody notices. A survey of azurerm while designing this found 5 resources with no test, and roughly 212 resources with no requires-import check, among them the Linux and Windows function apps, private endpoint, NAT gateway, Kusto cluster, the Cosmos children and the whole Data Factory family.

Resources without their core tests. The rule starts from the registration, so a resource with zero tests is visible. A data source needs basic. A resource needs basic and requiresImport, plus complete and update when it declares an Update handler, which the plugin SDK only allows when some field is not ForceNew. Test files are a separate package, so they are read from disk: a test counts when it is in a file named after the resource's file or its name spells the type name, with the older AzureRM prefix, a Resource suffix and DataSource affixes accepted. A step counts when a name segment starts with it, so _completeWithTags and _updateSku both count. The report names the tests to add.

 func (r Registration) SupportedResources() map[string]*pluginsdk.Resource {
 	return map[string]*pluginsdk.Resource{
-		"azurerm_example": resourceExample(), // only TestAccExample_basic exists
+		"azurerm_example": resourceExample(), // basic, requiresImport, complete, update

Shared code. The registration-entry walker AZP002 kept privately now lives in lib/tf, so AZP002 and AZT005 read registrations the same way. Each entry now also carries the registered value, which is what AZT005 resolves to the declaring file.

-		entries, allResolved := collectEntries(pass, fn.Body)
+		entries, allResolved := tf.RegistrationEntries(pass, fn.Body)

Things that are fine as they are. AZP002's behaviour, fixtures and tests are unchanged and pass on the shared walker. A resource whose Update handler is explicitly nil is treated as having none. Entries declared in another package are skipped rather than guessed at. Checking that every config step is followed by an ImportStep() is deliberately not part of this rule.

Enforcing one step at a time. Every step is asked for by default. The disable option turns steps off, as a list in the golangci settings or repeated on the command line, so a provider can start with basic alone and add requiresImport, then complete and update, as each backlog clears.

          AZT005:
            disable: [complete, update]

Run against azurerm it reports 647 entries: 600 resources and 47 data sources, with 430 missing complete, 252 requiresImport, 247 update and 170 basic. Spot checks against an independent grep survey found no false positives; every disagreement was a test the grep could not see, such as _Update with a capital or a declaration it could not resolve. Five resources have no test of any kind: the Front Door rules engine, the Palo Alto virtual network appliance, both Sentinel AWS data connectors and the domain service replica set.

The rule only reports. In azurerm the backlog is large enough that it belongs in the disable list or an azignore sweep at adoption, with enforcement on new code through --new-from-rev. Upsert-style singletons such as transparent data encryption and security alert policies, where a requires-import check cannot fail, are legitimate azignore cases.

Companion to #67, which adds AZT004 for acceptance tests not named TestAcc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant