Skip to content

add AZT004: acceptance tests must be named TestAcc - #67

Merged
katbyte merged 4 commits into
mainfrom
kt/azt004-azt005-acceptance-test-naming-and-coverage
Oct 9, 2026
Merged

katbyte merged 4 commits into
mainfrom
kt/azt004-azt005-acceptance-test-naming-and-coverage

Conversation

@katbyte

@katbyte katbyte commented Oct 9, 2026 •

Copy link
Copy Markdown
Owner

A new rule, AZT004, that reports acceptance tests whose name does not start with TestAcc.

Why: acceptance runs select tests by that prefix. A test named TestThing_basic that drives the acceptance framework is skipped by every acceptance run, so the resource it covers is never actually tested against Azure, however complete the test looks. A survey of azurerm found 65 such functions covering 16 resources and data sources: every Oracle resource, the Cosmos PostgreSQL role, node configuration and firewall rule, the IoT security device group, and two App Service resources. tfproviderlint's AT005 misses all of them because it only recognises direct resource.Test calls, not azurerm's wrapper.

Tests that exist but never run. AZT004 reports a top-level Test function in a test file that reaches the acceptance harness, itself or through helpers, sequential maps or a TF_ACC check, without the TestAcc prefix. Lowercase helpers and methods are left alone, since no runner selects them. When Acc already sits elsewhere in the name, as in TestWebAppAccActiveSlot_basic, the report says to move it rather than suggesting a doubled-up name.

-func TestThing_basic(t *testing.T) {
+func TestAccThing_basic(t *testing.T) {
 	data := acceptance.BuildTestData(t, "azurerm_thing", "test")
 	data.ResourceTest(t, ThingResource{}, []acceptance.TestStep{{Config: "basic"}})

Shared code. The harness-reachability analysis AZT003 kept privately moves into lib/acctest as an analyzer with a result, so AZT003 and AZT004 compute it once per package. It returns early for packages with no test files.

-	runs := map[*types.Func]bool{}
-	mentions := map[*types.Func][]*types.Func{}
-	for fn, fd := range decls {
+	runs, ok := pass.ResultOf[acctest.Analyzer].(acctest.Funcs)

Things that are fine as they are. AZT003's messages, fixtures and tests are unchanged and pass on the shared code. Its TestAccountName handling, where Acc starts a word, is untouched.

The rule only reports. This is the mirror of AZT003, which reports the opposite case. The companion rule for resources missing their core tests is #68.

@katbyte
katbyte force-pushed the kt/azt004-azt005-acceptance-test-naming-and-coverage branch from 7ac70e6 to 99ddd77 Compare October 9, 2026 21:02
@katbyte katbyte changed the title add AZT004 and AZT005: acceptance tests must be named TestAcc, and every resource needs its core tests add AZT004: acceptance tests must be named TestAcc Oct 9, 2026
@katbyte
katbyte force-pushed the kt/azt004-azt005-acceptance-test-naming-and-coverage branch from 941f3c5 to 6fbd08b Compare October 9, 2026 21:17
@katbyte
katbyte force-pushed the kt/azt004-azt005-acceptance-test-naming-and-coverage branch from 6fbd08b to 6356fd9 Compare October 9, 2026 21:18
@katbyte
katbyte merged commit 550c3eb into main Oct 9, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant