ci(tests): stop reporting success for tests that never ran - #193
Merged
Conversation
The object store and key manager tests skip themselves when their endpoint is unset, and nothing in the workflow set one. So four suites reported success having exercised nothing, in the same shape as the bug integration-postgres already carries a comment about: five repository tests went a whole chantier that way. Service containers for RustFS and OpenBao, and the same REQUIRE_ guard integration-postgres uses, so losing a container fails the job instead of quietly emptying it. RustFS needs its master key here too. Without one it refuses every encrypted write rather than storing plaintext, which is the right failure and one worth exercising rather than configuring around. What these cover is what a mock cannot answer: path style addressing, absence of a key, prefix listing, whether the store applies the rules it was asked for, and whether a rotation leaves an archive taken before it still readable.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #177.
The bug
Four test suites landed in #188 and #189 that report success without running.
aether-s3::object_store,aether-api::archive_bucket,aether-transit::key_providerandaether-api::wrapping_keyall skip themselves when their endpoint is unset, which is what lets a developer runcargo testwithout a store or a key manager. Nothing in the workflow ever set one, so in CI they skipped, and a skip reads as a pass.This is the same shape as the bug
integration-postgresalready carries a comment about:I reintroduced it two chantiers later, which is a reasonable argument that the guard belongs beside every test that can skip rather than in one job's comment.
The fix
integration-servicesjob.REQUIRE_OBJECT_STORE_ENDPOINTandREQUIRE_KEY_MANAGER_ADDRESS, the same mechanismREQUIRE_DATABASE_URLuses: set in CI, they turn a skip into a failure, so losing a service container fails the job instead of quietly emptying it.Verified in both directions rather than assumed. With the services up, all four suites run. With
REQUIRE_OBJECT_STORE_ENDPOINT=1and no endpoint:Two details worth the lines they cost
RustFS needs
RUSTFS_SSE_S3_MASTER_KEYhere too. Without one it refuses every encrypted write rather than storing plaintext. That is the right failure, and exercising it is more useful than configuring around it. The value is the same local stand-indocker-compose.yamluses and is not a secret.Neither image ships a health check, and an S3 endpoint answers an unauthenticated request with 403, so the wait loop treats answering at all as ready rather than a successful status. This is the same mistake that made the compose health check fail earlier in this chantier:
curl -fturns a perfectly healthy 403 into a failure.builddoes not depend on this job, matching howintegration-postgresis already wired.What it is worth
The verification sections of #188 through #192 claim these suites cover real infrastructure. That was true on my machine and false in CI. It is true in both now.