test: add integration suites that create real GCE instances

[tas50]

Why

The unit suite stubs the Compute Engine client, so it proves the driver builds the right request — never that GCE accepts it. Nothing in CI has ever reached the API. There was no kitchen.yml in the repository at all, so there was no way to run the driver end to end without writing one by hand first.

That gap is where the bugs live. Several fixes since 3.0.0 were found by driving the thing against a real project, and every one of them is invisible to a mocked test, because the mock encodes the same assumption the code does:

#152	A 10 GB boot disk request against a 20 GB image, which GCE refuses outright
#158	Metadata sent under both :"created-by" and "created-by", rejected as duplicate keys
#162	A legal instance name that produced an illegal disk name
#161	A missing custom_image silently creating a blank disk, so a run verified nothing

What this adds

Ten suites in integration/, each creating a real instance and asserting on the instance that the driver configured it as asked:

Suite	What it proves
default	Create, converge and destroy from an image family, and the driver's metadata reaching the instance alongside the user's.
region	With region and no zone, the driver lists the region's zones, picks one that is up, and the instance lands there.
extra-disk	A second persistent disk created standalone, waited on until READY, attached, and deleted again on destroy.
local-ssd	A local-ssd attached as SCRATCH at GCE's fixed 375 GB, with no size sent for it.
legacy-disk	The deprecated top-level disk_size / disk_type / autodelete_disk options still producing a working boot disk.
boot-disk-size	A 10 GB request against a 20 GB image raised to the image's size rather than refused.
metadata	Custom metadata, network tags and service_account_scopes reaching the instance, with short aliases such as storage-ro expanded.
preemptible	preemptible: true honoured, and auto-restart and live migration forced off however the suite asks for them.
long-instance-and-disk-names	A suite and disk name that overflow the 63-character budget falling back to a UUID instance name, still leaving room for a legal disk name.
windows	The WinRM path: the guest agent resetting a non-builtin account's password over the serial port, and the startup script opening 5985 in the guest.

Assertions live in the shell provisioner, not a verifier. The script is transferred over the driver's own transport and executed on the instance, so reaching the machine is part of every assertion, a non-zero exit fails the suite, and there is no verifier licence to satisfy. Everything the script needs to know travels as instance metadata — see the region suite — because nothing of the local environment survives to the guest.

export GCE_PROJECT=my-gcp-project
ssh-keygen -t ed25519 -N "" -f ~/.ssh/id_kitchen_gce

bundle exec rake integration:list
bundle exec rake integration:test
bundle exec rake integration:destroy   # after a failed run

Not part of rake default — these cost money.

CI

.github/workflows/integration.yml runs weekly against main and on demand, never on a pull request: secrets are unavailable to forks and every run costs money.

It authenticates with workload identity federation, exchanging GitHub's OIDC token for Google credentials, so no service account key is stored anywhere.

Three details that matter more than they look:

• Destroy everything runs with if: always(). A suite that leaks an instance on failure turns a red build into a recurring bill.
• concurrency: gce-integration with cancel-in-progress: false. A fresh project is often capped at 8–24 CPUs per region; two overlapping runs exhaust it and both fail.
• The WinRM firewall rule is created by the workflow. Nothing in the default VPC allows 5985, and the driver cannot open it — its startup script runs inside the guest. Without the rule the windows suite just waits out its timeout on a running instance.

Setup needs three repository secrets (GCE_PROJECT, GCE_WORKLOAD_IDENTITY_PROVIDER, GCE_SERVICE_ACCOUNT) and a workload identity pool scoped to this repo, documented in integration/README.md along with the roles the service account needs.

One caveat worth reading before enabling this: the driver does not manage SSH keys, so kitchen.yml puts the public half into ssh-keys metadata itself. If the project enforces OS Login, metadata keys are ignored and nothing connects. That is called out in integration/README.md.

Testing

The suites are configuration; what can be checked without a project is that they render, that the merges come out right, and that nothing regressed:

$ cd integration && bundle exec kitchen list
Instance                                  Driver                Provisioner  Verifier  Transport
default-ubuntu-2204                       Google Compute (GCE)  Shell        Shell     Ssh
region-ubuntu-2204                        Google Compute (GCE)  Shell        Shell     Ssh
extra-disk-ubuntu-2204                    Google Compute (GCE)  Shell        Shell     Ssh
local-ssd-ubuntu-2204                     Google Compute (GCE)  Shell        Shell     Ssh
legacy-disk-ubuntu-2204                   Google Compute (GCE)  Shell        Shell     Ssh
boot-disk-size-rockylinux-9               Google Compute (GCE)  Shell        Shell     Ssh
metadata-ubuntu-2204                      Google Compute (GCE)  Shell        Shell     Ssh
preemptible-ubuntu-2204                   Google Compute (GCE)  Shell        Shell     Ssh
long-instance-and-disk-names-ubuntu-2204  Google Compute (GCE)  Shell        Shell     Ssh
windows-windows-2022                      Google Compute (GCE)  Shell        Shell     Winrm

kitchen diagnose confirms the merges: the region suite really does end up with zone: empty and region: us-central1, and every suite that sets its own metadata keeps the ssh-keys entry from the top level rather than replacing it — which is the one merge that would have silently locked the runner out.

$ bundle exec cookstyle --chefstyle
18 files inspected, no offenses detected

$ bundle exec rake test
344 examples, 0 failures

$ yamllint -c .yamllint integration/kitchen.yml .github/workflows/integration.yml
$ npx markdownlint-cli2 integration/README.md CONTRIBUTING.md
Summary: 0 issues in 0 files
$ shellcheck integration/scripts/*.sh

Follow-ups, deliberately left out

• use_private_ip needs a bastion or an IAP tunnel to assert from CI; there is no point adding a suite that cannot connect.
• guest_accelerators needs GPU quota, which a fresh project does not have.
• Shared VPC (network_project, subnet_project) needs a second project.