diff --git a/administration/cloud_capture/getting_started.md b/administration/cloud_capture/getting_started.md index c4dccecd..59b21ac8 100644 --- a/administration/cloud_capture/getting_started.md +++ b/administration/cloud_capture/getting_started.md @@ -41,11 +41,50 @@ When used, the CloudFormation template will send your AWS account id to Kosli, s ### Enable Cloud Capture -When you have created the IAM role, using the CloudFormation template, you can activate Cloud Capture within the Kosli user-interface. Cloud Capture runs on a five-minute schedule, and once you have enabled it, Cloud Capture will pick up your environment the next time it runs - you should see environments and snapshots appearing within a few minutes. +When you have created the IAM role, using the CloudFormation template, you can activate Cloud Capture within the Kosli user-interface. Cloud Capture scans every five minutes unless you [choose another frequency](#scan-frequency), and once you have enabled it, Cloud Capture will pick up your environment the next time it runs - you should see environments and snapshots appearing within a few minutes. ### Excluding resources -If there are resources you do not wish to include within a Cloud Capture snapshot, for example an ECS cluster that you consider to be out of scope, you can add a tag to it indicating that the item should be skipped. Adding a tag with the name `kosli.capture` and the value `false` will ensure that Cloud Capture skips over that resource. +If there are resources you do not want Cloud Capture to snapshot, such as a test cluster or a service a vendor runs in your account, exclude them by tag. You choose the tags: in your organization's **Settings**, under **Cloud Capture**, the **Skip tags** panel takes any number of key and value pairs, and Cloud Capture skips every resource that carries one of them. + +1. Tag the resource in AWS, with any key and value you like. For example, to tag an ECS cluster and a Lambda function with `kosli-capture=false`: + + ```bash + aws ecs tag-resource \ + --resource-arn arn:aws:ecs:eu-west-1:123456789012:cluster/sandbox \ + --tags key=kosli-capture,value=false + + aws lambda tag-resource \ + --resource arn:aws:lambda:eu-west-1:123456789012:function:scratch \ + --tags kosli-capture=false + ``` + +2. In the **Skip tags** panel, enter the key `kosli-capture` and the value `false`, select **Add**, then **Save**. + +How the tags are matched: + +- A resource is skipped when any one of its tags equals any one of the skip tags, key and value both, exactly as AWS stores them. Tags are case-sensitive, so `Env=test` does not match `env=test`. A key on its own does not match: to skip both `env=test` and `env=staging`, add both. +- The tag is read on the resource Cloud Capture snapshots. An ECS cluster is skipped with everything in it, its services and tasks included; tags on a service or a task definition are not read. A Lambda function is skipped on its own, and the region's Lambda environment in Kosli is made up of the functions that remain. +- The change takes effect on the next scan. A Kosli environment that was already being captured keeps its last snapshot and receives no new ones; nothing is deleted. Remove the tag, or the skip tag, and the next run picks the resource up again. +- Skip tags, like the other Cloud Capture settings, can only be changed while Cloud Capture is enabled. +- The IAM role created by the CloudFormation template already grants the permissions needed to read tags, `ecs:DescribeClusters` and `lambda:ListTags`, so no change to the role is needed. + +Skip tags can also be set through the API, as part of the organization's Cloud Capture settings. The settings PUT replaces every setting, not only the one you are changing, so start from what is stored: read the current configuration, take the `enabled`, `default_regions`, `scan_interval_minutes` and `skip_tags` values from its `settings` object, add your tags to `skip_tags`, and send those four fields back. Sending the example below as it stands would also set your default regions and scan interval to the values shown. + +```bash +curl "https://app.kosli.com/api/v2/cloud-capture/" \ + -H "Authorization: Bearer " + +curl -X PUT "https://app.kosli.com/api/v2/cloud-capture//settings" \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "enabled": true, + "default_regions": {"aws": ["eu-west-1"]}, + "scan_interval_minutes": 5, + "skip_tags": {"aws": [{"key": "kosli-capture", "value": "false"}]} + }' +``` @@ -65,11 +104,21 @@ GCP has no equivalent of the CloudFormation "phone-home" feature, so the Terrafo ### Enable Cloud Capture -When you have created the role and added the account details to Kosli, you can activate Cloud Capture within the Kosli user-interface. Cloud Capture runs on a five-minute schedule, and once you have enabled it, Cloud Capture will pick up your environment the next time it runs - you should see environments and snapshots appearing within a few minutes. +When you have created the role and added the account details to Kosli, you can activate Cloud Capture within the Kosli user-interface. Cloud Capture scans every five minutes unless you [choose another frequency](#scan-frequency), and once you have enabled it, Cloud Capture will pick up your environment the next time it runs - you should see environments and snapshots appearing within a few minutes. ### Excluding resources -Support for excluding resources is coming soon. +Excluding resources by label is coming with GCP support. +## Scan frequency + +Cloud Capture scans your registered accounts every five minutes unless you choose otherwise. The frequency is a setting of your Kosli organization, so it applies to every account and region Cloud Capture scans for you. + +To change it, open your organization's **Settings**, go to **Cloud Capture**, and under **Scan frequency** pick an interval from the list: every 5, 10, 15 or 30 minutes, or every 1, 2, 6, 12 or 24 hours. Select **Save**, and the next scan runs at the new interval; nothing restarts and no snapshots are lost. Like the other Cloud Capture settings, the frequency can only be changed while Cloud Capture is enabled. + +Each scan reports what is running in every environment Cloud Capture manages for you, and Kosli records a new snapshot when that differs from the last one. The interval is therefore how long a change can go unnoticed: a deployment that lands and is rolled back inside one interval may never be seen. Five minutes suits most teams; an estate that changes rarely can scan hourly. + +Through the API, the frequency is `scan_interval_minutes` on the same settings resource as the skip tags above: any whole number of minutes from 5 to 1440 (a day), not only the presets the settings page offers, with 5 the value when it is left out. An interval set this way that is not one of the presets is shown on the settings page as its own option, so saving another setting there keeps it. The PUT replaces every setting, so read the current configuration first and send the other settings back unchanged. + diff --git a/administration/cloud_capture/overview.md b/administration/cloud_capture/overview.md index 06b4c211..13b98fa6 100644 --- a/administration/cloud_capture/overview.md +++ b/administration/cloud_capture/overview.md @@ -9,7 +9,7 @@ tag: "BETA" Cloud Capture is still in active development. Its capabilities and configuration format may change, and onboarding is done together with Kosli's Customer Success team. -Cloud Capture is a managed service that runs on Kosli's infrastructure and connects to your cloud platform to observe the resources deployed there. You grant Cloud Capture a set of permissions, and it uses them to run a `kosli snapshot` every few minutes against the infrastructure you have allowed it to scan. +Cloud Capture is a managed service that runs on Kosli's infrastructure and connects to your cloud platform to observe the resources deployed there. You grant Cloud Capture a set of permissions, and it uses them to run a `kosli snapshot` against the infrastructure you have allowed it to scan, every five minutes unless you [choose another frequency](/administration/cloud_capture/getting_started#scan-frequency). Kosli also supports reporting from your own cloud accounts by running the Kosli CLI on a schedule. Cloud Capture inverts this, with Kosli running the regular [snapshots](/getting_started/environments) so there is no software for you to install. @@ -41,7 +41,7 @@ Cloud Capture has been designed to operate with no on-going support from you. O Cloud Capture finds all supported resources within your AWS accounts, and determines which Kosli environment should hold the snapshots. Cloud Capture will create physical environments for you inside Kosli. -Cloud Capture can [filter out resources based on AWS tags](/administration/cloud_capture/getting_started#excluding-resources). +You can keep a resource out of Cloud Capture by tagging it with any tag you have listed in your Cloud Capture settings; see [Excluding resources](/administration/cloud_capture/getting_started#excluding-resources). As your cloud environment evolves, such as the addition of new AWS ECS clusters or the retirement of existing Lambdas, Cloud Capture automatically detects the changes. Because Cloud Capture creates physical environments as needed, when your infrastructure changes, Kosli will keep up. No changes to the configuration created during the initial setup are required.