diff --git a/.claude/napkin.md b/.claude/napkin.md index 69e20a5..b3f8ef8 100644 --- a/.claude/napkin.md +++ b/.claude/napkin.md @@ -4,6 +4,10 @@ | Date | Source | What Went Wrong | What To Do Instead | |------|--------|----------------|-------------------| | 2026-07-06 | self | Ran the HAR generator directly on a narrow dashboard HAR and it overwrote the broad raw command registry with only 24 commands | For additive HAR integrations, preserve the generated baseline and add/merge new commands instead of replacing unrelated captured endpoints | +| 2026-07-13 | self | Passed an absolute binary path to `direnv exec`, which only resolves the command through its constructed PATH | Build temporary authenticated smoke-test binaries into an existing PATH directory (for example `~/go/bin`) and invoke by name | +| 2026-07-13 | self | Assumed `/tmp/groundcover-cli` was unused when choosing a build output and later tried to remove it as a file; it was an existing directory | Inspect temporary output paths first, then use a uniquely named file and remove only that artifact | +| 2026-07-13 | self | Assumed the harness `stat` binary used Darwin/BSD `-f` syntax because the workstation is macOS; it exposed GNU-style behavior | Prefer portable `readlink` for symlink resolution here; do not infer embedded tool variants from the host OS | +| 2026-07-13 | self | Initially documented storage `custom_rules` as optional without checking the second captured PUT; omitting the field removed the existing rule | Treat storage PUT as full writable-document replacement: start from GET, retain every writable field and the complete rule list, then edit | ## User Preferences - Always commit any changes made to this napkin file. @@ -17,5 +21,6 @@ ## Domain Notes - `raw grafana …` commands CANNOT authenticate with the gcsa bearer token (RESOLVED: use a `glsa_` grafana service account token, see Patterns That Work). Embedded Grafana at app.groundcover.com/grafana/* is session-gated. A gcsa/bearer request to `/grafana/api/*` hits a catch-all returning the ~980KB Grafana SPA `index.html` (200 text/html), never JSON. Signal of a real Grafana response = `grafana-trace-id` response header + content-type application/json. - The gcsa bearer IS valid against the real GC API (api.groundcover.com/api/*) — verified via GET /api/monitors/recurring-silences and GET /api/dashboards (both return JSON). So the grafana breakage is the proxy path, not the key. +- Storage lifecycle settings use `GET`/`PUT /api/storage-management/:dataType` on `api.groundcover.com` and authenticate with the normal `gcsa_` bearer. Verified live for `logs`. Captured data types: `logs`, `traces`, `events`, `measurements`, `monitor_instance`; PUT replaces the writable settings document, carries the current `version`, and increments it on success. Captured proof: a second PUT that omitted `custom_rules` returned settings without the previously configured rule. - For dashboards over the CLI, use the GC-native SDK `dashboards` resource (/api/dashboards), NOT the embedded-Grafana `raw grafana …` passthrough. - The `glsa_` token is generated by groundcover's OFFICIAL CLI (github.com/groundcover-com/cli), which also installs a binary named `groundcover` — collides with THIS cli. Installer (`sh -c "$(curl -fsSL https://groundcover.com/install.sh)"`) drops it at `~/.groundcover/bin/groundcover` and PREPENDS that dir to PATH, shadowing ours (doesn't delete ours). Fix: invoke the official one by full path: `~/.groundcover/bin/groundcover auth login` then `~/.groundcover/bin/groundcover auth generate-service-account-token` (tenant admin only, token shown once). This CLI now prints this whole guide when a `raw grafana …` command runs without a token (internal/raw/grafana.go grafanaSetupGuide()). diff --git a/README.md b/README.md index 61f3dad..0d0e62d 100644 --- a/README.md +++ b/README.md @@ -167,6 +167,19 @@ groundcover raw grafana folders list groundcover raw grafana ds query --body-file query.json ``` +Storage management uses one endpoint per data type: + +```sh +groundcover raw storage-management get --data-type logs --raw \ + | jq '{retention,version,cold_move_duration,cold_volume,custom_rules:(.custom_rules // [])}' \ + > storage.json +# Edit storage.json, preserving every writable field and the complete rule list. +groundcover raw storage-management update --data-type logs --body-file storage.json +``` + +Supported data types are `logs`, `traces`, `events`, `measurements`, and `monitor_instance`. Updates use optimistic concurrency and replace the writable settings document: start from `get`, pass its current `version`, and preserve `retention`, `cold_move_duration`, `cold_volume`, and the full `custom_rules` list. Omitting `custom_rules` removes existing rules. + + Raw commands support: - `--body-json ''` diff --git a/internal/raw/command.go b/internal/raw/command.go index 599da86..80fa184 100644 --- a/internal/raw/command.go +++ b/internal/raw/command.go @@ -42,9 +42,10 @@ func Find(tokens []string) (Command, bool) { } func allCommands() []Command { - commands := make([]Command, 0, len(Commands)+len(ExtraCommands)) + commands := make([]Command, 0, len(Commands)+len(ExtraCommands)+len(StorageManagementCommands)) commands = append(commands, Commands...) commands = append(commands, ExtraCommands...) + commands = append(commands, StorageManagementCommands...) byName := map[string]Command{} for _, command := range commands { if _, exists := byName[command.Key()]; exists { diff --git a/internal/raw/runner_test.go b/internal/raw/runner_test.go index ea4a79f..7a42778 100644 --- a/internal/raw/runner_test.go +++ b/internal/raw/runner_test.go @@ -2,8 +2,11 @@ package raw import ( "bytes" + "encoding/json" + "io" "net/http" "net/http/httptest" + "reflect" "strings" "testing" "time" @@ -38,6 +41,111 @@ func TestFind(t *testing.T) { } } +func TestStorageManagementCommands(t *testing.T) { + tests := []struct { + name string + method string + }{ + {name: "get", method: http.MethodGet}, + {name: "update", method: http.MethodPut}, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + command, ok := Find([]string{"storage-management", tt.name}) + if !ok { + t.Fatalf("expected storage-management %s command", tt.name) + } + if command.Method != tt.method { + t.Fatalf("unexpected method %s", command.Method) + } + if command.Path != "/api/storage-management/:dataType" { + t.Fatalf("unexpected path %s", command.Path) + } + if len(command.PathParams) != 1 || command.PathParams[0] != "dataType" { + t.Fatalf("unexpected path params %#v", command.PathParams) + } + + requestURL, err := buildURL(command, config.Config{BaseURL: config.DefaultBaseURL}, Options{ + PathValues: map[string]string{"dataType": "monitor_instance"}, + }) + if err != nil { + t.Fatalf("buildURL failed: %v", err) + } + if got, want := requestURL.String(), "https://api.groundcover.com/api/storage-management/monitor_instance"; got != want { + t.Fatalf("unexpected URL\n got: %s\nwant: %s", got, want) + } + }) + } +} + +func TestRunStorageManagementUpdate(t *testing.T) { + var captured *http.Request + var capturedBody []byte + var readErr error + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + captured = r.Clone(r.Context()) + capturedBody, readErr = io.ReadAll(r.Body) + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(http.StatusOK) + _, _ = w.Write([]byte(`{}`)) + })) + defer srv.Close() + + command, ok := Find([]string{"storage-management", "update"}) + if !ok { + t.Fatal("expected storage-management update command") + } + bodyJSON := `{"retention":"30d","version":3,"cold_move_duration":"7d","cold_volume":"cold","custom_rules":[]}` + cfg := config.Config{ + APIKey: "my-api-key", + BackendID: "my-backend", + TenantUUID: "my-tenant", + BaseURL: srv.URL, + Timeout: time.Second, + } + var out bytes.Buffer + err := Run(command, cfg, Options{ + PathValues: map[string]string{"dataType": "logs"}, + BodyJSON: bodyJSON, + }, &out) + if err != nil { + t.Fatalf("Run returned unexpected error: %v", err) + } + if captured == nil { + t.Fatal("server was never hit") + } + if readErr != nil { + t.Fatalf("reading request body failed: %v", readErr) + } + if got, want := captured.Method, http.MethodPut; got != want { + t.Errorf("method = %q, want %q", got, want) + } + if got, want := captured.URL.Path, "/api/storage-management/logs"; got != want { + t.Errorf("path = %q, want %q", got, want) + } + if got, want := captured.Header.Get("Content-Type"), "application/json"; got != want { + t.Errorf("Content-Type = %q, want %q", got, want) + } + if got, want := captured.Header.Get("Authorization"), "Bearer my-api-key"; got != want { + t.Errorf("Authorization = %q, want %q", got, want) + } + if got, want := captured.Header.Get("X-Tenant-UUID"), "my-tenant"; got != want { + t.Errorf("X-Tenant-UUID = %q, want %q", got, want) + } + + var gotBody, wantBody map[string]any + if err := json.Unmarshal(capturedBody, &gotBody); err != nil { + t.Fatalf("decoding captured body failed: %v", err) + } + if err := json.Unmarshal([]byte(bodyJSON), &wantBody); err != nil { + t.Fatalf("decoding expected body failed: %v", err) + } + if !reflect.DeepEqual(gotBody, wantBody) { + t.Errorf("body = %#v, want %#v", gotBody, wantBody) + } +} + func TestFindGrafanaCommand(t *testing.T) { command, ok := Find([]string{"grafana", "dashboards", "get"}) if !ok { diff --git a/internal/raw/storage_management_commands.go b/internal/raw/storage_management_commands.go new file mode 100644 index 0000000..264e494 --- /dev/null +++ b/internal/raw/storage_management_commands.go @@ -0,0 +1,18 @@ +package raw + +var StorageManagementCommands = []Command{ + { + Name: []string{"storage-management", "get"}, + Method: "GET", + Path: "/api/storage-management/:dataType", + Description: "Get retention and indexing settings for a data type", + PathParams: []string{"dataType"}, + }, + { + Name: []string{"storage-management", "update"}, + Method: "PUT", + Path: "/api/storage-management/:dataType", + Description: "Update retention and indexing settings for a data type", + PathParams: []string{"dataType"}, + }, +} diff --git a/skills/groundcover-cli/SKILL.md b/skills/groundcover-cli/SKILL.md index 58b7331..3a76538 100644 --- a/skills/groundcover-cli/SKILL.md +++ b/skills/groundcover-cli/SKILL.md @@ -83,6 +83,7 @@ Reach for `groundcover raw …` for any of these; the SDK has the *parent* resou - **resources / RUM:** `resources apis errors|filters|latencies|list|requests`, `rum sessions filters|query`, `sources list` - **pipelines stats:** `pipelines logs current-stats`, `pipelines traces current-stats` (SDK only does config CRUD) - **tenant / billing / RBAC reads:** `rbac seatsUsage`, `rbac tenant ai-settings`, `rbac tenant settings`, `backend settings`, `billing method`, `agent token-budgets|token-usage|token-usage history|token-usage tenant` +- **storage management:** `storage-management get|update --data-type ` for retention, gcQL exception rules, and index-tier settings - **misc:** `graph`, `graph filters`, `views member`, `views member defaults`, `migrations`, `connectors list org|personal`, `synthetics rules`, `aggregations metrics config|config default`, `integrations data config` Run `groundcover raw list` / `groundcover raw list ` to confirm the exact name before invoking. @@ -214,6 +215,20 @@ groundcover raw grafana folders list groundcover raw grafana ds query --body-file query.json ``` +### Storage management + +Admins can read and update lifecycle settings for `logs`, `traces`, `events`, `measurements` (APM), and `monitor_instance` (monitor issues): + +```sh +groundcover raw storage-management get --data-type logs --raw \ + | jq '{retention,version,cold_move_duration,cold_volume,custom_rules:(.custom_rules // [])}' \ + > storage.json +# Edit storage.json, preserving every writable field and the complete rule list. +groundcover raw storage-management update --data-type logs --body-file storage.json +``` + +Updates use optimistic concurrency and replace the writable settings document. Always start from `get`, carry forward its current `version`, and preserve `retention`, `cold_move_duration`, `cold_volume`, and the complete `custom_rules` list; omitting `custom_rules` removes existing rules. A successful update increments `version`. Each observed custom rule contains `name`, `retention`, and a gcQL `filters` expression. + ### Grafana native dashboards