Skip to content

[Bug] gcp-cloud-sql-performance: analyze_performance.sh aborts on instances with no IOPS samples; IOPS and noise detection unimplemented #769

Open

Description

@Rohit-Ekbote

codebundles/gcp-cloud-sql-performance/analyze_performance.sh backs the Identify Cloud SQL Performance Issues task. Reviewing it surfaced one hard failure and three correctness/completeness problems, plus a configuration gap that makes its only detector untunable from the platform.

1. Task crashes on any instance with no disk IOPS samples

analyze_performance.sh:96-97

iops_avg=$(jq -n --argjson r "$read_ops" --argjson w "$write_ops" '
  (($r | add) + ($w | add)) / (if (($r | length) + ($w | length)) == 0 then 1 else (($r | length) + ($w | length)) end)')

The zero-length guard protects the divisor but not the numerator. On an empty series [] | add returns null, so the expression becomes null / 1, which is a jq error:

$ jq -n --argjson r '[]' --argjson w '[]' '(($r | add) + ($w | add)) / (if (($r | length) + ($w | length)) == 0 then 1 else (($r | length) + ($w | length)) end)'
jq: error (at <unknown>): null (null) and number (1) cannot be divided
exit=5

The script runs under set -euo pipefail, and the loop body executes in the current shell (process substitution, not a pipe), so this aborts the entire task — not just the offending instance. Every instance after it in sql_config.json goes unanalyzed.

This is reachable in normal operation: a newly created instance, an instance with no disk activity in the lookback window, or a transient Cloud Monitoring response returning no points for disk/read_ops_count.

Suggested fix: (($r | add) // 0) and (($w | add) // 0), or skip the instance when both series are empty.

2. iops_avg and iops_max are computed on inconsistent bases

analyze_performance.sh:96-98

  • iops_avg divides the summed read+write values by the combined sample count. With equal-length series this yields (mean_read + mean_write) / 2 — half the true combined per-interval IOPS.
  • iops_max is max(read) + max(write) — a genuine sum.

The two values land side by side in performance_report.json and cannot be meaningfully compared. iops_avg understates combined IOPS by roughly 2x.

3. NOISE_CV is declared but never used

analyze_performance.sh:25

: "${NOISE_CV:=2.0}"

NOISE_CV appears exactly once in the entire codebundle. No coefficient-of-variation calculation exists.

The README advertises the capability:

Performance: Flags instances showing throughput spikes or noisy traffic patterns via Cloud Monitoring network and disk IOPS metrics

No noise detection is implemented.

4. IOPS is collected and reported but never evaluated

iops_avg / iops_max are written into performance_report.json and compared against nothing. The sole issue-raising condition in the script is the throughput spike ratio at analyze_performance.sh:115. No IOPS value can produce an issue or affect the SLI, though the README lists IOPS as a detection dimension.

5. SPIKE_FACTOR is not exposed, and its default misfires on batch workloads

SPIKE_FACTOR (default 5) gates the only detector in the script:

if [ "$(awk -v r="$ratio" -v f="$SPIKE_FACTOR" 'BEGIN{print (r>f)?1:0}')" = "1" ]; then

It is absent from both:

  • the README's variable list, and
  • configProvided in .runwhen/templates/gcp-cloud-sql-performance-slx.yaml

so it is effectively hardcoded to 5 for any platform-generated SLX.

The detector compares throughput_max / throughput_avg over a 6h window at 300s alignment. For scheduled batch or ETL workloads, a job starting is a >5x throughput spike by design, so this fires on every batch window with no way to tune it from the platform. Exposing SPIKE_FACTOR in the README and the SLX template would make the task usable against batch-heavy Cloud SQL estates.

Suggested scope

Items 1 and 2 are correctness bugs worth fixing directly. Items 3 and 4 are a choice between implementing the advertised detection or correcting the README so it matches the code. Item 5 is a small template/README change.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions