Skip to content

Fix/perf push timeout - #951

Merged
ybezsonov merged 3 commits into
mainfrom
fix/perf-push-timeout
Oct 12, 2026
Merged

ybezsonov merged 3 commits into
mainfrom
fix/perf-push-timeout

Conversation

@ybezsonov

Copy link
Copy Markdown
Contributor

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

A ws-test run lost 20 min in one docker push: the checkpoint finished at 09:38:51, ECR
registered the image at 09:59:07, docker printed nothing in between. The same two-layer
push takes seconds. Each attempt now has 180 s and is retried up to 3 times; layers already
uploaded are not sent again.
Two platform nodes remained: Karpenter started a c6a.large per batch of pending pods, and the
platform PVCs (Prometheus, Pyroscope, Grafana) had bound in two zones before placement, so no
single node could hold them.

- platform-placement.sh creates an own 'platform' NodePool (>= 4 vCPU and 8 GiB, c6a.xlarge
  cheapest, WhenEmpty); 'pool' creates only the pool.
- The template creates the pool in stage B and exports MONITORING_NODEPOOL=platform before
  monitoring.sh, which then installs Prometheus server, Pyroscope and Grafana with that node
  selector, so their PVCs bind in the platform node's zone. Opt-in: other workshops do not set
  the variable and install as before (helm template: no selector, same arguments).
@ybezsonov
ybezsonov merged commit ba1a90a into main Oct 12, 2026
48 checks passed
@ybezsonov
ybezsonov deleted the fix/perf-push-timeout branch October 12, 2026 15:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant