Skip to content

Jenkins: retry checkout scm with backoff and debug diagnostics on failure - #5186

Open
causten wants to merge 4 commits into
developfrom
jenkins-checkout-retry
Open

Jenkins: retry checkout scm with backoff and debug diagnostics on failure#5186
causten wants to merge 4 commits into
developfrom
jenkins-checkout-retry

Conversation

@causten

@causten causten commented Aug 24, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Adds a checkoutWithRetry helper that replaces all 5 bare checkout scm calls in the Jenkinsfile
  • Uses the Git plugin's CloneOption(retryFetchCount: 3) for plugin-level fetch retries before Groovy sees the error
  • On Groovy-level failure, retries up to 3 times with exponential backoff (15 s, then 30 s)
  • On final failure, emits debug diagnostics: git remote -v, git config --list, ip route, and a curl probe to github.com

Motivation

PR-5185 build #1 failed with a git fetch error. The failure was transient (network blip during SCM checkout) but caused a full pipeline failure requiring a manual re-trigger. This change makes all checkout points self-healing for transient errors and captures actionable diagnostics when they aren't.

Test plan

  • Trigger a new Jenkins build on this PR and verify the setup stage completes successfully
  • Confirm no behavioral change to the build steps themselves (only the checkout wrapper changed)
  • If a flaky node is available, verify retry log lines appear on a simulated checkout failure

🤖 Generated with Claude Code

…lure

Replaces all bare `checkout scm` calls with a `checkoutWithRetry` helper
that uses the Git plugin's CloneOption retryFetchCount, retries up to 3
times with exponential backoff (15s, 30s), and emits network/git
diagnostics (remote config, routing, GitHub reachability) only on final
failure so transient fetch errors don't require manual re-triggers.

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings August 24, 2026 20:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens Jenkins SCM checkout behavior by introducing a reusable retry wrapper around Git checkouts, aiming to reduce transient-network related pipeline failures and emit actionable diagnostics when checkout ultimately fails.

Changes:

  • Added a checkoutWithRetry helper that performs Jenkins Git checkouts with plugin-level fetch retry plus Groovy-level retries and backoff.
  • Replaced the existing checkout scm usages in the pipeline with checkoutWithRetry() to apply consistent retry/diagnostic behavior.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread Jenkinsfile
userRemoteConfigs: scm.userRemoteConfigs,
])
return
} catch (Exception e) {
Comment thread Jenkinsfile Outdated
Comment on lines +178 to +181
sh 'git remote -v || true'
sh 'git config --list || true'
sh 'ip route || true'
sh 'curl -sv https://github.com 2>&1 | tail -20 || true'
Comment thread Jenkinsfile
Comment on lines +184 to +186
def delay = 15 * attempt
echo "checkout scm failed (attempt ${attempt}/${maxAttempts}): ${e.message}. Retrying in ${delay}s..."
sleep(time: delay, unit: 'SECONDS')
…hRetry

Rethrow FlowInterruptedException and InterruptedException immediately so
pipeline aborts and timeouts are not swallowed by the retry loop.

Filter token/password/auth/header keys from git config --list output to
avoid exposing the Authorization extraheader that the Git plugin writes
into .git/config when using credential helpers.

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

Jenkinsfile:187

  • delay = 15 * attempt is linear if maxAttempts is increased, which doesn’t match the stated exponential-backoff intent for this helper. Use a true exponential formula so increasing maxAttempts doesn’t accidentally reduce resiliency under persistent transient failures.
            def delay = 15 * attempt

Comment thread Jenkinsfile Outdated
Comment on lines +181 to +184
sh 'git remote -v || true'
sh 'git config --list | grep -v -i "token\\|password\\|auth\\|header" || true'
sh 'ip route || true'
sh 'curl -sv https://github.com 2>&1 | tail -20 || true'
…nostic

git remote -v and remote.*.url in git config can expose embedded HTTPS
credentials. Drop remote -v entirely (curl already probes GitHub reachability)
and extend the grep filter to exclude .url keys from git config --list output.

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>
@gh-app-migraphx-bot-pr-write

Copy link
Copy Markdown
Test Batch New Rate (f52549) Old Rate (3a503c)* Diff Status
torchvision-resnet50 64 3,299.53 3,264.92 1.06%
torchvision-resnet50_fp16 64 7,878.02 7,548.67 4.36%
torchvision-densenet121 32 2,491.98 2,483.99 0.32%
torchvision-densenet121_fp16 32 5,016.29 5,004.24 0.24%
torchvision-inceptionv3 32 2,059.98 2,058.51 0.07%
torchvision-inceptionv3_fp16 32 4,448.57 4,416.99 0.71%
cadene-inceptionv4 16 817.02 820.61 -0.44%
cadene-resnext64x4 16 785.45 782.78 0.34%
slim-mobilenet 64 8,376.43 8,386.36 -0.12%
slim-nasnetalarge 64 229.25 228.86 0.17%
slim-resnet50v2 64 3,229.71 3,180.91 1.53%
bert-mrpc-onnx 8 1,170.29 1,168.84 0.12%
bert-mrpc-tf 1 502.93 498.63 0.86%
pytorch-examples-wlang-gru 1 366.62 473.35 -22.55% 🔴
pytorch-examples-wlang-lstm 1 551.24 384.83 43.24% 🔆
torchvision-resnet50_1 1 1,047.98 1,046.63 0.13%
cadene-dpn92_1 1 449.88 437.32 2.87%
cadene-resnext101_1 1 364.14 365.89 -0.48%
onnx-taau-downsample 1 842.73 844.09 -0.16%
dlrm-criteoterabyte 1 32.26 32.42 -0.49%
dlrm-criteoterabyte_fp16 1 51.57 51.80 -0.46%
agentmodel 1 14,642.04 9,209.12 58.99% 🔆
unet_fp16 2 58.33 58.80 -0.80%
resnet50v1_fp16 1 1,452.99 1,366.11 6.36% 🔆
resnet50v1_int8 1 1,809.62 1,883.96 -3.95%
bert_base_cased_fp16 64 1,099.78 1,098.16 0.15%
bert_large_uncased_fp16 32 347.17 345.59 0.46%
bert_large_fp16 1 206.30 206.59 -0.14%
distilgpt2_fp16 16 2,095.25 2,092.89 0.11%
yolov5s 1 565.94 558.33 1.36%
tinyllama 1 45.77 45.83 -0.14%
vicuna-fastchat 1 44.22 44.20 0.05%
whisper-tiny-encoder 1 412.97 411.87 0.27%
whisper-tiny-decoder 1 409.44 408.48 0.23%
llama2_7b 1 20.86 20.84 0.10%
qwen1.5-7b 1 23.66 23.58 0.32%
phi3-3.8b 1 28.34 26.72 6.08% 🔆
llama3-8b 1 22.74 21.80 4.29%
whisper-large-encoder 1 10.17 10.18 -0.10%
whisper-large-decoder 1 105.78 105.30 0.45%
mistral-7b 1 23.77 23.78 -0.05%
FLUX.1-schnell 1 796.67 755.22 5.49% 🔆

Regressions detected 🔴

* No develop baseline was found for this PR's branch point; compared against the latest available develop run instead.

@gh-app-migraphx-bot-pr-write

Copy link
Copy Markdown
Test Status Result
bert-mrpc-onnx PASSED: MIGraphX meets tolerance
bert-mrpc-tf ERROR - check error output
traceback
Traceback (most recent call last):
File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 377, in
main()
File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 313, in main
import tensorflow as tf
File "/usr/local/lib/python3.12/dist-packages/tensorflow/init.py", line 40, in
from tensorflow.python import pywrap_tensorflow as _pywrap_tensorflow # pylint: disable=unused-import
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 37, in
self_check.preload_check()
File "/usr/local/lib/python3.12/dist-packages/tensorflow/python/platform/self_check.py", line 63, in preload_check
from tensorflow.python.platform import _pywrap_cpu_feature_guard
ImportError: libnuma.so.1: cannot open shared object file: No such file or directory
pytorch-examples-wlang-gru 🔴 FAILED: MIGraphX is not within tolerance - check verbose output
pytorch-examples-wlang-lstm 🔴 FAILED: MIGraphX is not within tolerance - check verbose output
dlrm-criteoterabyte PASSED: MIGraphX meets tolerance
agentmodel PASSED: MIGraphX meets tolerance
unet PASSED: MIGraphX meets tolerance
resnet50v1 PASSED: MIGraphX meets tolerance
bert_base_cased_fp16 PASSED: MIGraphX meets tolerance
bert_large_uncased_fp16 🔴 FAILED: MIGraphX is not within tolerance - check verbose output
bert_large PASSED: MIGraphX meets tolerance
yolov5s PASSED: MIGraphX meets tolerance
tinyllama PASSED: MIGraphX meets tolerance
vicuna-fastchat PASSED: MIGraphX meets tolerance
whisper-tiny-encoder PASSED: MIGraphX meets tolerance
whisper-tiny-decoder PASSED: MIGraphX meets tolerance
distilgpt2_fp16 🔴 FAILED: MIGraphX is not within tolerance - check verbose output
llama2_7b PASSED: MIGraphX meets tolerance
qwen1.5-7b PASSED: MIGraphX meets tolerance
phi3-3.8b PASSED: MIGraphX meets tolerance
llama3-8b PASSED: MIGraphX meets tolerance
whisper-large-encoder PASSED: MIGraphX meets tolerance
whisper-large-decoder PASSED: MIGraphX meets tolerance
mistral-7b PASSED: MIGraphX meets tolerance
FLUX.1-schnell PASSED: MIGraphX meets tolerance

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants