Skip to content

feat(cli): add global concurrency control - #356

Merged
Nan Liu (liunan-ms) merged 1 commit into
microsoft:mainfrom
ddstreet:add-concurrency-config
Sep 18, 2026
Merged

Nan Liu (liunan-ms) merged 1 commit into
microsoft:mainfrom
ddstreet:add-concurrency-config

Conversation

@ddstreet

Copy link
Copy Markdown
Contributor

Add --concurrency as the shared base limit for parallel operations. The value defaults to the number of logical CPUs, with CPU-bound, I/O-bound, and fast worker pools using 1x, 2x, and 4x that value respectively.

Handle automatic, explicit, and clamped values consistently, add test coverage, and regenerate the CLI reference documentation.

Copilot AI lite review requested due to automatic review settings September 17, 2026 19:49
Dan Streetman (ddstreet) pushed a commit to ddstreet/azure-linux-dev-tools that referenced this pull request Sep 17, 2026
Add --concurrency as the shared base limit for parallel operations. The
value defaults to the number of logical CPUs, with CPU-bound, I/O-bound,
and fast worker pools using 1x, 2x, and 4x that value respectively.

Handle automatic, explicit, and clamped values consistently, add test
coverage, and regenerate the CLI reference documentation.

Refs: microsoft#356

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Derived concurrency limits can overflow for very large positive --concurrency values.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds a global --concurrency flag with scaled CPU, I/O, and fast-worker limits.

Changes:

  • Adds defaulting, clamping, and CLI propagation.
  • Adds concurrency and CLI tests.
  • Regenerates CLI reference documentation.
File summaries
File Summary
internal/app/azldev/env.go Adds concurrency configuration and derived limits; unchecked multiplication can overflow for very large values.
internal/app/azldev/env_test.go Tests concurrency behavior.
internal/app/azldev/app.go Registers and applies the global flag.
internal/app/azldev/app_test.go Tests CLI flag handling.
docs/user/reference/cli/azldev.md Documents the global flag.
docs/user/reference/cli/azldev_version.md Documents the global flag.
docs/user/reference/cli/azldev_repo.md Documents the global flag.
docs/user/reference/cli/azldev_repo_query.md Documents the global flag.
docs/user/reference/cli/azldev_repo_compare.md Documents the global flag.
docs/user/reference/cli/azldev_project.md Documents the global flag.
docs/user/reference/cli/azldev_project_new.md Documents the global flag.
docs/user/reference/cli/azldev_project_init.md Documents the global flag.
docs/user/reference/cli/azldev_package.md Documents the global flag.
docs/user/reference/cli/azldev_package_list.md Documents the global flag.
docs/user/reference/cli/azldev_image.md Documents the global flag.
docs/user/reference/cli/azldev_image_test.md Documents the global flag.
docs/user/reference/cli/azldev_image_list.md Documents the global flag.
docs/user/reference/cli/azldev_image_inject-files.md Documents the global flag.
docs/user/reference/cli/azldev_image_customize.md Documents the global flag.
docs/user/reference/cli/azldev_image_build.md Documents the global flag.
docs/user/reference/cli/azldev_image_boot.md Documents the global flag.
docs/user/reference/cli/azldev_docs.md Documents the global flag.
docs/user/reference/cli/azldev_docs_markdown.md Documents the global flag.
docs/user/reference/cli/azldev_docs_agent.md Documents the global flag.
docs/user/reference/cli/azldev_docs_agent_show.md Documents the global flag.
docs/user/reference/cli/azldev_docs_agent_install.md Documents the global flag.
docs/user/reference/cli/azldev_config.md Documents the global flag.
docs/user/reference/cli/azldev_config_generate-schema.md Documents the global flag.
docs/user/reference/cli/azldev_config_dump.md Documents the global flag.
docs/user/reference/cli/azldev_component.md Documents the global flag.
docs/user/reference/cli/azldev_component_update.md Documents the global flag.
docs/user/reference/cli/azldev_component_test.md Documents the global flag.
docs/user/reference/cli/azldev_component_render.md Documents the global flag.
docs/user/reference/cli/azldev_component_query.md Documents the global flag.
docs/user/reference/cli/azldev_component_prepare-sources.md Documents the global flag.
docs/user/reference/cli/azldev_component_list.md Documents the global flag.
docs/user/reference/cli/azldev_component_history.md Documents the global flag.
docs/user/reference/cli/azldev_component_diff-sources.md Documents the global flag.
docs/user/reference/cli/azldev_component_changed.md Documents the global flag.
docs/user/reference/cli/azldev_component_build.md Documents the global flag.
docs/user/reference/cli/azldev_component_add.md Documents the global flag.
docs/user/reference/cli/azldev_completion.md Documents the global flag.
docs/user/reference/cli/azldev_completion_zsh.md Documents the global flag.
docs/user/reference/cli/azldev_completion_powershell.md Documents the global flag.
docs/user/reference/cli/azldev_completion_fish.md Documents the global flag.
docs/user/reference/cli/azldev_completion_bash.md Documents the global flag.
docs/user/reference/cli/azldev_advanced.md Documents the global flag.
docs/user/reference/cli/azldev_advanced_wget.md Documents the global flag.
docs/user/reference/cli/azldev_advanced_mock.md Documents the global flag.
docs/user/reference/cli/azldev_advanced_mock_shell.md Documents the global flag.
docs/user/reference/cli/azldev_advanced_mock_build-rpms.md Documents the global flag.
docs/user/reference/cli/azldev_advanced_mcp.md Documents the global flag.
docs/user/reference/cli/azldev_advanced_download-sources.md Documents the global flag.
Review details
  • Files reviewed: 53/53 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

// (network clones, file copies). Returns 2× the configured base concurrency.
func (env *Env) IOBoundConcurrency() int {
return max(1, 2*runtime.NumCPU()) //nolint:mnd // 2x CPU
return 2 * env.concurrency //nolint:mnd // 2x base concurrency
Copilot AI review requested due to automatic review settings September 17, 2026 19:52
@ddstreet

Copy link
Copy Markdown
Contributor Author

note that this is needed for many operations that involve remote actions (e.g. refresh-upstream-commits) where the system running azldev has a significant number of cpus. For example, on my 144-core system, when refreshing upstream commits for all components, azldev spawns 288 parallel threads hammering upstream fedora dist-git repos, which mostly all fail obviously due to upstream usage throttling.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Large concurrency values can overflow derived limits; add safe bounds or overflow-safe derivation and boundary tests.

Get a fresh assessment by requesting another Copilot review.

Review details
  • Files reviewed: 53/53 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment on lines +253 to +258
func (env *Env) SetConcurrency(concurrency int) {
if concurrency == 0 {
concurrency = defaultConcurrency()
} else if concurrency < 0 {
concurrency = 1
}
@trungams

Copy link
Copy Markdown
Member

The change LGTM, but looks like some test snapshots need to be updated

Add --concurrency as the shared base limit for parallel operations. The
value defaults to the number of logical CPUs, with CPU-bound, I/O-bound,
and fast worker pools using 1x, 2x, and 4x that value respectively.

Handle automatic, explicit, and clamped values consistently, add test
coverage, and regenerate the CLI reference documentation.

Refs: microsoft#356

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Derived concurrency limits can overflow for large base values and need safe bounds or saturation.

Review details

Suppressed comments (1)

internal/app/azldev/env.go:260

  • --concurrency accepts any int, but IOBoundConcurrency and FastConcurrency multiply this value without overflow protection. For example, --concurrency set to math.MaxInt produces negative worker limits (2*base/4*base), which parmap silently turns into one worker and can pass invalid values to the Python batch workers. Please reject/clamp the base to a safe upper bound (at least MaxInt/4) or saturate the derived limits, and cover the boundary in tests.
  • Files reviewed: 61/61 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

@liunan-ms Nan Liu (liunan-ms) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@liunan-ms
Nan Liu (liunan-ms) merged commit e832cb0 into microsoft:main Sep 18, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants