fix(api): bound OptimizationJob random search seed to the sampler range by rshdhere · Pull Request #4248 · kubeflow/trainer

rshdhere · 2026-10-15T18:48:01Z

Add Minimum=0 and Maximum=4294967295 to RandomAlgorithm.Seed in optimizationjob_types.go
Out-of-range seeds were admitted and forwarded as random_state to the Optuna suggestion service, whose RandomSampler raises "Seed must be between 0 and 2**32 - 1"
The service never caches the failed sampler, so every reconcile retried GetSuggestions and the OptimizationJob stalled with no trials and no Failed condition
Regenerate the CRD manifests, Helm chart CRD, OpenAPI spec, apply configuration and Python API model
Add an integration test that checks the API server accepts nil, 0 and 4294967295 and rejects -1 and 4294967296

What this PR does / why we need it:
spec.searchAlgorithm.random.seed accepted any int64, but the Optuna random sampler in the suggestion service only accepts seeds in [0, 2^32-1]. An out-of-range seed made every GetSuggestions call fail, so the OptimizationJob stalled with no trials and no Failed condition. This adds Minimum/Maximum validation to the seed so the API server rejects it at creation, regenerates the API artifacts, and adds an integration test for the boundary values.

Which issue(s) this PR fixes:
Fixes #4247

Checklist:

Docs included if any changes are user facing

- Add Minimum=0 and Maximum=4294967295 to RandomAlgorithm.Seed in optimizationjob_types.go - Out-of-range seeds were admitted and forwarded as random_state to the Optuna suggestion service, whose RandomSampler raises "Seed must be between 0 and 2**32 - 1" - The service never caches the failed sampler, so every reconcile retried GetSuggestions and the OptimizationJob stalled with no trials and no Failed condition - Regenerate the CRD manifests, Helm chart CRD, OpenAPI spec, apply configuration and Python API model - Add an integration test that checks the API server accepts nil, 0 and 4294967295 and rejects -1 and 4294967296 Signed-off-by: raashed <mohammedraashed10@gmail.com>

akshaychitneni · 2026-10-20T14:23:07Z

 // RandomAlgorithm is the random search algorithm.
 type RandomAlgorithm struct {
 	// seed is the seed for the random search algorithm.
+	// It must be between 0 and 4294967295, the range accepted by the Optuna random sampler.


could we not mention about algorithm being used in the api spec

Good point.

I've removed the sampler mention so it now just states the accepted range, and regenerated the manifests/OpenAPI/Python model. Thanks!

akshaychitneni · 2026-10-20T14:23:54Z

@@ -0,0 +1,92 @@
+/*


please rename the fine to optimizationjob_test.go so it can be reused for other webhook tests

Makes sense, renamed it to optimizationjob_test.go. Just a heads-up: #4062 also adds a file with that name, so whichever lands second will need a quick rebase to merge the two. Happy to handle that on my side if this one goes in later.

- Remove the mention of the underlying random sampler from the RandomAlgorithm.Seed API doc, keeping only the accepted range - Regenerate the CRD manifests, Helm chart CRD, OpenAPI spec, apply configuration and Python API model - Rename optimizationjob_seed_test.go to optimizationjob_test.go so other OptimizationJob webhook tests can live in the same file Signed-off-by: raashed <mohammedraashed10@gmail.com>

andreyvelich · 2026-10-20T23:32:25Z

/ok-to-test

andreyvelich

Thanks @rshdhere for the fix! This review was done with the help of AI tools.

andreyvelich · 2026-10-20T23:50:06Z

+	"github.com/kubeflow/trainer/v2/test/integration/framework"
+)
+
+var _ = ginkgo.Describe("OptimizationJob random search seed validation", ginkgo.Ordered, func() {


Since this file will host other OptimizationJob webhook tests, use a generic suite name matching trainjob_test.go:

Suggested change

var _ = ginkgo.Describe("OptimizationJob random search seed validation", ginkgo.Ordered, func() {

var _ = ginkgo.Describe("OptimizationJob Webhook", ginkgo.Ordered, func() {

Done in ace3938. Renamed it to OptimizationJob webhook to match trainjob_test.go.

andreyvelich · 2026-10-20T23:50:06Z

+	ginkgo.BeforeEach(func() {
+		ns = &corev1.Namespace{
+			ObjectMeta: metav1.ObjectMeta{
+				GenerateName: "optimizationjob-seed-",


Same here, align with trainjob-webhook-:

Suggested change

GenerateName: "optimizationjob-seed-",

GenerateName: "optimizationjob-webhook-",

Done in ace3938 as well. The prefix is now optimizationjob-webhook-, matching trainjob-webhook-.

…tionJob webhook tests - Rename the Describe block to "OptimizationJob Webhook" to match trainjob_test.go, since the file will host other OptimizationJob webhook tests - Use the optimizationjob-webhook- namespace prefix to align with trainjob-webhook- Signed-off-by: raashed <mohammedraashed10@gmail.com>

rshdhere · 2026-10-21T00:07:57Z

Thanks for the review @andreyvelich!
I've applied both suggestions in ace3938. Let me know if anything else needs changing.

andreyvelich · 2026-10-21T00:29:04Z

+	ginkgo.DescribeTable("validates the random search seed at admission",
+		func(seed *int64, errorMatcher gomega.OmegaMatcher) {


Can you accept OptimizationJob as input, like we do for TrainJob validation tests: https://github.com/rshdhere/trainer/blob/ace3938e61767a58c207a483a5e7e20c63887347/test/integration/webhooks/trainjob_test.go#L91
So we can add the rest of the tests in the followup PRs.

Suggested change

ginkgo.DescribeTable("validates the random search seed at admission",

func(seed *int64, errorMatcher gomega.OmegaMatcher) {

ginkgo.DescribeTable("Validate OptimizationJob on creation", func(optimizationJob func() *trainer.OptimizationJob, errorMatcher gomega.OmegaMatcher) {

gomega.Expect(k8sClient.Create(ctx, optimizationJob())).Should(errorMatcher)

You can update the MakeOptimizationJobWrapper to set default values for Objectives, SearchAlgorithm, Parameters, and TrainJobTemplate

…ests - Default Objectives, SearchAlgorithm, Parameters and TrainJobTemplate in MakeOptimizationJobWrapper to a minimal valid random search, so tests only override the fields they exercise - Make the OptimizationJob webhook table take a func() *trainer.OptimizationJob, matching the TrainJob validation tests, so future entries can vary any field Signed-off-by: raashed <mohammedraashed10@gmail.com>

rshdhere · 2026-10-21T00:39:21Z

done in cb96c3f. MakeOptimizationJobWrapper now defaults Objectives, SearchAlgorithm, Parameters and TrainJobTemplate to a minimal valid random search. The table now takes func() *trainer.OptimizationJob, the same as the TrainJob validation tests, so each entry only overrides the field it tests.

andreyvelich

Thanks @rshdhere!
/lgtm
/approve

google-oss-prow · 2026-10-21T00:50:34Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andreyvelich

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [andreyvelich]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

github-project-automation Bot added this to Kubeflow Trainer Oct 15, 2026

github-project-automation Bot moved this to Needs Triage in Kubeflow Trainer Oct 15, 2026

google-oss-prow Bot requested review from jinchihe and robert-bell October 15, 2026 18:48

google-oss-prow Bot added the size/M label Oct 15, 2026

akshaychitneni reviewed Oct 20, 2026

View reviewed changes

google-oss-prow Bot added size/L and removed size/M labels Oct 20, 2026

rshdhere requested a review from akshaychitneni October 20, 2026 17:50

google-oss-prow Bot added the ok-to-test label Oct 20, 2026

andreyvelich reviewed Oct 20, 2026

View reviewed changes

andreyvelich reviewed Oct 21, 2026

View reviewed changes

google-oss-prow Bot assigned andreyvelich Oct 21, 2026

google-oss-prow Bot added the lgtm label Oct 21, 2026

google-oss-prow Bot added the approved label Oct 21, 2026

rshdhere mentioned this pull request Oct 21, 2026

fix(api): reject OptimizationJob trainer env that trial TrainJobs cannot accept #4286

Open

1 task

google-oss-prow Bot merged commit 3e2e15d into kubeflow:master Oct 21, 2026
30 checks passed

google-oss-prow Bot added this to the v2.4 milestone Oct 21, 2026

github-project-automation Bot moved this from Needs Triage to Done in Kubeflow Trainer Oct 21, 2026

	var _ = ginkgo.Describe("OptimizationJob random search seed validation", ginkgo.Ordered, func() {
	var _ = ginkgo.Describe("OptimizationJob Webhook", ginkgo.Ordered, func() {

	GenerateName: "optimizationjob-seed-",
	GenerateName: "optimizationjob-webhook-",

		ginkgo.DescribeTable("validates the random search seed at admission",
		func(seed *int64, errorMatcher gomega.OmegaMatcher) {

Conversation

rshdhere commented Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andreyvelich commented Oct 20, 2026

Uh oh!

andreyvelich left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rshdhere commented Oct 21, 2026

Uh oh!

andreyvelich Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rshdhere commented Oct 21, 2026

Uh oh!

andreyvelich left a comment

Choose a reason for hiding this comment

Uh oh!

google-oss-prow Bot commented Oct 21, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rshdhere commented Oct 15, 2026 •

edited

Loading

andreyvelich Oct 21, 2026 •

edited

Loading