diff --git a/README.md b/README.md index 2eac11e..a018b40 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,10 @@ See the [releases page](https://github.com/bb4/bb4-optimization/releases) or [Ma **global simulated annealing** (same multistart pattern with SA locals via `GlobalSimulatedAnnealingConfig`), simulated annealing (`SimulatedAnnealingConfig`: adaptive acceptance by default, budget-safe best-so-far), **genetic / concurrent genetic** (`GeneticSearchConfig`: elitism, cull/crossover, neighbor modes, budget-safe evolve), - **state-space search**, and brute force + **state-space search**, and brute force, + and for constraint problems **min-conflicts** (`MinConflictsConfig`; fixes a conflicted position by its best move, using `Optimizee.conflictedPositions`) + and **tabu search** (`TabuSearchConfig`; takes the best move that is not tabu). `RestartingStrategy` (`RestartingConfig`) re-runs any strategy with Luby budgets. + See [docs/solve-rate-results.md](docs/solve-rate-results.md) for how often each strategy solves the constraint benchmark problems. - **`SimulatedAnnealingConfig` / `SimulatedAnnealingStepMode`** — cooling schedule and proposal radius; default `AdaptiveAcceptance` scales steps from Metropolis acceptance rate on continuous problems (discrete spaces automatically use Fixed radius); proposal radius is clamped to 1.5 - **`GlobalSimulatedAnnealingConfig`** — global sample budget + multistart SA from diverse / top-fitness seeds (prefer over single-chain SA on multimodal discrete problems; diff --git a/docs/new-strategies-plan.md b/docs/new-strategies-plan.md index 7f51574..bcc162d 100644 --- a/docs/new-strategies-plan.md +++ b/docs/new-strategies-plan.md @@ -216,3 +216,17 @@ These have defaults, so the plan can proceed without answers: bb4-puzzles later if one is needed. 3. Is a 20 seed solve rate section acceptable for the length of a full benchmark run (about 10 minutes now)? Default: yes, with the section as its own `--solve-rate` run if it makes the full run too long. + +## Outcome (phases A to F, bb4-optimization) + +Measured with the 20 seed solve rate in [solve-rate-results.md](solve-rate-results.md): + +- **MIN_CONFLICTS** solves all 80 runs of the four problems, including coloring-200, which no earlier strategy solved, + and queens-400 in about 0.35 s against 0.76 s for annealing. It joins the enum. It needs `conflictedPositions` to be at + its best. +- **TABU_SEARCH** solves all-interval-14 20/20 (the best earlier strategy: 10/20) and coloring-200 5/20 (earlier: 0/20), + but only 5/20 on queens-100 where annealing solves everything. It meets the rule on two problems, so it joins the enum, + with the note that it is not the first choice for queens. +- **RestartingStrategy** stays off the enum: no combination was shown to beat the strategies above, and it can be built in + code for any strategy. It is available for the puzzles to try. +- Not yet done: try min-conflicts and tabu on the Red puzzle in bb4-puzzles, and add planted 3-SAT and a Costas array. diff --git a/docs/solve-rate-results.md b/docs/solve-rate-results.md index 70eb3e1..8ecd969 100644 --- a/docs/solve-rate-results.md +++ b/docs/solve-rate-results.md @@ -8,33 +8,41 @@ Java 21.0.12.1, 2 processors, 20 seeds, 2000 ms or 5000000 evaluations per run | Problem | Strategy | Solved | Median ms (solved) | Median evals (solved) | Median broken (unsolved) | | --- | --- | ---: | ---: | ---: | ---: | -| queens-100 | SIMULATED_ANNEALING | 20/20 | 60 | 17,517 | - | -| queens-100 | GLOBAL_SIMULATED_ANNEALING | 20/20 | 62 | 18,800 | - | +| queens-100 | SIMULATED_ANNEALING | 20/20 | 53 | 17,517 | - | +| queens-100 | GLOBAL_SIMULATED_ANNEALING | 20/20 | 61 | 18,800 | - | | queens-100 | HILL_CLIMBING | 0/20 | - | - | 39.0 | | queens-100 | GLOBAL_HILL_CLIMBING | 0/20 | - | - | 36.0 | | queens-100 | GENETIC_SEARCH | 0/20 | - | - | 2.0 | | queens-100 | CONCURRENT_GENETIC_SEARCH | 0/20 | - | - | 2.0 | | queens-100 | ITERATED_LOCAL_SEARCH | 0/20 | - | - | 20.0 | | queens-100 | VARIABLE_NEIGHBORHOOD_SEARCH | 0/20 | - | - | 20.0 | -| queens-400 | SIMULATED_ANNEALING | 20/20 | 818 | 97,017 | - | -| queens-400 | GLOBAL_SIMULATED_ANNEALING | 20/20 | 993 | 94,068 | - | +| queens-100 | TABU_SEARCH | 5/20 | 349 | 104,610 | 1.0 | +| queens-100 | MIN_CONFLICTS | 20/20 | 8 | 3,178 | - | +| queens-400 | SIMULATED_ANNEALING | 20/20 | 761 | 97,017 | - | +| queens-400 | GLOBAL_SIMULATED_ANNEALING | 20/20 | 894 | 94,068 | - | | queens-400 | HILL_CLIMBING | 0/20 | - | - | 206.0 | | queens-400 | GLOBAL_HILL_CLIMBING | 0/20 | - | - | 199.0 | -| queens-400 | GENETIC_SEARCH | 0/20 | - | - | 155.5 | -| queens-400 | CONCURRENT_GENETIC_SEARCH | 0/20 | - | - | 155.0 | +| queens-400 | GENETIC_SEARCH | 0/20 | - | - | 154.5 | +| queens-400 | CONCURRENT_GENETIC_SEARCH | 0/20 | - | - | 154.0 | | queens-400 | ITERATED_LOCAL_SEARCH | 0/20 | - | - | 155.0 | | queens-400 | VARIABLE_NEIGHBORHOOD_SEARCH | 0/20 | - | - | 153.0 | +| queens-400 | TABU_SEARCH | 0/20 | - | - | 12.0 | +| queens-400 | MIN_CONFLICTS | 20/20 | 348 | 40,933 | - | | coloring-200 | SIMULATED_ANNEALING | 0/20 | - | - | 141.0 | | coloring-200 | HILL_CLIMBING | 0/20 | - | - | 107.5 | -| coloring-200 | GENETIC_SEARCH | 0/20 | - | - | 116.0 | -| coloring-200 | CONCURRENT_GENETIC_SEARCH | 0/20 | - | - | 115.0 | +| coloring-200 | GENETIC_SEARCH | 0/20 | - | - | 106.5 | +| coloring-200 | CONCURRENT_GENETIC_SEARCH | 0/20 | - | - | 105.5 | | coloring-200 | ITERATED_LOCAL_SEARCH | 0/20 | - | - | 131.0 | | coloring-200 | VARIABLE_NEIGHBORHOOD_SEARCH | 0/20 | - | - | 132.0 | +| coloring-200 | TABU_SEARCH | 5/20 | 26 | 50,278 | 2.0 | +| coloring-200 | MIN_CONFLICTS | 20/20 | 58 | 9,214 | - | | all-interval-14 | SIMULATED_ANNEALING | 1/20 | 2 | 1,771 | 1.0 | -| all-interval-14 | GLOBAL_SIMULATED_ANNEALING | 1/20 | 9 | 9,764 | 1.0 | +| all-interval-14 | GLOBAL_SIMULATED_ANNEALING | 1/20 | 16 | 9,764 | 1.0 | | all-interval-14 | HILL_CLIMBING | 0/20 | - | - | 2.0 | | all-interval-14 | GLOBAL_HILL_CLIMBING | 0/20 | - | - | 1.0 | -| all-interval-14 | GENETIC_SEARCH | 10/20 | 205 | 58,450 | 1.0 | -| all-interval-14 | CONCURRENT_GENETIC_SEARCH | 10/20 | 219 | 58,451 | 1.0 | +| all-interval-14 | GENETIC_SEARCH | 10/20 | 252 | 58,450 | 1.0 | +| all-interval-14 | CONCURRENT_GENETIC_SEARCH | 10/20 | 240 | 58,451 | 1.0 | | all-interval-14 | ITERATED_LOCAL_SEARCH | 5/20 | 3 | 2,436 | 1.0 | -| all-interval-14 | VARIABLE_NEIGHBORHOOD_SEARCH | 4/20 | 17 | 9,609 | 1.0 | +| all-interval-14 | VARIABLE_NEIGHBORHOOD_SEARCH | 4/20 | 13 | 9,609 | 1.0 | +| all-interval-14 | TABU_SEARCH | 20/20 | 46 | 38,561 | - | +| all-interval-14 | MIN_CONFLICTS | 20/20 | 9 | 7,762 | - | diff --git a/scala-source/com/barrybecker4/optimization/strategy/OptimizationStrategyType.scala b/scala-source/com/barrybecker4/optimization/strategy/OptimizationStrategyType.scala index 44584a3..be62375 100644 --- a/scala-source/com/barrybecker4/optimization/strategy/OptimizationStrategyType.scala +++ b/scala-source/com/barrybecker4/optimization/strategy/OptimizationStrategyType.scala @@ -7,6 +7,8 @@ import com.barrybecker4.optimization.strategy.annealing.{GlobalSimulatedAnnealin import com.barrybecker4.optimization.strategy.genetic.{ConcurrentGeneticSearchStrategy, GeneticSearchStrategy} import com.barrybecker4.optimization.strategy.hillclimbing.{GlobalHillClimbingStrategy, HillClimbingStrategy} import com.barrybecker4.optimization.strategy.iteratedlocal.{IteratedLocalSearchConfig, IteratedLocalSearchStrategy} +import com.barrybecker4.optimization.strategy.minconflicts.{MinConflictsConfig, MinConflictsStrategy} +import com.barrybecker4.optimization.strategy.tabu.{TabuSearchConfig, TabuSearchStrategy} import com.barrybecker4.optimization.strategy.particleswarm.ParticleSwarmStrategy import com.barrybecker4.optimization.strategy.statespace.{StateSpaceSearchConfig, StateSpaceSearchStrategy} @@ -33,7 +35,9 @@ enum OptimizationStrategyType { DIFFERENTIAL_EVOLUTION, PARTICLE_SWARM, ITERATED_LOCAL_SEARCH, - VARIABLE_NEIGHBORHOOD_SEARCH + VARIABLE_NEIGHBORHOOD_SEARCH, + TABU_SEARCH, + MIN_CONFLICTS def getDescription: String = this match case GLOBAL_SAMPLING => @@ -64,6 +68,11 @@ enum OptimizationStrategyType { "Iterated local search (ILS) for discrete problems such as permutations." case VARIABLE_NEIGHBORHOOD_SEARCH => "Variable neighborhood search (VNS) for discrete problems such as permutations." + case TABU_SEARCH => + "Tabu search for discrete problems: takes the best allowed move even when it is worse, and forbids undoing recent moves." + case MIN_CONFLICTS => + "Min-conflicts for constraint problems: repeatedly fixes a conflicted position by its best move. " + + "Needs an optimizee that reports conflictedPositions to work well." /** * True when this strategy can run on `optimizee` with `params`. @@ -81,6 +90,10 @@ enum OptimizationStrategyType { !optimizee.evaluateByComparison && params.isInstanceOf[NumericParameterArray] case HILL_CLIMBING | GLOBAL_HILL_CLIMBING => locallySearchable(params) + case TABU_SEARCH => + !optimizee.evaluateByComparison + case MIN_CONFLICTS => + !optimizee.evaluateByComparison && params.supportsLocalMoves case _ => true @@ -123,6 +136,14 @@ enum OptimizationStrategyType { IteratedLocalSearchStrategy.strategyFor(optimizee, IteratedLocalSearchConfig.IteratedLocalSearch, rnd) case VARIABLE_NEIGHBORHOOD_SEARCH => IteratedLocalSearchStrategy.strategyFor(optimizee, IteratedLocalSearchConfig.VariableNeighborhoodSearch, rnd) + case TABU_SEARCH => + val strategy = new TabuSearchStrategy(optimizee, rnd) + strategy.setConfig(TabuSearchConfig.Default) + strategy + case MIN_CONFLICTS => + val strategy = new MinConflictsStrategy(optimizee, rnd) + strategy.setConfig(MinConflictsConfig.Default) + strategy private def hasContinuousParameter(params: ParameterArray): Boolean = params match @@ -153,7 +174,9 @@ object OptimizationStrategyType { DIFFERENTIAL_EVOLUTION, PARTICLE_SWARM, ITERATED_LOCAL_SEARCH, - VARIABLE_NEIGHBORHOOD_SEARCH + VARIABLE_NEIGHBORHOOD_SEARCH, + TABU_SEARCH, + MIN_CONFLICTS ) /** @deprecated use [[valuesForUi]] */ diff --git a/scala-test/com/barrybecker4/optimization/benchmark/BenchmarkConfig.scala b/scala-test/com/barrybecker4/optimization/benchmark/BenchmarkConfig.scala index 93624e2..817202d 100644 --- a/scala-test/com/barrybecker4/optimization/benchmark/BenchmarkConfig.scala +++ b/scala-test/com/barrybecker4/optimization/benchmark/BenchmarkConfig.scala @@ -23,6 +23,9 @@ object BenchmarkConfig { SIMULATED_ANNEALING, GLOBAL_SIMULATED_ANNEALING, HILL_CLIMBING, GLOBAL_HILL_CLIMBING, GENETIC_SEARCH, CONCURRENT_GENETIC_SEARCH, ITERATED_LOCAL_SEARCH, VARIABLE_NEIGHBORHOOD_SEARCH) + /** The solve rate run adds the strategies made for constraint problems. */ + val CONSTRAINT_STRATEGIES: Seq[OptimizationStrategyType] = DEFAULT_STRATEGIES ++ Seq(TABU_SEARCH, MIN_CONFLICTS) + /** * Reads options of the form `--name=value`. The names are time-ms, max-evals, seeds, problems, strategies * (the last two are comma separated lists), report (the Markdown summary file, or `none`) and out (a file to @@ -41,8 +44,8 @@ object BenchmarkConfig { val solveRate = options.get("solve-rate").contains("true") val problemNames = options.get("problems").map(_.split(",").toSeq).getOrElse(Seq.empty) val strategies = options.get("strategies").map(_.split(",").toSeq.map(OptimizationStrategyType.valueOf)) - .getOrElse(DEFAULT_STRATEGIES) - val isFullRun = problemNames.isEmpty && strategies == DEFAULT_STRATEGIES + .getOrElse(if (solveRate) CONSTRAINT_STRATEGIES else DEFAULT_STRATEGIES) + val isFullRun = problemNames.isEmpty && strategies == (if (solveRate) CONSTRAINT_STRATEGIES else DEFAULT_STRATEGIES) BenchmarkConfig( timeLimitMillis = options.get("time-ms").map(_.toLong).getOrElse(defaultTimeLimit(solveRate)), maxEvaluations = options.get("max-evals").map(_.toLong).getOrElse(DEFAULT_MAX_EVALUATIONS), diff --git a/scala-test/com/barrybecker4/optimization/benchmark/BenchmarkConfigSuite.scala b/scala-test/com/barrybecker4/optimization/benchmark/BenchmarkConfigSuite.scala index 1c0283d..1515aa1 100644 --- a/scala-test/com/barrybecker4/optimization/benchmark/BenchmarkConfigSuite.scala +++ b/scala-test/com/barrybecker4/optimization/benchmark/BenchmarkConfigSuite.scala @@ -70,4 +70,11 @@ class BenchmarkConfigSuite extends AnyFunSuite { test("an ordinary run is not a solve rate run") { assert(!BenchmarkConfig.parse(Seq.empty).solveRate) } + + test("the solve rate run includes the strategies for constraint problems, and a full one writes its report") { + val config = BenchmarkConfig.parse(Seq("--solve-rate=true")) + assertResult(BenchmarkConfig.CONSTRAINT_STRATEGIES)(config.strategies) + assert(config.strategies.containsSlice(Seq(TABU_SEARCH, MIN_CONFLICTS))) + assertResult(Some(BenchmarkConfig.DEFAULT_SOLVE_RATE_REPORT_FILE))(config.reportFile) + } } diff --git a/scala-test/com/barrybecker4/optimization/strategy/OptimizationStrategyTypeSuite.scala b/scala-test/com/barrybecker4/optimization/strategy/OptimizationStrategyTypeSuite.scala index c7f6d70..b120d3a 100644 --- a/scala-test/com/barrybecker4/optimization/strategy/OptimizationStrategyTypeSuite.scala +++ b/scala-test/com/barrybecker4/optimization/strategy/OptimizationStrategyTypeSuite.scala @@ -83,7 +83,8 @@ class OptimizationStrategyTypeSuite extends AnyFunSuite { "DIFFERENTIAL_EVOLUTION", "PARTICLE_SWARM", "ITERATED_LOCAL_SEARCH", - "VARIABLE_NEIGHBORHOOD_SEARCH" + "VARIABLE_NEIGHBORHOOD_SEARCH", + "TABU_SEARCH" )) }