can't run examples

[bwheelz36]

With both the .py example and the note book example, I get the below error.
I'm using python 3.8, scipy version '1.5.2'

Process Process-4:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/home/brendan/Documents/temp/Bayesian-optimization-using-Gaussian-Process/modules/parallelstuff.py", line 35, in mworker
res = minimize(f, x0, args = fargs, **margs)
Process Process-5:
File "/home/brendan/python/PhaserVenv/lib/python3.8/site-packages/scipy/optimize/_minimize.py", line 617, in minimize
return _minimize_lbfgsb(fun, x0, args, jac, bounds,
File "/home/brendan/python/PhaserVenv/lib/python3.8/site-packages/scipy/optimize/lbfgsb.py", line 306, in _minimize_lbfgsb
sf = _prepare_scalar_function(fun, x0, jac=jac, args=args, epsilon=eps,
File "/home/brendan/python/PhaserVenv/lib/python3.8/site-packages/scipy/optimize/optimize.py", line 261, in _prepare_scalar_function
sf = ScalarFunction(fun, x0, args, grad, hess,
Traceback (most recent call last):
File "/home/brendan/python/PhaserVenv/lib/python3.8/site-packages/scipy/optimize/_differentiable_functions.py", line 95, in init
self._update_grad()
File "/home/brendan/python/PhaserVenv/lib/python3.8/site-packages/scipy/optimize/_differentiable_functions.py", line 171, in _update_grad
self._update_grad_impl()
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/home/brendan/python/PhaserVenv/lib/python3.8/site-packages/scipy/optimize/_differentiable_functions.py", line 91, in update_grad
self.g = approx_derivative(fun_wrapped, self.x, f0=self.f,
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/home/brendan/python/PhaserVenv/lib/python3.8/site-packages/scipy/optimize/_numdiff.py", line 388, in approx_derivative
raise ValueError("f0 passed has more than 1 dimension.")
File "/home/brendan/Documents/temp/Bayesian-optimization-using-Gaussian-Process/modules/parallelstuff.py", line 35, in mworker
res = minimize(f, x0, args = fargs, **margs)
ValueError: f0 passed has more than 1 dimension.
File "/home/brendan/python/PhaserVenv/lib/python3.8/site-packages/scipy/optimize/_minimize.py", line 617, in minimize
return _minimize_lbfgsb(fun, x0, args, jac, bounds,
File "/home/brendan/python/PhaserVenv/lib/python3.8/site-packages/scipy/optimize/lbfgsb.py", line 306, in _minimize_lbfgsb
sf = _prepare_scalar_function(fun, x0, jac=jac, args=args, epsilon=eps,
File "/home/brendan/python/PhaserVenv/lib/python3.8/site-packages/scipy/optimize/optimize.py", line 261, in _prepare_scalar_function
sf = ScalarFunction(fun, x0, args, grad, hess,
File "/home/brendan/python/PhaserVenv/lib/python3.8/site-packages/scipy/optimize/_differentiable_functions.py", line 95, in init
self._update_grad()
File "/home/brendan/python/PhaserVenv/lib/python3.8/site-packages/scipy/optimize/_differentiable_functions.py", line 171, in _update_grad
self._update_grad_impl()
File "/home/brendan/python/PhaserVenv/lib/python3.8/site-packages/scipy/optimize/_differentiable_functions.py", line 91, in update_grad
self.g = approx_derivative(fun_wrapped, self.x, f0=self.f,
File "/home/brendan/python/PhaserVenv/lib/python3.8/site-packages/scipy/optimize/_numdiff.py", line 388, in approx_derivative
raise ValueError("f0 passed has more than 1 dimension.")
ValueError: f0 passed has more than 1 dimension.

[bwheelz36]

Hey Adi, There's a version of the example file that I can run with my environment at
https://github.com/bwheelz36/Bayesian-optimization-using-Gaussian-Process/tree/OptOnFly

The key change is the last line in bayes_optimization, change
return -GPUCB
to
return -float(GPUCB)

In addition, I have to set
multiprocessingQ = False
basinhoppingQ = can be True or False

[CY-Zhang]

@bwheelz36 I got the same error as you did, I found out that it can also be solved by changing the line you mentioned
modules/bayes_optimization.py/line586
from
return -GPUCB
to
return -GPUCB.flatten()
or
return -GPUCB[0]

I think the problem was on the y_var parameter to negUCB, it has an empty dimension ([[y_var]]), so the resulting GPUCB also has that empty dimension, then scipy.optimize.minimize (of some versions) gets confused as it thinks it is minimizing an object of higher dimension. Maybe some scipy versions are smart enough to ignore the empty dimension.

[CY-Zhang]

I am also getting trouble from two other lines when running the demo under multiprocessing mode, they are:

modules/parallelstuff.py/ Line 37:
out_q.put([[res.x, res.fun[0][0]]])

and

modules/parallelstuff.py/ Line 340:
out_q.put(np.hstack((x, res[0][0])))

I have to change res[0][0] in both lines to res[0], otherwise, I got an error message: 'IndexError: invalid index to scalar variable.' I think this might be caused by some difference in different versions of scipy, as the res.fun in res returned from scipy.optimize.minimize is 1D in some version (like my 1.6.1), and some might have an extra empty dimension. I hope making this change won't cause any trouble, the demo now runs well for me under multiprocessing mode.

[bwheelz36]

@CY-Zhang, thanks for updating.
Using your changes, I can run the default example with multi processing... @adkoo if these changes are acceptable can we close this issue?