I forked this project and did some modifications to pre-computed search space for the case of discrete variables with constraints. What is the process to get green light for this changes to be merged to master (i.e. pull request approval process)?

I run optimization and get different results. Sometimes it hits the optimum (I have independent "slow" exhaustive search to know the optimum). Sometimes it does not hit it. Are there known rules of thumb wrt parameters to get more consistent results?

Hi everyone! I still have the problem to use the new version of GPyOpt with the following error:

import GPyOpt Traceback (most recent call last): File "", line 1, in File "build/bdist.macosx-10.5-x86_64/egg/GPyOpt/init.py", line 4, in

File "build/bdist.macosx-10.5-x86_64/egg/GPyOpt/core/init.py", line 4, in

File "build/bdist.macosx-10.5-x86_64/egg/GPyOpt/core/bo.py", line 7, in ImportError: No module named task.cost

Any idea to resolve this problem? THANKS

I tried to run the first example from the manual: http://nbviewer.ipython.org/github/SheffieldML/GPyOpt/blob/master/manual/GPyOpt_reference_manual.ipynb

The line:

import GPyOpt

fails with the following error:

ImportError: No module named 'core'

Happens on Python 3.4 / Windows 8 x64.

I think the problem is Python 3.x, but I cannot use another version because of other packages... any way to fix this?

± pip install gpy gpyopt --user --upgrade
Collecting gpy
  Downloading GPy-1.8.5.tar.gz (856kB)
    100% |████████████████████████████████| 860kB 1.3MB/s 
Collecting gpyopt
  Using cached GPyOpt-1.2.1.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/ahundt/pip-build-fcsazi50/gpyopt/setup.py", line 6, in <module>
        from GPyOpt.__version__ import __version__
      File "/tmp/ahundt/pip-build-fcsazi50/gpyopt/GPyOpt/__init__.py", line 7, in <module>
        from GPyOpt.core.task.space import Design_space
      File "/tmp/ahundt/pip-build-fcsazi50/gpyopt/GPyOpt/core/__init__.py", line 4, in <module>
        from .bo import BO
      File "/tmp/ahundt/pip-build-fcsazi50/gpyopt/GPyOpt/core/bo.py", line 9, in <module>
        from ..util.duplicate_manager import DuplicateManager
      File "/tmp/ahundt/pip-build-fcsazi50/gpyopt/GPyOpt/util/duplicate_manager.py", line 5, in <module>
        from ..core.task.space import Design_space
      File "/tmp/ahundt/pip-build-fcsazi50/gpyopt/GPyOpt/core/task/__init__.py", line 4, in <module>
        from .objective import SingleObjective
      File "/tmp/ahundt/pip-build-fcsazi50/gpyopt/GPyOpt/core/task/objective.py", line 8, in <module>
        import GPy
    ImportError: No module named 'GPy'
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/ahundt/pip-build-fcsazi50/gpyopt/

Hi GPyOpt developers. I have an issue where GPyOpt chooses an infeasible next point when the number of variables in my problem exceeds 8 and then immediately converges to a suboptimal infeasible point (with respect to the inequality constraints, the chosen point is still within the domain). What I am doing is optimizing multiple l2-norm regularization parameters, an n dimensional vector denoted x, where the unknown function f(x) is the solution to a regularized logistic regression with respect to some weights, w, under the inequality constraints x_1 <= x_2 <= x_3 <= ... <= x_n. The purpose of these constraints is to reduce the computation time since any permutation of the elements of x will result in the same value of f(x). Here is an example of how I have the bounds and constraints set up for an n = 5 problem before being passed into BayesianOptimization():

domain = [{'domain': (0.0, 1.0), 'type': 'continuous', 'name': x_1_b'},
                  {'domain': (0.0, 1.0), 'type': 'continuous', 'name': 'x_2_b'},
                  {'domain': (0.0, 1.0), 'type': 'continuous', 'name': 'x_3_b'},
                  {'domain': (0.0, 1.0), 'type': 'continuous', 'name': 'x_4_b'},
                  {'domain': (0.0, 1.0), 'type': 'continuous', 'name': 'x_5_b'}]

constraints = [{'constrain': 'x[:,0] - x[:,1]', 'name': 'x_1_2_c'},
                       {'constrain': 'x[:,1] - x[:,2]', 'name': 'x_2_3_c'},
                       {'constrain': 'x[:,2] - x[:,3]', 'name': 'x_3_4_c'},
                       {'constrain': 'x[:,3] - x[:,4]', 'name': 'x_4_5_c'}]

Other information: I am using the matern32 kernel (this phenomenon occurs with matern52 as well) with the maximum likelihood update to the Gaussian process hyperparameters and the lower confidence bound acquisition function (default values except jitter is 1E-4 to protect against singularity of the kernel). The optimization has worked fine with these constraints when n < 9 and also worked for n > 8 when the constraints were removed.

Hi,

I have a fairly complex function that should always return a number >=0. Print statements in this function indicate this is satisfied. Running opt.fx_opt gives a value close to 0, which should be the true minimum. However, opt.plot_acquisition shows negative numbers at the minimum. This shouldn't be an artifact of the underlying gaussian process model, as even the points in red where GPyOpt evaluates the function is negative. Why is this the case?

Hello, I'm new to python and gaussian processes, so I need a little help/insight. I am trying to use GPyOpt with physical x, y data and not a "known" function. The examples all use a known function. I am reading in a .csv file and I want to use bayesian optimization with the test data to let me know what experiment to run next (obtain next x,y data set). Please any advice and help is appreciated. Thank you.

While running GPyOpt on python3 I started to see the following failure:

Traceback (most recent call last):
  File "/Users/mattlavin/Projects/GPyOpt/GPyOpt/testing/test_parallelization.py", line 102, in test_run
    unittest_result = run_eval(problem_config= self.problem_config, f_inits= self.f_inits, method_config=m_c, name=name, outpath=self.outpath, time_limit=None, unittest = self.is_unittest)
  File "/Users/mattlavin/Projects/GPyOpt/GPyOpt/testing/driver.py", line 47, in run_eval
    verbosity       = m_c['verbosity'])
  File "/Users/mattlavin/Projects/GPyOpt/GPyOpt/methods/bayesian_optimization.py", line 458, in run_optimization
    super(BayesianOptimization, self).run_optimization(max_iter = max_iter, max_time = max_time,  eps = eps, verbosity=verbosity, save_models_parameters = save_models_parameters, report_file = report_file, evaluations_file= evaluations_file, models_file=models_file)
  File "/Users/mattlavin/Projects/GPyOpt/GPyOpt/core/bo.py", line 103, in run_optimization
    self._update_model()
  File "/Users/mattlavin/Projects/GPyOpt/GPyOpt/core/bo.py", line 198, in _update_model
    self._save_model_parameter_values()
  File "/Users/mattlavin/Projects/GPyOpt/GPyOpt/core/bo.py", line 209, in _save_model_parameter_values
    if self.model_parameters_iterations == None:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

It turns out that an array was being compared against None with == instead of is switching to is avoided the ambiguous type coercion.

I understand that "kernel" will become deprecated in the new version. How can one specify the Gpy Matern ARD 5/2 kernel with the newest API?

GPyOpt.methods.bayesian_optimization.BayesianOptimization(f, domain=None, constrains=None, cost_withGradients=None, model_type='GP', X=None, Y=None, initial_design_numdata=None, initial_design_type='random', acquisition_type='EI', normalize_Y=True, exact_feval=False, acquisition_optimizer_type='lbfgs', model_update_interval=1, evaluator_type='sequential', batch_size=1, num_cores=1, verbosity=True, verbosity_model=False, bounds=None, **kwargs)

Dear Developers, We were able to customise our kernel and BO.model, and used it with run_optimisation. The BO procedure works well, but we noticed that if we update our model with each acquisition, this also updates the hyperparameters of the GP process (maximum likelihood) and often results in overfitting. Increasing model update interval was not a good idea either. We cannot find how to a) impose upper/lower numerical bounds on the hyperparameter search and b) refit the model with each acquisitions but update the model hyperparameters more slowly. Ideally, we would use Bayesian updating of hyperparameters (with priors on the hyperparameters), is this available? This is currently the biggest issue for us. Many thanks for your assistance, Milica.

I am trying to write a new acquisition function using the GpyOpt package as provided in https://nbviewer.org/github/SheffieldML/GPyOpt/blob/master/manual/GPyOpt_creating_new_aquisitions.ipynb. After completing the writing of my acquisition function, I am unable to run Bayesian optimization loop using my new acquisition function. Please give me some suggestions regarding how to run my new acquisition function.

I do Bayesian optimisation in two dimensions. When 'ARD' is set to 'True', printing kernel outputs '(2, )'. I would like to check the respective lengthscale in 2D, is there a way to do this?

I often see that optimization terminates earlier than max_iter. I set the option of 'de_duplication' to "False" and eps to zero (or a negative value), so I do not think the consecutive occurrence of the same values of x is the reason.

I would appreciate it if someone could tell me what is causing this problem and how to solve it.

Hi,

Is it possible to model constrained via Gaussian process? (especially implicit constrained, like max tracking error, which is not the constrained of the hyperparameter) Then output the best parameter based on cost and constraint?

Thanks, and have a nice day!

Hey all,

According to the issues I read in this repository, the batch_size>1 only works with the local_penalization evaluator, and not with sequential or any other evaluator model.

I have set up a Bayesian Optimization model where I use a GP surrogate model m trained on my dataset as my objective function, as follows:

def obj_func(X):
    out,_ = m.predict(X)
    return(out)

bo_step = GPyOpt.methods.BayesianOptimization(f = obj_func, domain = bounds,
                                                    model_type='GP',normalize_Y = False,
                                                    evaluator_type = 'local_penalization',
                                                    acquisition_type='EI',batch_size=5, maximize=True, eps=1e-8)

when I run the optimization, I only get a single prediction instead of batch_size=5:

bo_step.run_optimization(max_iter=5)

If I use the external object evaluation example as a starting point and use bo_step.suggest_next_locations() functions, I get 5 suggestions, but it seems that it does not really maximize my objective function (below). However I am not sure if I can/should use this object since I already have a surrogate model function fit into my dataset.

x_next = bo_step.suggest_next_locations()

Any help or suggestion on this is highly appreciated.

Best,

Bulut

Hi, thanks for your contributions.

I notice that the optimizer will finish when the iteration number is over than 'max_iter', or the time is over 'max_time'. But I wonder that can I set a specific number, if the best_y in the optimization is smaller than this value, it means that a optimal result has been found, then BO can stop searching.

Hoping for your reply, thanks!