Bounds

bayesopt requires finite bounds on all variables. (categorical variables are, by nature, bounded in their possible values.) Pass the lower and upper bounds for real and integer-valued variables in optimizableVariable.

bayesopt uses these bounds to sample points, either uniformly or log-scaled. You set the scaling for sampling in optimizableVariable.

For example, to constrain a variable X1 to values between 1e-6 and 1e3, scaled logarithmically,

xvar = optimizableVariable('X1',[1e-6,1e3],'Transform','log')

bayesopt includes the endpoints in its range. Therefore, you cannot use 0 as a lower bound for a real log-transformed variable.

For an integer-valued log-transformed variable, you can use 0 as a lower bound. If a lower bound is 0 for an integer-valued variable, then the software creates the log-scaled space by using the log1p function instead of the log function to include 0 in the variable sampling range. log1p is a function that returns log(1+x) for an input x.

Deterministic Constraints — XConstraintFcn

Sometimes your problem is valid or well-defined only for points in a certain region, called the feasible region. A deterministic constraint is a deterministic function that returns true when a point is feasible, and false when a point is infeasible. So deterministic constraints are not stochastic, and they are not functions of a group of points, but of individual points.

Write a deterministic constraint function using the signature

tf = xconstraint(X)

Pass the deterministic constraint function in the bayesopt XConstraintFcn name-value argument. For example,

results = bayesopt(fun,vars,'XConstraintFcn',@xconstraint)

bayesopt evaluates deterministic constraints on thousands of points, and so runs faster when your constraint function is vectorized. See Vectorization.

For example, suppose that the variables named 'x1' and 'x2' are feasible when the norm of the vector [x1 x2] is less than 6, and when x1 <= x2. The following constraint function evaluates these constraints.

function tf = xconstraint(X)
tf1 = sqrt(X.x1.^2 + X.x2.^2) < 6;
tf2 = X.x1 <= X.x2;
tf = tf1 & tf2;

Conditional Constraints — ConditionalVariableFcn

Conditional constraints are functions that enforce one of the following two conditions:

Specify a conditional constraint by setting the bayesopt ConditionalVariableFcn name-value argument to a function handle, say @condvariablefcn. The @condvariablefcn function must have the signature

Xnew = condvariablefcn(X)

condvariablefcn sets Xnew to be equal to X, except it also sets the relevant variables in each row of Xnew to the correct values for the constraint.

function XTable = fitcdiscrCVF(XTable)
% Gamma must be 0 if discrim type is a quadratic
XTable.Gamma(ismember(XTable.DiscrimType, {'quadratic',...
        'diagQuadratic','pseudoQuadratic'})) = 0;
end

function Xnew = condvariablefcn(X)
Xnew = X;
Xnew.PolynomialOrder(Xnew.KernelFunction ~= 'polynomial') = NaN;

function X = condvariablefcn(X)
X.PolynomialOrder(Xnew.KernelFunction ~= 'polynomial') = NaN;

fun = @(X)mysvmfun(X,predictors,responce,c)

function objective = mysvmfun(X,predictors,response,c)
    args = {predictors,response, ...
        'CVPartition',c, ...
        'KernelFunction',X.KernelFunction};
    if ~isnan(X.PolynomialOrder)
        args = [args,{'PolynomialOrder',X.PolynomialOrder}];
    end
    objective = kfoldLoss(fitcsvm(args{:}));
end

Coupled Constraints

Coupled constraints are constraints that you can evaluate only by calling the objective function. These constraints can be stochastic or deterministic. Return these constraint values from your objective function in the second argument. See Bayesian Optimization Objective Functions.

The objective function returns a numeric vector for the coupled constraints, one entry for each coupled constraint. For each entry, a negative value indicates that the constraint is satisfied (also called feasible). A positive value indicates that the constraint is not satisfied (infeasible).

bayesopt automatically creates a coupled constraint, called the Error constraint, for every run. This constraint enables bayesopt to model points that cause errors in objective function evaluation. For details, see Objective Function Errors and predictError.

If you have coupled constraints in addition to the Error constraint:

Observe the coupled constraint values in each iteration by setting the bayesopt Verbose name-value argument to 1 or 2.

For an example, see Bayesian Optimization with Coupled Constraints.

Bayesian Optimization with Coupled Constraints

A coupled constraint is one that can be evaluated only by evaluating the objective function. In this case, the objective function is the cross-validated loss of an SVM model. The coupled constraint is that the number of support vectors is no more than 100. The model details are in Optimize Cross-Validated Classifier Using bayesopt.

rng default
grnpop = mvnrnd([1,0],eye(2),10);
redpop = mvnrnd([0,1],eye(2),10);
redpts = zeros(100,2);
grnpts = redpts;
for i = 1:100
    grnpts(i,:) = mvnrnd(grnpop(randi(10),:),eye(2)*0.02);
    redpts(i,:) = mvnrnd(redpop(randi(10),:),eye(2)*0.02);
end
cdata = [grnpts;redpts];
grp = ones(200,1);
grp(101:200) = -1;
c = cvpartition(200,'KFold',10);
sigma = optimizableVariable('sigma',[1e-5,1e5],'Transform','log');
box = optimizableVariable('box',[1e-5,1e5],'Transform','log');

function [objective,constraint] = mysvmfun(x,cdata,grp,c)
SVMModel = fitcsvm(cdata,grp,'KernelFunction','rbf',...
    'BoxConstraint',x.box,...
    'KernelScale',x.sigma);
cvModel = crossval(SVMModel,'CVPartition',c);
objective = kfoldLoss(cvModel);
constraint = sum(SVMModel.IsSupportVector)-100.5;

fun = @(x)mysvmfun(x,cdata,grp,c);

results = bayesopt(fun,[sigma,box],'IsObjectiveDeterministic',true,...
    'NumCoupledConstraints',1,'PlotFcn',...
    {@plotMinObjective,@plotConstraintModels},...
    'AcquisitionFunctionName','expected-improvement-plus','Verbose',0);