smt_optim.acquisition_strategies package

Submodules

smt_optim.acquisition_strategies.base module

class AcquisitionStrategy

Bases: ABC

abstractmethod get_infill(state) → list[ndarray]

abstractmethod validate_config(state) → None

smt_optim.acquisition_strategies.mfsego module

class MFSEGO(state: State, **kwargs)

Bases: AcquisitionStrategy

Multi-Fidelity Super Efficient Global Optimization (MF-SEGO) strategy.

This acquisition strategy can perform Efficient Global Optimization (EGO) (unconstrained optimization), SEGO (constrained optimization), and MF-SEGO (multi-fidelity unconstrained or constrained optimization).

It is compatible with various acquisition functions, including: - expected improvement, - log expected improvement, - probability of improvement, and - log probability of improvement.

The constraints are handled by maximizing the acquisition function with respect to predictions from constraint surrogate models, instead of using the Probability-of-Improvement approach.

In the multi-fidelity setting, the acquisition function is first maximized, followed by fidelity level selection. This strategy maintains a nested Design of Experiments (DoE), meaning that for each new fidelity level sampled, all lower-fidelity levels are also requested to be sampled.

MF-SEGO offers different fidelity selection criteria: - obj-only, - optimistic, - pessimistic, and - average.

Parameters:

• state (State) – Optimization state containing surrogate models, data, and problem definition.

• acq_func (callable, optional) – Acquisition function used to rank candidate points (default: log_ei).

• n_start (int, optional) – Number of multistart initializations for the inner optimizer. Default: 20.

• fidelity_crit ({"obj-only", "average", "optimistic", "pessimistic"}, optional) – Strategy used to select fidelity level.

• select_fidelity (bool, optional) – If False, always evaluate all fidelity levels.

• sp_method (str, optional) – Optimization method passed to SciPy (e.g., “SLSQP”, “COBYLA”). Default = “SLSQP”.

• sp_tol (float, optional) – Tolerance for the SciPy optimizer. Default = sqrt(machine epsilon).

• relax_constraints (float, optional) – Margin multiplier to relax the constraints using the variance predicted by the surrogate models (relax * sigma). Default is 0.0 (no relaxation).

Notes

When optimizing a high-dimensional problem, it is recommended to increase the number of starting points (n_start). The default setting may be insufficient for problems with higher dimensions or many constraints.

This acquisition strategy is designed to work with SMT’s surrogate models. In the multi-fidelity setting, SMT’s MFK model must be used.

build_scipy_constraints(state: State) → list[dict]

build_scipy_objective(acq_context: State) → Callable

get_fidelity(next_x: ndarray, state: State) → list[int]

Select the highest fidelity level to sample at the given point(s).

Parameters:

• next_x (np.ndarray) – The point(s) to sample at.

• state (State) – The current optimization state.

Returns:

levels – The selected fidelity level(s). If state.problem.num_fidelity is 1, returns the single fidelity level; otherwise, returns a list of fidelity levels, one for each point in next_x.

Return type:

list[int] or array of int

Notes

This method takes into account the problem’s cost model and the available surrogate models.

get_infill(acq_context: State) → list[ndarray]

Compute the next infill point(s) using the acquisition strategy.

Parameters:

acq_context (State) – Current optimization state, including surrogate models and data.

Returns:

List of selected infill points. Each entry corresponds to a point (and potentially a fidelity level, depending on configuration).

Return type:

list of ndarray

Notes

This method: - Optimizes the acquisition function using a multistart strategy - Applies the selected fidelity criterion if select_fidelity=True - Uses SciPy optimizers (controlled via sp_method, sp_tol)

validate_config(acq_context: State) → None

build_scipy_constraints(state: State, relax: float = 0.0) → list[dict]

compute_all_s2_red_norm(x_pred: ndarray, costs: list[float], surrogates: list, corr_method=None) → list[ndarray]

Compute the normalized the variance reduction of all models in the surrogates list.

Parameters:

• x_pred (np.ndarray of shape (num_points, num_dim)) – Prediction points.

• costs (list of float) – Evaluation cost of each fidelity level.

• surrogates (list of Surrogate) – List of SMT-Optim surrogate models.

• corr_method (str, optional) – corrected_predict_variances_all_levels correction method.

Return type:

List of np.ndarray of shape (num_points, num_level).

compute_norm_sigma2_red(x_pred: ndarray, norm_costs2: list[float], surrogate, corr_method=None) → ndarray

Normalize the variance reduction of each level by their corresponding normalized total squared costs.

Parameters:

• x_pred (np.ndarray of shape (num_points, num_dim)) – Prediction points.

• norm_costs2 (list of float) – normalized total squared costs.

• surrogate (Surrogate) – SMT-Optim surrogate model with a model attribute corresponding to a SMT MFK model.

• corr_method (str, optional) – corrected_predict_variances_all_levels correction method.

Return type:

np.ndarray of shape (num_points, num_level)

compute_norm_squared_cost(costs: list[float]) → ndarray

Compute the normalized total squared cost of each fidelity level.

The output corresponds to:

\[\text{cost}_{\ell} = \left(\sum_{\ell'=1}^{\ell}{c_{\ell'}} \; \bigg/ \; \sum_{\ell'=1}^{L}{c_{\ell'}}\right)^{2}.\]

where \(\ell\) corresponds to the fidelity level evaluated and \(L\) to the total number of fidelity levels.

Parameters:

costs (list of float) – Evaluation cost of each fidelity level.

Returns:

Normalized total squared cost of each fidelity level.

Return type:

np.ndarray

compute_sigma2_red(x_pred: ndarray, surrogate, method=None) → ndarray

Compute the variance contribution of each fidelity level viewed from the highest fidelity level.

For a given surrogate model, the output corresponds to

\[\sigma_{\text{red}}^2(\ell, \boldsymbol{x}_{i}) = \sum_{\ell'=1}^{\ell}\sigma^2_{(\delta, \ell')}(\boldsymbol{x}_{i}) \prod_{j=\ell'}^{L-1}{\rho^2_j}.\]

where \(\ell\) corresponds to the fidelity level evaluated and \(L\) to the total number of fidelity level.

Parameters:

• x_pred (np.ndarray of shape (num_points, num_dim)) – Prediction points.

• surrogate (Surrogate) – SMT-Optim surrogate model with a model attribute corresponding to a SMT MFK model.

• method (str, optional) – corrected_predict_variances_all_levels correction method.

Returns:

Variance contribution of each level viewed from the highest fidelity level.

Return type:

np.ndarray of shape (num_points, num_level)

corrected_predict_variances_all_levels(x_pred: ndarray, model, method: str = 'max') → tuple[ndarray, list]

Predict the variance at all fidelity levels for given prediction points x_pred.

Parameters:

• x_pred (np.ndarray of shape (num_points, num_dim)) – Prediction points.

• model (MFK) – SMT’s MFK model.

• method (str, optional) – Correction method for ill-conditioned models.

Returns:

• np.ndarray – Variances for all points at all fidelity levels.

• list[float] – The squared correlation coefficient for each level.

Notes

If method is set to None, no correction is performed. If set to max, the maximum variance for each level is subtracted to the variance of x_pred. If set to closest, for each prediction point and each fidelity level the variance of the closest training point is subtracted to the predicted variance.

select_fidelity_level(x_pred: ndarray, costs: list[float], all_surrogates: list[Surrogate], criterion: str = 'pessimistic', corr_method=None) → tuple[ndarray, ndarray]

Select the highest fidelity level to sample based on the criterion.

Parameters:

• x_pred (np.ndarray of shape(num_points, num_dim)) – Prediction points.

• costs (list of float) – Evaluation cost of each fidelity level.

• all_surrogates (list of Surrogate) – List of surrogate models.

• criterion (str, optional) – Fidelity criterion. The possible values are : “obj-only”, “optimistic”, “pessimistic”, “average”, and “cstr-only”.

• corr_method (str, optional) – corrected_predict_variances_all_levels correction method.

Returns:

• np.ndarray of shape (num_points,) – Highest fidelity level to sample based on the criterion.

• np.ndarray – The criterion value for each fidelity level. The index corresponds to the corresponding fidelity level.

Notes

The obj-only criterion only evaluates the variance reduction of the first objective model. The optimistic criterion selects the overall lowest fidelity level from all the models. The pessimistic criterion selects the overall highest fidelity level from all the models. The average criterion averages the variance reduction of all the models on a fidelity level basis, and selects the level with the highest averaged variance reduction. The cstr-only criterion only works for a single constraint.

smt_optim.acquisition_strategies.mosego module

class MOSEGO(state: State, acq_init: Callable = <function init_ehvi_2o>, n_start: int = 20, genetic_flag: bool = True, genetic_pop_size: int = 20, genetic_n_gen: int = 100, sp_method: str = 'SLSQP', sp_tol: float = np.float64(1.4901161193847656e-08), select_fidelity: bool = True, cr_override: float | None = None, var_red_corr: str | None = None, seed: int | None = None, relax_constraints: float = 0.0)

Bases: AcquisitionStrategy

Multi-objective Super Efficient Global Optimization acquisition strategy.

This acquisition strategy can perform multi-objective optimization on unconstrained, constrained, and multi-fidelity optimization problems.

The constraints are handled by maximizing the acquisition function with respect to predictions from constraint surrogate models, instead of using the Probability-of-Improvement approach.

In the multi-fidelity setting, the acquisition function is first maximized, followed by fidelity level selection. This strategy maintains a nested Design of Experiments (DoE), meaning that for each new fidelity level sampled, all lower-fidelity levels are also requested to be sampled.

Parameters

surrogate models, and dataset.

acq_initCallable, optional

Acquisition function initializer used to generate the acquisition function. By default, uses init_ehvi_2o.

n_startint, optional

Number of starting points used for local acquisition optimization. Default is 20.

genetic_flagbool, optional

Whether to use a genetic algorithm for global acquisition optimization. Default is True.

genetic_pop_sizeint, optional

Population size for the genetic optimizer. Default is 20.

genetic_n_genint, optional

Number of generations for the genetic optimizer. Default is 100.

sp_methodstr, optional

Local optimization method used for acquisition refinement. Default is “SLSQP”.

sp_tolfloat, optional

Tolerance for the local optimizer. Default is sqrt(np.finfo(float).eps).

select_fidelitybool, optional

Whether to optimize fidelity selection when using multi-fidelity optimization. Default is True.

cr_overridefloat or None, optional

Optional override value for the cost ratio used in fidelity selection. Default is None.

seedint or None, optional

Random seed for reproducibility. Default is None.

relax_constraintsfloat, optional

Margin multiplier to relax the constraints using the variance predicted by the surrogate models (relax * sigma). Default is 0.0 (no relaxation).

Notes

All objective configuration type must be set to minimize.

If genetic_flag is set to True, genetic_pop_size points are added to the n_start points generated with LHS. These points are obtained by solving the predicted Pareto Front (PF) using the surrogate models. genetic_flag is currently not available for mixed-variable problems.

get_fidelity(next_x: ndarray, state: State) → list[int]

Select the highest fidelity level to sample at the given point(s).

Parameters:

• next_x (np.ndarray) – The point(s) to sample at.

• state (State) – The current optimization state.

Returns:

levels – The selected fidelity level(s). If state.problem.num_fidelity is 1, returns the single fidelity level; otherwise, returns a list of fidelity levels, one for each point in next_x.

Return type:

list[int] or array of int

Notes

This method takes into account the problem’s cost model and the available surrogate models.

get_infill(state)

validate_config(state)

smt_optim.acquisition_strategies.vfpi module

class VFPI(state: State, **kwargs)

Bases: AcquisitionStrategy

epi(x: ndarray, lvl: int, state: State) → float

Evaluate the Extended Probability of Improvement (EPI) acquisition function.

This method computes an acquisition value that combines the probability of improvement at the highest fidelity with several multiplicative correction factors accounting for fidelity correlation, evaluation cost, sampling density, and probability of feasibility (for constrained optimization).

Parameters:

• x (ndarray of shape (1, n_dim)) – Input point at which the acquisition function is evaluated.

• lvl (int) – Fidelity level at which the acquisition function is computed. Lower values correspond to lower-fidelity (cheaper) models, starting from 0.

• state (State) – Optimization state

Returns:

Value of the EPI acquisition function at the given point and fidelity level.

Return type:

float

Notes

The EPI criterion is defined as a product of the following terms:

• Probability of Improvement (PI): Computed at the highest fidelity level using the predictive mean and variance.

• Fidelity Correlation Penalty: Accounts for the correlation between the selected fidelity level and the highest fidelity.

• Cost Ratio: Ratio of highest-fidelity cost to the cost at the selected level.

• Density Penalty: Optional factor that penalizes regions with high sampling density.

• Probability of Feasibility (PoF): Product of feasibility probabilities across all constraints, evaluated at the selected fidelity level.

The input x is reshaped internally to match the expected input format of the surrogate models.

See also

probability_of_improvement

Computes the probability of improvement.

fidelity_correlation

Computes correlation between fidelity levels.

sample_density

Estimates local sampling density (if enabled).

get_infill(state: State) → list[ndarray]

get_predicted_fmin(state)

Estimate the minimum predicted objective value using multistart optimization.

This method constructs a wrapper around the first surrogate objective model stored in state.obj_models and performs a multistart optimization over the unit hypercube \([0, 1]^d\), where \(d\) is the problem dimension. The optimization is carried out using the multistart_minimize routine.

Parameters:

state (State) – Optimization state

Returns:

The minimum predicted objective function value found across all multistart runs.

Return type:

float

Notes

• The search domain is assumed to be the unit hypercube.

• The optimization relies on self.n_start initial points and self.seed for reproducibility.

See also

multistart_minimize

Multistart optimization routine used to perform the search.

sample_density(x: ndarray, lvl: int, mfck) → ndarray

Compute a sampling density penalty based on correlation structure.

This method evaluates a multiplicative penalty that reflects the local sampling density at a given point x for a specified fidelity level. The penalty is derived from the correlation kernel of a multi-fidelity co-Kriging (MFCK) model and decreases in regions where training samples are dense.

Parameters:

• x (ndarray of shape (n_dim,) or (n_eval, n_dim)) – Input point(s) at which the density penalty is evaluated.

• lvl (int) – Fidelity level at which the density is computed.

• mfck (MFCK) – Multi-fidelity co-Kriging model from the SMT package.

Returns:

Density penalty values at the input locations. Lower values indicate regions with higher sampling density.

Return type:

ndarray of shape (n_eval, 1)

Notes

• Inputs are internally normalized using the model’s scaling and offset.

• The kernel hyperparameters (sigma2 and theta) are extracted from optimal_theta based on the fidelity level.

• This formulation encourages exploration by penalizing regions that are strongly correlated with existing samples.

validate_config(acq_context: State) → None

Module contents

class AcquisitionStrategy

Bases: ABC

abstractmethod get_infill(state) → list[ndarray]

abstractmethod validate_config(state) → None

class MFSEGO(state: State, **kwargs)

Bases: AcquisitionStrategy

Multi-Fidelity Super Efficient Global Optimization (MF-SEGO) strategy.

This acquisition strategy can perform Efficient Global Optimization (EGO) (unconstrained optimization), SEGO (constrained optimization), and MF-SEGO (multi-fidelity unconstrained or constrained optimization).

It is compatible with various acquisition functions, including: - expected improvement, - log expected improvement, - probability of improvement, and - log probability of improvement.

The constraints are handled by maximizing the acquisition function with respect to predictions from constraint surrogate models, instead of using the Probability-of-Improvement approach.

In the multi-fidelity setting, the acquisition function is first maximized, followed by fidelity level selection. This strategy maintains a nested Design of Experiments (DoE), meaning that for each new fidelity level sampled, all lower-fidelity levels are also requested to be sampled.

MF-SEGO offers different fidelity selection criteria: - obj-only, - optimistic, - pessimistic, and - average.

Parameters:

• state (State) – Optimization state containing surrogate models, data, and problem definition.

• acq_func (callable, optional) – Acquisition function used to rank candidate points (default: log_ei).

• n_start (int, optional) – Number of multistart initializations for the inner optimizer. Default: 20.

• fidelity_crit ({"obj-only", "average", "optimistic", "pessimistic"}, optional) – Strategy used to select fidelity level.

• select_fidelity (bool, optional) – If False, always evaluate all fidelity levels.

• sp_method (str, optional) – Optimization method passed to SciPy (e.g., “SLSQP”, “COBYLA”). Default = “SLSQP”.

• sp_tol (float, optional) – Tolerance for the SciPy optimizer. Default = sqrt(machine epsilon).

• relax_constraints (float, optional) – Margin multiplier to relax the constraints using the variance predicted by the surrogate models (relax * sigma). Default is 0.0 (no relaxation).

Notes

When optimizing a high-dimensional problem, it is recommended to increase the number of starting points (n_start). The default setting may be insufficient for problems with higher dimensions or many constraints.

This acquisition strategy is designed to work with SMT’s surrogate models. In the multi-fidelity setting, SMT’s MFK model must be used.

build_scipy_constraints(state: State) → list[dict]

build_scipy_objective(acq_context: State) → Callable

get_fidelity(next_x: ndarray, state: State) → list[int]

Select the highest fidelity level to sample at the given point(s).

Parameters:

• next_x (np.ndarray) – The point(s) to sample at.

• state (State) – The current optimization state.

Returns:

levels – The selected fidelity level(s). If state.problem.num_fidelity is 1, returns the single fidelity level; otherwise, returns a list of fidelity levels, one for each point in next_x.

Return type:

list[int] or array of int

Notes

This method takes into account the problem’s cost model and the available surrogate models.

get_infill(acq_context: State) → list[ndarray]

Compute the next infill point(s) using the acquisition strategy.

Parameters:

acq_context (State) – Current optimization state, including surrogate models and data.

Returns:

List of selected infill points. Each entry corresponds to a point (and potentially a fidelity level, depending on configuration).

Return type:

list of ndarray

Notes

This method: - Optimizes the acquisition function using a multistart strategy - Applies the selected fidelity criterion if select_fidelity=True - Uses SciPy optimizers (controlled via sp_method, sp_tol)

validate_config(acq_context: State) → None

class MOSEGO(state: State, acq_init: Callable = <function init_ehvi_2o>, n_start: int = 20, genetic_flag: bool = True, genetic_pop_size: int = 20, genetic_n_gen: int = 100, sp_method: str = 'SLSQP', sp_tol: float = np.float64(1.4901161193847656e-08), select_fidelity: bool = True, cr_override: float | None = None, var_red_corr: str | None = None, seed: int | None = None, relax_constraints: float = 0.0)

Bases: AcquisitionStrategy

Multi-objective Super Efficient Global Optimization acquisition strategy.

This acquisition strategy can perform multi-objective optimization on unconstrained, constrained, and multi-fidelity optimization problems.

The constraints are handled by maximizing the acquisition function with respect to predictions from constraint surrogate models, instead of using the Probability-of-Improvement approach.

In the multi-fidelity setting, the acquisition function is first maximized, followed by fidelity level selection. This strategy maintains a nested Design of Experiments (DoE), meaning that for each new fidelity level sampled, all lower-fidelity levels are also requested to be sampled.

Parameters

surrogate models, and dataset.

acq_initCallable, optional

Acquisition function initializer used to generate the acquisition function. By default, uses init_ehvi_2o.

n_startint, optional

Number of starting points used for local acquisition optimization. Default is 20.

genetic_flagbool, optional

Whether to use a genetic algorithm for global acquisition optimization. Default is True.

genetic_pop_sizeint, optional

Population size for the genetic optimizer. Default is 20.

genetic_n_genint, optional

Number of generations for the genetic optimizer. Default is 100.

sp_methodstr, optional

Local optimization method used for acquisition refinement. Default is “SLSQP”.

sp_tolfloat, optional

Tolerance for the local optimizer. Default is sqrt(np.finfo(float).eps).

select_fidelitybool, optional

Whether to optimize fidelity selection when using multi-fidelity optimization. Default is True.

cr_overridefloat or None, optional

Optional override value for the cost ratio used in fidelity selection. Default is None.

seedint or None, optional

Random seed for reproducibility. Default is None.

relax_constraintsfloat, optional

Margin multiplier to relax the constraints using the variance predicted by the surrogate models (relax * sigma). Default is 0.0 (no relaxation).

Notes

All objective configuration type must be set to minimize.

If genetic_flag is set to True, genetic_pop_size points are added to the n_start points generated with LHS. These points are obtained by solving the predicted Pareto Front (PF) using the surrogate models. genetic_flag is currently not available for mixed-variable problems.

get_fidelity(next_x: ndarray, state: State) → list[int]

Select the highest fidelity level to sample at the given point(s).

Parameters:

• next_x (np.ndarray) – The point(s) to sample at.

• state (State) – The current optimization state.

Returns:

levels – The selected fidelity level(s). If state.problem.num_fidelity is 1, returns the single fidelity level; otherwise, returns a list of fidelity levels, one for each point in next_x.

Return type:

list[int] or array of int

Notes

This method takes into account the problem’s cost model and the available surrogate models.

get_infill(state)

validate_config(state)

class VFPI(state: State, **kwargs)

Bases: AcquisitionStrategy

epi(x: ndarray, lvl: int, state: State) → float

Evaluate the Extended Probability of Improvement (EPI) acquisition function.

This method computes an acquisition value that combines the probability of improvement at the highest fidelity with several multiplicative correction factors accounting for fidelity correlation, evaluation cost, sampling density, and probability of feasibility (for constrained optimization).

Parameters:

• x (ndarray of shape (1, n_dim)) – Input point at which the acquisition function is evaluated.

• lvl (int) – Fidelity level at which the acquisition function is computed. Lower values correspond to lower-fidelity (cheaper) models, starting from 0.

• state (State) – Optimization state

Returns:

Value of the EPI acquisition function at the given point and fidelity level.

Return type:

float

Notes

The EPI criterion is defined as a product of the following terms:

• Probability of Improvement (PI): Computed at the highest fidelity level using the predictive mean and variance.

• Fidelity Correlation Penalty: Accounts for the correlation between the selected fidelity level and the highest fidelity.

• Cost Ratio: Ratio of highest-fidelity cost to the cost at the selected level.

• Density Penalty: Optional factor that penalizes regions with high sampling density.

• Probability of Feasibility (PoF): Product of feasibility probabilities across all constraints, evaluated at the selected fidelity level.

The input x is reshaped internally to match the expected input format of the surrogate models.

See also

probability_of_improvement

Computes the probability of improvement.

fidelity_correlation

Computes correlation between fidelity levels.

sample_density

Estimates local sampling density (if enabled).

get_infill(state: State) → list[ndarray]

get_predicted_fmin(state)

Estimate the minimum predicted objective value using multistart optimization.

This method constructs a wrapper around the first surrogate objective model stored in state.obj_models and performs a multistart optimization over the unit hypercube \([0, 1]^d\), where \(d\) is the problem dimension. The optimization is carried out using the multistart_minimize routine.

Parameters:

state (State) – Optimization state

Returns:

The minimum predicted objective function value found across all multistart runs.

Return type:

float

Notes

• The search domain is assumed to be the unit hypercube.

• The optimization relies on self.n_start initial points and self.seed for reproducibility.

See also

multistart_minimize

Multistart optimization routine used to perform the search.

sample_density(x: ndarray, lvl: int, mfck) → ndarray

Compute a sampling density penalty based on correlation structure.

This method evaluates a multiplicative penalty that reflects the local sampling density at a given point x for a specified fidelity level. The penalty is derived from the correlation kernel of a multi-fidelity co-Kriging (MFCK) model and decreases in regions where training samples are dense.

Parameters:

• x (ndarray of shape (n_dim,) or (n_eval, n_dim)) – Input point(s) at which the density penalty is evaluated.

• lvl (int) – Fidelity level at which the density is computed.

• mfck (MFCK) – Multi-fidelity co-Kriging model from the SMT package.

Returns:

Density penalty values at the input locations. Lower values indicate regions with higher sampling density.

Return type:

ndarray of shape (n_eval, 1)

Notes

• Inputs are internally normalized using the model’s scaling and offset.

• The kernel hyperparameters (sigma2 and theta) are extracted from optimal_theta based on the fidelity level.

• This formulation encourages exploration by penalizing regions that are strongly correlated with existing samples.

validate_config(acq_context: State) → None