smt_optim.acquisition_functions package

smt_optim.acquisition_functions package#

Submodules#

smt_optim.acquisition_functions.ehvi module#

ehvi_2o(mu: ndarray, s: ndarray, Y: ndarray) → float[source]#

Compute the Expected Hypervolume Improvement (EHVI) for bi-objective optimization.

Parameters:

  • mu (np.ndarray of shape (2,)) – Predictive mean of the two objective functions.

  • s (np.ndarray of shape (2,)) – Predictive standard deviation of the two objective functions.

  • Y (np.ndarray of shape (n, 2)) – Augmented Pareto front, sorted in ascending order with respect to the second objective.

Returns:

The expected hypervolume improvement.

Return type:

float

Notes

Follows the implementation discussed in: Yang, K., Emmerich, M., Deutz, A., & Bäck, T. (2019). Multi-objective Bayesian global optimization using expected hypervolume improvement gradient. Swarm and evolutionary computation, 44, 945-956.

init_ehvi_2o(state) → Callable[source]#

Initialize the Expected Hypervolume Improvement (EHVI) acquisition function for bi-objective Bayesian optimization.

The returned callable evaluates EHVI using the objective surrogate models stored in state. The Pareto front is extracted from the scaled dataset and augmented with reference points required by the two-objective EHVI computation (ehvi_2o).

Parameters:

state (State) –

Optimization state containing the problem definition, scaled dataset, and trained objective surrogate models.

The problem must have exactly two objectives. The scaled dataset must contain objective evaluations from which the current Pareto front can be computed.

Returns:

EHVI acquisition function with signature:

acquisition(x) -> float

where x is a candidate point and the returned value is the expected hypervolume improvement at x.

Return type:

Callable

Raises:

ValueError – If the optimization problem is not bi-objective.

Notes

The Pareto front is augmented with two additional points and sorted by the second objective before being passed to the EHVI computation.

Follows the implementation discussed in: Yang, K., Emmerich, M., Deutz, A., & Bäck, T. (2019). Multi-objective Bayesian global optimization using expected hypervolume improvement gradient. Swarm and evolutionary computation, 44, 945-956.

psi(a: float | ndarray, b: float | ndarray, mu: float | ndarray, s: float | ndarray) → float | ndarray[source]#

Helper function to be used with the EHVI (ehvi_2o) acquisition function.

Parameters:

  • a (float or np.array)

  • b (float or np.array)

  • mu (float or np.array)

  • s (float or np.array)

Return type:

float or np.ndarray

smt_optim.acquisition_functions.expected_improvement module#

expected_improvement(mu: float, s2: float, f_min: float) → float[source]#

Expected Improvement acquisition function.

Parameters:

  • mu (float) – Mean prediction.

  • s2 (float) – Variance prediction.

  • f_min (float) – Best minimum objective value in training data.

Returns:

Expected Improvement value.

Return type:

float

log1mexp(z: float) → float[source]#
log_ei(mu: float, s2: float, f_min: float) → float[source]#

Log Expected Improvement acquisition function.

LogEI is more numerically stable that the EI acquisition function especially when the GP’s variance is small. From: https://arxiv.org/abs/2310.20708.

Parameters:

  • mu (float) – Mean prediction

  • s2 (float) – Variance prediction

  • f_min (float) – Best minimum objective value in training data.

Return type:

float

vec_expected_improvement(mu: ndarray, s2: ndarray, f_min: float) → ndarray[source]#

Vectorized Expected Improvement acquisition function.

Parameters:

  • mu (np.ndarray) – Mean prediction of shape (num_points, 1).

  • s2 (np.ndarray) – Variance prediction of shape (num_points, 1).

  • f_min (float) – Best minimum objective value in training data.

Returns:

Expected Improvement values of shape (num_points, 1).

Return type:

np.ndarray

vec_log1mexp(z: ndarray) → ndarray[source]#
vec_log_ei(mu: ndarray, s2: ndarray, f_min: float) → ndarray[source]#

Vectorized Log Expected Improvement acquisition function.

LogEI is more numerically stable that the EI acquisition function especially when the GP’s variance is small. From: https://arxiv.org/abs/2310.20708.

Parameters:

  • mu (np.ndarray) – Mean prediction of shape (num_points, 1).

  • s2 (np.ndarray) – Variance prediction of shape (num_points, 1).

  • f_min (float) – Best minimum objective value in training data.

Return type:

np.ndarray

smt_optim.acquisition_functions.fidelity_correlation module#

fidelity_correlation(covariance: ndarray, li_var: ndarray, lj_var: ndarray) → ndarray[source]#

Compute the posterior correlation between two fidelity levels.

This function evaluates the pointwise Pearson correlation coefficient between the predictive distributions of two fidelity levels based on their posterior covariance and variances. The resulting correlation is clipped to the interval [0, 1].

Parameters:

  • covariance (ndarray of shape (n_eval,)) – Posterior covariance between the predictions at fidelity levels \(i\) and \(j\) for each evaluation point.

  • li_var (ndarray of shape (n_eval,)) – Posterior predictive variance at fidelity level \(i\).

  • lj_var (ndarray of shape (n_eval,)) – Posterior predictive variance at fidelity level \(j\).

Returns:

Absolute value of the correlation coefficient between fidelity levels \(i\) and \(j\), clipped to the interval [0, 1].

Return type:

ndarray of shape (n_eval,)

smt_optim.acquisition_functions.integrated_variance_reduction module#

current_variance(model, x)[source]#

Compute the conditional variance of the current model (without new points).

integrated_variance_reduction(model, points: ndarray, integration_points: ndarray = None, inv_block: bool = True) → ndarray[source]#

Integrated Variance Reduction (IVR) acquisition function. Evaluates IVR for one or multiple candidate points. Returns the absolute reduction in IMSE: |IMSE_current - IMSE_new|.

Parameters:

  • model (Surrogate) – The smt-optim surrogate model representing the objective function.

  • points (np.ndarray) – Candidate points for enrichment, shape (num_points, num_dim).

  • integration_points (np.ndarray, optional) – Monte-Carlo points for integration over the domain, shape (N_mc, num_dim). If None, a 500-point LHS grid is automatically generated (requires model to have xlimits or infers from training points).

  • inv_block (bool, optional) – Whether to use block matrix inversion (faster). Default True.

Returns:

The IVR values of shape (num_points, 1) (higher is better).

Return type:

np.ndarray

variance_update(model, point, x, inv_block=True)[source]#

Compute the “look-ahead” conditional variance after adding the new point to the model.

Parameters:

  • model (Surrogate) – smt-optim surrogate model (must wrap an SMT Kriging/MFK model).

  • point (np.ndarray) – Enrichment point (candidate).

  • x (np.ndarray()) – Evaluation points (Monte-Carlo integration grid).

  • inv_block (bool, optional) – Compute the augmented correlation matrix using block inversion. Default True.

Returns:

MSE – Look-ahead variance at evaluation points.

Return type:

np.ndarray

smt_optim.acquisition_functions.mpi module#

init_mpi(state) → Callable[source]#

Initialize the Minimum Probability of Improvement (MPoI / MPI) multi-objective acquisition function. DOI: https://doi.org/10.1145/3071178.3071276

Parameters:

  • state (State)

  • Returns – Callable acquisition function

  • -------

Notes

Uses the scaled dataset

smt_optim.acquisition_functions.probability_improvement module#

log_pi(mu: float, s2: float, f_min: float) → float[source]#

Log Probability of Improvement acquisition function.

LogPI is more numerically stable that the pi acquisition function especially when the GP’s variance is small. From: https://arxiv.org/abs/2310.20708.

Parameters:

  • mu (float) – Mean prediction

  • s2 (float) – Variance prediction

  • f_min (float) – Best minimum objective value in training data.

Return type:

float

logerfc(x: float) → float[source]#
probability_of_improvement(mu: float, s2: float, f_min: float) → float[source]#

Probability of Improvement (PI) acquisition function.

Parameters:

  • mu (float) – Mean prediction.

  • s2 (float) – Variance prediction.

  • f_min (float) – Best minimum objective value in training data.

Returns:

Probability of Improvement value.

Return type:

float

vec_probability_of_improvement(mu: ndarray, s2: ndarray, f_min: float) → ndarray[source]#

Probability of Improvement (PI) acquisition function.

Parameters:

  • mu (float) – Mean prediction.

  • s2 (float) – Variance prediction.

  • f_min (float) – Best minimum objective value in training data.

Returns:

Probability of Improvement value.

Return type:

float

Module contents#

expected_improvement(mu: float, s2: float, f_min: float) → float[source]#

Expected Improvement acquisition function.

Parameters:

  • mu (float) – Mean prediction.

  • s2 (float) – Variance prediction.

  • f_min (float) – Best minimum objective value in training data.

Returns:

Expected Improvement value.

Return type:

float

fidelity_correlation(covariance: ndarray, li_var: ndarray, lj_var: ndarray) → ndarray[source]#

Compute the posterior correlation between two fidelity levels.

This function evaluates the pointwise Pearson correlation coefficient between the predictive distributions of two fidelity levels based on their posterior covariance and variances. The resulting correlation is clipped to the interval [0, 1].

Parameters:

  • covariance (ndarray of shape (n_eval,)) – Posterior covariance between the predictions at fidelity levels \(i\) and \(j\) for each evaluation point.

  • li_var (ndarray of shape (n_eval,)) – Posterior predictive variance at fidelity level \(i\).

  • lj_var (ndarray of shape (n_eval,)) – Posterior predictive variance at fidelity level \(j\).

Returns:

Absolute value of the correlation coefficient between fidelity levels \(i\) and \(j\), clipped to the interval [0, 1].

Return type:

ndarray of shape (n_eval,)

init_ehvi_2o(state) → Callable[source]#

Initialize the Expected Hypervolume Improvement (EHVI) acquisition function for bi-objective Bayesian optimization.

The returned callable evaluates EHVI using the objective surrogate models stored in state. The Pareto front is extracted from the scaled dataset and augmented with reference points required by the two-objective EHVI computation (ehvi_2o).

Parameters:

state (State) –

Optimization state containing the problem definition, scaled dataset, and trained objective surrogate models.

The problem must have exactly two objectives. The scaled dataset must contain objective evaluations from which the current Pareto front can be computed.

Returns:

EHVI acquisition function with signature:

acquisition(x) -> float

where x is a candidate point and the returned value is the expected hypervolume improvement at x.

Return type:

Callable

Raises:

ValueError – If the optimization problem is not bi-objective.

Notes

The Pareto front is augmented with two additional points and sorted by the second objective before being passed to the EHVI computation.

Follows the implementation discussed in: Yang, K., Emmerich, M., Deutz, A., & Bäck, T. (2019). Multi-objective Bayesian global optimization using expected hypervolume improvement gradient. Swarm and evolutionary computation, 44, 945-956.

init_mpi(state) → Callable[source]#

Initialize the Minimum Probability of Improvement (MPoI / MPI) multi-objective acquisition function. DOI: https://doi.org/10.1145/3071178.3071276

Parameters:

  • state (State)

  • Returns – Callable acquisition function

  • -------

Notes

Uses the scaled dataset

integrated_variance_reduction(model, points: ndarray, integration_points: ndarray = None, inv_block: bool = True) → ndarray[source]#

Integrated Variance Reduction (IVR) acquisition function. Evaluates IVR for one or multiple candidate points. Returns the absolute reduction in IMSE: |IMSE_current - IMSE_new|.

Parameters:

  • model (Surrogate) – The smt-optim surrogate model representing the objective function.

  • points (np.ndarray) – Candidate points for enrichment, shape (num_points, num_dim).

  • integration_points (np.ndarray, optional) – Monte-Carlo points for integration over the domain, shape (N_mc, num_dim). If None, a 500-point LHS grid is automatically generated (requires model to have xlimits or infers from training points).

  • inv_block (bool, optional) – Whether to use block matrix inversion (faster). Default True.

Returns:

The IVR values of shape (num_points, 1) (higher is better).

Return type:

np.ndarray

log_ei(mu: float, s2: float, f_min: float) → float[source]#

Log Expected Improvement acquisition function.

LogEI is more numerically stable that the EI acquisition function especially when the GP’s variance is small. From: https://arxiv.org/abs/2310.20708.

Parameters:

  • mu (float) – Mean prediction

  • s2 (float) – Variance prediction

  • f_min (float) – Best minimum objective value in training data.

Return type:

float

log_pi(mu: float, s2: float, f_min: float) → float[source]#

Log Probability of Improvement acquisition function.

LogPI is more numerically stable that the pi acquisition function especially when the GP’s variance is small. From: https://arxiv.org/abs/2310.20708.

Parameters:

  • mu (float) – Mean prediction

  • s2 (float) – Variance prediction

  • f_min (float) – Best minimum objective value in training data.

Return type:

float

probability_of_improvement(mu: float, s2: float, f_min: float) → float[source]#

Probability of Improvement (PI) acquisition function.

Parameters:

  • mu (float) – Mean prediction.

  • s2 (float) – Variance prediction.

  • f_min (float) – Best minimum objective value in training data.

Returns:

Probability of Improvement value.

Return type:

float

vec_expected_improvement(mu: ndarray, s2: ndarray, f_min: float) → ndarray[source]#

Vectorized Expected Improvement acquisition function.

Parameters:

  • mu (np.ndarray) – Mean prediction of shape (num_points, 1).

  • s2 (np.ndarray) – Variance prediction of shape (num_points, 1).

  • f_min (float) – Best minimum objective value in training data.

Returns:

Expected Improvement values of shape (num_points, 1).

Return type:

np.ndarray

vec_probability_of_improvement(mu: ndarray, s2: ndarray, f_min: float) → ndarray[source]#

Probability of Improvement (PI) acquisition function.

Parameters:

  • mu (float) – Mean prediction.

  • s2 (float) – Variance prediction.

  • f_min (float) – Best minimum objective value in training data.

Returns:

Probability of Improvement value.

Return type:

float