Benchmark problem registry

Show code cell content

Hide code cell content

import numpy as np
import pandas as pd
import itables
itables.init_notebook_mode()

import matplotlib.pyplot as plt

from smt_optim.benchmarks.registry import list_problems, get_problem

Benchmark problems collection

The table below lists the available problems in SMT-Optim, along with some of their properties.

Show code cell source

Hide code cell source

problems = list_problems(
    num_dim=None,
    num_obj=None,
    num_cstr=None,
    num_fidelity=None,
)

rows = []

for prob in problems:
    rows.append({
        "Problem": prob.__class__.__name__,
        "Dimensions": prob.num_dim,
        "Constraints": prob.num_cstr,
        "Objectives": prob.num_obj,
        "Fidelity": prob.num_fidelity,
        "Tags": prob.tags,
    })

df = pd.DataFrame(data=rows).sort_values(by=["Objectives", "Fidelity", "Problem", "Dimensions", "Constraints"])
itables.show(df, showIndex=False)

Loading ITables v2.8.0 from the init_notebook_mode cell... (need help?)

Obtaining benchmark problems

The list_problems method returns a collection of problems list[BenchmarkProblems] given desired properties. The get_problem method returns the BenchmarkProblem of the corresponding name. Usage examples are provided in the documentation.

Variable dimension

The dimension of some problem can be set by the user. These problems can be identified with the n_variable tag. The dimension can be set with the set_dim class method. The code snippet below fetches the Ackley test problem, sets the number of variable to 5, and then to 10.

ackley = get_problem("Ackley")

ackley.set_dim(5)
print(f"bounds.shape[0] = {ackley.bounds.shape[0]}")

ackley.set_dim(10)
print(f"bounds.shape[0] = {ackley.bounds.shape[0]}")

bounds.shape[0] = 5
bounds.shape[0] = 10

Multi-fidelity problems

As multi-fidelity problems have multiple fidelity levels, the objective attribute returns a list of callable, and the constraints attribute return a list of lists of callable. Each callable corresponds to a fidelity level. These are ordered in ascending order of fidelity.

There are two cases when fetching an objective value at a given fidelity level, depending on whether the problem has multiple objectives:

Single objective:

value = problem.objective[fidelity_level: int](x: np.ndarray)

Multiple objectives:

value = problem.objective[objective_id: int][fidelity_level: int](x: np.ndarray)

Constraints:

To fetch the value of a specific constraint at a given fidelity level:

value = problem.constraints[constraint_id: int][fidelity_level: int](x: np.ndarray)

Mixed-variable problems

The bounds attribute of mixed-variable problems is set to None. Users should instead refer to the design_space attribute which provides the SMT’s DesignSpace object with respect to the problem. The code snippet below illustrates this for the mixed-variable Branin test problem.

mixvar_branin = get_problem("MixVarBranin")

print(f"------- bounds -------\n{mixvar_branin.bounds}")
print(f"---- design_space ----\n{mixvar_branin.design_space}")

------- bounds -------
None
---- design_space ----
Design space:
x0: Float (0, 1)
x1: Float (0, 1)
x2: Cat [0, 1]
x3: Cat [0, 1]