Overview of Optimization

Each optimization problem can be formed by the GPS:

\begin{array}{rlllll} \min & f(\mathbf{x})=\dots \\ \text{where} & \mathbf{x}=\begin{bmatrix} x_{1} \\ x_{2} \\ \vdots \\ x_{n}\end{bmatrix} \\ \text{s.t.} & g_{i}(\mathbf{x})\le 0, i=1,2,\dots,m \\ & h_{j}(\mathbf{x})=0, j=1,2,\dots,\ell \\ \end{array}

The vector \(\mathbf{x}\) is the design variable or optimization variable.

Design space

is the set of all solutions. While under constraints, only a subset of solutions satisfies all constraints. This subset is called feasible region.

For a certain solution \(\mathbf{x}\),

if \( g(\mathbf{x}) \) …	then, the constraint is called …
\(\gt0\)	violated constraint
\(=0\)	active constraint
\(\lt0\)	inactive constraint