Optimization

KKT Condition

KKT condition is the necessary & sufficient conditions for inequality constrained problems. Reference

1. Unconstraint Problem

Suppose the objective $f(x)$ is a convex function, then the necessary and sufficient condition is \(\nabla f(x^*) =0\) In a more general setting,

2. Equality Constrained Problem

\[\begin{aligned} &\min_{x \in R^{n}} f(x) \\\\ &\text{s.t. } c_{i}(x)=0, i=1, \ldots, m, \quad m \leq n \end{aligned}\]

3. Inequality Constrainted Problem

\[\begin{aligned} &\min_{x \in R^{n}} f(x) \\\\ &\text{s.t. } c_{i}(x) \geq 0, i=1, \ldots, m, \quad m \leq n \end{aligned}\] \[\min_{x \in R^{n}} f(x) \\\\ \text{s.t. } c_{i}(x) \geq 0, i=1, \ldots, m, \quad m \leq n\]