Optimization problem: Difference between revisions

Revision as of 10:05, 6 April 2021

Latest revision as of 15:30, 10 May 2025

(11 intermediate revisions by 9 users not shown)

Line 1:

{{Short description|Problem of finding the best feasible solution}}

{{Broader|Mathematical optimisationoptimization}}

In [[mathematics]], [[engineering]], [[computer science]] and [[economics]], an '''optimisationoptimization problem''' is the [[Computational problem|problem]] of finding the ''best'' solution from all [[feasible solution]]s.

OptimisationOptimization problems can be divided into two categories, depending on whether the [[Variable (mathematics)|variables]] are [[continuous variable|continuous]] or [[discrete variable|discrete]]:

* An optimisationoptimization problem with discrete variables is known as a ''[[discrete optimisationoptimization]]'', in which an [[Mathematical object|object]] such as an [[integer]], [[permutation]] or [[Graph (discrete mathematics)|graph]] must be found from a [[countable set]].

* A problem with continuous variables is known as a ''[[continuous optimisationoptimization]]'', in which an optimal value from a [[continuous function]] must be found. They can include [[Constrained optimisationoptimization|constrained problem]]s and multimodal problems.

== Search space ==

==Continuous optimisation problem==

In the context of an optimization problem, the '''search space''' refers to the set of all possible points or solutions that satisfy the problem's constraints, targets, or goals.<ref>{{Cite web |title=Search Space |url=https://courses.cs.washington.edu/courses/cse473/06sp/GeneticAlgDemo/searchs.html |access-date=2025-05-10 |website=courses.cs.washington.edu}}</ref> These points represent the feasible solutions that can be evaluated to find the optimal solution according to the objective function. The search space is often defined by the ___domain of the function being optimized, encompassing all valid inputs that meet the problem's requirements.<ref>{{Cite web |date=2020-09-22 |title=Search Space - LessWrong |url=https://www.lesswrong.com/w/search-space |access-date=2025-05-10 |website=www.lesswrong.com |language=en}}</ref>

The ''[[Canonical form|standard form]]'' of a [[Continuity (mathematics)|continuous]] optimisation problem is<ref>{{cite book|title=Convex Optimisation|first1=Stephen P.|last1=Boyd|first2=Lieven|last2=Vandenberghe|page=129|year=2004|publisher=Cambridge University Press|isbn=978-0-521-83378-3|url=https://web.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf#page=143|format=pdf}}</ref>

: <math>\begin{align}

The search space can vary significantly in size and complexity depending on the problem. For example, in a continuous optimization problem, the search space might be a multidimensional real-valued ___domain defined by bounds or constraints. In a discrete optimization problem, such as combinatorial optimization, the search space could consist of a finite set of permutations, combinations, or configurations.

In some contexts, the term ''search space'' may also refer to the optimization of the ___domain itself, such as determining the most appropriate set of variables or parameters to define the problem. Understanding and effectively navigating the search space is crucial for designing efficient algorithms, as it directly influences the computational complexity and the likelihood of finding an optimal solution.

==Continuous optimisationoptimization problem==

The ''[[Canonical form|standard form]]'' of a [[Continuity (mathematics)|continuous]] optimisationoptimization problem is<ref>{{cite book|title=Convex OptimisationOptimization|first1=Stephen P.|last1=Boyd|first2=Lieven|last2=Vandenberghe|page=129|year=2004|publisher=Cambridge University Press|isbn=978-0-521-83378-3|url=https://web.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf#page=143|format=pdf}}</ref>

: <math display=block>\begin{align}

&\underset{x}{\operatorname{minimize}}& & f(x) \\

&\operatorname{subject\;to}

Line 16 ⟶ 25:

\end{align}</math>

where

* {{math|''f'' : [[Euclidean space|ℝ^''n'']] → [[Real numbers|ℝ]]}} is the '''[[Loss function|objective function]]''' to be minimized over the {{mvar|n}}-variable vector {{mvar|x}},

* {{math|''g_i''(''x'') ≤ 0}} are called '''inequality [[Constraint (mathematics)|constraints]]'''

* {{math|''h_j''(''x'') {{=}} 0}} are called '''equality constraints''', and

* {{math|''m'' ≥ 0}} and {{math|''p'' ≥ 0}}.

If {{math|''m'' {{=}} ''p'' {{=}} 0}}, the problem is an unconstrained optimisationoptimization problem. By convention, the standard form defines a '''minimisationminimization problem'''. A '''maximisationmaximization problem''' can be treated by [[Additive inverse|negating]] the objective function.

==Combinatorial optimisationoptimization problem==

{{Main|Combinatorial optimization}}

Formally, a [[combinatorial optimisationoptimization]] problem {{mvar|A}} is a quadruple{{Citation needed|date=January 2018}} {{math|(''I'', ''f'', ''m'', ''g'')}}, where

* {{math|I}} is a [[Set (mathematics)|set]] of instances;

* given an instance {{math|''x'' ∈ ''I''}}, {{math|''f''(''x'')}} is the set of feasible solutions;

Line 32 ⟶ 42:

The goal is then to find for some instance {{mvar|x}} an ''optimal solution'', that is, a feasible solution {{mvar|y}} with

: <math display=block>m(x, y) = g \biglleft\{ m(x, y') \mid: y' \in f(x) \bigrright\} .</math>

: <math>m(x, y) = g \bigl\{ m(x, y') \mid y' \in f(x) \bigr\} .</math>

For each combinatorial optimization problem, there is a corresponding [[decision problem]] that asks whether there is a feasible solution for some particular measure {{math|''m''₀}}. For example, if there is a [[Graph (discrete mathematics)|graph]] {{mvar|G}} which contains vertices {{mvar|u}} and {{mvar|v}}, an optimization problem might be "find a path from {{mvar|u}} to {{mvar|v}} that uses the fewest edges". This problem might have an answer of, say, 4. A corresponding decision problem would be "is there a path from {{mvar|u}} to {{mvar|v}} that uses 10 or fewer edges?" This problem can be answered with a simple 'yes' or 'no'.

In the field of [[approximation algorithm]]s, algorithms are designed to find near-optimal solutions to hard problems. The usual decision version is then an inadequate definition of the problem since it only specifies acceptable solutions. Even though we could introduce suitable decision problems, the problem is more naturally characterisedcharacterized as an optimisationoptimization problem.<ref name=Ausiello03>{{citation

| last1 = Ausiello | first1 = Giorgio

| year = 2003

Line 46 ⟶ 55:

|display-authors=etal}}</ref>

==See also==

*[[ {{annotated link|Counting problem (complexity)]] }}

*[[ {{annotated link|Design Optimization]] }}

*[[Function problem]]

* {{annotated link|Ekeland's variational principle}}

*[[Glove problem]]

*[[ {{annotated link|Function problem]]}}

*[[Operations research]]

*[[ {{annotated link|Glove problem]]}}

*[[Satisficing]]: the optimum need not be found, just a "good enough" solution.

*[[ {{annotated link|Operations research]] }}

*[[Search problem]]

*[[ {{annotated link|Satisficing]]:}} − the optimum need not be found, just a "good enough" solution.

*[[Semi-infinite programming]]

*[[ {{annotated link|Search problem]] }}

*[[ {{annotated link|Semi-infinite programming]]}}

==References==

{{reflist}}

==External links==

*{{cite web |title=How Traffic Shaping Optimizes Network Bandwidth |work=IPC |date=12 July 2016 |access-date=13 February 2017 |url=https://www.ipctech.com/how-traffic-shaping-optimizes-network-bandwidth }}

* {{cite web |title=How Traffic Shaping Optimizes Network Bandwidth |work=IPC |date=12 July 2016 |access-date=13 February 2017 |url=https://www.ipctech.com/how-traffic-shaping-optimizes-network-bandwidth }}

{{Convex analysis and variational analysis}}

{{Authority control}}