Submodular set function

In mathematics, submodular functions are set functions which usually appear in approximation algorithms and in game theory (as functions modeling user preferences). These functions have a natural diminishing returns property which makes them suitable for many applications.

Definition

Submodular function is a set function ${\displaystyle f:2^{\Omega }\rightarrow R}$  which satisfies one of the following equivalent definitions^[1].

1. For every ${\displaystyle X\subseteq Y\subseteq \Omega }$  and ${\displaystyle x\in \Omega \backslash Y}$  we have that ${\displaystyle f(X\cup \{x\})-f(X)\geq f(Y\cup \{x\})-f(Y)}$ 
2. For every ${\displaystyle S,T\subseteq \Omega }$  we have that ${\displaystyle f(S)+f(T)\geq f(S\cup T)+f(S\cap T)}$ 
3. For every ${\displaystyle X\subseteq \Omega }$  and ${\displaystyle x_{1},x_{2}\in \Omega \backslash X}$  we have that ${\displaystyle f(X\cup \{x_{1}\})+f(X\cup \{x_{2}\})\geq f(X\cup \{x_{1},x_{2}\})+f(X)}$ 

A nonnegative submodular function is also a subadditive function, but a subadditive function need not be submodular.

Applications

Types

Monotone Submodular function

A submodular function ${\displaystyle f}$  is said to be monotone if for every ${\displaystyle T\subseteq S}$  we have that ${\displaystyle f(T)\leq f(S)}$ .

Examples of Monotone Submodular function

1. Linear functions

   Any function of the form ${\displaystyle f(S)=\sum _{i\in S}w_{i}}$  is called a linear function. Additionally if ${\displaystyle \forall i,w_{i}\geq 0}$  then f is monotone.

2. Budget-additive functions

   Any function of the form ${\displaystyle f(S)=\min(B,\sum _{i\in S}w_{i})}$  for each ${\displaystyle w_{i}\geq 0}$  and ${\displaystyle B\geq 0}$  is called budget additive.

3. Coverage function

   Let ${\displaystyle \Omega =\{E_{1},E_{2},\ldots ,E_{n}\}}$  be a collection of subsets of some ground set ${\displaystyle \Omega '}$ . The function ${\displaystyle f(S)=|\cup _{E_{i}\in S}E_{i}|}$  for ${\displaystyle S\subseteq \Omega }$  is called a coverage function.

4. Entropy

   Let ${\displaystyle \Omega =\{X_{1},X_{2},\ldots ,X_{n}\}}$  be a set of random variables. Then for any ${\displaystyle S\subseteq \Omega }$  we have that ${\displaystyle H(S)}$  is a submodular function, where ${\displaystyle H(S)}$  is the entropy of the set of random variables ${\displaystyle S}$ 

5. Matroid rank functions

   Let ${\displaystyle \Omega =\{e_{1},e_{2},\dots ,e_{n}\}}$  be the ground set on which a matroid is defined. Then the rank function of the matroid is a submodular function.

Non-monotone Submodular function

A submodular function ${\displaystyle f}$  which is not necessarily monotone is called as Non-monotone Submodular function.

Symmetric Non-monotone Submodular function

A submodular function ${\displaystyle f}$  is called symmetric if for every ${\displaystyle S\subseteq \Omega }$  we have that ${\displaystyle f(S)=f(\Omega -S)}$ 

Examples of Symmetric Non-Monotone Submodular function

1. Graph cuts

   Let ${\displaystyle \Omega =\{v_{1},v_{2},\dots ,v_{n}\}}$  be the vertices of a graph. For any set of vertices ${\displaystyle S\subseteq \Omega }$  let ${\displaystyle f(S)}$  denote the number of edges ${\displaystyle e=(u,v)}$  such that ${\displaystyle u\in S}$  and ${\displaystyle v\in \Omega -S}$ .

2. Mutual information

   Let ${\displaystyle \Omega =\{X_{1},X_{2},\ldots ,X_{n}\}}$  be a set of random variables. Then for any ${\displaystyle S\subseteq \Omega }$  we have that ${\displaystyle f(S)=I(S;\Omega -S)}$  is a submodular function, where ${\displaystyle I(S;\Omega -S)}$  is the mutual information.

Asymmetric Non-monotone Submodular function

A submodular function ${\displaystyle f}$  is called Asymmetric if it is not necessarily symmetric.

Examples of Symmetric Non-Monotone Submodular function

1. Directed graph cuts

   Let ${\displaystyle \Omega =\{v_{1},v_{2},\dots ,v_{n}\}}$  be the vertices of a directed graph. For any set of vertices ${\displaystyle S\subseteq \Omega }$  let ${\displaystyle f(S)}$  denote the number of edges ${\displaystyle e=(u,v)}$  such that ${\displaystyle u\in S}$  and ${\displaystyle v\in \Omega -S}$ .

Continuous extensions

Lovasz extension

This extension has been named after László Lovász. Consider any vector ${\displaystyle {\mathbf {x}}=\{x_{1},x_{2},\dots ,x_{n}\}}$  such that each ${\displaystyle 0\leq x_{i}\leq 1}$ . Then the Lovasz extension is defined as ${\displaystyle f^{L}({\mathbf {x}})=\mathbb {E} (f(\{i|x_{i}\geq \lambda \}))}$  where the expectation is over choosing ${\displaystyle \lambda }$  uniformly in ${\displaystyle [0,1]}$ . It can be shown that Lovasz extension is a convex function.

Multilinear extension

Consider any vector ${\displaystyle {\mathbf {x}}=\{x_{1},x_{2},\ldots ,x_{n}\}}$  such that each ${\displaystyle 0\leq x_{i}\leq 1}$ . Then the multilinear extension is defined as ${\displaystyle F({\mathbf {x}})=\sum _{S\subseteq \Omega }f(S)\prod _{i\in S}x_{i}\prod _{i\notin S}(1-x_{i})}$ 

Convex Closure

Consider any vector ${\displaystyle {\mathbf {x}}=\{x_{1},x_{2},\dots ,x_{n}\}}$  such that each ${\displaystyle 0\leq x_{i}\leq 1}$ . Then the convex closure is defined as ${\displaystyle f^{-}({\mathbf {x} })=\min(\sum _{S}\alpha _{S}f(S):\sum _{S}\alpha _{S}1_{S}={\mathbf {x} },\sum _{S}\alpha _{S}=1,\alpha _{S}\geq 0)}$ . It can be shown that ${\displaystyle f^{L}({\mathbf {x}})=f^{-}({\mathbf {x}})}$ 

Concave Closure

Consider any vector ${\displaystyle {\mathbf {x}}=\{x_{1},x_{2},\dots ,x_{n}\}}$  such that each ${\displaystyle 0\leq x_{i}\leq 1}$ . Then the concave closure is defined as ${\displaystyle f^{+}({\mathbf {x} })=\max(\sum _{S}\alpha _{S}f(S):\sum _{S}\alpha _{S}1_{S}={\mathbf {x} },\sum _{S}\alpha _{S}=1,\alpha _{S}\geq 0)}$ .

Properties

Operations which preserve Submodularity

1. Non-negative linear combinations. Consider any submodular function ${\displaystyle f_{1},f_{2},\ldots ,f_{k}}$  and non negative numbers ${\displaystyle \alpha _{1},\alpha _{2},\ldots ,\alpha _{k}}$ . Then ${\displaystyle g(S)=\sum _{i=1}^{k}\alpha _{i}f_{i}(S)}$  is a submodular function.
2. Consider any monotone submodular function ${\displaystyle f}$  and a non negative number ${\displaystyle c}$ . Then ${\displaystyle g(S)=min(f(S),c)}$  is also a submodular function.
3. Consider any submodular function ${\displaystyle f}$ . Then ${\displaystyle g(S)=f(\Omega -S)}$  is also a submodular function.

Optimization problems

Submodular functions have properties which are very similar to convex and concave functions. Hence a lot of optimization problems can be cast as maximizing or minimizing submodular functions subject to various constraints.

1. Minimization of submodular functions.

   Under the simplest case the problem is to find set ${\displaystyle S\subseteq \Omega }$  which minimizes submodular function subject to no constraints. A series of results ^[2][3][4][5] have established the polynomial time solvability of this problem. Finding minimum cut in a graph is a special case of this problem.

2. Maximization of submodular functions.

   Unlike minimization, maximization of submodular functions is usually NP-hard. A host of problems such as max cut, maximum coverage problem can be cast as special cases of this problem under suitable constraints. Typically the approximation algorithms for these problems are based on either greedy or local search type of algorithms.

   1. Maximizing a Symmetric Non-monotone Submodular function subject to no constraint. This problem admits a 1/2 approximation algorithm^[6]. Finding max cut is a special case of this problem.
   2. Maximizing a Monotone Submodular function subject to cardinality constraint. This problem admits a 1-1/e approximation algorithm^[7]. Maximum coverage problem is a special case of this problem.

See also

• Supermodular function
• Polymatroid
• Matroid

Citations

1. (Schrijver 2003, §44, p. 766)
2. M. Grotschel, L. Lovasz, and A. Schrijver, The ellipsoid method and its consequences in combinatorial optimization,Combinatorica,1 (1981),pp. 169–197.
3. W. H. Cunningham, On submodular function minimization, Combinatorica,5 (1985),pp. 185–192.
4. S. Iwata, L. Fleischer, and S. Fujishige, A combinatorial strongly polynomial algorithm for minimizing submodular functions,J. ACM,48 (2001),pp. 761–777
5. A. Schrijver, A combinatorial algorithm minimizing submodular functions in strongly polynomial time,J. Combin. Theory Ser. B,80 (2000),pp. 346–355.
6. U. Feige, V. Mirrokni and J. Vondr´ak. Maximizing non-monotone submodular functions, Proc. of 48th FOCS (2007), 461–471.
7. G. L. Nemhauser, L. A. Wolsey and M. L. Fisher. An analysis of approximations for maximizing submodular set functions I, Mathematical Programming 14 (1978), 265–294

References

General References

• Schrijver, Alexander (2003), Combinatorial Optimization, Springer, ISBN 3-540-44389-4
• Lee, Jon (2004), A First Course in Combinatorial Optimization, Cambridge University Press, ISBN 0-521-01012-8
• Fujishige, Satoru (2005), Submodular Functions and Optimization, Elsevier, ISBN 0-444-52086-4
• Narayanan, H. (1997), Submodular Functions and Electrical Networks, ISBN 0-444-82523-1

External links