Submodular set function

In mathematics, a submodular set function (also known as a submodular function) is a set function whose value, informally, has the property that the difference in the value of the function that a single element makes when added to an input set decreases as the size of the input set increases. Submodular functions have a natural diminishing returns property which makes them suitable for many applications, including approximation algorithms and game theory (as functions modeling user preferences).

Definition

If ${\displaystyle \Omega }$  is a set, a submodular function is a set function ${\displaystyle f:2^{\Omega }\rightarrow \mathbb {R} }$ , where ${\displaystyle 2^{\Omega }}$  denotes the power set of ${\displaystyle \Omega }$ , which satisfies one of the following equivalent definitions^[1].

1. For every ${\displaystyle X,Y\subseteq \Omega }$  with ${\displaystyle X\subseteq Y}$  and every ${\displaystyle x\in \Omega \backslash Y}$  we have that ${\displaystyle f(X\cup \{x\})-f(X)\geq f(Y\cup \{x\})-f(Y)}$ .
2. For every ${\displaystyle S,T\subseteq \Omega }$  we have that ${\displaystyle f(S)+f(T)\geq f(S\cup T)+f(S\cap T)}$ .
3. For every ${\displaystyle X\subseteq \Omega }$  and ${\displaystyle x_{1},x_{2}\in \Omega \backslash X}$  we have that ${\displaystyle f(X\cup \{x_{1}\})+f(X\cup \{x_{2}\})\geq f(X\cup \{x_{1},x_{2}\})+f(X)}$ .

A nonnegative submodular function is also a subadditive function, but a subadditive function need not be submodular.

Types of submodular functions

Monotone

A submodular function ${\displaystyle f}$  is monotone if for every ${\displaystyle T\subseteq S}$  we have that ${\displaystyle f(T)\leq f(S)}$ . Examples of monotone submodular functions include:

Linear functions

Any function of the form ${\displaystyle f(S)=\sum _{i\in S}w_{i}}$  is called a linear function. Additionally if ${\displaystyle \forall i,w_{i}\geq 0}$  then f is monotone.

Budget-additive functions

Any function of the form ${\displaystyle f(S)=\min(B,\sum _{i\in S}w_{i})}$  for each ${\displaystyle w_{i}\geq 0}$  and ${\displaystyle B\geq 0}$  is called budget additive.

Coverage functions

Let ${\displaystyle \Omega =\{E_{1},E_{2},\ldots ,E_{n}\}}$  be a collection of subsets of some ground set ${\displaystyle \Omega '}$ . The function ${\displaystyle f(S)=|\cup _{E_{i}\in S}E_{i}|}$  for ${\displaystyle S\subseteq \Omega }$  is called a coverage function.

Entropy

Let ${\displaystyle \Omega =\{X_{1},X_{2},\ldots ,X_{n}\}}$  be a set of random variables. Then for any ${\displaystyle S\subseteq \Omega }$  we have that ${\displaystyle H(S)}$  is a submodular function, where ${\displaystyle H(S)}$  is the entropy of the set of random variables ${\displaystyle S}$ 

Matroid rank functions

Let ${\displaystyle \Omega =\{e_{1},e_{2},\dots ,e_{n}\}}$  be the ground set on which a matroid is defined. Then the rank function of the matroid is a submodular function.

Non-monotone

A submodular function which is not monotone is called non-monotone.

Symmetric

A non-monotone submodular function ${\displaystyle f}$  is called symmetric if for every ${\displaystyle S\subseteq \Omega }$  we have that ${\displaystyle f(S)=f(\Omega -S)}$ . Examples of symmetric non-monotone submodular functions include:

Graph cuts

Let ${\displaystyle \Omega =\{v_{1},v_{2},\dots ,v_{n}\}}$  be the vertices of a graph. For any set of vertices ${\displaystyle S\subseteq \Omega }$  let ${\displaystyle f(S)}$  denote the number of edges ${\displaystyle e=(u,v)}$  such that ${\displaystyle u\in S}$  and ${\displaystyle v\in \Omega -S}$ .

Mutual information

Let ${\displaystyle \Omega =\{X_{1},X_{2},\ldots ,X_{n}\}}$  be a set of random variables. Then for any ${\displaystyle S\subseteq \Omega }$  we have that ${\displaystyle f(S)=I(S;\Omega -S)}$  is a submodular function, where ${\displaystyle I(S;\Omega -S)}$  is the mutual information.

Asymmetric

A non-monotone submodular function which is not symmetric is called asymmetric.

For example, a directed graph cut is an asymmetric non-monotone submodular function. Let ${\displaystyle \Omega =\{v_{1},v_{2},\dots ,v_{n}\}}$  be the vertices of a directed graph. For any set of vertices ${\displaystyle S\subseteq \Omega }$  let ${\displaystyle f(S)}$  denote the number of edges ${\displaystyle e=(u,v)}$  such that ${\displaystyle u\in S}$  and ${\displaystyle v\in \Omega -S}$ .

Continuous extensions

Lovász extension

This extension is named after mathematician László Lovász. Consider any vector ${\displaystyle {\mathbf {x}}=\{x_{1},x_{2},\dots ,x_{n}\}}$  such that each ${\displaystyle 0\leq x_{i}\leq 1}$ . Then the Lovász extension is defined as ${\displaystyle f^{L}({\mathbf {x}})=\mathbb {E} (f(\{i|x_{i}\geq \lambda \}))}$  where the expectation is over ${\displaystyle \lambda }$  chosen from the uniform distribution on the interval ${\displaystyle [0,1]}$ . The Lovász extension is a convex function.

Multilinear extension

Consider any vector ${\displaystyle {\mathbf {x}}=\{x_{1},x_{2},\ldots ,x_{n}\}}$  such that each ${\displaystyle 0\leq x_{i}\leq 1}$ . Then the multilinear extension is defined as ${\displaystyle F({\mathbf {x}})=\sum _{S\subseteq \Omega }f(S)\prod _{i\in S}x_{i}\prod _{i\notin S}(1-x_{i})}$ .

Convex closure

Consider any vector ${\displaystyle {\mathbf {x}}=\{x_{1},x_{2},\dots ,x_{n}\}}$  such that each ${\displaystyle 0\leq x_{i}\leq 1}$ . Then the convex closure is defined as ${\displaystyle f^{-}({\mathbf {x} })=\min(\sum _{S}\alpha _{S}f(S):\sum _{S}\alpha _{S}1_{S}={\mathbf {x} },\sum _{S}\alpha _{S}=1,\alpha _{S}\geq 0)}$ . It can be shown that ${\displaystyle f^{L}({\mathbf {x}})=f^{-}({\mathbf {x}})}$ .

Concave closure

Consider any vector ${\displaystyle {\mathbf {x}}=\{x_{1},x_{2},\dots ,x_{n}\}}$  such that each ${\displaystyle 0\leq x_{i}\leq 1}$ . Then the concave closure is defined as ${\displaystyle f^{+}({\mathbf {x} })=\max(\sum _{S}\alpha _{S}f(S):\sum _{S}\alpha _{S}1_{S}={\mathbf {x} },\sum _{S}\alpha _{S}=1,\alpha _{S}\geq 0)}$ .

Closure properties

The class of submodular functions is closed under non-negative linear combinations. Consider any submodular function ${\displaystyle f_{1},f_{2},\ldots ,f_{k}}$  and non-negative numbers ${\displaystyle \alpha _{1},\alpha _{2},\ldots ,\alpha _{k}}$ . Then the function ${\displaystyle g}$  defined by ${\displaystyle g(S)=\sum _{i=1}^{k}\alpha _{i}f_{i}(S)}$  is submodular. Furthermore, for any submodular function ${\displaystyle f}$ , the functions defined by ${\displaystyle g(S)=\min(f(S),c)}$ , where ${\displaystyle c}$  is a non-negative real number and ${\displaystyle g(S)=f(\Omega \setminus S)}$  are also submodular.

Optimization problems

Submodular functions have properties which are very similar to convex and concave functions. For this reason, an optimization problem which concerns optimizing a convex or concave function can also be described as the problem of maximizing or minimizing a submodular function subject to some constraints.

The simplest minimization problem is to find a set ${\displaystyle S\subseteq \Omega }$  which minimizes a submodular function subject to no constraints. This problem is computable in polynomial time.^[2][3][4][5] Computing the minimum cut in a graph is a special case of this general minimization problem.

Unlike minimization, maximization of submodular functions is usually NP-hard. Many problems, such as max cut and the maximum coverage problem, can be cast as special cases of this general maximization problem under suitable constraints. Typically, the approximation algorithms for these problems are based on either greedy algorithms or local search algorithms. The problem of maximizing a symmetric non-monotone submodular function subject to no constraints admits a 1/2 approximation algorithm.^[6] Computing the maximum cut of a graph is a special case of this problem. The more general problem of maximizing an arbitrary non-monotone submodular function subject to no constraints also admits a 1/2 approximation algorithm.^[7] The problem of maximizing a monotone submodular function subject to a cardinality constraint admits a ${\displaystyle 1-1/e}$  approximation algorithm.^[8] The maximum coverage problem is a special case of this problem. The more general problem of maximizing a monotone submodular function subject to a matroid constraint also admits a ${\displaystyle 1-1/e}$  approximation algorithm.^[9][10]

See also

• Supermodular function
• Polymatroid
• Matroid

Citations

1. (Schrijver 2003, §44, p. 766)
2. M. Grötschel, L. Lovasz and A. Schrijver, The ellipsoid method and its consequences in combinatorial optimization, Combinatorica 1 (1981), pp. 169–197.
3. W. H. Cunningham, On submodular function minimization, Combinatorica,5 (1985),pp. 185–192.
4. S. Iwata, L. Fleischer, and S. Fujishige, A combinatorial strongly polynomial algorithm for minimizing submodular functions, J. ACM 48 (2001), pp. 761–777.
5. A. Schrijver, A combinatorial algorithm minimizing submodular functions in strongly polynomial time, J. Combin. Theory Ser. B 80 (2000), pp. 346–355.
6. U. Feige, V. Mirrokni and J. Vondrák, Maximizing non-monotone submodular functions, Proc. of 48th FOCS (2007), pp. 461–471.
7. N. Buchbinder, M. Feldman, J. Naor and R. Schwartz, A tight linear time (1/2)-approximation for unconstrained submodular maximization, Proc. of 53rd FOCS (2012), pp. 649-658.
8. G. L. Nemhauser, L. A. Wolsey and M. L. Fisher, An analysis of approximations for maximizing submodular set functions I, Mathematical Programming 14 (1978), 265–294.
9. G. Calinescu, C. Chekuri, M. Pál and J. Vondrák, Maximizing a submodular set function subject to a matroid constraint, SIAM J. Comp. 40:6 (2011), 1740-1766.
10. Y. Filmus, J. Ward, A tight combinatorial algorithm for submodular maximization subject to a matroid constraint, Proc. of 53rd FOCS (2012), pp. 659-668.

References

• Schrijver, Alexander (2003), Combinatorial Optimization, Springer, ISBN 3-540-44389-4
• Lee, Jon (2004), A First Course in Combinatorial Optimization, Cambridge University Press, ISBN 0-521-01012-8
• Fujishige, Satoru (2005), Submodular Functions and Optimization, Elsevier, ISBN 0-444-52086-4
• Narayanan, H. (1997), Submodular Functions and Electrical Networks, ISBN 0-444-82523-1