Content deleted Content added
m v2.05b - Bot T5 CW#16 - Fix errors for CW project (Unicode control characters - Reference before punctuation) |
→Definition and formulation: codimension is wrong here. |
||
| (21 intermediate revisions by 14 users not shown) | |||
Line 1:
{{Short description|Machine learning framework}}
'''Neural operators''' are a class of [[deep learning]] architectures designed to learn maps between infinite-dimensional [[
The primary application of neural operators is in learning surrogate maps for the solution operators of [[
== Operator learning ==
Understanding and mapping relationships between function spaces has many applications in engineering and the sciences. In particular, [[Abstract differential equation|one can cast the problem]] of solving partial differential equations as identifying a map between function spaces, such as from an initial condition to a time-evolved state. In other PDEs this map takes an input coefficient function and outputs a solution function. Operator learning is a [[machine learning]] paradigm to learn solution operators mapping the input function to the output function.
Using traditional machine learning methods, addressing this problem would involve discretizing the infinite-dimensional input and output function spaces into finite-dimensional grids and applying standard learning models, such as neural networks. This approach reduces the operator learning to finite-dimensional function learning and has some limitations, such as generalizing to discretizations beyond the grid used in training.
Line 24 ⟶ 13:
== Definition and formulation ==
Architecturally, neural operators are similar to feed-forward neural networks in the sense that they are composed of alternating [[
Neural operators seek to approximate some operator <math>\mathcal{G} : \mathcal{A} \to \mathcal{U}</math> between function spaces <math>\mathcal{A}</math> and <math>\mathcal{U}</math> by building a parametric map <math>\mathcal{G}_\phi : \mathcal{A} \to \mathcal{U}</math>. Such parametric maps <math>\mathcal{G}_\phi</math> can generally be defined in the form
Line 30 ⟶ 19:
<math>\mathcal{G}_\phi := \mathcal{Q} \circ \sigma(W_T + \mathcal{K}_T + b_T) \circ \cdots \circ \sigma(W_1 + \mathcal{K}_1 + b_1) \circ \mathcal{P},</math>
where <math>\mathcal{P}, \mathcal{Q}</math> are the lifting (lifting the codomain of the input function to a higher dimensional space) and projection (projecting the codomain of the intermediate function to the output
<math>(\mathcal{K}_\phi v_t)(x) := \int_D \kappa_\phi(x, y, v_t(x), v_t(y))v_t(y)dy, </math>
Line 44 ⟶ 33:
<math>v_{t+1}(x) \approx \sigma\left(\sum_j^n \kappa_\phi(x, y_j, v_t(x), v_t(y_j))v_t(y_j)\Delta_{y_j} + W_t(v_t(y_j)) + b_t(x)\right).</math>
The above approximation, along with parametrizing <math>\kappa_\phi</math> as an implicit neural network, results in the graph neural operator (GNO).<ref name="Graph NO">{{cite arXiv |last1=Li |first1=Zongyi |last2=Kovachki |first2=Nikola |last3=Azizzadenesheli |first3=Kamyar |last4=Liu |first4=Burigede |last5=Bhattacharya |first5=Kaushik |last6=Stuart |first6=Andrew |last7=Anima |first7=Anandkumar |title=Neural operator: Graph kernel network for partial differential equations |date=2020 |class=cs.LG |eprint=2003.03485 }}</ref>
There have been various parameterizations of neural operators for different applications.<ref name="FNO" /><ref name="Graph NO" /> These typically differ in their parameterization of <math>\kappa</math>. The most popular instantiation is the Fourier neural operator (FNO). FNO takes <math>\kappa_\phi(x, y, v_t(x), v_t(y)) := \kappa_\phi(x-y)</math> and by applying the [[convolution theorem]], arrives at the following parameterization of the kernel integral operator:
Line 59 ⟶ 48:
where <math>\|\cdot \|_\mathcal{U}</math> is a norm on the output function space <math>\mathcal{U}</math>. Neural operators can be trained directly using [[backpropagation]] and [[gradient descent]]-based methods.
Another training paradigm is associated with physics-informed machine learning. In particular, [[physics-informed neural networks]] (PINNs) use complete physics laws to fit neural networks to solutions of PDEs. Extensions of this paradigm to operator learning are broadly called physics-informed neural operators (PINO),<ref name="PINO">{{cite arXiv |last1=Li |first1=Zongyi | last2=Hongkai| first2=Zheng |last3=Kovachki |first3=Nikola | last4=Jin | first4=David | last5=Chen | first5= Haoxuan |last6=Liu |first6=Burigede | last7=Azizzadenesheli |first7=Kamyar |last8=Anima |first8=Anandkumar |title=Physics-Informed Neural Operator for Learning Partial Differential Equations |date=2021 |class=cs.LG |eprint=2111.03794 }}</ref>
== See also ==
* [[Neural network (machine learning)|Neural network]]
* [[Physics-informed neural networks]]
* [[Neural field]]
== References ==
Line 67 ⟶ 62:
== External links ==
*[https://github.com/neuraloperator/neuraloperator/ neuralop] – Python library of various neural operator architectures
[[Category:Deep learning]]
| |||