Neural operators: Difference between revisions

Revision as of 03:18, 21 January 2024

Revision as of 12:29, 22 March 2024

Line 4:

'''Neural operators''' are a class of [[deep learning]] architectures designed to learn maps between infinite-dimensional [[function space]]s.<ref name="patel1">{{cite arXiv |last1=Patel |first1=Ravi G. |last2=Desjardins |first2=Olivier |title=Nonlinear integro-differential operator regression with neural networks |date=2018 |class=cs.LG |eprint=1810.08552}}</ref> Neural operators represent an extension of traditional [[artificial neural network]]s, marking a departure from the typical focus on learning mappings between finite-dimensional Euclidean spaces or finite sets. Neural operators directly learn [[Operator (mathematics)|operators]] between function spaces; they can receive input functions, and the output function can be evaluated at any discretization.<ref name="NO journal">{{cite journal |last1=Kovachki |first1=Nikola |last2=Li |first2=Zongyi |last3=Liu |first3=Burigede |last4=Azizzadenesheli |first4=Kamyar |last5=Bhattacharya |first5=Kaushik |last6=Stuart |first6=Andrew |last7=Anandkumar |first7=Anima |title=Neural operator: Learning maps between function spaces |journal=Journal of Machine Learning Research |date=2021 |volume=24 |pages=1–97 |arxiv=2108.08481 |url=https://www.jmlr.org/papers/volume24/21-1524/21-1524.pdf}}</ref>

The primary application of neural operators is in learning surrogate maps for the solution operators of [[partial differential equation]]s (PDEs),<ref name="NO journal" /> which are critical tools in modeling the natural environment.<ref name="Evans">{{cite book |author-link=Lawrence C. Evans |first=L. C. |last=Evans |title=Partial Differential Equations |publisher=American Mathematical Society |___location=Providence |year=1998 |isbn=0-8218-0772-2 }}</ref><ref>X,{{cite S.press (2023, September 6).release |title=How aiAI models are transforming weather forecasting: A showcase of data-driven systems. Phys.org. |url=https://phys.org/news/2023-09-ai-weather-showcase-data-driven.html |work=phys.org |publisher=European Centre for Medium-Range Weather Forecasts |date=6 September 2023 }}</ref> Standard PDE solvers can be time-consuming and computationally intensive, especially for complex systems. Neural operators have demonstrated improved performance in solving PDEs <ref>Kadri{{cite Umay,news Y.|last1=Russ O.|first1=Dan (2023,|last2=Abinader September 20).|first2=Sacha |title=Microsoft and&nbsp;accenture&nbsp; Accenture partner to tackle methane emissions with AI technology. Microsoft Azure Blog. |url=https://azure.microsoft.com/en-us/blog/microsoft-and-accenture-partner-to-tackle-methane-emissions-with-ai-technology/ |work=Microsoft Azure Blog |date=23 August 2023 }}</ref> compared to existing machine learning methodologies while being significantly faster than numerical solvers.<ref name="patel2">{{cite journal |last1=Patel |first1=Ravi G. |last2=Trask |first2=Nathaniel A. |last3=Wood |first3=Mitchell A. |last4=Cyr |first4=Eric C. |title=A physics-informed operator regression framework for extracting data-driven continuum models |journal=Computer Methods in Applied Mechanics and Engineering |date=January 2021 |volume=373 |pages=113500 |doi=10.1016/j.cma.2020.113500|arxiv=2009.11992 |bibcode=2021CMAME.373k3500P }}</ref><ref name="FNO">{{cite arXiv |last1=Li |first1=Zongyi |last2=Kovachki |first2=Nikola |last3=Azizzadenesheli |first3=Kamyar |last4=Liu |first4=Burigede |last5=Bhattacharya |first5=Kaushik |last6=Stuart |first6=Andrew |last7=Anima |first7=Anandkumar |title=Fourier neural operator for parametric partial differential equations |date=2020 |class=cs.LG |eprint=2010.08895 }}</ref><ref>Hao,{{cite K.news (2021,|last1=Hao October|first1=Karen 20). Ai|title=AI has cracked a key mathematical puzzle for understanding our world. MIT Technology Review. |url=https://www.technologyreview.com/2020/10/30/1011435/ai-fourier-neural-network-cracks-navier-stokes-and-partial-differential-equations/ |work=MIT Technology Review |date=30 October 2020 }}</ref><ref>Ananthaswamy,{{cite A.,news &amp;|last1=Ananthaswamy Quanta Magazine moderates comments to&nbsp;facilitate an informed, substantive. (2021, September 10).|first1=Anil |title=Latest neuralNeural netsNets solveSolve world’sWorld’s hardestHardest equationsEquations fasterFaster thanThan everEver before. Quanta Magazine.Before |url=https://www.quantamagazine.org/latest-neural-nets-solve-worlds-hardest-equations-faster-than-ever-before-20210419/ |work=Quanta Magazine |date=19 April 2021 }}</ref> Neural operators have also been applied to various scientific and engineering disciplines such as turbulent flow modeling, computational mechanics, graph-structured data,<ref>{{cite journal |last1=Sharma, A.,|first1=Anuj |last2=Singh, S. &|first2=Sukhdeep |last3=Ratna, |first3=S. |title=Graph Neural Network Operators: a Review. Multimed|journal=Multimedia Tools Appland Applications |date=15 August (2023). https://|volume=83 |issue=8 |pages=23413–23436 |doi.org/=10.1007/s11042-023-16440-4 }}</ref> and the geosciences.<ref>{{cite journal |last1=Wen |first1=Gege |last2=Li |first2=Zongyi |last3=Azizzadenesheli |first3=Kamyar |last4=Anandkumar |first4=Anima |last5=Benson |first5=Sally M. |title=U-FNO—An enhanced Fourier neural operator-based deep-learning model for multiphase flow |journal=Advances in Water Resources |date=May 2022 |volume=163 |pages=104180 |doi=10.1016/j.advwatres.2022.104180 }}</ref> In particular, they have been applied to learning stress-strain fields in materials, classifying complex data like spatial transcriptomics, predicting multiphase flow in porous media,<ref>{{cite journal |last1=Choubineh |first1=Abouzar |last2=Chen |first2=Jie |last3=Wood |first3=David A. |last4=Coenen |first4=Frans |last5=Ma |first5=Fei |title=Fourier Neural Operator for Fluid Flow in Small-Shape 2D Simulated Porous Media Dataset |journal=Algorithms |date=2023 |volume=16 |issue=1 |pages=24 |doi=10.3390/a16010024 }}</ref> and carbon dioxide migration simulations. Finally, the operator learning paradigm allows learning maps between function spaces, and is different from parallel ideas of learning maps from finite-dimensional spaces to function spaces,<ref name="meshfreeflownet">{{cite journal |doi=10.1109/SC41405.2020.00013 }}</ref><ref name="deeponet">{{cite journal |last1=Lu |first1=Lu |last2=Jin |first2=Pengzhan |last3=Pang |first3=Guofei |last4=Zhang |first4=Zhongqiang |last5=Karniadakis |first5=George Em |title=Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators |journal=Nature Machine Intelligence |date=18 March 2021 |volume=3 |issue=3 |pages=218–229 |doi=10.1038/s42256-021-00302-5 }}</ref> and subsumes these settings when limited to fixed input resolution.

</ref> and the geosciences.<ref>Gege Wen, Zongyi Li, Kamyar Azizzadenesheli, Anima Anandkumar, Sally M. Benson,

U-FNO—An enhanced Fourier neural operator-based deep-learning model for multiphase flow,

Advances in Water Resources,

Volume 163,

2022,

104180,

ISSN 0309-1708,

https://doi.org/10.1016/j.advwatres.2022.104180.

(https://www.sciencedirect.com/science/article/pii/S0309170822000562)

</ref> In particular, they have been applied to learning stress-strain fields in materials, classifying complex data like spatial transcriptomics, predicting multiphase flow in porous media,<ref>Choubineh A, Chen J, Wood DA, Coenen F, Ma F. Fourier Neural Operator for Fluid Flow in Small-Shape 2D Simulated Porous Media Dataset. Algorithms. 2023; 16(1):24. https://doi.org/10.3390/a16010024

</ref> and climate modeling through long-term weather forecasting<ref>Yang, Q., Hernandez-Garcia, A., Harder, P., Ramesh, V., Sattegeri, P., Szwarcman, D., ... & Rolnick, D. (2023). Fourier Neural Operators for Arbitrary Resolution Climate Data Downscaling. arXiv preprint arXiv:2305.14452.</ref> and carbon dioxide migration simulations. Finally, the operator learning paradigm allows learning maps between function spaces, and is different from parallel ideas of learning maps from finite-dimensional spaces to function spaces,<ref name="meshfreeflownet">{{cite journal | vauthors=((Esmaeilzadeh, S., Azizzadenesheli, K., Kashinath, K., Mustafa, M., Tchelepi, H. A., Marcus, P., Prabhat, M., Anandkumar, A., others)) | title=Meshfreeflownet: A physics-constrained deep continuous space-time super-resolution framework | pages=1–15 | publisher=IEEE | date=19 October 2020| arxiv=2005.01463 }}</ref><ref name="deeponet">{{cite journal | vauthors=((Lu, L., Jin, P., Pang, G., Zhang, Z., Karniadakis, G. E.)) | title=Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators | volume=3 | issue=3 | pages=218–229 | publisher=Nature Publishing Group UK London | date=19 October 2021}}</ref> and subsumes these settings when limited to fixed input resolution.

== Operator learning ==