Neural operators: Difference between revisions

Revision as of 03:38, 19 November 2023

Revision as of 04:01, 19 November 2023

Line 11:

'''Neural operators''' are a class of [[deep learning]] architectures designed to learn maps between infinite-dimensional [[Function space|function spaces]]. Neural operators represent an extension of traditional [[Artificial neural network|artificial neural networks]], marking a departure from the typical focus on learning mappings between finite-dimensional Euclidean spaces or finite sets. Neural operators directly learn [[Operator (mathematics)|operators]] between function spaces; they can receive input functions, and the output function can be evaluated at any discretization.<ref name="NO journal">{{cite journal |last1=Kovachki |first1=Nikola |last2=Li |first2=Zongyi |last3=Liu |first3=Burigede |last4=Azizzadenesheli |first4=Kamyar |last5=Bhattacharya |first5=Kaushik |last6=Stuart |first6=Andrew |last7=Anandkumar |first7=Anima |title=Neural operator: Learning maps between function spaces |journal=Journal of Machine Learning Research |date=2021 |volume=24 |page=1-97 |arxiv=2108.08481 |url=https://www.jmlr.org/papers/volume24/21-1524/21-1524.pdf}}</ref>

The primary application of neural operators is in learning surrogate maps for the solution operators of [[Partial differential equation|partial differential equations]] (PDEs)<ref name="NO journal" />, which are critical tools in modeling the natural environment.<ref name="Evans"> {{cite book |author-link=Lawrence C. Evans |first=L. C. |last=Evans |title=Partial Differential Equations |publisher=American Mathematical Society |___location=Providence |year=1998 |isbn=0-8218-0772-2 }}</ref> <ref> X, S. (2023, September 6). How ai models are transforming weather forecasting: A showcase of data-driven systems. Phys.org. https://phys.org/news/2023-09-ai-weather-showcase-data-driven.html </ref> Standard PDE solvers can be time-consuming and computationally intensive, especially for complex systems. Neural operators have demonstrated improved performance in solving PDEs <ref>Kadri Umay, Y. O. (2023, September 20). Microsoft and&nbsp;accenture&nbsp;partner to tackle methane emissions with AI technology. Microsoft Azure Blog. https://azure.microsoft.com/en-us/blog/microsoft-and-accenture-partner-to-tackle-methane-emissions-with-ai-technology/ </ref> compared to existing machine learning methodologies while being significantly faster than numerical solvers.<ref name="FNO">{{cite arXiv |last1=Li |first1=Zongyi |last2=Kovachki |first2=Nikola |last3=Azizzadenesheli |first3=Kamyar |last4=Liu |first4=Burigede |last5=Bhattacharya |first5=Kaushik |last6=Stuart |first6=Andrew |last7=Anima |first7=Anandkumar |title=Fourier neural operator for parametric partial differential equations |date=2020 |class=cs.LG |eprint=2010.08895 }}</ref><ref>Hao, K. (2021, October 20). Ai has cracked a key mathematical puzzle for understanding our world. MIT Technology Review. https://www.technologyreview.com/2020/10/30/1011435/ai-fourier-neural-network-cracks-navier-stokes-and-partial-differential-equations/ </ref><ref> Ananthaswamy, A., &amp; Quanta Magazine moderates comments to&nbsp;facilitate an informed, substantive. (2021, September 10). Latest neural nets solve world’s hardest equations faster than ever before. Quanta Magazine. https://www.quantamagazine.org/latest-neural-nets-solve-worlds-hardest-equations-faster-than-ever-before-20210419/ </ref>. TheyNeural operators have also revolutionizedbeen applied to various scientific and engineering disciplines with their applicationssuch inas turbulent flow modeling, computational mechanics, graph-structured data <ref>Sharma, A., Singh, S. & Ratna, S. Graph Neural Network Operators: a Review. Multimed Tools Appl (2023). https://doi.org/10.1007/s11042-023-16440-4

</ref>, geoscienceand the geosciences.​<ref> Gege Wen, Zongyi Li, Kamyar Azizzadenesheli, Anima Anandkumar, Sally M. Benson,

U-FNO—An enhanced Fourier neural operator-based deep-learning model for multiphase flow,

Advances in Water Resources,

Line 21:

https://doi.org/10.1016/j.advwatres.2022.104180.

(https://www.sciencedirect.com/science/article/pii/S0309170822000562)

</ref>, andIn weatherparticular, forecasting.they Theyhave excelbeen inapplied tasks like modeling turbulent flows with unprecedented speed and accuracy,to learning stress-strain fields in materials, classifying complex data like spatial transcriptomics, predicting multiphase flow in porous media <ref> Choubineh A, Chen J, Wood DA, Coenen F, Ma F. Fourier Neural Operator for Fluid Flow in Small-Shape 2D Simulated Porous Media Dataset. Algorithms. 2023; 16(1):24. https://doi.org/10.3390/a16010024

</ref>, and improving climate modeling through advancedlong-term weather forecasting <ref> Yang, Q., Hernandez-Garcia, A., Harder, P., Ramesh, V., Sattegeri, P., Szwarcman, D., ... & Rolnick, D. (2023). Fourier Neural Operators for Arbitrary Resolution Climate Data Downscaling. arXiv preprint arXiv:2305.14452.</ref> and carbon dioxide migration simulations. Finally, the operator learning paradigm allows learning maps between function spaces, and is different from parallel ideas of learning maps from finite-dimensional spaces to function spaces<ref name="meshfreeflownet">{{cite journal | vauthors=((Esmaeilzadeh, S., Azizzadenesheli, K., Kashinath, K., Mustafa, M., Tchelepi, H. A., Marcus, P., Prabhat, M., Anandkumar, A., others)) | title=Meshfreeflownet: A physics-constrained deep continuous space-time super-resolution framework | pages=1–15 | publisher=IEEE | date=19 October 2020| arxiv=2005.01463 }}</ref><ref name="deeponet">{{cite journal | vauthors=((Lu, L., Jin, P., Pang, G., Zhang, Z., Karniadakis, G. E.)) | title=Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators | volume=3 | issue=3 | pages=218–229 | publisher=Nature Publishing Group UK London | date=19 October 2021}}</ref>, and subsumes these settings when limited to fixed input resolution.

== Operator learning ==