PyTorch: Difference between revisions

Revision as of 18:00, 19 April 2025

Revision as of 18:11, 19 April 2025

Line 1:

{{Short description|Machine learning library}}

{{Use dmy dates|date=April 2025}}

{{Infobox software

| name = PyTorch

Line 22 ⟶ 23:

{{Machine learning}}

'''PyTorch''' is a [[machine learning]] [[Library (computing)|library]] based on the [[Torch (machine learning)|Torch]] library,<ref>{{cite news|url=https://www.infoworld.com/article/3159120/artificial-intelligence/facebook-brings-gpu-powered-machine-learning-to-python.html|title=Facebook brings GPU-powered machine learning to Python|last=Yegulalp|first=Serdar|date=19 January 2017|work=InfoWorld|access-date=11 December 2017}}</ref><ref>{{cite web|url=https://www.oreilly.com/ideas/why-ai-and-machine-learning-researchers-are-beginning-to-embrace-pytorch|title=Why AI and machine learning researchers are beginning to embrace PyTorch|last=Lorica|first=Ben|date=3 August 2017|publisher=O'Reilly Media|access-date=11 December 2017}}</ref><ref>{{Cite book|title=Deep Learning with Python|last=Ketkar|first=Nikhil|date=2017|publisher=Apress, Berkeley, CA|isbn=9781484227657|pages=195–208|language=en|doi=10.1007/978-1-4842-2766-4_12|chapter=Introduction to PyTorch}}</ref> used for applications such as [[computer vision]] and [[natural language processing]],<ref>{{Cite web|url=https://www.datacamp.com/tutorial/nlp-with-pytorch-a-comprehensive-guide|title=NLP with PyTorch: A Comprehensive Guide|author=Moez Ali|date=Jun 2023|website=datacamp.com|language=en|access-date=1 April 2024-04-01}}</ref> originally developed by [[Meta AI]] and now part of the [[Linux Foundation]] umbrella.<ref>{{Cite news|url=https://www.oreilly.com/ideas/when-two-trends-fuse-pytorch-and-recommender-systems|title=When two trends fuse: PyTorch and recommender systems|last=Patel|first=Mo|date=7 December 2017-12-07|work=O'Reilly Media|access-date=18 December 2017-12-18|language=en}}</ref><ref>{{Cite news|url=https://techcrunch.com/2017/09/07/facebook-and-microsoft-collaborate-to-simplify-conversions-from-pytorch-to-caffe2/|title=Facebook and Microsoft collaborate to simplify conversions from PyTorch to Caffe2|last=Mannes|first=John|work=[[TechCrunch]]|access-date=18 December 2017-12-18|language=en|quote=FAIR is accustomed to working with PyTorch – a deep learning framework optimized for achieving state of the art results in research, regardless of resource constraints. Unfortunately in the real world, most of us are limited by the computational capabilities of our smartphones and computers.}}</ref><ref>{{Cite web|url=https://venturebeat.com/2017/11/29/tech-giants-are-using-open-source-frameworks-to-dominate-the-ai-community/|title=Tech giants are using open source frameworks to dominate the AI community|last=Arakelyan|first=Sophia|date=29 November 2017-11-29|website=[[VentureBeat]]|language=en-US|access-date=18 December 2017-12-18}}</ref><ref>{{Cite web |title=PyTorch strengthens its governance by joining the Linux Foundation |url=https://pytorch.org/blog/PyTorchfoundation/ |access-date=13 September 2022-09-13 |website=pytorch.org |language=en}}</ref> It is one of the most popular [[deep learning]] frameworks, alongside others such as [[TensorFlow]],<ref>{{Cite web|url=https://github.com/cncf/velocity|title=Top 30 Open Source Projects.|website=Open Source Project Velocity by CNCF|access-date=2023-10-12 October 2023}}</ref> offering [[free and open-source software]] released under the [[modified BSD license]]. Although the [[Python (programming language)|Python]] interface is more polished and the primary focus of development, PyTorch also has a [[C++]] interface.<ref>{{Cite web|url=https://pytorch.org/cppdocs/frontend.html|title=The C++ Frontend|website=PyTorch Master Documentation|access-date=29 July 2019-07-29}}</ref>

A number of pieces of [[deep learning]] software are built on top of PyTorch, including [[Tesla Autopilot]],<ref>{{Cite web|last=Karpathy|first=Andrej|title=PyTorch at Tesla - Andrej Karpathy, Tesla|website=[[YouTube]] |date=6 November 2019 |url=https://www.youtube.com/watch?v=oBklltKXtDE}}</ref> [[Uber]]'s Pyro,<ref>{{Cite news|url=https://eng.uber.com/pyro/|title=Uber AI Labs Open Sources Pyro, a Deep Probabilistic Programming Language|date=3 November 2017-11-03|work=Uber Engineering Blog|access-date=2017-12-18 December 2017|language=en-US}}</ref> [[Hugging Face]]'s Transformers,<ref>{{Citation|title=PYTORCH-TRANSFORMERS: PyTorch implementations of popular NLP Transformers|date=1 December 2019-12-01|url=https://pytorch.org/hub/huggingface_pytorch-transformers/|publisher=PyTorch Hub|access-date=1 December 2019-12-01}}</ref><ref>{{Cite web|url=https://pytorch.org/ecosystem/|title=Ecosystem Tools|website=pytorch.org|language=en|access-date=2020-06-18 June 2020}}</ref> and Catalyst.<ref>{{Citation|title=GitHub - catalyst-team/catalyst: Accelerated DL & RL|date=5 December 2019-12-05|url=https://github.com/catalyst-team/catalyst|publisher=Catalyst-Team|access-date=5 December 2019-12-05}}</ref><ref>{{Cite web|url=https://pytorch.org/ecosystem/|title= Ecosystem Tools|website=pytorch.org|language=en|access-date=4 April 2020-04-04}}</ref>

PyTorch provides two high-level features:<ref>{{cite web |url=https://pytorch.org/about/ |title=PyTorch – About |website=pytorch.org |access-date=2018-06-11 June 2018 |archive-url=https://web.archive.org/web/20180615190804/https://pytorch.org/about/ |archive-date=2018-06-15 June 2018 |url-status=dead }}</ref>

* Tensor computing (like [[NumPy]]) with strong acceleration via [[graphics processing unit]]s (GPU)

* [[Deep neural networks]] built on a tape-based [[automatic differentiation]] system

==History==

Meta (formerly known as Facebook) operates both PyTorch and Convolutional Architecture for Fast Feature Embedding ([[Caffe (software)|Caffe2]]), but models defined by the two frameworks were mutually incompatible. The Open Neural Network Exchange ([[Open Neural Network Exchange|ONNX]]) project was created by Meta and [[Microsoft]] in September 2017 for converting models between frameworks. Caffe2 was merged into PyTorch at the end of March 2018.<ref>{{cite web|url=https://medium.com/@Synced/caffe2-merges-with-pytorch-a89c70ad9eb7|title=Caffe2 Merges With PyTorch|date=2 April 2018-04-02}}</ref> In September 2022, Meta announced that PyTorch would be governed by the independent PyTorch Foundation, a newly created subsidiary of the [[Linux Foundation]].<ref>{{cite web |url=https://arstechnica.com/information-technology/2022/09/meta-spins-off-pytorch-foundation-to-make-ai-framework-vendor-neutral/ |title=Meta spins off PyTorch Foundation to make AI framework vendor neutral |date=2022-09-12 September 2022 |website=[[Ars Technica]] |last=Edwards |first=Benj}}</ref>

PyTorch 2.0 was released on 15 March 2023, introducing [[TorchDynamo]], a Python-level [[compiler]] that makes code run up to 2x faster, along with significant improvements in training and inference performance across major [[cloud computing|cloud platforms]].<ref>{{cite web|title=Dynamo Overview |url=https://pytorch.org/docs/stable/torch.compiler_dynamo_overview.html }}</ref><ref>{{cite news |title=PyTorch 2.0 brings new fire to open-source machine learning |url=https://venturebeat.com/ai/pytorch-2-0-brings-new-fire-to-open-source-machine-learning/ |access-date=16 March 2023 |work=VentureBeat |date=15 March 2023}}</ref>

Line 37 ⟶ 38:

==PyTorch tensors==

{{main|Tensor (machine learning)}}

PyTorch defines a class called Tensor (<code>torch.Tensor</code>) to store and operate on homogeneous multidimensional rectangular arrays of numbers. PyTorch Tensors are similar to [[NumPy]] Arrays, but can also be operated on a [[CUDA]]-capable [[Nvidia|NVIDIA]] GPU. PyTorch has also been developing support for other GPU platforms, for example, AMD's [[ROCm]]<ref>{{cite web|url=https://rocm.docs.amd.com/projects/install-on-linux/en/latest/how-to/3rd-party/pytorch-install.html|title=Installing PyTorch for ROCm|date=9 February 2024-02-09|website=rocm.docs.amd.com}}</ref> and Apple's [[Metal (API)|Metal Framework.]]<ref>{{Cite web |title=Introducing Accelerated PyTorch Training on Mac |url=https://pytorch.org/blog/introducing-accelerated-pytorch-training-on-mac/ |access-date=4 June 2022-06-04 |website=pytorch.org |language=en}}</ref>

PyTorch supports various sub-types of Tensors.<ref>{{cite web |url=https://www.analyticsvidhya.com/blog/2018/02/pytorch-tutorial/ |title=An Introduction to PyTorch – A Simple yet Powerful Deep Learning Library |website=analyticsvidhya.com |access-date=2018-06-11 June 2018|date=2018-02-22 February 2018 }}</ref>

Note that the term "tensor" here does not carry the same meaning as tensor in mathematics or physics. The meaning of the word in machine learning is only superficially related to its original meaning as a certain kind of object in [[linear algebra]]. Tensors in PyTorch are simply multi-dimensional arrays.