Content deleted Content added
m →History: spell out acronym, add link, improve wording |
|||
| (5 intermediate revisions by 5 users not shown) | |||
Line 23:
{{Machine learning}}
'''PyTorch''' is
PyTorch utilises [[tensor]]s as a intrinsic datatype, very similar to [[NumPy]]. Model training is handled by an [[automatic differentiation]] system, Autograd, which constructs a [[directed acyclic graph]] of a forward pass of a model for a given input, for which automatic differentiation utilising the [[chain rule]], computes model-wide gradients.<ref>{{Cite web|title=Overview of PyTorch Autograd Engine|website=PyTorch Blog|date=8 June 2021|url=https://pytorch.org/blog/overview-of-pytorch-autograd-engine|url-status=live}}</ref> PyTorch is capable of transparent leveraging of [[SIMD]] units, such as [[General-purpose computing on graphics processing units|GPGPU]]s.
A number of pieces of [[deep learning]] software are built on top of PyTorch, including [[Tesla Autopilot]],<ref>{{Cite web|last=Karpathy|first=Andrej|title=PyTorch at Tesla - Andrej Karpathy, Tesla|website=[[YouTube]]|date=6 November 2019|url=https://www.youtube.com/watch?v=oBklltKXtDE|access-date=2 June 2020|archive-date=24 March 2023|archive-url=https://web.archive.org/web/20230324144838/https://www.youtube.com/watch?v=oBklltKXtDE|url-status=live}}</ref> [[Uber]]'s Pyro,<ref>{{Cite news|url=https://eng.uber.com/pyro/|title=Uber AI Labs Open Sources Pyro, a Deep Probabilistic Programming Language|date=3 November 2017|work=Uber Engineering Blog|access-date=18 December 2017|language=en-US|archive-date=25 December 2017|archive-url=https://web.archive.org/web/20171225034106/https://eng.uber.com/pyro/|url-status=live}}</ref> [[Hugging Face]]'s Transformers,<ref>{{Citation|title=PYTORCH-TRANSFORMERS: PyTorch implementations of popular NLP Transformers|date=1 December 2019|url=https://pytorch.org/hub/huggingface_pytorch-transformers/|publisher=PyTorch Hub|access-date=1 December 2019|archive-date=11 June 2023|archive-url=https://web.archive.org/web/20230611061047/https://pytorch.org/hub/huggingface_pytorch-transformers/|url-status=live}}</ref><ref>{{Cite web|url=https://pytorch.org/ecosystem/|title=Ecosystem Tools|website=pytorch.org|language=en|access-date=18 June 2020|archive-date=18 July 2023|archive-url=https://web.archive.org/web/20230718105354/https://pytorch.org/ecosystem/|url-status=live}}</ref> and Catalyst.<ref>{{Citation|title=GitHub - catalyst-team/catalyst: Accelerated DL & RL|date=5 December 2019|url=https://github.com/catalyst-team/catalyst|publisher=Catalyst-Team|access-date=5 December 2019|archive-date=22 December 2019|archive-url=https://web.archive.org/web/20191222162045/https://github.com/catalyst-team/catalyst|url-status=live}}</ref><ref>{{Cite web|url=https://pytorch.org/ecosystem/|title=Ecosystem Tools|website=pytorch.org|language=en|access-date=4 April 2020|archive-date=18 July 2023|archive-url=https://web.archive.org/web/20230718105354/https://pytorch.org/ecosystem/|url-status=live}}</ref>▼
▲A number of
==History==
In 2001, Torch was written and released under a [[GNU General Public License|GPL license]]. It was a machine-learning library written in C++, supporting methods including neural networks, [[Support vector machine|
Meta (formerly known as Facebook) operates both PyTorch and Convolutional Architecture for Fast Feature Embedding ([[Caffe (software)|Caffe2]]), but models defined by the two frameworks were mutually incompatible. The Open Neural Network Exchange ([[Open Neural Network Exchange|ONNX]]) project was created by Meta and [[Microsoft]] in September 2017 for converting models between frameworks. Caffe2 was merged into PyTorch at the end of March 2018.<ref>{{cite web|url=https://medium.com/@Synced/caffe2-merges-with-pytorch-a89c70ad9eb7|title=Caffe2 Merges With PyTorch|date=2 April 2018|access-date=2 January 2019|archive-date=30 March 2019|archive-url=https://web.archive.org/web/20190330143500/https://medium.com/@Synced/caffe2-merges-with-pytorch-a89c70ad9eb7|url-status=live}}</ref> In September 2022, Meta announced that PyTorch would be governed by the independent PyTorch Foundation, a newly created subsidiary of the [[Linux Foundation]].<ref>{{cite web |url=https://arstechnica.com/information-technology/2022/09/meta-spins-off-pytorch-foundation-to-make-ai-framework-vendor-neutral/ |title=Meta spins off PyTorch Foundation to make AI framework vendor neutral |date=12 September 2022 |website=[[Ars Technica]] |last=Edwards |first=Benj |access-date=13 September 2022 |archive-date=13 September 2022 |archive-url=https://web.archive.org/web/20220913034850/https://arstechnica.com/information-technology/2022/09/meta-spins-off-pytorch-foundation-to-make-ai-framework-vendor-neutral/ |url-status=live }}</ref>
Line 40 ⟶ 38:
==PyTorch tensors==
{{main|Tensor (machine learning)}}
PyTorch defines a class called Tensor (<code>torch.Tensor</code>) to store and operate on homogeneous multidimensional rectangular arrays of numbers. PyTorch Tensors are similar to [[NumPy]] Arrays, but can also be operated on by a [[CUDA]]-capable [[Nvidia|NVIDIA]] [[Graphics processing unit|GPU]]. PyTorch has also been developing support for other GPU platforms, for example, AMD's [[ROCm]]<ref>{{cite web|url=https://rocm.docs.amd.com/projects/install-on-linux/en/latest/how-to/3rd-party/pytorch-install.html|title=Installing PyTorch for ROCm|date=9 February 2024|website=rocm.docs.amd.com}}</ref> and Apple's [[Metal (API)|Metal Framework.]]<ref>{{Cite web |title=Introducing Accelerated PyTorch Training on Mac |url=https://pytorch.org/blog/introducing-accelerated-pytorch-training-on-mac/ |access-date=4 June 2022 |website=pytorch.org |language=en |archive-date=29 January 2024 |archive-url=https://web.archive.org/web/20240129141050/https://pytorch.org/blog/introducing-accelerated-pytorch-training-on-mac/ |url-status=live }}</ref>
PyTorch supports various sub-types of Tensors.<ref>{{cite web |url=https://www.analyticsvidhya.com/blog/2018/02/pytorch-tutorial/ |title=An Introduction to PyTorch – A Simple yet Powerful Deep Learning Library |website=analyticsvidhya.com |access-date=11 June 2018 |date=22 February 2018 |archive-date=22 October 2019 |archive-url=https://web.archive.org/web/20191022200858/https://www.analyticsvidhya.com/blog/2018/02/pytorch-tutorial/ |url-status=live }}</ref>
Line 78 ⟶ 76:
# Output: tensor(-2.1540)
print(a[1, 2]) # Output of the element in the third column of the second row (zero
# Output: tensor(0.5847)
print(a.max())
# Output: tensor(0.8498)
</syntaxhighlight>
The following code-block defines a neural network with linear layers using the <code>nn</code> module. <syntaxhighlight lang="python3" line="1"> from torch import nn # Import the nn sub-module from PyTorch
Line 91 ⟶ 92:
self.flatten = nn.Flatten() # Construct a flattening layer.
self.linear_relu_stack = nn.Sequential( # Construct a stack of layers.
nn.Linear(28 * 28, 512), # Linear Layers have an input and output shape
nn.ReLU(), # ReLU is one of many activation functions provided by nn
nn.Linear(512, 512),
| |||