PyTorch

from Wikipedia, the free encyclopedia
PyTorch

Pytorch logo.png
Basic data

developer Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan
Publishing year 2016
Current  version 1.6.0
( July 28, 2020 )
operating system Linux , MacOS , Windows
programming language Python , C , CUDA
License Open source
pytorch.org

PyTorch is on machine learning oriented open source - program library for the programming language Python , based on the in Lua written library Torch that has existed since of 2002. PyTorch was developed by the Facebook research team for artificial intelligence . The non-profit organization OpenAI announced at the end of January 2020 that it would rely on PyTorch for machine learning.

Features and version history

The program library can be used to create accelerated tensor analyzes with GPUs and to create neural networks based on a tape-based Autograd system. Proven Python libraries such as NumPy , SciPy and Cython can be used. In deep learning , the program library is characterized by great flexibility and high speed. ONNX for exchanging models with other program libraries is supported. TorchScript documents can be converted into PyTorch models by a compiler . TorchScript can be executed independently of Python and has been included in PyTorch since version 1.2.

The 1.3 version released in October 2019 enables the use of PyTorch on the mobile platforms Android and iOS (PyTorch Mobile). An 8-bit quantization model is also intended to make deployment on servers and edge devices more efficient. As the traditional implementation auswies of tensors defects tensors can since version as Named tensor be named. Version 1.4 from January 2020 added pruning for artificial neural networks and parallel training of models with remote procedure call . A connection to Java was also added. TorchServe has been used as an open source server for PyTorch since version 1.5, which was released in mid-April 2020 . In addition to Facebook, the cloud provider Amazon Web Services (AWS for short) is also involved in the service. Since this version, a C ++ front-end API can also be used.

Libraries and Platforms

PyTorch is composed of several libraries and platforms that are used for machine learning . Below is a list of the individual elements of PyTorch with an overview of the most important functions:

Surname Functions
AllenNLP Design and development of natural language processing models
advertorch Dealing with deviating training data
BoTorch Modules for Artificial Neural Networks , GPU and Autograd support
ELEVEN Solving computer games
fastai Image recognition / text recognition / spreadsheet / collaborative filtering
flair Natural language processing (computational linguistics)
glow Compiler optimization and code generation of neural network graphs
GPyTorch Gaussian processes with calculations by the graphics processor
Horovod Distributed deep learning with performance optimizations through transmissions between the nodes based on the message passing interface .
ignite Training of Artificial Neural Networks
ParlAI Exchange of large data sets for training and testing deep learning applications
pennylane Quantum computing in the field of machine learning , automatic differentiation and optimization
PySyft Data security / data processing
PyTorch geometric Recognition of geometric patterns
PyTorch Lightning automation
Pyro Statistics , prognosis and probability calculations
scorch Implementation of scikit-learn functions
TensorLy Tensor analysis / tensor algebra
Translate Machine translation

Important libraries in PyTorch for machine learning are torchvision for image recognition , torchtext for text recognition and torchaudio for speech and audio recognition .

example

The following example is to show the functionality of the library using a simple example 

 1 import torch
 2 dtype = torch.float
 3 device = torch.device("cpu") # Hiermit werden alle Berechnungen auf der CPU ausgeführt
 4 # device = torch.device("cuda:0") # Hiermit werden alle Berechnungen auf der GPU ausgeführt
 5 
 6 # Erstellung eines Tensors und Befüllung des Tensors mit Zufallszahlen
 7 a = torch.randn(2,3,device=device,dtype=dtype)
 8 print(a) # Ausgabe des Tensors A
 9 # Ausgabe: tensor([[-1.1884,  0.8498, -1.7129],
10 #                  [-0.8816,  0.1944,  0.5847]])
11 
12 # Erstellung eines Tensors und Befüllung des Tensors mit Zufallszahlen
13 b = torch.randn(2,3,device=device,dtype=dtype)
14 print(b) # Ausgabe des Tensors B
15 # Ausgabe: tensor([[ 0.7178, -0.8453, -1.3403],
16 #                  [ 1.3262,  1.1512, -1.7070]])
17 
18 print(a*b) # Ausgabe einer Multiplikation der beiden Tensoren
19 # Ausgabe: tensor([[-0.8530, -0.7183,  2.58],
20 #                  [-1.1692,  0.2238, -0.9981]])
21 
22 print(a.sum()) # Ausgabe der Summe aller Elemente im Tensor A
23 # Ausgabe: tensor(-2.1540)
24 
25 print(a[1,2]) # Ausgabe des Elements in der dritten Spalte der zweiten Zeile
26 # Ausgabe: tensor(0.5847)
27 
28 print(a.min()) # Ausgabe des Minimumwertes im Tensor A
29 # Ausgabe: tensor(-1.7129)

Web links

literature

German

  • Tariq Rashid: Programming neural networks yourself: An understandable introduction with Python, O'Reilly, 2017, ISBN 9783960101031
  • Delip Rao, Brian McMahan: Natural Language Processing with PyTorch: Creating Intelligent Language Applications with Deep Learning, Dpunkt.Verlag, 2019, ISBN 9783960091189
  • Sebastian Raschka: Machine Learning with Python: the practical manual for data science, predictive analytics and deep learning, mitp Verlag, 2017, ISBN 9783958454231
  • Ramon Wartala: Practical introduction to deep learning: Creating your own deep learning applications with Python, Caffe, TensorFlow and Spark, O'Reilly, 2018, ISBN 9783960101574

English

  • Chitra Vasudevan: Concepts and Programming in PyTorch, Chitra Vasudevan, 2018, ISBN 9789388176057
  • Sherin Thomas, Sudhanshu Passi: PyTorch Deep Learning Hands-On: Build CNNs, RNNs, GANs, reinforcement learning, and more, quickly and easily, Packt Publishing Ltd, 2019, ISBN 9781788833431
  • Pradeepta Mishra: PyTorch Recipes: A Problem-Solution Approach, Apress, 2019 ISBN 9781484242582
  • Samuel Burns: Python Deep Learning: Develop Your First Neural Network in Python Using Tensorflow, Keras, and Pytorch, Independently Published, 2019, ISBN 9781092562225
  • Vishnu Subramanian: Deep Learning with PyTorch: A practical approach to building neural network models using PyTorch, Packt Publishing, 2018, ISBN 9781788626071
  • Hyatt Saleh: Applied Deep Learning with PyTorch: Demystify neural networks with PyTorch, Packt Publishing, 2019, ISBN 9781789807059
  • Eli Stevens, Luca Antiga: Deep Learning with Pytorch, MANNING PUBN, 2019, ISBN 9781617295263
  • David Julian: Deep Learning with PyTorch Quick Start Guide: Learn to train and deploy neural network models in Python, Packt Publishing, 2018, ISBN 9781789539738

Individual evidence

  1. ↑ Release 1.6.0 . July 28, 2020 (accessed July 29, 2020).
  2. ↑ Serdar Yegulalp: Facebook brings GPU-powered machine learning to Python . In: InfoWorld . ( infoworld.com [accessed January 26, 2018]).
  3. ↑ Ben Lorica: Why AI and machine learning researchers are beginning to embrace PyTorch . In: O'Reilly Media . August 3, 2017 ( oreilly.com [accessed January 26, 2018]).
  4. ^ Nikhil Ketkar: Introduction to PyTorch . In: Deep Learning with Python . Apress, Berkeley, CA, 2017, ISBN 978-1-4842-2765-7 , pp. 195–208 , doi : 10.1007 / 978-1-4842-2766-4_12 ( springer.com [accessed January 26, 2018]).
  5. ^ Mo Patel: When two trends fuse: PyTorch and recommender systems . In: O'Reilly Media . December 7, 2017 ( oreilly.com [accessed January 26, 2018]).
  6. ^ John Mannes: Facebook and Microsoft collaborate to simplify conversions from PyTorch to Caffe2 . In: TechCrunch . ( techcrunch.com [accessed January 26, 2018]).
  7. ↑ Tech giants are using open source frameworks to dominate the AI ​​community | VentureBeat. Retrieved January 26, 2018 (American English).
  8. ↑ heise online: Machine Learning: OpenAI will use Facebook's PyTorch in the future. Retrieved May 18, 2020 .
  9. ↑ Alexander Neumann: PyTorch: A deep learning framework from Facebook. Retrieved January 26, 2018 (German).
  10. ↑ What is ONNX (Open Neural Network Exchange)? Retrieved August 30, 2019 .
  11. ↑ a b heise online: Machine Learning: PyTorch 1.2 improves scripting and export. Retrieved August 30, 2019 .
  12. ↑ heise online: Machine Learning: PyTorch 1.3 relies on mobile devices. Retrieved October 14, 2019 .
  13. ↑ heise online: Machine Learning: PyTorch 1.4 opens for Java. Retrieved May 18, 2020 .
  14. ↑ heise online: Machine Learning: TorchServe serves PyTorch 1.5 as a new open source server. Retrieved May 18, 2020 .
  15. ↑ PyTorch. Retrieved August 30, 2019 .
  16. ↑ https://www.bigdata-insider.de/was-ist-horovod-a-846416/. Retrieved August 30, 2019 .