Parallel computer
A parallel computer is a computer in which arithmetic operations can be carried out simultaneously on several main processors or graphics processors .
Areas of application
The following areas of application can be roughly distinguished:
Massively parallel processing
Distributed problems such as B. Weather forecasts . It is z. B. the earth's surface is divided into grid squares and a processor takes over the calculation for each grid square. In order to take into account the influences between neighboring grid squares, the processes on the different processors must exchange data with one another and be synchronized.
Computers designed for this type of task can contain several thousand main processors . The term massively parallel computer is used for this . In addition to weather forecasts, there are applications for such architectures, e.g. B. in all areas of simulations (see e.g. chemoinformatics , computer physics )
Pipelining
Problems in which large amounts of data are processed in several successive steps, so-called pipelining . For example, the modules of a compiler (preprocessor, lexical analysis, semantic analysis, optimization, code generation) can be implemented as processes running in parallel.
Each process forwards its output to the subsequent process and is therefore free to process the next data while the subsequent process step is being carried out by another process. Each of the processes can be assigned to a processor, so that an extensive real parallelization of the processing is achieved.
Problems of this kind are usually suitable for computers that have comparatively few processors.
In this case, one often speaks of multithreading or concurrency . However, for multithreading (thread) it is not absolutely necessary that the various processes (threads) each run on their own processor. It would also be conceivable to run the various processes in parallel on a single processor. However, the speed advantages cannot then be realized.
optimization
Parallel computers increase their working speed by distributing the computing effort among their processors. In order to utilize the full capabilities of a parallel computer, the programming must be adapted accordingly. A number of programming interfaces are available for this (see section below).
The distribution of the computing load over several processes always requires additional effort in order to organize and coordinate this distribution. This effort usually increases disproportionately with the number of processes . Depending on the existing dependencies, it is not always possible to parallelize processes. Implementing a task in such a way that it efficiently uses a parallel computer therefore requires a deep understanding of the problem, and a cost-benefit assessment must always be made in order to find an optimum for parallelization. It is important to use the scarce resources - computing time, memory access, data buses - efficiently. The sequential program overhead should always be minimal ( Amdahl's law ).
The type of networking of the hardware components also has an impact on efficiency . For many problems, good results can be achieved with the following topologies :
- Cube (configuration of 8 computers. Networking corresponds to a cuboid ).
- Hyper-Cube (n-dimensional cuboid)
GPU cluster
In recent years graphics cards have come onto the market, some of which have more than 2000 cores. Actually designed for the calculation of color gradients, textures, etc., these cores can also be used for the parallelization of other calculations. For this reason, computers that contain clusters of several hundred graphics processors or graphics cards have now also been built for massively parallel applications. This enables computer architectures to be achieved that contain a few hundred thousand processors instead of a few thousand.
Programming interfaces
- OpenCL Open Computing Language (interface for inconsistent parallel computers)
- CUDA Compute Unified Device Architecture (interface to use the graphics card for parallel calculations)
- OpenHMPP Hybrid Multicore Parallel Programming
- OpenACC
- OpenMP Open Multi-Processing
See also
- Computer network
- Field Programmable Gate Array (FPGA)
- Gustafson's Law
- Coprocessor
- Multi-core processor
- Multiprocessor system
- Exchange of messages
- Supercomputers
- Transputer
