Grid computing

from Wikipedia, the free encyclopedia

Grid computing is a form of distributed computing in which a virtual supercomputer is created from a cluster of loosely coupled computers. It is designed to solve computationally intensive problems. Today, grid computing is used in many areas, in some cases also commercially, for example in pharmaceutical research and economics , in electronic commerce and in web services . It is also used for risk management in building dynamics and financial management.

Grid computing differs from typical computer clusters in the significantly looser coupling, the heterogeneity and the geographical dispersion of the computers. Furthermore, a grid is mostly intended for a special application and often uses standardized program libraries and middleware . In the database area in particular, grid computing can be combined with in-memory technologies.

The basics of grid computing come from Ian Foster , Carl Kesselman and Steven Tuecke (2001).

Basics

definition

The first attempt at a definition comes from Ian Foster and Carl Kesselman in the book "The Grid: Blueprint for a New Computing Infrastructure":

"A computational grid is a hardware and software infrastructure that provides dependable, consistent, pervasive, and inexpensive access to high-end computational capabilities."

"A computational grid is a hardware and software infrastructure that enables reliable, consistent, inexpensive access to the capacities of high-performance computers, accessible from anywhere."

Since this definition was written before the actual creation of grids, it was clearly refined by Ian Foster in the second edition of the book:

“The sharing that we are concerned with is not primarily file exchange but rather direct access to computers, software, data, and other resources, as is required by a range of collaborative problem-solving and resource-brokering strategies emerging in industry, science, and engineering. This sharing is, necessarily, highly controlled, with resource providers and consumers defining clearly and carefully just what is shared, who is allowed to share, and the conditions under which sharing occurs. A set of individuals and / or institutions defined by such sharing rules form what we call a virtual organization. "

“The resource sharing that we're dealing with here isn't primarily about sharing files, but rather providing direct access to computers, software, data, and other resources, as used in a number of collaborative, problem-solving, and resource-broking strategies that are currently emerging in industry, science and engineering. This sharing of resources is, of necessity, controlled to the greatest possible extent, with the providers and consumers of the resources clearly stipulating which resources are shared, who is allowed to share them, and under what conditions the sharing takes place. A lot of individuals and / or institutions that result from such resource sharing guidelines form what we call a Virtual Organization. "

The main difference to the original definition is that the sharing of resources is determined by virtual organizations. These also play a central role in today's implementations of the grid. Also, not only high-performance computers are now referred to as resources, but general resources such as physical experiments.

There are other attempts at a uniform definition in the literature (cf.).

Systems such as cluster computing , peer-to-peer computing or meta computing are incorrectly referred to as grid systems. Although these have aspects of grid computing, there are essential points missing to describe them as a grid. Ian Foster has addressed this problem in a 3-point checklist. The properties of a grid system are briefly defined as follows:

A grid ...
... coordinates resources that are not subordinate to a central authority ...
A grid coordinates and integrates resources and users from different administrative domains within the same company or within different countries. In this context, u. a. taken into account security, billing and membership (cf.). Resources can include clusters, mass storage devices, databases, applications or measuring devices.
... and uses open, standardized protocols and interfaces, ...
A grid uses open and general protocols and interfaces that guarantee basic functions for authentication, authorization, resource determination and resource access.
... to provide non-trivial quality of service .
A grid uses existing resources in a coordinated manner in order to provide different quality of service, depending on, for example, the response time, the throughput, the availability or security. Or, in order to meet complex user expectations, several resource types are reallocated so that the benefit of the combined system is significantly greater than the sum of its parts.

Volunteer computing projects (e.g. SETI @ home ) are generally not counted among the grid computing systems, since the computing power there, unlike grid computing, is not provided by organizations but by anonymous and possibly not trusted clients.

origin

Concepts for the distribution of computationally intensive tasks already existed in the 1960s. Most of today's research on grid systems, however, has its origins in early experiments with high-speed networks. In this context, the projects FAFNER and I-WAY should be mentioned in particular.

The term Grid has its origin in the comparison of this technology to the power grid (Engl. Power Grid ). Accordingly, the grid should just as easily provide a user with resources such as B. Provide computing power or storage space via the Internet, how it is possible to draw electricity from a socket. The user submits his order via standardized interfaces to the grid, after which the resource allocation is automatic.

I-WAY

The I-WAY (Information Wide Area Year) project was carried out in 1995 within the gigabit test environment at the University of Illinois, in which 17 facilities in the USA and Canada were connected to a high-speed network. Its task was to connect different supercomputers using existing networks. I-WAY supported the development of the Globus project to a great extent.

Goal setting

The motivating objective that led to the development of grid technology was the joint, coordinated use of resources and the joint solution of problems within dynamic, cross-institutional, virtual organizations (cf.). This means that after the accounting data and rights have been determined, direct access to, for example, computing services, applications, data or instruments should be made possible jointly. In this context, a virtual organization (VO) is a dynamic association of individuals and / or institutions that pursue common goals when using the grid. Although the focus of much of the work is on distributed computing, the primary goal, analogous to the development of the Internet, is the development of a uniform, global grid.

classification

Roughly speaking, grid computing can be divided into classes, such as

  • Computational grids (Computing Grids) : access to distributed computing resources
  • Data grid (DataGrid) : access to distributed databases
  • Resource Grids
  • Service grids
  • Knowledge grids

The class of the computing grid is comparable to the power grid , i.e. the power grid : For this purpose, the consumer of computing power establishes a connection to the computing network, similar to the electricity consumer to the power supply network. Everything that happens behind the socket is hidden from the consumer there; he simply uses the service offered.

In the data grid class, not only do the (high-performance) computer systems of those involved cooperate in order to provide computing power, but databases are also linked. A grid portal usually provides access to such grids .

In addition, the provision of network resources is "gridified", i. H. an automatic selection made from a pool of resources based on certain QoS parameters. Ideally, the choice of resources should be application-driven, i.e. dependent on the application in the computing grid or data grid.

Architecture and implementations

General

There are several concepts for the architecture of a grid . It is peculiar to every known concept that, in addition to the requesting authority and the actual performance requirement made there, there must be a coordinating authority for the agglomeration of computing power and for the merging of the partial services. In addition, a strict hierarchy is required which allows or excludes the agglomeration of computing power according to objective criteria. Each computer in the “ grid ” is initially a unit that is hierarchically equal to the other computers (peer-2-peer).

The typical tasks in which grid computing offers itself as a strategy are those that overwhelm the performance of individual computers. This includes, for example, the integration , evaluation and presentation of very large amounts of data from scientific and medical research. In the routine, the techniques are also used in meteorology and computationally intensive simulations in industry. Particle physics in particular with large-scale experiments (e.g. the Large Hadron Collider ) as a scientific application is a pioneer in the further development and establishment of grid technologies.

The typical problems that grid computing brings with it are the increasing effort as part of the available power for coordination. Because of the coordination effort, the computing power never increases linearly with the number of computers involved. This aspect takes a backseat in complex numerical tasks.

OGSA

One possible software architecture for grids is the Open Grid Services Architecture (OGSA) co-developed by Ian Foster . This is already being described in its predecessor, the Open Grid Services Infrastructure (OGSI). Their basic idea is the representation of the components involved (computers, storage space, microscopes, ...) as grid services in an open component architecture.

With the convergence of the web services of the W3C and the OGSA standard of the Open Grid Forum (OGF), grid services on a technical level, as implemented in Globus Toolkit 4, for example, have become web services that the technical functionalities of Enable grid middleware. In this context, OGSA suggests the use of WSRF (the Web Services Resource Framework ) as a basic building block for service grids. In this way, the web services, the use of which enables uniform access procedures to the individual services of a grid, also get a status: They are provided with stateful resources (such as files, Java objects or POJOs , data records in a database ) associated. This is what makes it possible to carry out functions that span several transactions. The combination of a web service with such a statusful resource forms a so-called WS resource .

Virtual organizations

A central and hardware-independent concept behind the grid philosophy is that of virtual organizations (VO, see there). Resources (or services ) are dynamically assigned to virtual organizations.

Implementations / grid middleware

hardware

In practical terms, all that hardware is needed is a computer with a network connection. On these grid computers, software takes on the task of solving a sub-task, which was generated, for example, with the help of software that can split a large task into a number of sub-tasks for all nodes in the grid and summarize the partial results again.

Scientific projects

EGEE / EGI

The EGEE project ( Enabling Grids for E- science, formerly Enabling Grids for E-science in Europe), which ended in March 2006, is the largest grid project in the European Union and is now used worldwide. It has been continued under the name EGEE2 since April 2006. The project was funded by the EU with 32 million euros in the first project phase and represents the world's largest grid infrastructure. In 2010 EGEE was replaced by European Grid Infrastructure (EGI).

Participants include CERN (Switzerland), Karlsruhe Institute of Technology (KIT, Germany), Rutherford Appleton Laboratory (RAL, United Kingdom), Istituto Nazionale di Fisica Nucleare (INFN, Italy) and Academia Sinica (ASCC, Taiwan). See also: Enabling Grids for E-science .

Northern Ugrid

The open grid Nordugrid based on the grid middleware ARC emerged from a merger of five Scandinavian institutes.

XtreemOS

Building and Promoting a Linux-Based Operating System to Support Virtual Organizations for Next Generation Grids is a project funded by the European Union in the 6th Framework Program . In addition to 17 European project partners, two from China are also involved in XtreemOS . It started in July 2006 and will run for four years.

Neugrid

Neugrid is a project funded by the European Union in the 7th Framework Program. Its infrastructure enables science to research neurodegenerative diseases.

National grid infrastructure initiatives

As in various other countries (e.g. US: Cyberinfrastructure, UK: UK e-Science / OMII, NL: BIG-Grid) there are also grid initiatives in Germany and Austria to set up a national grid infrastructure.

See also

literature

Web links

Individual evidence

  1. ^ I. Foster and C. Kesselman: The Grid: Blueprint for a new computing infrastructure. Morgan Kaufmann, Tech. Rep., 1998.
  2. ^ I. Foster and C. Kesselman: The Grid: Blueprint for a new computing infrastructure. 2nd edition, Morgan Kaufmann, Tech. Rep., 2003. ISBN 978-1-55860-933-4
  3. ^ A b I. Foster, C. Kesselman, and S. Tuecke: The Anatomy of the Grid: Enabling Scalable Virtual Organizations. International Journal of High Performance Computing Applications, vol. 15, no. 3, pp. 200-222, 2001.
  4. ^ I. Foster, C. Kesselman, JM Nick, and S. Tuecke: The physiology of the grid. IBM Corporation, Poughkeepie, NY 12601, Tech. Rep., June 2002.
  5. ^ I. Foster: What is the Grid? A three point checklist. July 2002.
  6. ^ I. Foster, C. Kesselman, G. Tsudik, and S. Tuecke: A security architecture for computational grids. in ACM Conference on Computer and Communications Security, 1998, pp. 83-92.
  7. ↑ M. Humphrey and M. Thompson: Security implications of typical grid computing usage scenarios. in Proceedings of Intl Symposium on High Performance Distributed Computing. (HPDC), San Francisco, CA., August 2001, pp. 7-9.
  8. ↑ David P. Anderson : Why volunteer computing is not grid computing. ( DOC , 84 kB) Boinc , accessed on December 22, 2015 (English).
  9. ↑ T. DeFanti, I. Foster, M. Papka, R. Stevens, and T. Kuhfuss: Overview of the i-way: Widearea visual supercomputing. in International Journal of Supercomputer Applications, vol. 10, 1996, pp. 123-130.
  10. ^ C. Catlett: In search of gigabit applications. Communications Magazine, IEEE, vol. 30, no. 4, pp. 42-51, 1992.