Benchmark (computer)

from Wikipedia, the free encyclopedia
This article or section needs to be revised. Details should be given on the talk page. Please help to improve it , and then remove this flag.

Benchmarkings are standardized measurement and evaluation procedures that can be used to determine the performance of EDP systems or system classes and to compare them with one another according to certain criteria. The benchmark tests for hardware performance in computer systems are well known. Benchmarks are also used to compare the performance of programming languages, including interpreters and compilers and their runtime systems on the same hardware. Then one speaks of a software benchmark. A benchmark in general is always a program that solves a problem (often from mathematics, complex). Examples are solving linear systems of equations ( LINPACK ) or the calculation of Mersenne prime numbers .

history

  • 1970s: measurement in MIPS and FLOPS
  • Early 1980s: Measurement in Dhrystones and Whetstones
  • Since 1989: Standard Performance Evaluation Corporation (SPEC) application benchmark tests . It is a series of applications (e.g. neural networks, finite elements, video coding, etc.) which are divided into integer-related (SpecInt) and floating point-related (SpecFp) programs. Accordingly, the final result of the tests always forms two numbers that indicate the performance in relation to a reference machine (benchmark). With Spec2000 this is e.g. B. a Sun UltraSparcIIi / 256MB.

Hardware benchmarks traditionally only include CPU and memory access performance. Graphics card, mass storage, etc. are not tested.

The SPEC update took place in August 2006: SPECcpu 2006. A new version is being planned. A “SPEC CPU Benchmark Search Program” was started in 2008 for this purpose.

The best known benchmarks are:

Recently the focus of benchmarks has expanded. In particular, the inclusion of energy consumption as a metric has become established and is used in various SPEC (e.g. SPECpower) and TPC benchmarks.

In addition, it deals with how one can capture current trends in IT with the help of benchmarks. For example, there are currently several projects that deal with cloud benchmarking or benchmarking peer-to-peer systems.

Hardware benchmark example

Table of some Dhrystone values ​​as an example for hardware benchmarks :

Type microprocessor operating system Compiler Dhrystones / sec., No reg Dhrystones / sec., Reg
IBM PC / XT 8088-4.77Mhz PC / IX cc 257 287
VAX 11/780 UNIX 5.2 cc 1,515 1,562
Compaq PIII / 450 NT4 TCC 3.1 / 286/287 71,428
Compaq PIII / 450 NT4 wcc 10.5 / -otexan-fp5 250,000
FMS AthlonXP1800 SUSE 8.2 gcc 2,016,129
FMS AthlonXP1800 SUSE 8.2 gcc -O3 5,050,505

Software benchmarks

Software benchmarks are used to compare the performance of different programming systems in terms of execution speed. When creating a software benchmark, the same algorithm is implemented in different programming languages ​​and the execution times of the programs are compared with one another. It is necessary to bring together experts for the various programming languages ​​who can implement the algorithm as optimally as possible for the respective programming language. The Ackermann function is a classic example of such benchmarks .

Application benchmarks

Application benchmarks are used to assess the performance of computer systems when all hardware and software components interact. For this purpose, realistic application scenarios are created. The runtime of these automatically reproduced application scenarios allows a uniform comparison of different computer systems. Important parameters such as energy consumption and battery life for mobile devices can also be determined in a comparable way.

Examples of application benchmarks are

Manipulations

If the benchmark software used is known, there is scope for manipulation. In practice, device drivers , compilers or even instruction sets can be optimized in such a way that common benchmarks run particularly quickly.

See also

Individual evidence

  1. ↑ Frequently Asked Questions on the Linpack Benchmark and Top500 netlib.org , August 5, 2007, Accessed October 22, 2013
  2. ↑ Sysmark 2012 - Bapco announces new benchmark that AMD doesn't like. In: golem.de , June 22, 2011. Accessed October 10, 2013
  3. ↑ New 3DMark appears in: heise.de , January 31, 2013. Accessed October 10, 2013
  4. ↑ Tricky - Oddities in VideoLogic's Apocalypse 3D In: c't 4/97. Retrieved from heise.de on October 10, 2013

Web links