Program library

A program library (short library , English library , short lib ) referred to in the program is a collection of subprograms / routines that offer solutions to thematically associated problems. In contrast to programs, libraries are not independently executable units, but they contain auxiliary modules that are requested by programs.

In a broader sense, program libraries (sometimes also called “component library” or “class library”) are all types of libraries that provide / contain program code (components). In this respect, a distinction is made between program libraries and a. according to the type of program code, e.g. B. source texts, macros, object or bytecode, machine code, etc. Accordingly, libraries are used at different times, some only in the context of software development (from tools of the development environment ), others only for the execution of programs, still others as a hybrid of both. Such libraries often contain not only subprograms, but program code parts of all program types .

Frameworks are a special form of program libraries .

access

Possible access to functions of a program library are defined by the programming interface (API). This is the entirety of the publicly available functions and classes; in contrast to the private units of the library that are not accessible.

Some proprietary program libraries are not published in the source code because they represent company secrets. To protect against decompilation , an obfuscator is then often used and all symbols ( variable and jump address names ) are removed.

Storage forms

Depending on the operating system and the development environment, program libraries and their contents can be saved in different forms and structures, for example:

The library is a file directory , ^a its elements / components are individual files . ^a
The library is a file that ^a identifies the components contained therein are of the programs of the development environment or a special library management software and processes.

Examples: A so-called 'DLL' from Microsoft or a PO file from IBM mainframes as a library for source texts, object modules or executable load modules.

Different types of libraries are managed in a common file ^a , the development environment can distinguish / process this. Example: An 'MDB' in MS Access contains libraries with source code, macros, pre-translated pseudocode and other code types.
There is no 'library'; the components are saved and executed as individual files, e.g. B. as "EXE files".

^a managed by the usual file management system of the operating system

Static libraries

A “static library” is a program library that contains modules / subprograms that are linked to the compilation of another program by a so-called linker . The linker generates an executable file or (depending on the operating system) a load module in a load library, in which the modules called by it are permanently (statically) integrated / attached, usually for a main program .

An optimizing linker only looks for those components (subroutines or data) from the assigned object modules (library files) that are actually called (referenced) by the program (and for which there is no overwriting implementation in the program ) and then appends them the program. The resulting file is correspondingly larger. Simple linkers simply add the complete object module or the complete library and thereby enlarge the program even more.

A static library is generally itself the result of source texts, divided into several modules, the compilations ( object modules ) of which are then put together by the linker to form the library.

Dynamic libraries

Components from dynamic libraries are only loaded into the main memory via a so-called loader while a program is running . This is done either by an explicit instruction by the program or implicitly by a so-called runtime loader if the program has been dynamically linked . Nowadays this is mostly done with the support of the operating system, since (see below) dynamic libraries are dealt with separately with regard to swapping and displaying in the program address space. The loader is therefore mostly an operating system component today.

Dynamically linked programs do not have to worry about loading the required dynamic components themselves. With dynamic linking , library and compilation are only loosely linked. Instead of copying the necessary symbols ( variable and jump address names ), as is the case with static linking , they are only referenced. A so-called runtime link takes care of loading the dynamic libraries and searching for symbols . The referenced symbols are resolved either immediately (before the actual program is started or when loading from the library) or lazy ( lazy) when the symbol is used for the first time.

A typical use case for explicitly opening libraries (looking up the address of symbols by their name to use the symbol) is with plug-in architectures. Another scenario is an optional library that is used if it is present, but the lack of it is not an error.

One advantage of dynamic libraries is that programs that use a dynamic library benefit from bug fixes in the library without having to be recompiled. For example, if an error is found in the OpenSSL library and corrected (the corresponding library has been replaced), it is sufficient to restart the programs that use this library to correct the error in these programs as well.

Since the files in which dynamic libraries are stored are only read and executed during use, but not changed, operating systems with virtual memory only have to load dynamic libraries once and can then display them in the address space of all processes using them. This is advantageous, for example, in multitasking systems if the libraries are very large overall and are used by many processes at the same time. (If the dynamic library has statuses or stored data, this can be intercepted with copy-on-write .)

However, many modern operating systems do not load the dynamic libraries immediately, but instead display them directly from the hard disk if necessary - they are treated like swapped-out pages . Similarly, pages that are not required and that belong to a dynamic library and have not been changed are simply discarded. If necessary, they can be reloaded from the hard drive.

Examples

Program libraries provide components in certain contexts to which they (must) fit in terms of their construction and their interfaces. Accordingly, there are program libraries, for example, in the following contexts:

Features of the operating system

Other system and utility programs

Sub-components of development environments and programming languages

Components of standard software products

Components of individual software that are managed by the users in individually defined libraries.

GUI toolkits for creating graphical user interfaces

Libraries in various programming languages

Libraries in programming languages contain services that are not implemented in the compiler, but are programmed in the language itself and are available to the programmer together with the compiler or completely separately from it. In the first case, the library is usually specified in the language description. In the second case one speaks of an external library.

Libraries specified in the language description differ greatly in scope.

language	Parts / packages	Headers / classes	Functions / methods / constructors
C (C89 + Amendments)	1	18th	142
C (C99)	1	24	482
C ++	1	32 + 18 (C89)
Java 2 (JDK 1.2)	62	1,287	≈ 18,000
Java 6	202	3,850	21,881
.Net 1.0	41	3,581	35,470
.Net 1.1	43	3,818	37,556
.Net 2.0	51	7,419	74,607
.Net 3.0	80	10,639	102,613
.Net 3.5	98	11,417	109,657

Libraries in different operating systems

Windows

The operating systems Windows and OS / 2 is a library file that binds dynamically as DLL (for D se Dynamic L ink L ibrary called). Accordingly, these files usually have the file extension .dll . Their file format is New Executable (16-bit), Linear Executable (32-bit OS / 2) or Portable Executable (32- or 64-bit Windows).

Windows distinguishes between several types of DLLs:

Entry-level DLL
ActiveX -DLL
.NET Assembly ( CLI )
Windows Metadata ( CLI , file extension .winmd)

Entry-level DLLs contain functions, while ActiveX-DLLs contain classes .

DLLs up to Windows 98 and Windows NT 4.0 cannot be controlled - any program can exchange them and could possibly damage the operating system. Windows Me , Windows 2000 and the subsequent versions have system protection that also includes the DLLs.

advantages

In addition to code, data (for example dialog resources) can also be used jointly by several processes.
DLLs are often linked statically, but can also be linked dynamically . Dynamic here means that the DLL is explicitly loaded by the program at runtime and the functions that are in the DLL are linked to the program "manually". This makes it possible to change the functionality of the program at runtime by exchanging the DLL.
DLLs can be maintained independently of the main program. This means that functions in the DLL can be changed without the program's knowledge. Then the DLL is simply exchanged (the old DLL file is overwritten) without having to change the main program.
Since the DLL has to be included with the main program as an independent file, code providers can better ensure that programmers who use the functions of their DLL also pay for it. The functionality of the DLL does not disappear (as with a library) in the code of the program. Free software advocates see this advantage as a disadvantage.

disadvantage

Changes in DLLs often result in changes in the program. This easily leads to version conflicts that are often very difficult to track down. One of the basic ideas of the DLLs was to share program code between several programs in order to save memory. In practice, however, many programs write DLLs into the Windows system directory during installation that no other program can use except for this particular program. In addition, the development, and in particular the connection, is more complex than for the static library.

In practice, however, this no longer has any meaning, since since at least Windows 2000 most applications have included private libraries with their installation, i.e. a version conflict in the sense of a DLL conflict is excluded. Under Windows, libraries that are stored in the application's program folder have a higher priority than those that are available system-wide.

Unix-like systems

On Unix-like operating systems (such as Linux ), static libraries use the file suffix .a(from "archive") and can be viewed and edited with the UNIX programs arand nm. In systems that offer package management, static libraries are often located in a separate development package together with the header files .

For dynamic libraries, the term shared library (English for "shared library") is common. The Executable and Linking Format (ELF) , the standard format used by Linux, FreeBSD and Solaris , is very popular . The extension .so(from shared object , "jointly used object") has become established for these files . As a rule, the library name is followed by the version number of the binary interface (ABI version) so that several versions of a library can be installed at the same time. Meta-information about an existing one .socan also be readelfqueried, for example .

Shared libraries on Linux (with the exception of some low-level libraries) usually begin with the prefix “ lib”. The actual library file contains the full version number. Symbolic links enable access via the soname and access without specifying the version. Example:

libfoo.so -> libfoo.so.1
libfoo.so.1 -> libfoo.so.1.2.3
libfoo.so.1.2.3

The soname is in this case " libfoo.so.1". The number after " .so." only changes if the library interface changes in a new version.

The runtime linker selects a version with a compatible interface based on version information in the program to be executed or in the library to be loaded . Since both programs can depend on libraries and libraries can depend on other libraries, software can indirectly depend on many libraries. Since many Unix systems manage the libraries centrally for the operating system and application, there are package administrations that calculate dependencies and can automatically install the required libraries. Examples of such package management systems are APT and YUM .

On some systems, the UNIX program can be used to lddfind out which libraries a program directly depends on .

In the case of software that is to be distributed in the binary version, for example commercial closed-source software or portable / distribution-independent software, it must be ensured that all required libraries are available. The user system on which this software is to be used must then be able to act as a compatible binary platform . This can be done as follows:

Specification of one or more compatible operating system distributions, for example: "Runs under Debian 5.0 (Lenny)".
Use of a distribution-specific package so that the required libraries are reloaded by the package manager. The disadvantage is the enormous effort involved in providing a package for the many existing distributions.
Delivery of all dynamic libraries. These are copied to a separate directory together with the program, documentation, etc. so as not to collide with versions that were installed system-wide by the package management. Subsequent adaptation of the library search path for the program using LD_LIBRARY_PATHa script or manually.
No dynamic libraries and static linking , which is technically difficult or can be prevented by license conflicts. You also lose distribution-based update support.
Via private libraries that are integrated via a relative library path. To do this, the application must be created with the linker option $ORIGIN.

Object and component-oriented approaches can be implemented here by instantiating and returning the corresponding object or component in a function.

advantages

Sharing resources
Avoidance of redundant private libraries
An enormous number of free and supplied libraries are available
Libraries can be repaired independently of the main program and program updates become smaller
Optimizations in one place can accelerate the whole system
Can quickly hide changing system calls
Abstracting low-level problems

disadvantage

Problems in essential libraries make systems unusable. With modern distributions, however, appropriate scripts in the package management usually ensure that this cannot happen.
In the event of incompatible changes to the binary interface , all dependent programs must be recompiled. Version numbers and the parallel use of packages reduce the problem of binary compatibility, but it does not solve it.
In the event of incompatible changes to the programming interface , all dependent programs must be adapted accordingly.
The creation of portable or cross-distribution programs becomes more difficult.

Java

Java is its own platform and uses a library concept that is not tied to the operating system. Basically, no distinction is made between program and library. All classes are compiled in the form of .class files and are loaded when required. As a rule, libraries, if they consist of several classes, are combined in a Java archive . The Java API itself is also available in the form of Java archives.

Since the individual classes of a library are only loaded at runtime, there may be delays when using the library for the first time until the class has been loaded and initialized. In Java for real-time systems , but also in Java SE , the class loader can use appropriate method calls to load certain or all of the necessary libraries when starting and also no longer unload them, so that there is no unexpected delay in use.

Libraries on z / OS

In z / OS as well as in the previous systems System / 360 and System / 390 , program libraries (like all types of libraries) are / were managed in the form of Partitioned Data Sets (PDS). For each program code type (see below) i. d. Usually own libraries are used and their elements are designated as members , regardless of type . The members are set there by various system programs in the development environment and used from there.

For each program code type, one (1) library is used for an entire company, for each corporate division or for specific areas of application, as required. The entries are differentiated by the member name. For example, the following types of program libraries are used:

Code / library types e.g. E.g. with z / OS (with 3GL language)

Source text libraries: These contain the source code of programs or subprograms, i. H. Declarations , functions and, depending on the programming language, other parts of code. Special versions of source texts are often managed in separate libraries; Examples are:

Copybook libraries; they contain fragments of source code, for example data declarations for certaintypes of data sets or functional code parts for use in many programs.

For example, macro libraries contain the instructions for the assembly language, which convert the macro call into ASSembler code during assembly and insert it in the source code.

The entries in source text libraries are usually created and modified with the help of text editors and are used by compilers / assemblers as input to generate compiled program code.

In object code libraries used by the translators object modules set, the compilation is usually a module. These modules are the input for the subsequent linking or binding required; this library is made known to the system program, also known as the Linkage Editor , as SYSLIB.

When binding, so-called loading modules are created , which are stored in a loading library , etc. a. also called “Core Image Library”. Load libraries can
- optionally (and usually) the main programs to be called (via JCL ) or the subprograms to be reloaded individually via the load command in separate libraries .

For execution , different libraries may be assigned via the STEPLIB, JOBLIB or LINKLIST specification and the individual modules are loaded from them by the operating system .

Libraries under Amiga OS

With AmigaOS , all libraries are used as shared libraries. They are requested by the program at runtime from the system, which then provides the base address of the library in memory (up to OS3.9) or the corresponding interface (from OS4.0). The program then uses relative addresses to get to the actual functions (after the base address) via a jump table in front of the base address. These functions are reentrant (reentrant).

Even if the library is changed, the existing entries in the jump tables are always the same. New entries may only be added at the end of the tables. This ensures downward compatibility.

As a special feature of AmigaOS, a minimum version number can be specified when opening a library, thus ensuring that the desired functionality is actually available. If this version is not found, the calling program can safely switch back to a simpler functionality such as that provided in the older library version.

Library files have the extension .library and are usually located in the LIBS directory : the system partition. When searching for a library, the operating system also checks the program directory of the requesting program.

Web links

Wiktionary: program library - explanations of meanings, word origins, synonyms, translations

Mike Hearn: Writing shared libraries Pitfalls and Best Practices for Creating ELF Libraries, 2004

Individual evidence

↑ ^a ^b Rick Anderson: The End of DLL Hell . microsoft.com. January 11, 2000. Archived from the original on June 5, 2001. Retrieved January 15, 2012: “ Private DLLs are DLLs that are installed with a specific application and used only by that application. "

↑ Arnaud Desitter: Using static and shared libraries across platforms; Row 9: Library Path ( English ) ArnaudRecipes. July 20, 2011. Archived from the original on July 20, 2011. Retrieved on January 26, 2012: “ win32 runtime library path :. and then PATH "

↑ Eric Brown: LSB 4.0 certifications aim to heal Linux fragmentation ( English ) linuxfordevices.com. December 8, 2010. Archived from the original on December 24, 2013. Info: The archive link was automatically inserted and not yet checked. Please check the original and archive link according to the instructions and then remove this notice. Retrieved on November 16, 2011: “ […] LSB helps to reduce fragmentation, it does not eliminate it. "The issue of packaging and broader dependencies is still a big one (for me) at least," writes Kerner. "The same RPM that I get for Fedora won't work on Ubuntu, and Ubuntu DEB packages won't work on SUSE etc etc." [...] " @1@ 2

^ Eskild Hustvedt: Playing well with distros ( English ) Linux Game Publishing . November 24, 2009. Archived from the original on September 21, 2011. Retrieved January 15, 2012.

↑ ^a ^b Eskild Hustvedt: Our new way to meet the LGPL ( English ) February 8, 2009. Archived from the original on April 13, 2014. Retrieved on March 9, 2011: “ You can use a special keyword $ ORIGIN to say 'relative to the actual location of the executable '. Suddenly we found we could use -rpath $ ORIGIN / lib and it worked. The game was loading the correct libraries, and so was stable and portable, but was also now completely in the spirit of the LGPL as well as the letter! "

↑ Evan Jones: Portable Linux Binaries ( English ) February 13, 2008. Retrieved January 10, 2012: “ Linux is not well known for its binary portability. Libraries vary from system to system, and the kernel interfaces have a tendency to change. [...] Recently, I needed to build a binary on one system, and run it on another. It only used standard C library functions, so I expected it to be easy. It was not. [...] "

↑ Christoph Baus: Yet another Unix nightmare: statically linking libstdc ++ ( English ) May 31, 2005. Archived from the original on February 10, 2010. Retrieved on January 15, 2012.

↑ ^a ^b Ulrich Drepper : Static Linking Considered Harmful ( English ) redhat.com . Archived from the original on May 27, 2010. Retrieved January 13, 2012: “ There are still too many people out there who think (or even insist) that static linking has benefits. This has never been the case and never will be the case. [...] "

↑ Mike Hearn: Random Collection of Current Linux Problems Binary Portability ( English ) Autopackage .org. 2006. Archived from the original on May 18, 2009. Retrieved on January 23, 2012: “ This page was prepared for the OSDL meeting in December 2005. It describes many of the problems inherent to Linux we've encountered whilst distributing complex software in binary form to end users. It also offers a few suggestions for improvements. "

↑ Troy Hepfner: Linux Game Development Part 2 - Distributable Binaries ( English ) gamedev.net. October 1, 2007. Archived from the original on October 13, 2007. Retrieved on December 19, 2011: “ Creating an executable that works on almost all Linux distributions is a challenge. There are a number of factors that contribute to the problem [...] "

↑ Simon Peter: AppImageKit Documentation 1.0 ( English , PDF; 38 kB) PortableLinuxApps.org. Pp. 2-3. 2010. Archived from the original on November 29, 2010. Info: The archive link has been inserted automatically and has not yet been checked. Please check the original and archive link according to the instructions and then remove this notice. Retrieved on July 29, 2011: " Not easy to move an app from one machine to another " @1@ 2

[endofdllhell-1] Rick Anderson: The End of DLL Hell . microsoft.com. January 11, 2000. Archived from the original on June 5, 2001. Retrieved January 15, 2012: “ Private DLLs are DLLs that are installed with a specific application and used only by that application. "

[sharedlibpolicy-2] Arnaud Desitter: Using static and shared libraries across platforms; Row 9: Library Path ( English ) ArnaudRecipes. July 20, 2011. Archived from the original on July 20, 2011. Retrieved on January 26, 2012: “ win32 runtime library path :. and then PATH "