Open Web Index

from Wikipedia, the free encyclopedia

Under the name Open Web index one summarizes a number of concepts and projects together, all of which have the goal of the contents World Wide Web not only on the databases of the global search engines - corporations find, but set the this access to the general public. Search engines of any kind can then use this database as an application.

Definitions of terms

A web index is an organized directory of the contents of the WWW. In the simplest case, it is a list of words in which for each word it is noted on which websites or URLs it can be found (comparable to the search index at the end of a book). Such an index is the basis of every search engine . Contents of the World Wide Web described by words (or other unambiguous character strings) can only be found if such an ordered directory is available.

An open web index is one that offers a publicly accessible interface with a publicly accessible description. In principle, anyone can then use this to search for content. Such Index is generated by software modules that web crawler or crawler are called.

In contrast, a proprietary index is one in which access is restricted to certain users or groups - mostly companies . Typical examples are the databases of the global search engines - corporations .

background

There are only a few search engines in the world that are based on their own comprehensive web index: USA ( Google , Bing ), China ( Baidu ) and Russia ( Yandex ). At the same time, the search engine provider Google is dominant in Germany and worldwide, sometimes in a kind of monopoly position. The Franco-German Quaero project was declared over in 2013.

Based on this experience and the analysis of the past development of the Internet, the German initiators demand the gradual creation of an open Internet index that should cover as large parts of the publicly accessible areas of the World Wide Web as possible. In the meantime, the Internet search and the necessary index service represent a basic infrastructure .

Implementation initiatives

There are several initiatives that pursue the goal of an open web index in slightly different ways: They all try to first create such an index and then make it publicly accessible. Below is a brief overview; without claim of completeness.

  • The oldest and internationally probably best known approach dates back to around 2011, based in the USA: the Common Crawl Initiative. However, Common Crawl does not create a searchable index, but rather gathers data from the web and makes it available.
  • Since around 2014 there has been a German project that started out from the HAW Hamburg, Department Information, and was taken up by Suma eV ; Abbreviated proper name OWI (for OPEN WEB INDEX), an initiative or a concept for the creation of an independent web index as a basis for offers on the Internet , such as map services , search engines or comparison services. The aim is to summarize the data distributed on the web under the umbrella of a central organization, which ideally could be located in the EU.
  • Since January 2019 there has been another organization in Germany that works with Suma eV and has the goal of combining and making accessible the distributed data of the web in a decentralized joint project consisting of data centers, civil society organizations and companies: abbreviated proper name OSF , for: OpenSearchFoundation.

Web links

Individual evidence

  1. ^ Christian Kunz: European counter model to Google? The Open Web Index - SEO Southwest. Accessed December 21, 2017 (German).
  2. ↑ Network Research: “Open Web Index” initiative - presented by N. Huss and A. Ude. Berliner Stammtisch with Nikolaus Huss and Albrecht Ude. www.netzwerkrecherche.org, March 21, 2016, accessed December 23, 2017 .
  3. ↑ Market shares of leading search engines in Germany from 2014 to 2016. de.statista.com, accessed on December 23, 2017 .
  4. ↑ Market shares of the most used search engines on the desktop according to page views worldwide in selected months from January 2015 to November 2017. de.statista.com, accessed on December 23, 2017 .
  5. ^ Christian Stöcker and Holger Dambeck: German-French search engine: Quaero has burst. The European answer to Google should be Quaero - but the Franco-German search engine project will probably not work. Now the Germans are practically working on something new on their own: Theseus. And Europe is threatened with a high-tech fiasco in the race with Google and Yahoo. www.spiegel.de, December 19, 2006, accessed on December 23, 2017 .
  6. ↑ December 31, 2013: le program Quaero s'achève. Quaero.org, January 28, 2014, accessed December 23, 2017 (French).
  7. ↑ Initiators. www.openwebindex.eu, 2017, accessed on December 23, 2017 .
  8. ^ Christian Stöcker: Initiative against Google: Scientists call for a European web index. The European search engine Quaero was an expensive disaster. Information and media scientists from Germany are now calling for an alternative model: a publicly financed web index. www.spiegel.de, February 18, 2015, accessed on December 23, 2017 .
  9. ↑ Regina Bruckner: Measurement of the world beyond Google. Our idea of ​​the world is shaped by artificial intelligence, algorithms and search engines, a Swedish foundation wants to correct them. www.derstandard.de, November 18, 2017, accessed on December 23, 2017 : “Mager wants to find out what role values ​​and visions play in European search engines: Open-Web-Index, Yacy and Startpage try to offer themselves as alternatives. The scientist does not believe that it could be too late in view of Google's market power. She was very keen on providing hundreds of millions of public funds to European initiatives. "Search is now a basic infrastructure." "
  10. ^ Initiative: "Common Crawl". Retrieved March 14, 2019 .
  11. ↑ Initiative OPEN-WEB-INDEX c / o SUMA-EV - Association for free access to knowledge: Imprint. www.openwebindex.eu, accessed on March 14, 2017 .
  12. ↑ a b Initiative OPEN-WEB-INDEX c / o SUMA-EV - Association for free access to knowledge: The Open-Web-Index. www.openwebindex.eu, 2017, accessed on December 21, 2017 : "The OPEN-WEB-INDEX (OWI) is a concept for an open digital infrastructure."
  13. ^ Roland Freist: Open Web Index. Open index internet business models. www.mittelstandswiki.de, January 20, 2017, accessed December 23, 2017 .
  14. ^ Initiative: "Open Search Foundation". Retrieved March 14, 2019 .