DMOZ

from Wikipedia, the free encyclopedia
Open Directory Project
DMOZ logo
www.dmoz.org
motto Humans do it better
description Web directory
languages multilingual
Published 1998
status closed since March 17th, 2017, will be continued at curlie.org
Curlie logo

The Open Directory Project ( ODP ), also known as dmoz (for Directory Mozilla), was the largest human- edited web directory on the World Wide Web . The contents of the project are free and have been edited and updated by volunteer editors , the so-called editors .

On February 28, 2017 it was announced that the directory would be closed by the operator AOL on March 17, 2017 . Nothing was said about the reasons.

The follow-up project was developed under the name Curlie and has been online since the end of 2017. The domain curlie.org has an Alexa Rank of 122,206 as of August 2019.

history

From "Gnuhoo" to the "Open Directory Project"

The ODP was originally founded under the name Gnuhoo and went online on June 5, 1998. The name was later changed to Newhoo after a Slashdot article pointed out that Gnuhoo was not based on free software and that Gnuhoo was using the trademark of the GNU project without permission.

Newhoo became the Open Directory Project after it was purchased by Netscape in October 1998 and published its content under an Open Content license. In November 1998 Netscape including the ODP was bought by AOL , which merged with Time Warner in 2000 . The short form “Dmoz” or “dmoz.org” was derived from the first hosting address directory.mozilla.org.

On October 20, 2006, the ODP experienced serious technical problems. The functions for suggesting links and changes as well as the internal editing functions for changing catalog entries were blocked. On December 18, 2006, access for editors was released again; on January 13, 2007 the public interface for proposing sites and changes and on February 23, 2007 the possibility to apply for the new editor.

motivation

The motivation for founding the ODP was the frustration over the lack of topicality, i. H. the long delay in adding new sites and the high percentage of dead links in editorially maintained directories. But also criticism of commercializing the Internet more and more and reducing it to an advertising medium instead of giving free access to information a higher priority.

Assuming that humans are superior to automated catalog and search systems, the user should be provided with a qualitatively better, condensed search result. The expression for this was the slogan Humans do it better to distinguish it from automated search engine concepts.

However, the inadequate timeliness and long processing time of the ODP was also criticized. The delay in the recording resembled in many categories of those who formerly of Yahoo was common. The editor community was struggling to keep pace with the growth of the Internet , and many of the category branches lacked editors to handle the large number of registrations.

Imitators

The idea inspired at least two other web directories , also funded by private companies and edited by volunteer editors: The Go Directory (formerly operated by Disney ) and Zeal (purchased by LookSmart ). Both have now been discontinued. However, none of these directories published their data as open content .

structure

The Open Directory Project was based on a classified tree structure in the hypertext system. From the point of view of the origin, categories were derived thematically or regionally and inherited down to the sub-categories in 89 languages ​​and dialects (as of June 2014). Each entry could be assigned to a suitable category according to the highest relevance for a topic or region. Multiple entries were also possible if they were relevant in several places.

use of data

The ODP content was freely accessible to the public free of charge. In addition, a free right of use for further distribution as complete or partial publication was granted. The data was available for download in an early form of the RDF format. A new version was usually released every week. The data of the self-contained ODP set (dump) contained both the URL entries and the directory structure including the links between categories.

The dissemination of the data took place under Open Directory Project license conditions. These provided for a visible ODP source reference with links for suggesting new entries and for applying to work with the ODP on each individual page that contains the content of the ODP. Due to this regulation, the content was often taken over from other sites. Among other things, Google also used the Dmoz data for its own web directory until 2011.

Editors

The ODP described itself as a project by volunteer editors who systematically catalog information published on the Internet and make it publicly available. Editors are only responsible for certain categories and branches. For this purpose, as many different editors as possible, each with the best knowledge in a special field and area of ​​interest, were entrusted by consensus.

Despite the possibility of proposing pages, the directory grew mainly through own research by editors. This also included cross-sectional activities, such as creating new sub-categories, moving entries into more suitable categories and updating entries and comments.

The community of editors built up

  • the publicly available guidelines
  • Trust in the competence, responsibility and reliability of the individual
  • a multi-eye principle, with mutual coordination of cases of doubt and when training new editors
  • Assistance from higher-level editors (metaeditors) and administrators

on. In addition to maintaining the existing database, the work of editors mainly included the publication of new entries in the category they were working on. Working in leisure time was not mandatory and not tied to fixed times.

The application process as a new editor or for new categories and extension of the editing rights comprised three example links and descriptions as well as questions on the topic. Quality and the prevention of misuse play an overriding role, also because of the transfer of the data under open license to other data users. A minimum previous education or references or similar were not required.

Just as editors could resign from their rights and leave the ODP, active editors could be excluded from the project at short notice if the guidelines were violated due to abuse, unreliability and irresponsible behavior.

Data acquisition and maintenance

DMOZ entries consisted of a short title (usually the name of the website or company) and a brief description of the content, offer and special features of a website.

The inclusion criteria have not changed since the directory was started. The entry should be accompanied by an informative, assignable, serious and relevant information offer with its own content on a website. The information content of a website is determined by the degree of its own content in relation to advertising (banners, Adwords, Adlinks), external integrated pages and information copied from other pages. Websites that glorify violence, are racist, pornographic or encourage criminal activities were not included or removed from the database.

In addition, there were a number of mechanisms and tools intended to improve data quality and the relevance of categories as well as to rule out abuse. For this purpose, a form was available in each category for external inclusion suggestions, one for updating requests for existing entries and one for checking suspected abuse (by metaeditors).

In addition to the work of the editors, the company's own web crawler named Robozilla was sent over all entries in the directory at intervals in order to find dead, impaired or suspicious links. Robozilla temporarily removed them from the directory and marked them for review by editors who can take further action.

Entries from expired domains were largely automatically removed from the directory in order to ensure that they were up to date and to prevent false or misleading content through hijacking and "parking" of Internet addresses.

Over time, the number of permanent employees assigned to the project by Netscape and AOL decreased. This was compensated by a series of tools that were created by editors and introduced in a qualified manner in consultation with the administration. To make the work easier, link checkers, specialized web crawlers , spell checkers, improved search engines and a number of bookmarklets for editors have been created.

Outsourcing

Over the years, editors have started a number of related projects autonomously, some of which are based on the software also used by the ODP. However, none of them has so far been as successful as the ODP itself.

One of these projects was integrated directly into the ODP as a directory in the directory: The branch for children and young people, kidmoz . There, selected sites specifically for the target group of children and young people were provided with appropriate descriptions and age information and listed. Organizationally, this part of the directory was relatively independent, even if the guidelines applied to both directories and a number of editors were active in both directories. It was founded in November 2000.

Another project specializing in collecting all data related to music and musicians is MusicMoz . This currently only exists in English, but a German-language branch is in preparation.

facts and figures

Number of entries in the German-speaking part of the ODP

On January 31, 2014, the ODP recorded over 4.2 million entries (almost 500,000 of them in the German-language part of the catalog) sorted into over 1,000,000 categories. In January 2016 there were just under 4 million entries, the total number of editors was over 90,000. In July 2016 there were 3,938,044 entries from a total of 91,441 editors.

The basis for the inclusion and maintenance of entries were the publicly accessible guidelines and instructions and whether a page information for the topic is useful and a valuable addition.

In September 2019, Curlie.org recorded 3,444,397 pages (426,509 of them in the German-language part of the catalog) in 91 languages ​​and 1,033,965 categories from a total of 92,019 editors.

Web links

Follow-up projects

Individual evidence

  1. www.dmoz.org ( Memento from November 14, 2017 in the Internet Archive )
  2. dmoz.org ( Memento from October 11, 2016 in the Internet Archive )
  3. dmoz.de. Retrieved October 17, 2018 .
  4. Dmoz will be closed - WebABC.info . In: WebABC.info . March 2, 2017 ( webabc.info [accessed March 23, 2017]).
  5. ^ RIP DMOZ: The Open Directory Project is closing . In: Search Engine Land . February 28, 2017 ( searchengineland.com [accessed March 1, 2017]).
  6. The follow-up project Curlie is online. Retrieved December 9, 2017 .
  7. Alexa Rank from curlie.org, August 2019
  8. Slashdot: The GnuHoo BooBoo ( Memento from February 28, 2014 in the Internet Archive )
  9. ^ Articles of Association ( memento of October 21, 2016 in the Internet Archive ) of Netscape with voluntary commitments to the web community
  10. Open Directory License ( Memento from October 20, 2016 in the Internet Archive )
  11. Herbert Braun: Google closes its labs. In: Heise Developer. July 21, 2011, accessed on July 30, 2011 : “At the same time, Google tacitly closed its Google Directory web catalog. This service, which was launched eleven years ago as a competitor to the Yahoo catalog, upgraded data from Netscape's dmoz directory with a ranking within the categories. "
  12. Questions and answers on applying to be an editor at the Open Directory Project ( Memento from October 18, 2016 in the Internet Archive )
  13. a b Open Directory: Editing Guidelines ( Memento of October 18, 2016 in the Internet Archive )
  14. Newsletter March 2001 ( Memento from March 1, 2017 in the Internet Archive ) with a presentation of the new project
  15. curlie.org