World Wide Web Hikers

The World Wide Web Wanderer (also known just as Wanderer ) was developed in June 1993 by Matthew Gray as the first web crawler for the World Wide Web at the Massachusetts Institute of Technology to measure the size and growth of the Internet. It was based on the Perl programming language .

The Wanderer created an index that came to be known as the Wandex and continued doing its job until the project was discontinued in 1995.