Related topics

New Search Tool Hits the Web

May 19, 1996 GMT

SAN FRANCISCO (AP) _ In the beginning, the World Wide Web was chaos, and that was okay.

But engineers and librarians knew an index was possible, and so they labored to bring order to the `library without a card catalog’ that was the Web. They created Lycos and Yahoo! and AltaVista, and it was good.

But not, some believed, as good as it might be. On Monday a new index of the Web will go public, one that purports to search every last one of the estimated 50 million or more Web pages currently in existence.

The new search tool, named HotBot, is a partnership between Inktomi Corp. of Berkeley, Calif., which built it, and HotWired Ventures of San Francisco, which runs it. Inktomi grew out of a university research project. HotWired was launched by the founders of Wired magazine as an online news and entertainment site.

With the search service, Wired becomes one of the first media firms to use something beyond traditional news and feature articles as a lure for delivering advertising messages on the Web.

``There’s certainly more traffic on a search service than on HotWired,″ David Pritchard, marketing director for HotBot. ``We’re trying to help advertisers reach an audience that is in a specific mindset and also, we hope, make advertising more relevant to individuals.″

HotWired is also the first to apply Inktomi’s technology for locating information on the World Wide Web.

Inktomi (pronounced ``ink-to-me″) is a Lakota name for a mythological spider. Like all search programs (called search engines in Internet-speak), it uses web-crawler or spider programs to wander the Web, indexing each page they comes across.

But whereas most current indexes work from a single, extremely large, fast -- and expensive -- computer, Inktomi takes a bunch of smaller computers and networks them together to do the work.

``By using a bunch of little machines instead of one big machine we get no limits on growth, better cost performance and better fault tolerance because if one node fails the rest can cover for them,″ said the company’s co-founder Eric Brewer. He is also a professor of computer science at the University of California at Berkeley.

Currently the largest Web index is AltaVista, the search tool created by Digital Equipment Corp. in Maynard, Mass., to show the might of its AlphaServer 8400 machines. The search engine proved so popular that Digital turned it into a product.


AltaVista index covers 30 million Web pages, a little more than half of those in existence. AltaVista’s Chuck Malkiel says it gets everything but pages that specifically say they don’t want to be indexed and corporate sites not meant for public view.

HotWired and Inktomi dispute that figure. They also said HotBot will index the entire Web weekly.

``If you put up a page yesterday, there will be a one in seven chance it’s in our index today,″ said Kevin Brown, spokesman for Inktomi. ``By the end of the week, it will be there.″

The HotBot site will also allow for extremely precise queries and will let users personalize their searches, then will remember how they searched this time in case they want to use the same kinds of perimeters next time, Brown said.

HotBot is specifically going up against AltaVista to be the most popular `brute force’ search tool, gulping down everything it sees and letting the user decide what they want to use.

Both work in a fundamentally different way than Yahoo!, still the most popular Web index of all.

While other indexes are created by computer search programs, Yahoo! uses actual people to sift through millions of Web sites and give users a manageable outline of what’s there. Not every page Web is searched, but Yahoo! creates a framework of those it looks at.

Inktomi’s arrangement with HotWired is exclusive as a stand-alone search engine. But it plans to pursue ventures with other companies that can use portions of its technology on the Web or on internal data networks that have the same technical standards, sometimes called intranets.


HotBot’s Web address is