An aeon ago, we all searched the Web with Yahoo, Excite, Altavista and others. The results, helpful at first, deteriorated over time. If a user searched for, say, “Viagra”, search engines decided that the best results were pages that contained the word “Viagra” as many times as possible. Naturally, spammers created pages stuffed with fake keywords to lure in the innocent. And results became increasingly crowded with unwanted stuff.
Along came Google with a revolutionary thought: Relevance, or links from other Web sites as an indicator of importance. So if the most number of pages on the Internet pointed to Pfizer’s (Viagra’s manufacturer) site, that was probably the most authoritative one. Google called its algorithm “PageRank”. Its results were so good that its market share grew exponentially year after year. Today, seven out of ten searches are via Google. Merriam-Webster and the Oxford English Dictionary list “googling” as a verb.
Competitors have ploughed many millions into dethroning the king. But Google owes much to a combination of bad competition, exploding Web content and the rise of online advertising. No new player is going to get that lucky.
But that’s not the end of it.
Google’s algorithm, while much more complex and powerful than the original PageRank, is starting to show its age. Spammers know they must game Google, and they do. The rise of social networks and the real-time Web offers users the ability to search conversations minutes, even seconds old, and Google is mostly at sea here.
New entrants have realised the futility of going head-to-head with Google, and are happy creating better niche products. Here are four promising new engines that complement, not replace, Google:
From the wonderful people who brought you Windows: Bing
Snidely called “But It’s No Google”, Microsoft’s shiny new search engine comes closest to doing what Google does, sometimes better. Bing’s “decision engine” goes beyond basic search results in areas like travel, shopping or personal finance. This may put off people who prefer not to be “nudged” by their search engines, but most will like it. Bing has also innovated in the way results are presented: Hovering over a result will pull up a summary of the page or present related phrases.
Compute this: Wolfram|Alpha
Unlike traditional search, which lists sites where answers may lie, Wolfram|Alpha, the “computational knowledge engine,” actually computes answers for you. It has been amassing expert knowledge: 10 trillion+ “pieces of data” linked through over 50,000+ algorithms and models. Searching for “Infosys Wipro employees” not only throws up the exact number of employees as reported most recently by both companies, but also draws a comparative graph covering the last few years.
I heard it on the grapevine: Scoopler
Standard search algorithms, including Google’s, aren’t nimble enough to instantly index the “real-time Web” — instant messages, link- or photo-sharing, exchanges on social networks. Scoopler is a start-up that searches services like Twitter, Flickr and Digg for what users are talking about that very instant. A search for “iPhone” throws up a constantly-updating feed of conversational mentions. Most of the results are from Twitter, though, reflecting its domination of the real-time Web.
Machines and algorithms are only as good as they are made to be. Mahalo goes to the source, with the human approach. The company encourages users to “claim” and then “curate” expert pages around topics. Mahalo hopes that having people organise and present the results will come across as warmer and more meaningful to searchers. If you’re interested in, say, “Sushi” or “Bob Dylan” and don’t want reams of useless search results, try Mahalo’s well-curated pages on both subjects, stocked with lots and lots of information.
to read on Deep Web search engines
(This story appears in the 03 July, 2009 issue of Forbes India. You can buy our tablet version from Magzter.com. To visit our Archives, click here.)